SHARE
Assembled builds the unified support operations platform trusted by Stripe, Canva, Robinhood, Notion, and hundreds of other companies to manage AI and human agents across chat, email, and voice. With support quality as their top priority, Assembled needed a retrieval layer that could match the stakes, where a wrong answer erodes customer trust, and a missed document means a frustrated end-user.
After integrating ZeroEntropy's reranking into their retrieval pipeline and validating it on live production traffic, Assembled migrated to 100% of their reranking volume through ZeroEntropy.
The Challenge
Assembled's AI agents handle customer support across three channels: chat, email, and phone, each with different retrieval demands. Behind each one sits a hybrid search system combining dense vector search with keyword-based retrieval. After both retrieval paths return candidate chunks, the reranker decides which results actually make it to the LLM.
The reranker is the last line of defense before generation. If it gets the ranking wrong, the model hallucinates or gives a bad answer, and Assembled's customers see it immediately.
"Quality is probably the most important thing for us right now," said John Wang, CTO and co-founder of Assembled.
As Assembled scaled across verticals and use cases, they needed a reranker that was consistently more accurate, particularly in the long tail of domain-specific queries where retrieval failures are hardest to detect.
Evaluation
Assembled runs one of the more rigorous retrieval evaluation setups in their category. Their golden dataset has been curated over eighteen months across ten to fifteen different customers, with each customer contributing ten to fifty expert-annotated examples. Domain experts review production queries and label the exact source documents the system should have retrieved. The core metric is straightforward: where does the correct source appear in the top-k results, if it appears at all.
On top of this, Assembled maintains regression sets and hallucination benchmarks to catch degradation across model updates.
When evaluating ZeroEntropy, Assembled didn't rely on offline benchmarks alone. They ran long-term experiments across live production traffic, with an engineer monitoring performance over weeks to verify that ZeroEntropy's accuracy held steady at the level they had originally evaluated.
Results
After validating performance through production A/B experiments, Assembled migrated 100% of their reranking traffic to ZeroEntropy, moving from an initial partial rollout to full production deployment.
Consistent accuracy across channels. ZeroEntropy reranking performs reliably across Assembled's three distinct retrieval paths: real-time chat, latency-sensitive phone, and the heavy-duty email pipeline that scores every article in a customer's knowledge base.
Production-validated performance. Long-running experiments on live traffic confirmed that ZeroEntropy matched or exceeded the accuracy benchmarks set during initial evaluation, giving Assembled the confidence to consolidate their entire reranking volume onto a single provider.
Domain-specific improvements with zerank-2 prompting. Assembled leverages zerank-2's instruction-following capabilities to tailor reranking behavior for specific customer verticals. By prompting the reranker with domain context, the team improved retrieval relevance without any model changes, particularly on queries where the semantic intent was difficult to capture from keywords alone.
"I'm excited that you guys exist and are doing the good work here." said Wang.
What's Next
Assembled and ZeroEntropy are exploring several areas for deeper collaboration. Assembled's embedding layer can still be further optimized, and the team is evaluating ZeroEntropy's upcoming models as a potential upgrade.
On the retrieval intelligence side, Assembled is building feedback loops that identify knowledge gaps, surfacing cases where a customer's knowledge base simply doesn't contain the right documents, and prompting them to create better content.
About Assembled
Assembled is the unified support operations platform that combines workforce management with AI-powered issue resolution. Founded by three early Stripe employees, the company has raised $70.7M and serves customers including Stripe, Etsy, DoorDash, and Robinhood. Assembled's AI agents automate over 50% of support tickets while improving customer satisfaction. Learn more at assembled.com.
About ZeroEntropy
ZeroEntropy builds state-of-the-art retrieval models for AI applications. Its reranking, embedding, and search APIs power the most demanding retrieval pipelines, from clinical reasoning systems to enterprise customer support. ZeroEntropy's open-weight models consistently outperform alternatives on both public benchmarks and real-world production workloads. Backed by Y Combinator and Initialized Capital.
Learn more at zeroentropy.dev.
Get started with
RELATED ARTICLES
