
September 2023 Releases
Your AI agent is live and interacting with customers, but now the real work begins: improving its performance.
More often than not, your first version won’t get everything right, and neither will the second. Once real customers start interacting with your agent, unexpected questions and edge cases inevitably come up. Every interaction is a chance to build trust, and it only takes one frustrating experience to lose it.
Your agent may give answers that are technically correct, but irrelevant to customer requests. By the time customer complaints surface, the opportunity to influence their decisions has passed. Even if you become aware that agent responses are misaligned with customer intent, it remains difficult to determine which Knowledge Base gaps are responsible or how to prioritize coverage. At scale, this means untangling messy, unstructured conversations.
That’s where Regal Improve’s Knowledge Base Coverage comes in.
The system automatically evaluates how well your KB supports real customer inquiries using the retrieval layer of the RAG pipeline for evaluation of coverage and relevance. It analyzes how customers actually ask questions, clusters semantically similar requests together, and measures whether the right knowledge is retrieved for each cluster. Results are broken down by topic and subtopic, exposing systematic retrieval gaps and their impact on the quality and accuracy of agent responses.
Customer conversations are inherently dynamic, and Regal Improve’s Knowledge Base Coverage tracks changes in retrieval performance over time and at scale. This makes it easy to verify that KB updates increase coverage and if your documentation is keeping up with evolving customer requests.
Regal Improve's Knowledge Base Coverage turns missing information into actionable insights. By evaluating the quality of every Knowledge Base retrieval and mapping it to real conversational topics, teams can see exactly where context breaks down, and where improvements are needed.
%201%20(2).png)
In the dashboard above, the system automatically surfaces which topics your knowledge base covers well, and where it falls short. Pricing-related questions perform as expected; topics like Pricing for Premium Plans show a high percentage of quality retrievals from the existing KB (78.13%), indicating that documentation is equipped to support customer inquiries in this area. This aligns with what most teams would anticipate: pricing is frequently asked about and well-documented.
However, the real value emerges when you look at the gaps.
Conversation data reveals that customers frequently ask about compatibility between premium plans and enterprise add-ons, yet this subtopic shows a high rate of poor-quality retrievals. In practice, this means AI agents are often presented with incomplete or irrelevant content at a critical decision point, right when customers are evaluating upgrades.
Poor matches indicate either a true KB content gap, or that the content exists but doesn’t align with how customers phrase their questions. In this case, teams can review existing documentation and infer that adding or expanding documentation around cross-package compatibility, plan constraints, and add-on eligibility would directly address the issue. By strengthening coverage in these areas, agents are better equipped to answer upgrade-related questions confidently and consistently.
Beyond highlighting the gap, Regal Improve informs the next action. By moving to proactive insights, you can identify emerging knowledge gaps before they result in customer dissatisfaction and missed upsell opportunities.
Beyond identifying what documentation is needed, Regal Improve makes it possible to track the measurable impact of every KB update.
Let’s use the example from above: After adding additional documentation on cross-package compatibility, you must confirm that your agent has improved retrieval performance, so it is better equipped for customer inquiries. After a week, you revisit the dashboard:
.png)
When improvements are made, a rising high-match percentage confirms the KB update is closing retrieval gaps during real customer conversations. Here, added content around plan and add-on compatibility moved from a poorly covered area (100% poor match quality) to the top covered subtopic (84.2% high match quality), while also lifting high-match performance across the Pricing & Packaging parent topic.
At the same time, a new gap has surfaced around Holiday Promotion Discounts. Because this is a time-bound offer, it can be addressed with a KB article that’s easy to retire once the promotion ends. Alternatively, the agent prompt can be updated with basic information about the promo.
This precision eliminates guesswork. Instead of broad, generic updates, you make targeted fixes with measurable impact on match quality scores.
In sales, faster, more accurate answers lead to shorter sales cycles, higher conversion rates, more upsells, and better customer satisfaction, turning more conversations into closed deals. And in support, increased coverage across topics yields higher containment rate, lower time-to-resolution, and more accurate answers to customer inquiries.
Knowledge Base Coverage works best where documentation gaps are hard to detect and costly. High-volume sales teams in education, insurance, financial services, and healthcare can pinpoint which complex topics need better documentation. Customer service teams managing large knowledge bases spot gaps that would take weeks to find manually. Fast-growing companies validate their knowledge base and keep pace with changing products.
Every unanswered question is an indication of what customers care about most. As your customers’ needs evolve, Regal Improve helps you maintain a current understanding of what they’re actually asking, providing real-time visibility into emerging priorities and shifting interests.
Not all gaps are created equal. Knowledge Base Coverage displays retrieval match quality by conversational subtopic, showing the distribution of strong, moderate, and poor matches. Topics can then be sorted by call volume or match quality to quickly compare performance and prioritize remediation for the highest-impact areas.

In the dashboard above, when comparing the subtopics for "Request info on Enterprise Pricing" and "Inquiry about HIPAA Compliance," Enterprise Pricing had higher conversation volume (84 vs. 50 moments) and a significantly higher percentage of poor-quality matches (80.95% vs. 66.00%). This means that not only is the agent fielding more questions about Enterprise Pricing, but when customers ask these questions, it's providing substantially lower-quality responses than for HIPAA inquiries.
Instead of prioritizing adding documentation to improve HIPAA Compliance coverage, you can focus on creating targeted Enterprise Pricing content that addresses the specific questions appearing in more customer conversations. This is more likely to drive higher conversion rates on enterprise deals, shorter sales cycles for your most valuable customer segment, and increased upsell from prospects expressing interest in your premium offering.
The best AI agents don't just handle conversations, they learn from them. With Knowledge Base Coverage, every interaction your agent completes feeds into a continuous improvement loop.
.png)
The system identifies knowledge gaps, you bridge those gaps with expanded or restructured content, and your AI agent immediately applies that knowledge, then surfaces the next priority gap to address.
Your AI agents get smarter with every interaction, handling more inquiries without requiring transfer to a human agent, so your team can focus on higher-value work, such as relationship building and revenue generation. The result? You can more confidently scale sales and support while keeping complex documentation up to date and providing high-quality customer experiences, all without scaling costs.
Schedule a demo to learn how Regal Improve can close the gaps holding your AI agent back.
Ready to see Regal in action?
Book a personalized demo.



