SEPTEMBER 2023 RELEASES
September 2023 Releases
Introducing Evaluations: Pinpoint AI Agent Failures Before They Go Live
You need confidence that every AI agent will behave exactly as intended before it interacts with live customers.
But enterprise-level testing is complex. Multi-turn conversations, branching logic, and hundreds of scenarios make it hard to identify end-to-end failures quickly. Doing so manually is very time-consuming.
The typical tradeoff is to run regression tests in small bulks and go live, hoping the agent handles edge cases correctly.
That tradeoff is what our Simulations feature solves for.
As an extension of our Simulations feature, Evaluations further speed up the process of pre-launch and regression testing, while giving greater visibility into the edge cases—the mishandled questions, repetition, miscommunications, incorrect function calls.
Evaluations allow you to validate agent behavior more quickly, so you can make more immediate and impactful updates to your AI agent prompt, custom actions, and knowledge bases and launch with confidence.
Introducing Evaluations
Evaluations are scenario-specific automated pass/fail assessments that use an LLM to review simulated conversations against the success criteria you define.
They provide a granular QA layer on top of running simulations in bulk. You get a quick view of which cases passed and failed, so you know exactly where to prioritize prompt, knowledge base, or custom action updates.
%20(1).png)
Key Aspects of Simulation Evaluations
1. Unlock granular automated QA with scenario-specific assessments
Each test case is evaluated against precise scenario occurrences (did the AI handle this objection correctly, did the AI get clarification on a vague answer), not the outcome of the entire conversation.
For example, you might test an appointment scheduling flow for “Unrelated Inquiry Handling,” where the success criteria is that the AI addresses the contact’s unrelated question, gracefully lets them know they can’t help, and then continues on the scheduling path.
It’s possible that the AI disregards the question completely or mistakenly tries to answer it, but still successfully schedules an appointment.
In this case, the success criteria was for handling the inquiry. This case would register as failed.
Scenario-specific assessments like this allow for fast, granular evaluation of every point in a conversation so you can tailor improvements to scoped, tangible issues.
2. Ensure consistency with unified scoring logic across the platform
The logic used to score simulations comes from the same engine logic that powers Regal scorecards. This ensures alignment between pre-launch testing and post-launch QA, bringing one unified way to measure improvements, identify regressions, and act on QA insights, regardless of when or where you’re doing so.
3. Built for complexity
Because LLMs can interpret variations in phrasing, intent, and context that strict rule-based testing might miss, your AI agent is tested and evaluated against realistic, nuanced conversations.
How it works: An LLM is employed as the customer-side contact in every simulation. An LLM is also employed to carry out the evaluation itself, determining pass or fail scores using the Success Criteria you define in the Simulation prompt.
.png)
This all plays into how fast you’re able to evaluate test cases and take action on AI agent improvements.
Identify Needed Prompt Updates Instantly
By running tests in bulk and getting instant pass-fail scores, you’ll know where to prioritize AI Agent updates.
This reduces QA back-and-forth, and helps speed up the iteration and deployment cycle: You can quickly identify where the AI is underperforming and address gaps pre-launch, assuring better performance at deployment. This lessens the need (and gets you time back) from having to manually QA the agent post-launch.
For example:
You have 10 test cases for an auto insurance lead qualification flow. Considering the use case, the tests would likely cover:
- Objections around pricing and competitors
- Answer clarifications (did the contact provide clear details on contact info and driving history)
- Unrelated or unsupported questions (the contact asks about auto and home bundles, but you don’t offer home insurance)
- Actions like sending an application link, rescheduling, or transferring the call
You run all 10, and two test cases fail:
- “Unsupported Service Inquiry”
- “Fear of Financial Risk”
.png)
You’ll see the summary of why the case failed, alongside the original success criteria:
%20(1).png)
In a matter of seconds, you know that you need to address your objection handling prompt and guardrails to better address unsupported service questions and speak clearly to pricing.
There are many failure points you can identify this quickly:
Conversation Flow:
- Sequencing: Does the agent follow the intended script or logical path?
- Objection handling: Are pushbacks and clarifications handled correctly?
- Questions: Does the AI acknowledge questions appropriately and with the right branded tone?
- Information accuracy: Are answers correct, relevant, and contextually aligned?
- Prompt/response completeness: Are all required questions asked and confirmations captured?
- Relevance: Does the AI go off-topic or field questions it shouldn’t?
Actions Taken:
- Transfers: Are they triggered according to business rules?
- Data capture: Are required fields collected at each step when intended?
- Workflow triggers: Are backend actions executed as intended?
- Call termination: Are end-of-call processes applied correctly?
- Knowledge Base retrieval: Is knowledge retrieved from the right KB at the right points, for the right questions?
Branching and Conditional Logic:
- Conditional paths: Does the agent follow the correct branch based on input? Does the agent ever hallucinate answers, including information from another, separate path?
Improve AI Agent Workflows Faster
Evaluations don’t just flag failures.
They provide a human-readable explanation of why a test failed and give you a sample conversation displaying this failure. And since you know the precise scenario that failed, this enables immediate iteration on prompts, branching logic, and function calls.
For Example:
The home insurance scenario above failed because the agent followed its prompt correctly, maintaining a helpful nature, but lacked explicit instructions on how to exit out-of-scope inquiries.
By adding a single instruction in the Objection Handling prompt, you address inquiries about home insurance.
.png)
From here, you re-run the simulation, and the test passes:
.png)
Driving Continued Improvement
Even after validation, AI agents aren’t perfectly predictable.
LLMs can’t match human unpredictability 1-to-1, so live calls will always introduce new edge cases and unexpected variances.
That makes Simulations and Evaluations critical for regression testing: when you create new agent versions or make improvements over time, you can continue to rerun your existing test suite and evaluations in one click, instead of starting from scratch every time.
This approach lets your enterprise turn insights into actionable improvements immediately, aligning simulation outcomes with live performance metrics, shortening iteration cycles, and ensuring agents perform reliably across every scenario.
Close the Loop Before Deploying
Every failed test scenario is an opportunity to improve.
Evaluations turn each test case into precise insights, letting you understand exactly what went wrong and where to intervene. Plus, since it’s done in an automated way at scale, it massively reduces the need for manual review.
This closes the gap between simulation and live performance, enabling you to iterate quickly, reduce QA cycles, and deploy AI agents that are reliable across complex workflows.
By incorporating scenario-specific feedback into every pre-launch evaluation, you ensure that your AI agents are not only tested—but truly optimized for customers.
Explore Evaluations today to aggregate, analyze, and act on test results with confidence.
Latest Blog Posts
%20(1).png)
August 2025 Releases
New features include automated test evaluations, the Regal Improve insights dashboard, PII-safe recording actions, 25 ultra-realistic voice options, bulk Knowledge Base URL upload, and expanded quiet hours compliance—designed to accelerate iteration, improve agent performance, and safeguard customer trust.
%20(1).png)
Apple iOS 26 Caller Screening: An Enterprise Guide
Apple’s iOS 26 introduces new call-screening controls that will reshape outbound performance. This guide explains what’s changing, how adoption may impact enterprise contact centers, and the proactive steps leaders can take now to protect answer rates and customer trust.
RAG Hygiene: How to Scale and Maintain AI Agent Knowledge
Learn how to maintain a clean, reliable RAG system for AI Agents. Discover best practices for structuring source docs, chunking content, titling for retrieval, avoiding redundancy, and keeping knowledge bases fresh to ensure accurate, scalable performance.
Announcing Regal + Rime: Human-Like Voices at Enterprise Scale
Regal is partnering with Rime to deliver ultra-realistic, emotionally nuanced AI voices for enterprise contact centers. Discover new voice options that capture human qualities like pacing, resonance, and subtle expression, and learn how Rime’s technology powers conversations that build trust at scale.
.png)
%20(1).png)
.png)
The RAG Playbook: Structuring Scalable Knowledge Bases for Reliable AI Agents
Learn how to structure a knowledge base that keeps AI Agents accurate and on-script. Discover why human-style KBs fail, and apply best practices like single-topic chunking, concrete instructions, and explicit outcomes to reduce hallucinations and scale reliable RAG performance.
.png)
.png)
.png)
%20(1).png)
Announcing Regal + Cartesia: Expanding Voice Options for Enterprise AI
Regal is partnering with Cartesia to deliver ultra-low latency, high-fidelity AI voices for enterprise contact centers. Explore new voice options, hear them in action, and learn about Cartesia's industry-leading voice AI offerings.
.png)
.png)
Anatomy of an AI Voice: What Makes It Sound Human
This article outlines the core characteristics that influence how voice AI is perceived on live calls. From mechanical traits like speed and volume, to more emotional and conversational behaviors, we’re going to look at what those characteristics mean, why they matter, and how they impact your bottom line.
.png)
AI Voice Customization 101: Settings That Work Best
Learn how to configure your AI Voice Agent for real performance. This guide covers the most important voice settings in Regal, what ranges top brands use in production, and how adjusting speed, tone, and responsiveness impact cost, containment, and overall customer experience.
%20(1).png)
June 2025 Releases
June’s releases give you more control to test, improve, and scale AI agents—while also making life easier for human agent teams. From real-time debugging tools to capacity control and a floating Chrome extension, we’re unlocking faster workflows, better control, and stronger performance across the board.
.png)
Caller ID vs. CNAM vs. Branded Caller ID: What’s the Difference?
Caller ID, CNAM, and Branded Caller ID all help identify an incoming call and, therefore, increase call answer rates. Find out how they differ and how outbound phone sales teams can utilize each to reach prospects and customers effectively.
.png)
%20(1).png)
%20(1).png)
%20(1).png)
%20(1).webp)
.avif)
.png)
The Automated Phone System Revolution: Why 73% of Enterprises Are Getting It Wrong (And How to Get It Right)
Explore how automated phone systems are transforming enterprise communication strategies. Learn how to avoid common pitfalls, optimize customer journeys, and achieve ROI through strategic deployment across industries like healthcare, insurance, and education.
%20(1).png)
.png)
.webp)
.webp)
.webp)
.webp)
.webp)
.webp)
.webp)
.webp)
How are Generative AI Voice Agents Different from AI Voice Assistants?
When it comes to comparing AI Agents vs. AI Assistants, Siri & Alexa handle simple tasks, but Gen AI Voice Agents—like Regal’s AI Phone Agent—drive real business impact with human-like conversations, automation, and seamless integration.
.webp)
.webp)
%20(1).webp)
7 Best Use Cases for AI Voice Agents in Your Contact Center
As AI technology continues to evolve, the use cases for AI Voice Agents in contact centers will only increase. By answering these six key questions, you can identify where AI agents fit best today in your contact center and plan for future integrations.
.webp)
.webp)
.webp)
.webp)
%20(1).webp)
.webp)
%202.webp)
AI Agents for Education: 8 Use Cases for More Meaningful Student Outcomes
AI Agents for Education are transforming student engagement—boosting enrollment, improving retention, and making support more human. Discover 8 game-changing use cases that free up your staff while delivering better student outcomes.
Regal Named One of Forbes America’s Best Startup Employers 2025!
Regal is officially one of Forbes America’s Best Startup Employers 2025, ranking #164 out of 500. This recognition is a testament to our incredible team, our innovative work culture, and our unwavering commitment to advancing AI technology.
AI in Education: The Future of Student Engagement & Enrollment
AI in education is helping to streamline admissions, automate student engagement, and enhance higher ed outreach. Discover key education technology trends to boost enrollment and learn why automated student engagement tools are the future.
.webp)
.webp)
.webp)
.webp)
.webp)
.avif)
.avif)
5 Ways to Improve Agent Productivity & 5X Outbound Performance
In this blog, we share tips to improve agent productivity for handling high volumes of calls, whether it's inbound support teams or outbound sales teams. The guiding principle is simple — focus your agents on tasks that absolutely requires their attention, and automate everything else.
.avif)
8 Strategies to Improve Call Answer Rates by 40%
Discover 8 proven strategies to improve call answer rates by 40%. Learn how personalized outreach, Branded Caller ID, and AI-driven solutions can transform your outbound contact center and boost engagement. Increase your call answer rates and drive more revenue today!
.avif)
Treat your customers like royalty
Ready to see Regal in action?
Book a personalized demo.
Thank you! Click here if you are not redirected.
Oops! Something went wrong while submitting the form.
