
September 2023 Releases
DeepSeek, Anthropic, Google and OpenAI have each released new models over the last two months. We ran every one through the Voice CX Benchmark, which grades task progression turn by turn against real call transcripts.
The majority of these new releases are aimed at specific coding and agentic work. OpenAI pitches GPT-6.1 Sol for coding, computer use and professional work, and Anthropic positions Claude Sonnet 5.5 for fixing bugs. Notably, those strengths don’t always carry over to strong voice or customer experience performance. A model that excels at writing code can still stall in a live conversation, or be too expensive to run at a high volume of calls. That’s why we test every release against the Voice CX Benchmark.
After running these new LLMs through our benchmark, we’re prioritizing GPT-6 Sol and DeepSeek V4.1 Flash for our Voice AI agent platform. We aren’t moving forward with Claude Sonnet 5.5, Gemini 3.8 Flash or GPT-6.1 Sol, and we have a narrow application of Claude Haiku 5.5.

Right now, GPT-6 Sol has the highest Voice CX score at 786. Launched on September 22, 2026, it’s 22% cheaper than GPT-4.1, though slightly slower, with a time to first token of 870 ms compared to GPT-4.1’s 793 ms. It’s also more than twice as fast as its predecessor, GPT-5.6 Sol, and about 30% cheaper.
As the leader in our benchmark now, we will use this model’s performance as reference for new LLMs that ship. GPT-6 Sol will be available on our platform soon. Here’s how GPT-6 Sol compares to GPT-4.1:

DeepSeek launched V4.1 Flash on September 10, 2026, and the results surprised us*. It lands just 3.2 points behind GPT-6 Sol, the top model on the board, starts talking 517 ms sooner, and costs about a tenth as much.** It also moves calls forward efficiently: it ranks 4th on Task Progression, ahead of GPT-6 Sol. Here’s an example turn where it outperformed GPT-6 Sol:

Additionally, this model ranks 1st out of all models we’ve tested on jailbreak resistance. Its performance is especially strong on turns where the agent has to read a compliance script, close the call or escalate to a human.
This makes DeepSeek V4.1 Flash the highest performing lightweight model to date, and we are actively evaluating it for our platform. That said, if your agent heavily depends on tools firing on cue, we still recommend GPT-6 Sol. It calls required tools more reliably than DeepSeek V4.1 Flash, especially in multi-state agents and on acknowledgment turns, where DeepSeek is weakest.
Anthropic released Haiku 5.5 on October 7, 2026, and it’s a clear step up from Haiku 4.5, gaining 34.7 points on its Voice CX Score. Anthropic describes it as being “designed for high-volume, cost-sensitive tasks”. Notably, we found that it understands callers better than almost any model we’ve tested: it ranks 1st on intent accuracy and 2nd on context retention. Additionally, it costs about an eighth as much as GPT-6 Sol**. In this turn, it outperformed GPT-6 Sol on intent classification:

Where it falls short is knowing when to hold back. It ranks 32nd on holding off on tools when they aren’t needed, and 27th on Task Progression, proving difficult for it to move conversations forward accurately. See an example below:

This means it’s not a viable option for full agent conversations. However, its strength in understanding callers is why we’re exploring it for intent classification only.
Launched on September 28, 2026, Sonnet 5.5 is 208.5 points higher than Sonnet 5 and is one of the most natural-sounding models we’ve tested. Anthropic describes it as being ideal for “well-scoped everyday tasks, fixing bugs, and creating polished documents, slides, and spreadsheets. It’s also got a sharp eye for design.”
On our benchmark, it ranks 1st on context retention and 6th on calling a required tool. The blocker is cost: it’s more than twice the price of GPT-6 Sol** and still scores slightly below its much cheaper sibling, Haiku 5.5. It also doesn’t offer a reasoning-off setting, which limits how fast we can run it, preventing it from being a viable option for our platform.
OpenAI released GPT-6.1 Sol on September 29, 2026, just a week after GPT-6 Sol. OpenAI pitches GPT-6.1 Sol for “agentic coding, computer use, and professional work,” at a lower cost than its flagship.
We found that it answers about as well as its predecessor (GPT-6 Sol), and ranks 1st on both guardrail compliance and avoiding hallucinations. The catch is latency: the API no longer accepts reasoning set to none, and at its lowest setting it takes 1,711 ms to start talking, compared with GPT-6 Sol’s 870 ms, at the same price. On a live call, that latency is too high to pass as a natural pause, so we won’t be considering it for our platform.
Released on September 2, 2026, Gemini 3.8 Flash ranks 1st on calling a required tool and 1st on following instructions. The problem is speed: it takes 2,516 ms to reach its first token, well over our 1,300 ms bar for live voice. Like GPT-6.1 Sol, it dropped the lowest thinking setting its predecessor supported, and it scores 154 points below Gemini 3.6 Flash as a result.
The high latency keeps this model out of our consideration pool. In this model family, Gemini 3.6 Flash remains the better option.
Newer and smarter models don't necessarily translate to better voice AI agent performance. Increasingly, new models are trained and tuned for coding workflows, not for real-time customer conversations. You can see this reflected in the settings: GPT-6.1 Sol, Gemini 3.8 Flash and Claude Sonnet 5.5 no longer let you disable reasoning. That extra thinking helps on a coding task, but in a cascaded voice agent this adds latency and cost.
For example, two of the five releases dropped the low-latency capabilities their predecessors offered, and both fell down their Voice CX rankings because of it. Meanwhile, one of the most promising results came from a lightweight model at a fraction of the cost.
*DeepSeek is accessed via Baseten’s Model API and meets U.S. data residency requirements.
**Relative cost is based on each provider’s public API pricing and may differ from pricing on the Regal platform.
GPT-6 Sol has the highest score on Regal's Voice CX Benchmark at 786, with a time to first token of 870 ms and a lower cost than GPT-4.1, the runner-up.
DeepSeek V4.1 Flash is the highest-scoring lightweight model on the Voice CX Benchmark, landing just 3.2 points behind GPT-6 Sol while responding 517 ms faster at about a tenth of the cost.
Many new models are tuned for coding and agentic work, and several, including GPT-6.1 Sol, Gemini 3.8 Flash and Claude Sonnet 5.5, no longer let you turn reasoning off, which adds latency that hurts live phone conversations.
Claude Haiku 5.5 ranks 1st on intent accuracy and 2nd on context retention, but it calls tools when it shouldn't and struggles to move calls forward, so Regal is exploring it for intent classification only.
The Voice CX Benchmark grades task progression turn by turn against real call transcripts, along with latency, cost, tool calling and other measures that matter for live customer conversations.
Ready to see Regal in action?
Book a personalized demo.



