A bare Claude model beat 6 of 10 AI support agents in G2's test; Yuma AI led accuracy at 97%

G2 now grades AI customer-service agents on outcomes rather than demos, and a Y Combinator-backed independent leads the trust metrics. Most packaged rivals, including two listed incumbents, scored below a bare Claude model on the same tasks.

Richard TangRichard Tangthree-minute read
Share
Guillaume Luccisano, founder and CEO of Yuma AI, speaking into a microphone on a panel stage beneath a slide naming him and three other panelists
Guillaume Luccisano, founder and CEO of Yuma AI, on a Shopify x Yuma panel in New York on ecommerce CX operations, with his name card on the slide behind him.

G2, the software-review site, has started grading AI customer-service agents the way an auditor grades a new hire: identical tasks, a simulated company, and a machine check that the refund actually happened 1. In G2's current standings, Yuma AI, a Y Combinator-backed Boston startup whose agents resolve ecommerce tickets inside Gorgias and Zendesk helpdesks, ranks first of ten agents on accuracy at 97% and second overall 12.

It also leads policy compliance at 95%, the score that decides whether a brand can let software issue refunds at all 12. The finding that reaches beyond Yuma sits lower in the table: a bare Claude model with no support product around it outscored most of the packaged agents 12.

The refund has to actually happen

G2 built a simulated support company with realistic order data, a written policy and 38 working tools for refunds and account changes, then gave every product the same 50 tasks 1. Language-model judges scored whether each answer was supported by evidence and followed the written policy, and deterministic checks confirmed the required outcome happened in the systems 1. Vendors could flag setup mistakes but could not change their scores 1.

Most products add nothing the model lacks

The most telling row is not a product. A raw Claude Haiku 4.5, Anthropic's$350B — Anthropic, private, latest valuation $350B model with no support product around it, finished ahead of six of the ten agents on task success 2.

A bare Claude model outscores six of ten packaged support agents on G2's task test

Four products beat the raw Claude baseline; Yuma leads them on accuracy and compliance

050100150200250Retell243Yuma240LiveAgent240Chipp238Claude Haiku 4.5 (model baseline)233Tidio219BoldDesk218GPT-4o Mini (model baseline)210Zoho203Jotform174HubSpot Breeze93Freshworks85
Data
Value
Retell243
Yuma240
LiveAgent240
Chipp238
Claude Haiku 4.5 (model baseline)233
Tidio219
BoldDesk218
GPT-4o Mini (model baseline)210
Zoho203
Jotform174
HubSpot Breeze93
Freshworks85
Task-success points out of 250 across 50 support tasks in G2's AI CX Agent Evaluations; standings as of November 21, 2026. Baselines are general-purpose models run with no support product around them. Source: G2.2

Retell, a voice platform that also tops G2's separate voice ranking, leads the table with weaker trust scores than Yuma 23. Yuma's design likely explains that split. Its agent follows a written handbook, runs under hard caps on refunds and coupons, and hands the ticket to a person wherever the handbook is silent 14.

The handoff probably also explains the three-point gap to Retell: passing on a task costs points, while guessing costs a refund nobody authorized 12. As Yuma's founder and CEO Guillaume Luccisano puts it, support leads "lose sleep over the ones it answers wrong: the refund outside the return window" 1.

The giants have not sat the test

A November 17, 2026, buyer's guide counts 44 AI-native support platforms, a market consolidating under bigger owners 5. Zendesk bought Forethought; NICE, a contact-center software giant, bought Cognigy, an enterprise AI platform; SoundHound−8.93% — SoundHound, down 8.93 percent today, a voice-AI company, bought LivePerson 5.

The listed companies this story touches, Salesforce−0.18% — Salesforce, down 0.18 percent today, HubSpot+2.91% — HubSpot, up 2.91 percent today, Freshworks−1.79% — Freshworks, down 1.79 percent today, NICE and SoundHound, have no equivalent result on the board 2. HubSpot's and Freshworks' agents sit at the bottom of the table, while the verified trust headline belongs to a private independent paid only for tickets it fully resolves 125.

The board is still filling in

The November standings are already G2's second published run at this market: October's covered nine agents on 46 tasks, November's moved to ten agents on 50, and G2 plans to refresh results several times a year 16. The gaps are the watchable part. Zendesk, Gorgias and Forethought are marked under evaluation; Decagon~$1.5B — Decagon, private, latest valuation ~$1.5B, a rival support-agent maker, declined; Salesforce and Sierra, an AI support startup, have not been evaluated 2.

Yuma's homepage carries the freshest line in the file: G2 "tested Yuma again on a retail task set", a run whose results are not yet published 4. Support budgets are likely to follow the agents that can show compliance on someone else's test, and the retail run is the next check on whether Yuma's lead holds.

Get Applications by email, free

How AI is changing the software people and companies use. In our daily email, every weekday.

Loading CRM

Reader comments

0 comments

    More in Applications

    All in this topic →

    Applications

    Verisk's CEO sold $1 million of stock a day after touting growth built on $1.9 billion of buybacks

    His Form 4 flags no trading plan, and the per-share growth he pitched rests on $1.9 billion of buybacks while free cash flow shrinks. That matters to anyone holding VRSK for the AI-data story: the earnings line is being financed, not earned.

    Applications

    Tyler Technologies' CEO Sold $12.8 Million of Stock, Three Times His Filed Notice

    The Form 144 flagged $4.25 million; the Form 4 shows $12.8 million with no trading plan marked. Tyler has $1.4 billion authorized to buy its own stock, and investors are paying near 52 times earnings for single-digit growth.

    Applications

    GTA VI takes 2% of Roblox's Q4 hours and 4.5% of its bookings, BofA estimates

    Grand Theft Auto VI's record $3.79 billion first day landed on the quarter Roblox guided down 10 to 15 percent. Bank of America's launch-day math shows why the damage concentrates: the hours leaving are the platform's highest-spending hours.

    Applications

    Apple deleted a job listing for its own AI agent, and Anthropic fell 12% toward its $83 IPO price

    The listings described an agent that writes code, browses the web and drives apps, and Anthropic fell as much as 12% on the report a day after its market debut. The money underneath is Apple's own multiple: 37 times forward earnings resting on a capital budget that just fell 29%.

    Applications

    OpenAI Puts HubSpot's CRM in ChatGPT as HubSpot Guides 12% Growth With AI Credits Unpriced

    The engineering post landed November 19, two days after ChatGPT for Sales made HubSpot one of four CRM plugins in a suite whose core works even on ChatGPT Free. HubSpot's growth now rides on usage credits it has not priced, against 12% guidance and 660 jobs cut.

    Applications

    Salesforce booked $94 million in restructuring in a filing that calls the charges nearly over

    Four consecutive filings promise no significant additional restructuring charges while the bill since fiscal 2024 reaches $2.2 billion, and the July quarter lifted the estimate of cash still to pay by $40 million. Salesforce reports on December 2 with the stock down 16 percent in a week.

    Subscribe to TheInference for free