---
title: "A bare Claude model beat 6 of 10 AI support agents in G2's test; Yuma AI led accuracy at 97%"
description: "G2 now grades AI customer-service agents on outcomes rather than demos, and a Y Combinator-backed independent leads the trust metrics. Most packaged rivals, including two listed incumbents, scored below a bare Claude model on the same tasks."
publisher: "The Inference"
section: "Applications"
published: 2026-11-21T18:14:40.062Z
modified: 2026-11-21T18:14:40.062Z
canonical: https://theinference.org/article/a-bare-claude-model-beat-6-of-10-ai-support-agents-in-g2-s-test-yuma-ai-led-accuracy-at-97
language: en
keywords: "AI Agents, Customer Support, Benchmarks, Enterprise Software"
---

# A bare Claude model beat 6 of 10 AI support agents in G2's test; Yuma AI led accuracy at 97%

> G2 now grades AI customer-service agents on outcomes rather than demos, and a Y Combinator-backed independent leads the trust metrics. Most packaged rivals, including two listed incumbents, scored below a bare Claude model on the same tasks.

G2, the software-review site, has started grading AI customer-service agents the way an auditor grades a new hire: identical tasks, a simulated company, and a machine check that the refund actually happened [1]. In G2's [current standings](https://www.g2.com/categories/ai-agents/agent-evaluations), [Yuma AI](https://theinference.org/markets/companies/yuma-ai), a Y Combinator-backed Boston startup whose agents resolve ecommerce tickets inside Gorgias and Zendesk helpdesks, ranks first of ten agents on accuracy at 97% and second overall [1][2].

It also leads policy compliance at 95%, the score that decides whether a brand can let software issue refunds at all [1][2]. The finding that reaches beyond Yuma sits lower in the table: a bare Claude model with no support product around it outscored most of the packaged agents [1][2].

## The refund has to actually happen

G2 built a simulated support company with realistic order data, a written policy and 38 working tools for refunds and account changes, then gave every product the same 50 tasks [1]. Language-model judges scored whether each answer was supported by evidence and followed the written policy, and deterministic checks confirmed the required outcome happened in the systems [1]. Vendors could flag setup mistakes but could not change their scores [1].

## Most products add nothing the model lacks

The most telling row is not a product. A raw Claude Haiku 4.5, [Anthropic](https://theinference.org/markets/companies/anthropic)'s model with no support product around it, finished ahead of six of the ten agents on task success [2].

*Chart: **A bare Claude model outscores six of ten packaged support agents on G2's task test** Twelve bars of task-success points out of 250: Retell 243, Yuma 240, LiveAgent 240, Chipp 238, the Claude Haiku 4.5 model baseline 233, Tidio 219, BoldDesk 218, the GPT-4o Mini model baseline 210, Zoho 203, Jotform 174, HubSpot Breeze 93 and Freshworks 85.*

*Task-success points out of 250 across 50 support tasks in G2's AI CX Agent Evaluations; standings as of November 21, 2026. Baselines are general-purpose models run with no support product around them. Source: G2. [2]*

Retell, a voice platform that also tops G2's separate voice ranking, leads the table with weaker trust scores than Yuma [2][3]. Yuma's design likely explains that split. Its agent follows a written handbook, runs under hard caps on refunds and coupons, and hands the ticket to a person wherever the handbook is silent [1][4].

The handoff probably also explains the three-point gap to Retell: passing on a task costs points, while guessing costs a refund nobody authorized [1][2]. As Yuma's founder and CEO Guillaume Luccisano puts it, support leads "lose sleep over the ones it answers wrong: the refund outside the return window" [1].

## The giants have not sat the test

A November 17, 2026, [buyer's guide](https://dynamicbusiness.com/featured/tech-tuesday/tech-tuesday-the-complete-guide-to-ai-native-customer-support-platforms-in-2026.html) counts 44 AI-native support platforms, a market consolidating under bigger owners [5]. Zendesk bought Forethought; NICE, a contact-center software giant, bought Cognigy, an enterprise AI platform; [SoundHound](https://theinference.org/markets/companies/soundhound-ai), a voice-AI company, bought LivePerson [5].

The listed companies this story touches, [Salesforce](https://theinference.org/markets/companies/salesforce), [HubSpot](https://theinference.org/markets/companies/hubspot), [Freshworks](https://theinference.org/markets/companies/freshworks), NICE and SoundHound, have no equivalent result on the board [2]. HubSpot's and Freshworks' agents sit at the bottom of the table, while the verified trust headline belongs to a private independent paid only for tickets it fully resolves [1][2][5].

## The board is still filling in

The November standings are already G2's second published run at this market: October's covered nine agents on 46 tasks, November's moved to ten agents on 50, and G2 plans to refresh results several times a year [1][6]. The gaps are the watchable part. Zendesk, Gorgias and Forethought are marked under evaluation; [Decagon](https://theinference.org/markets/companies/decagon), a rival support-agent maker, declined; Salesforce and Sierra, an AI support startup, have not been evaluated [2].

[Yuma's homepage](https://yuma.ai/) carries the freshest line in the file: G2 "tested Yuma again on a retail task set", a run whose results are not yet published [4]. Support budgets are likely to follow the agents that can show compliance on someone else's test, and the retail run is the next check on whether Yuma's lead holds.

## Takeaway

G2's outcome-based test of AI support agents found a bare Claude model beating six of ten packaged products, while Yuma AI led on accuracy at 97% and policy compliance at 95%.

## Sources

1. [EIN Presswire, "Yuma AI ranks #1 for accuracy and policy compliance in G2's new hands-on evaluation of AI customer service agents", 10 November 2026](https://www.einnews.com/pr_news/955783702/yuma-ai-ranks-1-for-accuracy-and-policy-compliance-in-g2-s-new-hands-on-evaluation-of-ai-customer-service-agents)
2. [G2, "AI Agent Evaluations: AI Agents" leaderboard, data from 21 November 2026](https://www.g2.com/categories/ai-agents/agent-evaluations)
3. [G2 AI (ai.g2.com), agent evaluations homepage, accessed 21 November 2026](https://ai.g2.com/)
4. [Yuma AI homepage, accessed 21 November 2026](https://yuma.ai/)
5. [Dynamic Business, "Tech Tuesday: The complete guide to AI-native customer support platforms in 2026", 17 November 2026](https://dynamicbusiness.com/featured/tech-tuesday/tech-tuesday-the-complete-guide-to-ai-native-customer-support-platforms-in-2026.html)
6. [EIN Presswire via The News Leader, "Tidio Ranks No. 1 in G2's AI CX Agent Evaluations", 7 October 2026](https://www.newsleader.com/press-release/story/129863/tidio-ranks-no-1-in-g2s-ai-cx-agent-evaluations/)


---

A bare Claude model beat 6 of 10 AI support agents in G2's test; Yuma AI led accuracy at 97% — The Inference. Canonical: https://theinference.org/article/a-bare-claude-model-beat-6-of-10-ai-support-agents-in-g2-s-test-yuma-ai-led-accuracy-at-97
