--- title: "Nanonets' own benchmark says the best AI agent finishes just 38% of multi-step enterprise tasks" description: "Nanonets, a private document-AI company whose platform automates paperwork for 10,000-plus teams, published an open benchmark grading AI systems on executing finance processes under rules buried in conversation; the best result was 38% of tasks fully right. Against $2.7 trillion of forecast 2026 AI spending, it is a rare inside-industry measurement of what enterprise agents actually deliver." publisher: "The Inference" section: "Platforms" published: 2026-10-14T21:08:10.037Z modified: 2026-10-14T21:08:10.037Z canonical: https://theinference.org/article/nanonets-own-benchmark-says-the-best-ai-agent-finishes-just-38-of-multi-step-enterprise-tasks language: en keywords: "AI Agents, Benchmarks, Enterprise Software, Document AI, Finance Automation" --- # Nanonets' own benchmark says the best AI agent finishes just 38% of multi-step enterprise tasks > Nanonets, a private document-AI company whose platform automates paperwork for 10,000-plus teams, published an open benchmark grading AI systems on executing finance processes under rules buried in conversation; the best result was 38% of tasks fully right. Against $2.7 trillion of forecast 2026 AI spending, it is a rare inside-industry measurement of what enterprise agents actually deliver. ## A number sitting in public since July The best AI system [Nanonets](https://nanonets.com/research) could find finishes only 38% of multi-step enterprise process tasks fully right. The number sits on the research page of Nanonets, a private San Francisco document-AI company, under TRACE, Task-Relevant Applied Constraint Execution, [an openly published benchmark](https://nanonets.com/research/trace) [1]. It is a ceiling on the work behind $2.7 trillion of forecast AI spending in 2026 [3]. The write-up's title states the finding: models find the rules, then fail to apply them [2]. The dataset and its scoring code were last updated July 22, 2026 [6][7]. News and web searches on October 14, 2026, found no outlet reporting the result. ## The money assumes the opposite Gartner says software vendors of every type are "[rapidly embedding agentic AI](https://www.hpcwire.com/bigdatawire/this-just-in/gartner-forecasts-worldwide-ai-spending-to-grow-49-5-in-2026/)" in their existing products [3]. Nanonets says more than 10,000 teams already run invoice matching, claims and month-end close on its platform [4]. If the best system passes 38% of process tasks, most budgeted automation work still comes back wrong, or comes back to a human. If 38% holds, two groups lose: enterprises deploying agents on vendor claims, and software revenue models priced for near-total back-office automation. The winner is whoever first posts a verified execution score far above it. ## The grader also sells the agents Nanonets is not a neutral referee. It grades the field, running TRACE and [a public leaderboard that ranks 29 intelligent document processing models](https://www.idp-leaderboard.org/), with its own OCR-3 at number one [1][5]. It also sells into that field: a blog marketing "AI-native" platforms against legacy vendors ABBYY and Kofax [4], and a banner for Trail, "the company brain for AI agents" [1]. ## What the test asks TRACE is deliberately harsh. Each sample is a Record-to-Report conversation, the back-and-forth of a month-end accounting close, about 100,000 tokens of text long, with company rules dropped mid-sentence. Five samples carry 47 questions graded on 388 criteria, scored all-or-nothing: a task passes only if the answer is right and every rule that applies is honored. A second AI model does the grading, with gpt-5.5 named in the sample setup [6]. ## Reading in the 80s, execution at 38 On Nanonets' own document leaderboard the same models read pages far more reliably: Nanonets OCR-3 at 85.9 overall, [OpenAI](https://theinference.org/markets/companies/openai)'s GPT-5.4 at 83.5, Google's Gemini-3-Pro at 82.8 [5]. For a buyer, the two scores together mean a model can read a page right roughly 86 times in 100 and still fail most multi-step processes it runs under a company's rules [1]. *Chart: **Reading documents is nearly solved; the best execution score is 38** Bar chart comparing document-reading benchmark scores in the 80s for four models with the best TRACE process-execution score of 38%* *Overall score on Nanonets' IDP Leaderboard (mean of three document benchmarks, read October 14, 2026) vs best task_pass_rate on TRACE (share of 47 process tasks fully correct, all-or-nothing scoring; dataset updated July 22, 2026). Sources: Nanonets Research [1]; IDP Leaderboard [5]. [1][5]* ## Small benchmark, interested party The caveats are real: five samples, 47 questions, one context size, self-published by a vendor with agents to sell, machine-graded [6]. But [the dataset is open under CC-BY-4.0](https://huggingface.co/datasets/nanonets/trace-benchmark-dataset) with the evaluation code on GitHub, so anyone can rerun the field instead of trusting the number [6][7]. Almost nobody has: the dataset card shows 44 downloads in the past month [6], the repository shows 39 stars and no forks [7], against 288,000 downloads for Nanonets' own OCR model [8]. *Chart: **Almost nobody has rerun the benchmark: 44 downloads against 288,000 for Nanonets' own model** Column chart showing 44 monthly downloads of the TRACE benchmark dataset, 39 stars on its GitHub evaluation repo, and 288,000 downloads of Nanonets' OCR-3 model* *Interest in the benchmark against interest in the vendor's own model: TRACE dataset downloads in the past month (Hugging Face, read October 14, 2026), stars on the evaluation repo (GitHub, no forks), and downloads of the nanonets OCR model (Hugging Face). Sources: Hugging Face [6]; GitHub [7]; Hugging Face [8]. [6][7][8]* ## Watch the pull requests The next reading is dated and public: [submissions of new systems land as GitHub pull requests](https://github.com/NanoNets/trace-benchmark) [7]. No verified score far above 38 has been posted. Until one is, the defensible budget line for enterprise agents is that most multi-step process work fails on the industry's own test. ## Takeaway A vendor-published open benchmark says the best AI system fully completes just 38% of multi-step enterprise process tasks, even as the same models read documents in the 80s; until a verified higher score lands in the open repo, enterprise agent budgets are built on a number no one has beaten. ## Sources 1. [Nanonets, Research index page, accessed 14 October 2026](https://nanonets.com/research) 2. [Nanonets Research, TRACE benchmark page: LLMs search & retrieve rules, but still don't apply them, accessed 14 October 2026](https://nanonets.com/research/trace) 3. [BigDATAwire, Gartner Forecasts Worldwide AI Spending to Grow 49.5% in 2026, 18 September 2026](https://www.hpcwire.com/bigdatawire/this-just-in/gartner-forecasts-worldwide-ai-spending-to-grow-49-5-in-2026/) 4. [Nanonets, Blog index (AI Agents for Enterprise Data Processing), accessed 14 October 2026](https://nanonets.com/blog/) 5. [IDP Leaderboard, Intelligent Document Processing Leaderboard, accessed 14 October 2026](https://www.idp-leaderboard.org/) 6. [Hugging Face, nanonets/trace-benchmark-dataset dataset card, updated 22 July 2026](https://huggingface.co/datasets/nanonets/trace-benchmark-dataset) 7. [GitHub, NanoNets/trace-benchmark evaluation code and submission repo, last commit 22 July 2026](https://github.com/NanoNets/trace-benchmark) 8. [Hugging Face, nanonets organization page, accessed 14 October 2026](https://huggingface.co/nanonets) --- Nanonets' own benchmark says the best AI agent finishes just 38% of multi-step enterprise tasks — The Inference. Canonical: https://theinference.org/article/nanonets-own-benchmark-says-the-best-ai-agent-finishes-just-38-of-multi-step-enterprise-tasks