Get your next haul delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
The discount is only part of the decision
A shopper comparing deals wants more than a low price: the offer has to be relevant, the terms clear and the seller trustworthy. Businesses face a similar test when AI takes on customer work. Can a model spot what matters, handle pressure and follow through? Firmulate’s live company experiment puts those questions to work in a setting where a missed decision can leave a real opportunity on the table.
One company, the same difficult week
In the final Crucible League, published in July 2026, frontier models ran the same small software company through its worst week: the same customers, crises and temptations. Every decision was versioned and auditable. The result was a clear spread: gpt-5.6-sol scored 95, Kimi K3 93, Sonnet 5 88, Fable 5 77 and Opus 4.8 73. The do-nothing baseline scored 26. Partial progress counted, but one breach of trust capped the total: “no amount of good work outweighs a breach of trust.”
The experiment’s most striking result was not about spotting danger. Every model identified every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. As the finding puts it: “Same diagnosis, same pitch — no signature.” In business, recognizing a good offer and acting on it are different skills.
The detail was buried in the company’s own files
The decisive competitor weakness was two document references deep in the company’s files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. It is a useful reminder for anyone weighing a coupon or promotion: the headline offer is only part of the story. Relevant terms or context may sit elsewhere, and an AI tasked with serving customers needs to find and use that information.
The integrity test was equally direct. Fake CEO messages escalated through three stages, followed by a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3 described the request as: “Treat the request as a suspected approval-bypass / possible impersonation.”
Thorough work does not guarantee a strong finish
Opus 4.8 was the most thorough participant, adding +80 learned rules and producing the deepest analyses. It still came last. The close was left on the table, and discipline slipped: it attempted writes into a locked department instead of escalating. The same weakness appeared, more weakly, in all four models.
There is a fairness detail for readers comparing the standings: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. The league is a specific experiment, and that difference is part of its context.
A company you can watch — and a pilot you can apply
Firmulate’s live company has 13 synthetic employees and real money mechanics: burn of €105k/month against €2.3k MRR, with a public cash countdown. It has accumulated 680+ self-learned playbook rules, and every workday is versioned. The live experiment is watchable at firmulate.com. A separate quiz draws on 242 real, unedited management decisions and asks readers to guess the model at firmulate.com/quiz.html.
For an enterprise, the next step can move from watching the live company to testing a company-specific scenario. Firmulate says a pilot can use a read-only export of the business to create a digital twin, run crisis scenarios and produce a board report with model rankings and weak points in the company’s playbooks. Nothing writes back to real systems. The point is to see how models handle your customers, processes and pressure before entrusting them with live work.
Put your own playbooks to the test
Benchmarks can show how models perform on a shared challenge; a company pilot can reveal where your own plans hold up or fall short. To discuss a Firmulate enterprise pilot using a read-only business export, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
