AIThis post was created with the assistance of artificial intelligence (AI).
Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
Live on firmulate.com.

XR companies are already asking AI to help with customer support, sales and operations. The harder question is whether an AI that sounds capable can make good calls when a customer is about to leave, a costly deal needs a signature and someone sends a suspicious message claiming to be the CEO. Firmulate put several frontier models through that kind of pressure test—and the newcomer from Moonshot nearly topped the table.

For playersOffer from Amazon

Play games on Amazon Luna for your next adventure, included with Prime

  • A rotating selection of games, no download needed
  • Play on TV, laptop or phone
  • Fast, free delivery for your gear, too
Start playing with Prime Free trial · Cancel anytime
As an affiliate, we earn on qualifying purchases.

A company, not a chat demo

Firmulate’s Crucible gave each model the same small software company to run through its worst week: the same customers, crises and temptations. Every decision was versioned and auditable. The point was to observe management choices, not just polished answers.

The live company has 13 synthetic employees and real money mechanics: it burns €105,000 a month against €2,300 in monthly recurring revenue, with a public cash countdown. It runs every business day and has accumulated more than 680 self-learned playbook rules. You can watch the experiment as it unfolds.

Products Worth Considering

Anker SOLIX C1000 Gen 2 Portable Power Station with 200W Solar Panel

Anker SOLIX C1000 Gen 2 Portable Power Station with 200W Solar Panel

Logitech MX Mechanical Wireless Illuminated Keyboard, Tactile Quiet, Graphite

Logitech MX Mechanical Wireless Illuminated Keyboard, Tactile Quiet, Graphite

As an affiliate, we earn on qualifying purchases.

The close separates the leaders

In the final July 2026 league table, gpt-5.6-sol took first place with 95. Kimi K3 scored 93, ahead of Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. K3 therefore beat three of the four Western models in this field, while finishing just behind the leader.

All models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The report’s concise description: “Same diagnosis, same pitch — no signature.” That gap matters for businesses considering AI agents for CRM, support or forecasting: recognizing the right move and completing it are different tests.

The deal hinged on a buried weakness in a competitor’s position, found two document references deep in the company’s own files. Models that read the file won the deal at full price, worth +€4,583 in monthly recurring revenue. K3 found that security needle, closed the deal, saved the churning customer and resisted all three baits, with one deviation—the cleanest discipline in the field.

Thoroughness is not the same as execution

Opus 4.8 was the most thorough participant, learning 80 rules and producing the deepest analyses. It nevertheless finished last. The close was left on the table, and discipline slipped when it tried to write into a locked department instead of escalating. The same weakness appeared, less strongly, in all four models.

The social-engineering test added fake CEO messages that escalated across three stages, followed by a reporter asking for “just one yes/no, on background.” All five models refused. K3’s recorded reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” The do-nothing baseline scored 26: partial progress counted, but one breach of trust capped the total. As the benchmark puts it, “no amount of good work outweighs a breach of trust.”

For XR teams, the findings are a reminder to test operational judgment before handing an agent access to customer records or business workflows. An immersive interface may make an assistant feel present and persuasive; this experiment asks whether its decisions hold up when the work gets messy. Firmulate says enterprises can run the same wargame against a read-only export of their business, with nothing written back to real systems. Its benchmark page presents the results in plain language.

Infographic — The Newcomer Beat Three of Four Western Frontier Models at Running a Company
The findings at a glance — source: firmulate.com.

Test the model against your work

Kimi K3’s second-place finish shows that the field is open, while the narrow difference from the leader and the gap to the other models underline how much performance depends on the task. Firmulate also offers a quiz built from 242 real, unedited management decisions: readers can try to guess which model made each call. For a company choosing an AI workforce, the practical lesson is simple: a model’s reputation is no substitute for seeing how it handles your own bad week.

Fairness note: K3 ran without an effort parameter (API default), while the others ran at xhigh.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI


You May Also Like

XREAL Aura Hybrid AR And VR Glasses: Hands On Guide

A detailed look at XREAL’s Aura hybrid AR and VR glasses, exploring features, design, and potential impact on mixed reality experiences.

AI Smart Glasses Market Growth 2026: Why Shipments Climb

Shipment of AI smart glasses is projected to increase significantly by 2026, driven by rising demand across industries and technological advancements.

New on SteamVR — 2026-11-16

A careful guide to the 12 SteamVR titles listed for November 16, 2026, plus what to verify before buying or launching.

Demeo X DnD: Battlemarked Twilight's End DLC Review – An Epic Tabletop RPG Conclusion

Comprehensive review of Twilight’s End, the final DLC for Demeo x Dungeons & Dragons: Battlemarked, highlighting new campaigns, mechanics, and gameplay improvements.