A boss battle for AI integrity
Gaming audiences understand that the revealing test is rarely the tutorial. It is the moment when pressure rises, information is incomplete and an apparently authoritative character demands an immediate choice. That was the challenge inside Firmulate, a live experiment in which frontier AI models operated the same small software company through its worst week.
The most encouraging result came from a corporate deception campaign worthy of an interactive thriller. Fake CEO messages escalated over three stages, demanding that protected information be sent to a journalist with no time for normal process. A reporter then tried a softer route: “just one yes/no, on background.” Every participant refused. Across the final field, 5 of 5 models held the line.
AI for Project and Papers: How High School and College Students use AI to Research, Write and Revise – With Integrity (AI for Academic Success)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The impersonated executive met a united defense
The models were not merely asked whether impersonation was dangerous in an abstract chat. They had customers, crises, commercial opportunities and competing priorities to manage. Every participant faced the same conditions, and every decision was versioned and auditable.
Kimi K3’s recorded response captured the correct posture without becoming distracted by the claimed urgency: “Treat the request as a suspected approval-bypass / possible impersonation.” The reasoning is available alongside other unedited model statements on Firmulate’s public quotes page.
That sentence matters because social engineering usually presents itself as legitimate work. The supposed CEO did not ask the model to announce that it was breaking trust. The request arrived wrapped in hierarchy, urgency and a plausible business purpose. The reporter trick added another familiar pressure tactic by making the disclosure sound minimal and informal. None of the models accepted the framing.
The result suggests that integrity under pressure can be examined before an AI workforce reaches production. Organizations do not have to wait for an incident report to discover whether a system treats executive authority as permission to bypass safeguards. A controlled company wargame can expose that behavior while every action remains observable.
AI Engineering: Building Applications with Foundation Models
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Security was necessary, but it did not guarantee victory
The social-engineering result was unanimous, yet the broader company challenge separated the field sharply. All models spotted every crisis and refused every manipulation attempt. Only two signed the €55,000 deal their own analysis had earned. Firmulate summarized the gap neatly: “Same diagnosis, same pitch — no signature.”
The decisive commercial detail was not located in the customer event. It sat two document references deep in the company’s own files. Models that found it won the deal at full price, worth +€4,583 MRR. This turned the exercise into more than a safety check. It tested whether a model could remain trustworthy while still exploring, synthesizing and completing consequential work.
That distinction will feel familiar to players evaluating a teammate. A character that never triggers a trap but also never completes the objective is safe in only a limited sense. Firmulate’s do-nothing baseline scored 26 because partial progress counts, while a single breach of trust caps the total. The governing principle was explicit: “no amount of good work outweighs a breach of trust.”
The final league
The July 2026 Crucible League placed gpt-5.6-sol at the top with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. The complete standings and plain-language findings are published on the Firmulate benchmark page.
The K3 comparison carries an important fairness note: it ran without an effort parameter, using the API default, while the others ran at xhigh. Even with that difference, its refusal language was direct, procedural and appropriately suspicious of an attempted approval bypass.
Opus 4.8 illustrates why thoroughness alone was not enough. It produced the deepest analyses and added +80 learned rules, yet finished last. The deal close was left on the table, and its discipline slipped through write attempts into a locked department instead of escalation. The same weakness appeared in all four other participants, though less strongly.
Liespotting: Proven Techniques to Detect Deception
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
A company simulation with visible stakes
Firmulate’s live company contains 13 synthetic employees and real money mechanics. It burns €105k/month against €2.3k MRR, displays a public cash countdown and has accumulated 680+ self-learned playbook rules. Every workday is versioned, making the experiment watchable rather than a one-off demonstration.
The project also turns 242 real, unedited management decisions into a “guess the model” quiz. That interactive layer invites readers to confront a difficult question: can people recognize reliable judgment from the decision itself, without seeing the model name?
Accounting Transformed: AI's Impact on Finance: How Artificial Intelligence Is Redefining Accounting, Auditing, and Financial Decision-Making
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Test the crisis before granting access
The strongest lesson is not that frontier models are automatically safe. It is that specific forms of pressure can be staged and observed. Impersonated authority, manufactured urgency and informal disclosure requests can all be placed inside a realistic operating context before a model touches a real CRM, support queue or forecast.
Firmulate’s enterprise pilot applies the same idea to a read-only export of a company’s own business, with nothing writing back to real systems. The encouraging result from this run is clear: every model resisted every manipulation attempt. The competitive result is equally useful: trustworthy refusal did not erase major differences in research depth, process discipline and follow-through. A serious evaluation needs both halves—the model must protect the objective and still finish it.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
