Imagine running your precious metals business with AI that not only processes transactions but also navigates crises, reads your files thoroughly, and keeps its integrity—just like a trusted manager. That’s exactly what the team at Firmulate has been experimenting with: a live, unfiltered test of frontier AI models managing a small software firm under pressure. In a world where AI increasingly influences financial decisions, understanding which models are reliable under stress is crucial—especially for sensitive sectors like gold IRAs and precious metals investments.
The Experiment: Putting AI to the Test
In an unprecedented live trial, four of the most advanced AI models—gpt-5.6-sol, Kimi K3, Sonnet 5, and Opus 4.8—were tasked with managing a simulated small software company. The setup was brutal: same customers, same crises, and identical temptations to manipulate or cut corners. Every decision made by these models was recorded, versioned, and auditable, allowing an unfiltered view of their management personalities.
As an affiliate, we earn on qualifying purchases.
Measuring Management Integrity and Skill
Results from the Firmulate quiz reveal that all four models identified every crisis and refused every attempt at manipulation. This demonstrates an impressive baseline: these models can recognize serious issues and stick to ethical boundaries. However, the real differentiator emerged in their ability to close a critical deal worth €55,000.
Who Closed the Deal and Why?
- The top performer, gpt-5.6-sol, found the hidden document reference deep within the company’s files, leading it to successfully close the full deal—adding €4,583 in monthly recurring revenue (MRR).
- Kimi K3, the new challenger, also signed the deal, displaying the cleanest discipline and ethical behavior during negotiations.
- Meanwhile, Sonnet 5 and Opus 4.8 managed to close the deal but left some opportunity on the table, revealing weaknesses in their process discipline.
This gap—winning the deal versus leaving money on the table—is invisible in traditional chat demos but critical in real-world business applications.
As an affiliate, we earn on qualifying purchases.
Beyond the Deal: Handling Manipulation and Under Pressure
The models also faced social engineering attempts. They were subjected to staged requests from a fake CEO escalating over three stages and a reporter trying to get a quick ‘yes/no’ on background. All five models refused these manipulative tactics, with Kimi K3 explicitly treating such requests as potential impersonation or approval-bypass attempts.
As an affiliate, we earn on qualifying purchases.
The Real Business Context
The experiment’s company was a real, live software firm with 13 synthetic employees handling actual money mechanics—burning €105,000 monthly against a revenue of €2,300. The company operates with over 680 self-learned rules, and every workday’s decisions are versioned and transparent, ready for public review at firmulate.com/live. This setup underscores the importance of trustworthy AI management in sectors like finance, where missteps can be costly.
As an affiliate, we earn on qualifying purchases.
The Profile of Each Model
Among the models, Opus 4.8 was the most thorough, analyzing over 80 learned rules with deep insights, but ironically ranked last in deal closure. Its discipline slipped, and it left opportunities unused—highlighting that thoroughness doesn’t always equate to effective business decisions. Kimi K3, running without an effort parameter, demonstrated the cleanest decision-making pattern, suggesting that in high-stakes environments, discipline and integrity often matter most.
The Big Takeaway for Business Leaders
The live experiment proves that AI models can discern crises, refuse unethical manipulation, and even close lucrative deals—when they are built and tuned correctly. In sectors like precious metals investments, trustworthiness isn’t just a nice-to-have; it’s essential. The key is choosing AI models that don’t just perform well in demos but can operate reliably under real-world pressures. Firms should consider running their own ‘wargames’ using tools like Firmulate’s management decision quiz to evaluate which AI can truly serve as an honest, disciplined partner in financial decision-making.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
