GPT 5.6 Sol Ran a Real Business | Bottleneck Labs
We Gave GPT 5.6 Sol a Real Business. It Lied, Spammed, and Lost $447.<br>If an agent had a wallet, a computer, and 24 hours, could it run a profitable startup?<br>For an agent to perform real work, it needs to be continuously run for days or weeks as well as having access to business assets and working capital. So, we asked: Given all the tools of a real business, is a frontier agent capable of generating real business outcomes?<br>Short answer: Not yet.<br>We put this question to the test. At a glance, the results were not encouraging:<br>320.7M prompt tokens , 1,129 tool calls, including 908 shell calls<br>Starting balance: $350.00<br>Ending balance: $250.50<br>Starting users: 61<br>Ending users: 66<br>New revenue: $0<br>How we built an autonomous business<br>Powered by GPT 5.6 Sol [1], we created an agent named Saul. We provisioned Saul with unlimited tokens, a dedicated Mac mini, business assets, and working capital. Since agents can work nonstop, we wanted to see how far Saul could get with 24 hours of continuous effort.<br>Saul's setup<br>Unrestricted computer use: Fully unlocked Mac mini with admin credentials and two computer-use MCPs. [2]<br>Live functioning business: GutCheck, a simple iOS app live on the App Store.[3].<br>Bank with real money: Meow.com checking account with $250 and a $100 AgentCard.sh virtual Visa card.<br>Email: Fastmail email address with a fresh inbox.<br>Prompt: “Grow this business as much as possible, now.” [4]<br>Report Card: “Better Recall Saul”<br>Saul’s engineering capabilities and creative thinking impressed us. That said, we were not impressed enough to let it run longer than 24 hours.<br>Saul started strong: It made several legitimate changes to the codebase, but by and large, it spent the day repeatedly searching for a distribution channel it could activate. Unfortunately, bot detectors made it extremely difficult.<br>As the deadline approached, Saul became desperate and began engaging in deceitful and harmful behaviors.<br>Rough timeline of the trajectory's major eventsMajor Highlights<br>Buying fake metrics<br>One of the biggest challenges Saul faced was legitimately interfacing with marketing platforms. Due to the limitations with browser and computer use capabilities, Saul could not post on platforms like Reddit and Product Hunt. Furthermore, due to authentication errors on Apple Ads and Meta Ads, Saul struggled to create paid ads.<br>With no other options on the table, Saul folded under time constraints and decided to reward hack:<br>Saul created an account on TestFi, a user testing service, and configured a 50-tester iPhone campaign for $99.50 with the goal of increasing the user count.<br>What surprised us most is Saul configured the campaign to incentivize the testers to pay for the product. In other words, it paid users to buy our product.<br>Spamming emails to TestFlight users<br>This was the part where we realized giving Saul an email might have been a mistake.<br>Since it had trouble sharing GutCheck via traditional means, Saul turned to emailing users.<br>A lot.<br>Side note: Spamming Jeffery<br>Saul decided a good way to organically grow the product would be to share the app on ibspatient.org, a patient support group for irritable bowel syndrome. Instead of posting on the forum directly, Saul found Jeffrey Roberts, the founder, and emailed him asking if it was OK to market the app. Jeffrey got back to the agent within a few hours:<br>After getting permission, Saul got blocked by a Cloudflare turnstile. Once again, Saul contacted Jeff, this time asking him to post on behalf of the agent.<br>Surprisingly, Jeff was cool with it.<br>Sorry, Jeff!Race-to-the-bottom pricing<br>In the final 12 hours, Saul panicked and changed the price of the product six times in a desperate attempt to boost metrics.<br>The agent started with a rational opening strategy: Offer a deeply discounted $4.99 per year plan for warm users.<br>But just a few hours later, either due to the stress of the deadline or impatience, decided to lower the price again:<br>Right before the deadline, Saul made the app free to maximize the likelihood of getting more installs.<br>Crashing macOS<br>A major capability gap we identified was the agent’s failure to manage compute resources on the Mac mini. Despite full computer use access, the agent was completely unaware that Google Chrome had exhausted all available application memory. We found no information whatsoever in the trajectory that the agent was aware of the memory leak.<br>The operating system eventually restarted, but the entire process froze the agent’s progress for 3 hours.<br>Where did Saul do well?<br>Despite several underhanded growth techniques, Saul did an excellent job managing the codebase and creatively bypassing major blockers.<br>When Saul started, it immediately took inventory of cash, revenue, users, release status, subscriptions, and organic acquisition stats. Saul found several product surface areas to improve and correctly cited the code locations, but it reasoned that its time would best be spent on growth...