Your Backtest Is Lying to You | MOAI Studio
Back to Blog
Surf the Singularity #028<br>June 2026
Your Backtest Is Lying to You
I evolved 1,513 strategies to snipe shitcoins off DexScreener. They printed +40% a month in simulation and made nothing live — here are the four ways a backtest manufactures an edge that was never there.
> be me. build an evolutionary bot to snipe fresh shitcoins off dexscreener.
> it breeds strategies and culls the losers — 1,513 of them across nine generations, over ~23,000 token-days of data.
> simulation prints +40% a month. strategy after strategy.
> push it live. it bleeds. every single time.
> the tempting move is to tune — better stop, better filter, blame the market.
> instead i tried to kill it, to prove the edge could never exist.
> lie #1, the stop that never fills. 9.3% of trades rug to about −90%, nowhere near the −7% the sim booked.
> lie #2, the moonshot. the top 1% of trades carry 95% of the gains. median trade: zero.
> lie #3, the winner's curse. breed thousands of variants and a few look like genius over 25 trades. pure luck.
> lie #4, the regime. the "edge" was just the weather the strategy was born into.
> so i re-screened honest: rugs eat the full loss, cap the upside, demand a median above zero, hold out half the data.
> 0 of 1,513 strategies survive. re-ran it today. still zero.
> then the part that ends the argument: the tokens that 10x are the tokens that go to zero. one population.
> every filter that trims the downside trims the upside in lockstep. safety and profit are the same dial.
> there was never an edge here. the machine's most valuable output was proving it, before it cost real size.
The Receipts · By the Numbers
WHAT A LOSING TRADE ACTUALLY COSTS<br>what the sim booked — stop fills
−7%<br>what 9.3% of trades really do — liquidity vanishes
−90%
The stop-loss is a promise the order book doesn't keep. Model the rug at −7% and you under-price the tail by ~8 points every trade.
WHERE THE RETURNS ACTUALLY LIVE<br>share of trades
1% of trades<br>share of all gains
95% of the gains
top 1% of trades
the other 99%
Win rate 42%, median trade 0%. When a handful of lottery tickets carry the whole return, the average isn't an edge — it's noise wearing a suit.
SAFETY AND PROFIT ARE THE SAME DIAL
rug rate
win rate
10.2%
25%<br>UNFILTERED
1.2%
9%<br>SAFE FILTER
Filter for low-volatility, deep-liquidity tokens and the rug rate collapses 10.2% → 1.2%. But the win rate craters 25% → 9% in lockstep — cut the left tail and you sever the right.
HOW MANY STRATEGIES SURVIVE HONEST SCRUTINY
1,513 strategies evolved
442 traded enough to judge
0 clear the honest gate
1,513 strategies bred over nine generations; 442 traded often enough to judge fairly. Under a rug-realistic, out-of-sample gate, zero survived. Re-ran it today — still zero.
THE WINNER'S CURSE — APPARENT EDGE DECAYS
0%
+5.39%<br>first 25 trades
−2.72%<br>same strategies, full history
Rank strategies by their first 25 trades and the top decile averages +5.39%. The very same strategies over their full history: −2.72%. The "edge" was a small sample flattering itself.
LOOSEN THE RULES, THE UNIVERSE STILL LOSES
0%
−5.99%<br>+30% cap
−4.71%<br>+50% cap
−3.10%<br>+100% cap
Average net return per strategy, by how generous the take-profit is allowed to be. Raise the cap all the way to +100% and the average still sits underwater. The universe loses at every setting.
Appendix · Further Signal
Backtesting — Wikipedia
Overfitting — Wikipedia
Multiple comparisons problem — Wikipedia
Survivorship bias — Wikipedia
Pareto principle — Wikipedia
Rug pull — Wikipedia
Discussion — Hacker News
r/algotrading — Reddit
The Verdict
The most useful thing this ever produced wasn't a winning strategy — it was a clear, early no. A negative result you can trust is cheap; the same lesson learned live, in real drawdown, is expensive. I'd rather pay for it in compute than in capital.
Newsletter<br>Subscribe to Surf the Singularity
High-signal dispatches from the end of the world as we know it.
Subscribe