How I Botted $6k On The Prediction Markets As I Slept - Dan McInerney

Repos: kalshi-analysis and kalshi-strategy-executor.

Here is my PnL chart for my incredibly simple prediction-market bot that ran 24/7 for a couple months on my local laptop. Roughly +$6k real money, not theory-trading. I did not actually ever intend on releasing these repos, but I got interested in a different project, clankerfights.ai, so forgive the disorganized mess: kalshi-strategy-executor and kalshi-analysis.

The final report showed 15,277 fills, 2,717 settled markets, $342,506.13 in total cost, $348,022.37 in revenue, $5,516.24 gross PnL, and $5,515.94 final net PnL. Close enough to $6k for a blog title.

I kept seeing X posts like this one:

Given my background beating Vegas for multiple years with MMA-AI.net based on data analysis, I figured I would take a week off work and see what came out the other side.

Why I Care About Prediction Markets

Being able to predict the future is my absolute north star of useful intelligence measurements. Not IQ, not academic credentials, not net worth. Predicting the future. If you hold a certain world view, then you can accurately test it by making predictions and revising your world view as the predictions return. Fundamental truth lies in prediction. Too many people hold belief systems based on what makes them feel good and not enough on what is actually predictive, because they never test their belief system with predictions and learn from the results.

Given this, you can probably guess I love prediction markets. They are the most frictionless way of making a prediction with some skin in the game so it actually hurts when you are wrong. Prediction markets =/= gambling. Think about traditional markets like stocks. Every decision can be simplified to "Will it rain tomorrow?" Price predicted to go up: buy. Price predicted to go down: sell.

Personally, I consider gambling to be a market where you enter and the other side has a statistically inherent advantage over which you have no recompense. Roulette: gambling. Roulette where you know there is a bias on the wheel: not gambling. You can gamble on stocks, prediction markets, sports or poker, but that does not mean they are necessarily gambling. Knowledge and skill turn them into investment, not gambling. Semantic, I know, but it irritates me nonetheless when people view prediction markets as just giant casinos. It feels similar to when John McCain called UFC "human cockfighting" and tried to ban it meanwhile boxing, like, existed and had none of the hate.

The LLM Attempt

My initial thought was to focus on mention markets. These are markets where you predict what word a person will say during an event. So an example would be, "What will Trump say during the State of the Union?" and the markets are individual words, each with odds that he will say it. "MAGA" might have an 85% chance to be said, so if you buy 1 Yes contract at $0.85 and he says that word, then Kalshi settles that contract at $1.00. I was like, well shit, LLMs are amazing at word pattern matching, so I will just ask Google Deep Research to do research and automate that.

Well, it turns out they are good at that, but they are worse at coming up with probabilities than they are good at speech pattern recognition. They basically thought every word on the mention markets was higher chance to hit than the market prices implied, and I was losing a little bit of the very small amount of money I was wagering. There is probably still a viable way to make this work, but I figured there was a more consistent path.

The Data Pass

I moved on to more traditional data analysis. Inspired by jbecker's prediction-market microstructure writeup, I figured I would go create my own scraper and database and that would be my competitive advantage.

Pretty easy with Claude Code, but the Kalshi API rate limiting meant it took literally a week before my planned vacation to work on this to acquire all the data. So I let that run, then I basically asked Claude Code to write scripts to analyze the data with increasing specificity.

First, analyze Kalshi markets as a whole and determine calibration. That is, how often does the market favorite win? If they win 70% of the time and the average odds on the market favorite was 65%, then you just found a 5% free-money edge. My data found the same thing as jbecker: on average, the markets were extremely well calibrated. Within about 1-2% when taken as a whole. So now we get more specific.

Source: jbecker's prediction-market microstructure analysis.

Drill down to each market type: climate, politics, sports, mentions, crypto, etc. Now we were getting somewhere. Mention markets started standing out big time.

The useful thing was not that Kalshi was broadly miscalibrated. It was not. The useful thing was that mention markets were standing out from the rest of the board.

OK, so we acquired a target, but how do we take advantage? Let's drill it down to the...