Live at the Witch Trials: If an LLM is too expensive it won't be next year
Saturday, May 16, 2026
If an LLM is too expensive it won't be next year
AI can cost more than human workers now<br>This is an Axios headline but the tldr is that once an LLM can do your job it will undercut you the next year.<br>This is the article and it focuses on token and infrastructure costs. Not cleaning up messes or other times a person needs to be in the loop. For example "Uber's chief technology officer already blew through his full 2026 AI budget due to token costs"<br>Example of someone thinking tokens will stay expensive.<br>No shade, it is just my belief they are wrong on this one claim.
There is a 7 year and a 4 year trend of the price drop to 1/10th a year for a set quality of token. As in $1 million become $1 hundred cost in 4 years.
This article looks at gpt 3.5 level tokens from late 2022 to late 2024
https:// and sees over 90% deflation a year<br>Example price decline in 2 years
But the deflation keeps going Qwen3.5 1.7B–2B is well above gpt 3.5 level in tests and in usage. This you can run really fast locally on any newish laptop or smartphone. Tokens at this quality level are now just electricity costs.
That's less than 4 years data but gpt 2 level has followed the same trend for 7 years. 7 more years of the trend turns $10 million cost into $1.
Lindy Effect says a reasonable first guess is if a trend has gone on for X length of time assume it will keep going for another X unless theres a good reason for it to stop. Coding and Video Image generation are still improving so fast there is not great reason to assume they will suddenly plateau yet. And the same for efforts to make smaller models smarter using big models. Teacher->pupil, chain of thought, distillation, pruning, low rank factorisation all keep getting better in a way that suggests that smaller models will keep getting better even if big ones plateau.
This is not an argument that LLMs in general will keep getting better though they will and that will help. Or that GPUs will keep improving though they will and that will help. Or that open weight models will stay about 9 months behind state of the art apis though they will and that will help a lot. Or that methods to condense smart models into smaller less demanding machines will improve though they will. It is that all these and a few other trends combined will work together to make a million tokens with a certain score on metrics and in a users opinion an order of magnitude cheaper every year.
Slop Tsunami comes down to the cost of tokens coming down.
The cost of tokens will come down. And at such a rate that it makes Moore's Law look tame. It won't go on forever but it is on a seven year trend with good reason to see it keeping going for at least 3 more years. At which point todays best current token will be 1/1000th the cost.
Posted by
Iamreddave
at
3:27 AM
Email ThisBlogThis!Share to XShare to FacebookShare to Pinterest
Labels:<br>llm,<br>prediction
No comments:
Post a Comment
Newer Post
Older Post
Home
Subscribe to:<br>Post Comments (Atom)
Labels
screwing the inscrutable<br>(90)
Funny Ha Ha<br>(84)
economics<br>(72)
Science<br>(70)
Maths<br>(55)
prediction<br>(52)
Rant<br>(43)
Prediction markets<br>(22)
Geek<br>(21)
security<br>(20)
progress<br>(18)
review<br>(16)
business plan<br>(14)
law<br>(12)
glpk<br>(10)
Homemade ventilator<br>(9)
hacking<br>(8)
quiz<br>(8)
Ruby<br>(7)
lottery<br>(6)
Long Now<br>(5)
PPX<br>(5)
analyticsx<br>(4)
collison<br>(4)
Rubik's<br>(3)
About Me
Iamreddave
View my complete profile
Stuff
Beer + Knitting
Real Wealth
A World Wide Web of Electricity
Our Friends Electric
Gruntled
Ginno
megalithomania
Watson Tricks
Blog Archive
2026
(12)
May
(1)
If an LLM is too expensive it won't be next...