They Don't Know How It Works | MOAI Studio
Back to Blog
Surf the Singularity #006<br>January 2026
They Don't Know How It Works
This is the dirty secret of AI: It's not engineering. It's alchemy.
> modern AI is alchemy wearing an engineering badge
> OpenAI and Anthropic both concede the interpretability problem is open
> pour in data, stir, out comes a poem or a python script
> ask why it produced that specific output
> nobody can give a full answer
> the field is growing something it can't fully read
> the same black box already screens hiring, lending, and security
> its reasoning lives in a billion-dimensional vector space
> intelligence without comprehension
> understanding got traded for capability
> fine until the box flags a person as a threat
> there is no appeal to a gradient descent
> every prior tool was understood before it shipped. this one just runs.
Appendix · Further Signal
Explainable artificial intelligence — Wikipedia
Mapping the Mind of a Large Language Model — Anthropic
Black box — Wikipedia
AI alignment — Wikipedia
Hallucination (artificial intelligence) — Wikipedia
Neural network (machine learning) — Wikipedia
Discussion — Hacker News
r/MachineLearning — Reddit
The Verdict
The machines work. We just don't know why. This is unprecedented in the history of technology. Every other tool humanity has built, we understood. Not this one.
Newsletter<br>Subscribe to Surf the Singularity
High-signal dispatches from the end of the world as we know it.
Subscribe