JEPA and the World Model Bet | Aryan Yadav
← Back to Blog
Yann LeCun at Ecole Polytechnique in 2018. Photo by Jeremy Barande / Ecole polytechnique Universite Paris-Saclay, CC BY-SA 2.0, via Wikimedia Commons.
The most expensive argument in AI right now is not "open weights versus closed weights" or "agents versus copilots."
It is whether intelligence begins with language.
The mainstream answer has been yes, or close enough. Take a transformer, feed it text, scale the hell out of it, bolt on vision, tool use, memory, RL, product screenshots, and vibes. The model starts as autocomplete and somehow becomes useful enough that everyone pretends the ontology is clean.
It is not clean.
Yann LeCun's counter-bet is JEPA, short for Joint Embedding Predictive Architecture. The claim is simple enough to be dangerous: real intelligence does not start with producing words. It starts with building internal models of the world, then using those models to predict what will happen if you act.
This is why LeCun's new company, AMI Labs, reportedly raised $1.03 billion in March 2026 to build world models. That is an absurd amount of capital for what is still mostly a research program. But absurdity is normal now. We gave autocomplete a trillion-dollar cap table. The bar for financial sanity left the building a while ago.
The thesis<br>LLMs learned language. JEPA tries to learn consequence. The difference matters because agents do not just need to answer. They need to choose actions before reality slaps them for choosing badly.
Why Next-Pixel Prediction Fails
The podcast starts in the obvious place: next-token prediction worked weirdly well for language. You give the model text, hide the next token, and train it to predict the missing piece. Do that at scale and you get GPT-style models with useful internal representations.
So why not do the same thing with video?
Take a few frames. Predict the next frame. Train on the pixels. In theory, the model learns how the world moves.
In practice, it learns blur.
The reason is not mysterious. If a ball might bounce left or right, and your model is forced to output one pixel-level future, the safe answer is the average of possible futures. Average a left bounce and a right bounce and you get a smear. Repeat that autoregressively and your future becomes soup.
Language can dodge some of this because tokens are discrete. "Left" and "right" are different outputs. Video does not have a neat 50,000-token vocabulary. A frame is millions of continuous values, most of them irrelevant.
Leaves moving behind a car have lots of pixels. They should not get lots of intelligence.
Fig. 01 · The wrong target eats the model
TWO SELF-SUPERVISED TARGETS
LANGUAGE<br>Predict the next token
the ball bounced to the
left<br>right<br>Discrete choices. Ambiguity can stay explicit.
VIDEO<br>Predict the next pixels
average future<br>Continuous pixels. Ambiguity becomes blur.
Generative video prediction asks the model to render every uncertainty. JEPA changes the target so the model can focus on the state that matters.
JEPA Moves the Loss Into Meaning Space
JEPA does not ask the model to generate the missing image, frame, text, or action. It asks the model to predict an embedding.
The setup is clean:
Encode X. Encode Y. Train a predictor to map embedding(X) to embedding(Y).
That is the whole trick. The system is still predictive, but it predicts in latent space. It predicts the compressed representation of the future, not the raw sensory surface.
This is why the architecture is not simply "non-generative" as a branding move. It is non-generative because generating the surface is often the wrong job. If I ask you what happens after a cup gets nudged off a table, you do not need to hallucinate every photon. You need the useful state transition: cup falls, gravity wins, floor gets involved.
The floor is not impressed by prose.
Fig. 02 · JEPA in one diagram
JOINT EMBEDDING PREDICTIVE ARCHITECTURE
Context X<br>visible video patches
Target Y<br>missing future state
Encoder<br>state vector
Target encoder<br>target vector
Predictor<br>predicts embedding(Y)
Loss in embedding space<br>match the meaningful future
No pixel reconstruction. No caption required. Learn the representation that predicts what matters.
The target is not "draw the missing patch." The target is "predict the representation of the missing patch." That one sentence is most of the philosophical shift.
The Old Problem: Collapse
Joint embedding systems have a dumb failure mode. If the objective says "make the two embeddings similar," the model can cheat by returning the same embedding for everything.
Dog? Same vector.
Truck? Same vector.
Founder pretending the roadmap is under control? Same vector, unfortunately plausible.
This is representation collapse. The loss looks happy. The model learned nothing.
The history here matters. LeCun worked on Siamese networks in the 1990s. Later contrastive systems avoided collapse by using...