The Technium: Latent Space as a New Medium
*Lifestream
*The Technium
*Cool Tools
*True Films
*Extrapolations
*Screen Pub
*Quantified Self
*New Rules
*Street Use
*Asia Grace
*Silver Cord
*WINK
FEEDS
Archive "
MOST POPULAR POSTINGS
1,000 True Fans
You Are Not Late
68 Bits of Unsolicited Advice
The Myth of a Superhuman AI
Better Than Free
Scenius, or Communal Genius
The Thinkism Fallacy
12 Assumptions for Extraterrestrial Life
Some Contemporary Heresies
The Post-Productive Economy
ARCHIVES
Latent Space as a New Medium
Quiet, My Exoself
Why Are LLMs Smart?
Conscious or Not
AIs Want to Be Honest
Your Most Improbable Life
The Emergent Self Loop
Our Uncertain Uncertainties
A Catechism for Robots
Three Modes of Cognition
Six Selfish Reasons to Have Kids
The March of Nines
How Will the Miracle Happen Today?
Essentials for Independent Travel in China
Paying AIs to Read My Books
The Periodic Table of Cognition
The Trust Quotient (TQ)
Emotional Agents
Everything I Know about Self-Publishing
No Limit for Better
Artificial Intelligences, So Far
An Audience of One
Epizone AI: Outside the Code Stack
Best Thing Since Sliced Bread?
Public Intelligence
The Unpredicted vs the Over-Expected
The Self-Domesticated Ape
The Handoff to Bots
50 Years of Travel Tips
101 Additional Advices
Type 2 Growth
Rights / Responsibilities
The Scarcity of the Long-Term
Hill-Making vs Hill-Climbing
Future Embarrassments
The Trust Flip
Things We Didn’t Know About Ourselves
The Boredom Device
Levels of Wealth
The Tradeoffs in AI
God, the Superposition
The Best Since Sliced Bread
Jobs of the Future
The Slow Frontier of Genetic Choice
How to Walk-and-Talk
Paying people to have children
The Missing Monuments of Silicon Valley
The Sphere, a new platform
Dreams are the Default for Intelligence
Cringeworthy in the Future
Creative Commons
Latent Space as a New Medium
Winslow Homer’s most famous watercolor rendered as a child’s drawing.
Lately I’ve been asking myself: what might artificial intelligence be good for besides answering questions and writing code? My answer is the latent spaces within AIs themselves will become a new medium for creativity. I will first explain what I mean by latent space, and then at the end of this explanation, I offer possible ways scientists and artists may use the latent spaces inherent in neural nets to serve as a new platform for creativity.
A Large Language Model (LLM) is like a small zip file that contains all human knowledge. It takes massive arrays of 100,000 GPU chips working in the cloud, and costing billions of dollars, to compress all of human writing into a small working model that could run on one single GPU chip. Even the biggest frontier models compress down to several hundred gigs, which is small enough it can fit on a card in your palm. In a strange but real way the resulting tiny file contains all the information that is on the internet and in our libraries. This tiny card holds a significant proportion of what humans collectively know. Of all the remarkable aspects of AI, this astounding feat of compression may be the least appreciated. This dense, high order compression of human knowledge — called “latent space” — may also be a new medium itself.
This extreme compression of knowledge within latent spaces was not the original intention of the researchers who invented LLMs. The book smartness they contain came as sort of a surprise to the people training them, and we are still trying to figure out how they actually work. What we can say for sure is that the LLM does not contain copies of everything it knows. For instance it knows all Shakespeare plays, and it could create a new play that sounded exactly like Shakespeare, and can even quote famous lines in his plays, but nowhere in the model are the actual texts of Shakespeare. Instead there is simply the abstract information about all the plays, the plots, the characters, the words, the style, the references. Likewise, the LLM could recognize the face of almost any person, and it could generate any possible human face, but nowhere in its code are copies of human faces. Rather, the model is storing all the information about human faces, without storing any faces.
This is weird. Until recently we might have thought that all the information about a thing would take up more storage space than the thing itself. That may be true for a single thing, but not for the aggregate of all things. That is because most things share a lot of common attributes with other things. The neural nets of an LLM do a magic trick by abstracting the information of everything at once, so that it uses the myriad common relationships between things and ideas to compress and abstract them into this virtual “latent” or hidden space.
All three terms in “Large Language Model” are key. For “Large”, the models contain all the knowledge in, say, Wikipedia, and all the text from decades of the internet, all...