TRiP/My_TRiP_through_AI-Chapter3.md at main · carlovalenti/TRiP · GitHub
//blob/show" data-turbo-transient="true" />
Skip to content
Search or jump to...
Search code, repositories, users, issues, pull requests...
-->
Search
Clear
Search syntax tips
Provide feedback
--><br>We read every piece of feedback, and take your input very seriously.
Include my email address so I can be contacted
Cancel
Submit feedback
Saved searches
Use saved searches to filter your results more quickly
-->
Name
Query
To see all available qualifiers, see our documentation.
Cancel
Create saved search
Sign in
//blob/show;ref_cta:Sign up;ref_loc:header logged out"}"<br>Sign up
Appearance settings
Resetting focus
You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
{{ message }}
carlovalenti
TRiP
Public
Notifications<br>You must be signed in to change notification settings
Fork<br>12
Star<br>78
FilesExpand file tree
main
/My_TRiP_through_AI-Chapter3.md
Copy path
Blame<br>More file actions
Blame<br>More file actions
Latest commit
History<br>History<br>History
87 lines (45 loc) · 5.39 KB
main
/My_TRiP_through_AI-Chapter3.md
Copy path
Top
File metadata and controls<br>Preview
Code
Blame
87 lines (45 loc) · 5.39 KB
Raw<br>Copy raw file<br>Download raw file
Edit and raw actions
Ready, pack, go!
An excerpt from "My TRiP through AI" by Carlo Valenti — Chapter 3
TRiP is my 15,000-line transformer engine in C. This is the chapter where it gets built.
REPO of TRiP engine: https://github.com/carlovalenti/TRiP/
Year 2023 was almost over, our son Paolo was born, and my understanding of AI had reached a really refreshing level, I could even say: absolute zero.
Internet material can be very hard to chase and follow, when you are busy with diapers and broken nights; and right when you manage to put the poo-poo aside and take some time, you discover that most resources don't let you past the sacred technical veil, and just repeat the pop-science mantras. I was so frustrated that I began talking like Miguel.
Luckily1, I began to connect the dots; concepts clarified; vectors appeared with their names, and tensors passed by to say hello.
The time has come to tell you this: at the end of the book there's a "technical appendix" — don't let the name scare you — where I happily walk you through the concepts and mysteries of "transformer" language models, the real revolution behind modern AI. If you're not familiar with AI, I strongly recommend reading it NOW: you'll find there the keys to making sense of the rest of the book.
At the beginning of 2024, I also discovered a group of researchers who wanted exactly what I wanted, to understand what's inside:
"A surprising fact about modern large language models is that nobody really knows how they work internally", they write.
"Hmmm. If they don't, I sure 'don't don't'... but maybe I can."
I quickly came to the conclusion: if I wanted to understand AIs, I needed to build one.
And, long story short, I did.
I packed, and in eighteen months2, during my lunch breaks at work and weekend nights, I programmed a full transformer3, and I named it "TRiP"4:
it could write text, and then could chat; you could show pictures, and it would describe them; and then — my ultimate goal — it could learn: patiently, from scratch, or just a bit more. But the one who learned a lot was me; and that was exactly what I was looking for.
Sometimes, in the weekend nights, Sofia couldn't sleep; she nestled up to me, and asked what I was doing.
"I am building a… a tiny robot."
"Really? Where is it?"
"It's… It's not here, it's far from us, but see, one day I'll let you talk to it, and maybe we will ask it to tell you a story. Ok?"
When TRiP became able to fulfill the promise, we actually made an attempt:
"TRiP, please tell a story about unicorns!"
AIs need big computers, and mine was small: the story came out, but sooo sllooowwllly, that ALL unicorns died in the process. Sofia laughed, and luckily that sufficed.
Telling a story was exactly the very first thing that TRiP became able to do. I suggested the intro:
"Hi! I am TRiP, an engine for transformer AIs!"
With the brio of an English gentleman under sedatives, TRiP continued:
"Icemen are my friends. We must keep things safe. We will manage, we are good, we will keep things safe."
"What the…?"
OK. It was late August 2024, the weather was hot, so Icemen looked appropriate after all, as well as TRiP's concern for AI safety. I was so happy! But, unlike Sofia, that was not enough for me; and I continued to improve TRiP's abilities for one more year.
Before you ask: no, I did not unlock the mysteries of Artificial Intelligence. In fact, I learned this: that a car is not its engine. It's the steering wheel, the frame, the seats, the wheels, the fuel, and so on. If you...