GitHub - maddiedreese/gbc-transformer · GitHub

/" data-turbo-transient="true" />

Skip to content

Search or jump to...

Search code, repositories, users, issues, pull requests...

-->

Search

Clear

Search syntax tips

Provide feedback

--><br>We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Cancel

Submit feedback

Saved searches

Use saved searches to filter your results more quickly

-->

Name

Query

To see all available qualifiers, see our documentation.

Cancel

Create saved search

Sign in

/;ref_cta:Sign up;ref_loc:header logged out"}"<br>Sign up

Appearance settings

Resetting focus

You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

{{ message }}

maddiedreese

gbc-transformer

Public

Notifications<br>You must be signed in to change notification settings

Fork

Star<br>52

main

BranchesTags

Go to file

CodeOpen more actions menu

Folders and files<br>NameNameLast commit message<br>Last commit date<br>Latest commit

History<br>3 Commits<br>3 Commits

docs

docs

src

src

tools

tools

.gitignore

.gitignore

FLASHCART_README.txt

FLASHCART_README.txt

LICENSE

LICENSE

Makefile

Makefile

NOTICE.md

NOTICE.md

README.md

README.md

stories260K.bin

stories260K.bin

tok512.bin

tok512.bin

View all files

Repository files navigation

gbc-transformer

TinyStories-260K running locally on a stock Game Boy Color.

This is a proof-of-concept GBDK-2020 ROM that runs a quantized transformer<br>language model on the Game Boy Color CPU. Prompt entry happens on the handheld<br>with the D-pad/buttons and an on-screen keyboard. The ROM tokenizes the prompt,<br>runs transformer prefill, then autoregressively generates a short continuation.

It is intentionally tiny, slow, and imperfect. The current build proves the<br>end-to-end path on hardware; it is not a polished text generator yet.

What Works

Desktop FP32 reference runner for stories260K.bin.

Desktop row-wise INT8/Q8 packer and zero-float Q8 runner.

GBC ROM with the Q8 model embedded as MBC5 bank-switched cartridge data.

On-device BPE tokenization using the 512-token TinyStories tokenizer.

Integer/fixed-point transformer inference with RoPE, attention, RMSNorm,<br>SwiGLU-style MLP, and greedy argmax decoding.

Cartridge SRAM KV cache so base WRAM stays below 8 KB.

On-screen keyboard with explicit ENTER.

Runtime progress markers (P for prefill, G for generation).

Current Limitations

Extremely slow on real hardware. The C implementation works, but the hot<br>matmul loops need SM83 assembly to become demo-friendly.

Output quality is rough/gibberish. The GBC path uses heavy quantization and<br>fixed-point approximations.

Context is capped at 16 tokens.

Greedy decoding only; no top-k or temperature sampling yet.

Built for one model shape: TinyStories-260K (dim=64, hidden_dim=172,<br>layers=5, heads=8, kv_heads=4, vocab=512).

Hardware

Game Boy Color

EZ-FLASH Junior or another MBC5-compatible flashcart

microSD card mounted at /Volumes/bootfs for make sd-card

No soldering or internal Game Boy modification is required.

Dependencies

Python 3

C compiler for desktop tools (clang or gcc)

GBDK-2020 4.5.0 or compatible

curl and unzip for fetch helpers

By default the Makefile looks for GBDK at:

tools/vendor/gbdk-4.5.0

You can also point it elsewhere:

make GBDK_HOME=/path/to/gbdk gbc-rom

Fetch Model Assets

The TinyStories checkpoint and tokenizer are downloaded from Andrej Karpathy's<br>karpathy/tinyllamas Hugging Face repository:

make fetch-assets

This creates:

stories260K.bin

tok512.bin

Expected sizes:

stories260K.bin: 1,056,540 bytes

tok512.bin: 6,227 bytes

Build

Desktop references:

make run-fp32<br>make run-q8<br>make run-gbc-sim

Game Boy ROM:

make gbc-rom

Output:

build/TINYSTORY.GB

Prepare EZ-FLASH Junior SD Card

Fetch the official EZ-FLASH Junior stable kernel:

make fetch-ezflash

Then copy the ROM and kernel to the SD card:

make sd-card

The SD root will contain:

ezgb.dat

TINYSTORY.GB

README.txt

GBC-TINYSTORIES/ reference folder

In the EZ-FLASH menu, run TINYSTORY.GB from the SD card root.

Credits

TinyStories-260K and tokenizer: Andrej Karpathy's<br>karpathy/tinyllamas

GBDK-2020: https://github.com/gbdk-2020/gbdk-2020

EZ-FLASH Junior kernel: https://www.ezflash.cn/download/

About

No description, website, or topics provided.

Resources

Readme

License

MIT license

Uh oh!

There was an error while loading. Please reload this page.

Activity

Stars

52<br>stars

Watchers

watching

Forks

fork

Report repository

Releases

No releases published

Packages

Uh oh!

There was an error while loading. Please reload this page.

Contributors

Uh oh!

There was an error while loading. Please reload this page.

Languages

67.0%

Python<br>26.5%

Makefile<br>4.5%

Shell<br>2.0%

You can’t perform that action at this time.