egpu for mac - tinygrad docs

Skip to content

Initializing search

GitHub

dtypes

nn (Neural Networks)

Environment Variables

Runtime

5. Use it!

Developer

tinybox

TinyGPU¤

TinyGPU app lets you use AMD and NVIDIA GPUs on macOS over USB4/Thunderbolt with tinygrad.

Requirements¤

macOS (13.0+)

USB4/Thunderbolt port

A supported GPU (AMD RDNA3+ or NVIDIA Ampere+)

Setup¤

1. Connect your GPU¤

Plug the supported GPU into your Mac over USB4/Thunderbolt.

2. Initiate the driver install¤

Note: If tinygrad is cloned but not installed, run commands with PYTHONPATH=.

curl -fsSL https://raw.githubusercontent.com/tinygrad/tinygrad/master/extra/setup_tinygpu_osx.sh | sh

This downloads TinyGPU.app and triggers a system prompt to install the driver extension.

3. Enable the driver¤

You should see a system prompt: "TinyGPU" would like to use a new driver extension . Click Open System Settings and toggle TinyGPU on.

If you missed the prompt, go to System Settings > General > Login Items & Extensions > Driver Extensions and toggle TinyGPU on.

4. Compiler Setup¤

AMD¤

curl -fsSL https://raw.githubusercontent.com/tinygrad/tinygrad/master/extra/setup_hipcomgr_osx.sh | sh

NV¤

Install Docker Desktop if you don't have it.

curl -fsSL https://raw.githubusercontent.com/tinygrad/tinygrad/master/extra/setup_nvcc_osx.sh | sh

Make sure ~/.local/bin is on your PATH:

export PATH="$HOME/.local/bin:$PATH"

5. Use it!¤

DEV={AMD|NV} python3 -m tinygrad.llm

Note: Use JITBEAM=2 to search for faster kernels (one-time search cost, results cached).

Back to top

Made with

Material for MkDocs