GitHub - martinkunc/cudametal · GitHub
/" data-turbo-transient="true" />
Skip to content
Search/
Sign in<br>Sign upAppearance settings
You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.
Dismiss alert
{{ message }}
martinkunc
cudametal
Public
Notifications<br>You must be signed in to change notification settings
Fork
Star
main
BranchesTags
Go to file
CodeOpen more actions menu
Latest commit
History<br>12 Commits<br>12 Commits
Folders and files<br>NameNameLast commit message<br>Last commit date<br>bin
bin
bugreports/metal-threadgroup-corruption-grid128
bugreports/metal-threadgroup-corruption-grid128
examples
examples
repos_build_artifacts
repos_build_artifacts
runtime/libcudametalrt
runtime/libcudametalrt
spike
spike
spirv-to-msl
spirv-to-msl
test
test
.gitignore
.gitignore
DESIGN.md
DESIGN.md
README.md
README.md
View all files
Repository files navigation
cudametal
Compiles a useful subset of CUDA C++ (.cu) into a real Metal binary and<br>runs it on macOS/Apple Silicon GPUs -- no NVIDIA GPU needed.
For simple example look into examples/rescale.
How it works
bin/cudametalcc takes a .cu file and runs it through a real compiler<br>pipeline, entirely built from public, documented tools (no attempt to<br>reverse-engineer NVIDIA's or Apple's own internals):
>>, ...)<br>-> clang -cc1 -x cuda (unpatched upstream Clang, device IR,<br>spirv64 target)<br>-> llvm-link, opt (always-inline + globaldce)<br>-> llc -mtriple=spirv64-unknown-unknown -> spirv-dis<br>-> spirv-to-msl/kernel_spirv_to_msl.py (this project's own from-<br>scratch SPIR-V -> MSL<br>translator)<br>-> xcrun metal / metallib (a real .metallib)<br>-> clang -x cuda --cuda-host-only -fcuda-include-gpubinary<br>(embeds the .metallib; Clang's own<br>CUDA host-side codegen auto-generates<br>the __cudaRegisterFatBinary/<br>__cudaRegisterFunction calls)<br>-> link against runtime/libcudametalrt (a libcudart-compatible host<br>shim: cudaMalloc/cudaMemcpy/<br>>> launch, backed by real<br>Metal buffers and command<br>queues)<br>-> a native, runnable macOS executable">foo.cu (real CUDA syntax: __global__, threadIdx, blockIdx, >>, ...)<br>-> clang -cc1 -x cuda (unpatched upstream Clang, device IR,<br>spirv64 target)<br>-> llvm-link, opt (always-inline + globaldce)<br>-> llc -mtriple=spirv64-unknown-unknown -> spirv-dis<br>-> spirv-to-msl/kernel_spirv_to_msl.py (this project's own from-<br>scratch SPIR-V -> MSL<br>translator)<br>-> xcrun metal / metallib (a real .metallib)<br>-> clang -x cuda --cuda-host-only -fcuda-include-gpubinary<br>(embeds the .metallib; Clang's own<br>CUDA host-side codegen auto-generates<br>the __cudaRegisterFatBinary/<br>__cudaRegisterFunction calls)<br>-> link against runtime/libcudametalrt (a libcudart-compatible host<br>shim: cudaMalloc/cudaMemcpy/<br>>> launch, backed by real<br>Metal buffers and command<br>queues)<br>-> a native, runnable macOS executable
kernel_spirv_to_msl.py is a purpose-built translator for exactly the<br>Kernel/OpenCL-flavor SPIR-V this pipeline produces (SPIRV-Cross only<br>accepts Shader/Vulkan-flavor SPIR-V, which Clang's CUDA path never<br>emits). See DESIGN.md for the full design and supported-feature list,<br>and spirv-to-msl/README.md for the translator's own running bug/fix<br>log.
Translating a real third-party CUDA project
repos_build_artifacts/ holds build configs for fetching and building<br>real, independently-written CUDA projects through cudametal -- a much<br>better way to find translator gaps than writing kernels to order (see<br>spirv-to-msl/README.md's "Status update" log for what this has already<br>found and fixed).
Example: Karpathy's llm.c
llm.c is a real, independently<br>written GPT-2 training implementation in raw CUDA. Its fp32-only<br>training program (train_gpt2_fp32.cu) builds and trains a real<br>GPT-2-124M model end to end through cudametal, completely unmodified.
Fetch, drop in cudametal's own Makefile.cudametal, and build:
.../checkout/<br>+ building (CUDAMETAL_DIR=/Users/mkunc/src/cudametal)<br>rm -f train_gpt2 train_gpt2cu_metal train_gpt2fp32cu_metal<br>cc -Ofast ... train_gpt2.c -lm -o train_gpt2<br>bin/cudametalcc train_gpt2_fp32.cu -o train_gpt2fp32cu_metal<br>...<br>cudametalcc: built train_gpt2fp32cu_metal (kernels: encoder_forward_kernel3,<br>encoder_backward_kernel, layernorm_forward_kernel3, permute_kernel,<br>permute_kernel_backward, unpermute_kernel, unpermute_kernel_backward,<br>softmax_forward_kernel5, residual_forward_kernel, gelu_forward_kernel,<br>gelu_backward_kernel, matmul_backward_bias_kernel4,<br>layernorm_backward_kernel2, softmax_autoregressive_backward_kernel,<br>adamw_kernel2, fused_classifier_kernel3, matmul_forward_kernel4)<br>--- train_gpt2cu (bf16, cublasLt fused epilogue) is a known,<br>documented cudametal gap -- run 'make -f Makefile.cudametal<br>build-cuda-bf16' directly to see it fail and capture why.
built:<br>repos_build_artifacts/llm.c/checkout/train_gpt2<br>repos_build_artifacts/llm.c/checkout/train_gpt2fp32cu_metal">$ bin/fetch_build_llm_c<br>+...