Compiling a Go program into a native binary for Nintendo Switch™ - Ebitengine

Compiling a Go program into a native binary for Nintendo Switch™

Hajime Hoshi<br>2022-01-03

This is an English translation of my article in Japanese.

tl;dr

Previously, we compiled a Go program into a WebAssembly and then converted it into C++ files to make it run on Nintendo Switch. Now, I have succeeded in compiling a Go program into a native binary for Nintendo Switch, and also running a game there. I replaced system calls with C function calls using the -overlay option. Also, I have developed a new package Hitsumabushi to generate JSON content for this.

Caution

This article and the open-source projects in this article are based only on publicly available information. Hajime is responsible for this article's content. Please do not ask Nintendo about this article.

Background

I have been developing a 2D game engine called Ebiten in my spare time. I have succeeded in porting this to Nintendo Switch and the Nintendo Switch version of "Bear's Restaurant" was released in 2021.

Copyright 2021 Odencat Inc.

The method was to compile a Go program into a WebAssembly (Wasm) binary and then convert it to C++ files. See the presentation slides from GoConference 2021 Autumn for more details. The advantages were low uncertainty, low maintenance cost, and high portability. Once I developed the tool, its maintenance cost was pretty small as Wasm's specification is stable. On the other hand, the disadvantages were bad performance and long compiling time. Not only that performance was worse than native, but GC also suspended the game due to a single thread.

Compiling a Go program into a native binary for Nintendo Switch without using Wasm was quite uncertain and a rocky road. Of course, Go doesn't support Nintendo Switch officially. And naturally, Nintendo Switch's source code and binary formats are not open. Even if I hit an issue, it'd be possible that there would not be any clues to help me solve it. However, if I knew that I were to succeed, performance would be better than ever, and compiling speed would be as fast as Go. So I thought it was worth a shot and have been doing some experiments intermittently for one year.

Strategy

The strategy is basically to replace system calls with C function calls in the runtime and the standard library. The system calls part is OS-dependent, and if I replace it with something portable, Go should work everywhere in theory. It seems pretty easy, doesn't it? Well, it was a lot more challenging than I expected...

The graphic below describes what I had to do. The left side is a structure of a structural overview of standard Go compiling. System calls work on specific systems and of course, this doesn't work on Nintendo Switch. So I had to replace them with standard C function calls like the right side.

Replacing system calls with C function calls

And, there is another action item to adjust the binary format that the Go compiler generates to fit with Nintendo Switch. So in summary, the action items were as follows:

Replacing system calls with standard C function and/or pthread function calls

Adjust the ELF format that the Go compiler generates

For replacing system calls, of course, system calls do not correspond one-to-one with C functions. And, there are too many system calls to implement. So, I replaced system calls one by one by finding which ones refused to work on an actual Nintendo Switch device.

The Go compiler can generate only formats that the Go compiler officially supports. For example, when a target is Linux, the format is ELF. Can Nintendo Switch support ELF? To make a long story short, yes, I managed it. I won't describe the details about 2. here*1.

What I have to do is create a .a file via the Go compiler with GOOS=linux GOARCH=arm64 and -buildmode=c-archive, and then link it with other object files and libraries via Nintendo Switch compiler. The reason why I don't use -buildmode=default is that there are some items I have to do around an entry point. IMO, in general, it is more portable to depend on the platform for an entry point.

System calls are defined basically in the standard library, especially runtime and syscall packages. So, how did I rewrite them? In this project, I adopted the -overlay option.

Hitsumabushi - rewriting the runtime with the -overlay option

go build's -overlay is an option that overwrites Go files to be compiled. I overwrote Go files in the runtime with this option. This is the official document's explanation:

-overlay file<br>read a JSON config file that provides an overlay for build operations.<br>The file is a JSON struct with a single field, named 'Replace', that<br>maps each disk file path (a string) to its backing file path, so that<br>a build will run as if the disk file path exists with the contents<br>given by the backing file paths, or as if the disk file path does not<br>exist if its backing file path is empty. Support for the -overlay flag<br>has some...