Wrapping C libraries in Nim - Peter's DevLog
Hide
Scraps
RSS
Peter's DevLog
I'm a computer scientist from Norway with interests in tinkering, open source, and game development.
This log is intended as a way to both share my projects and keep myself from forgetting them.
If you have any commentary please direct them to me by e-mail: devlog@peterme.net, any e-mail content can be reproduced on this site without further notice.
View all entries
-->
Wrapping C libraries in Nim
17th April 2023 - Guide<br>, Nim<br>, Programming
As we discovered in my last article, Nim will by default generate C code and then call on a C compiler to actually produce a binary. This might seem like an odd choice, especially in the age of LLVM. However it’s actually not uncommon for languages to compile or transpile into another language. Initially the choice to not use LLVM was simply because it wasn’t as mature back when Nim was created. Though going through C has some quite substantial benefits. For example you’re able to run Nim anywhere you can run C, which means you can run it pretty much anywhere. From the tiniest micro-controllers, to mobile apps, to normal desktop/server targets1. There is another benefit though which this article focuses on, namely the ability to super easily interface with C libraries or programs. This is what’s called a foreign function interface or FFI and just means we’re calling functions from another language, in this case Nim from C, or C from Nim. Of course the Nim language has quite a few extra features and systems compared to C, so when using a library written in C we often want to do a little bit of translation work in order to make the interface a bit more familiar to Nim users2. Especially when it comes to things like memory management, and things like async. In this article I’ll outline how this works in practice, and show some examples from wrapping the Constrained Application Protocol (CoAP) library in Nim.
Telling Nim what’s available
Making use of a C library in Nim is pretty easy, all you need to do is tell Nim the signature of a procedure, then tell it that this is something that exists in C and you’re good to go. Take a very simple example like this:
proc hello(x: string) {.importc.}
hello("world")
This compiles just fine from Nims point of view, and will generate C code which calls the procedure hello with a Nim string object. Of course if this procedure doesn’t exist anywhere in the C standard library, or in any library we import through switches this will create a undefined reference to 'hello' error from the C compiler3. You can further tell Nim which header this procedure comes from, or which dynamic library to link against to get it with additional pragmas. This will cause Nim to automatically import or link to these when the procedure is used. When wrapping a C library you likely want to use the c prefixed types in Nim. For example cstring, cint, cuint, csize, clong, and cfloat which all map to their corresponding type without the prefix in C. You will also have to get used to using a lot of ptr types. Type signatures and objects have to match exactly between C and Nim, but you can use Nims much better type system to improve on the basic signatures. For example a C procedure which should take an int where the special numbers you are allowed to pass are defined using #define will happily accept an enum with cint sized members. This means that instead of accepting any integer you can now only pass the valid options. Similarly if your C procedure takes (or returns) a pointer to the start of an array you could use a ptr UncheckedArray[T] in Nim. This would allow you to use the value in loops and index lookups. It is generally also helpful to use distinct types to make the entire library more type safe.
Writing all these definitions by hand might work well if you only have a handful of procedures you need. But if you share your code with someone else who might want to use different procedures they have to wrap them themselves. Another issue is that Nim will 100% trust you when you tell it what exists in C. So if you accidentally or purposfully give Nim incorrect information you will end up either with C compiler errors, or even weird runtime behaviour and crashes. For example if a procedure takes a pointer to a specific object, and you accidentially write the wrong object in your definition neither the Nim or C compiler will pick up on this. This means that you end up with weird undefined behaviours which can cause crashes and hard to debug issues. Of course if the library you’re wrapping changes you will have to go through and painstakingly make sure that every piece of your wrapper is still correct. If you don’t you will end up back at square one.
Outsourcing the job
So instead of writing all these definitions by hand it is a good idea to let the computer do these conversions for us. After all computers...