Making a Shading Language for my Offline Renderer · A Graphics Guy's Note
As a graphics programmer, I don’t usually spend too much time on something that is not strictly related to computer graphics or game engine. However, I did spend four months in my spare time last year building a shading language for my renderer SORT, which I call Tiny Shading Language (TSL).
In the beginning, I didn’t know how it would end up eventually due to the lack of knowledge about how compilers work in general, this is not something graphics programmers touch regularly. However, it does turn out without too much work, this thing can be done by one person in a few months.
In this blog, I will briefly cover some of my thoughts in designing this shading language library. To be more specific, this blog is about how the system is designed and how it works with an offline CPU renderer, instead of the detailed language implementation.
The following is a screenshot of the example tutorial that comes with the TSL library.<br>The patterns on the surface of the two spheres are procedurally generated in TSL.
Motivation
Whenever people heard about my new shading language, the first thing they asked was always, ‘why do you want to make your own shading language since there is already OSL’.
This is a fairly good question, I had the same doubts for more than half a year before putting my hands on it.<br>There are generally several reasons,
By working on my own shading language, I can learn everything from scratch by myself. This is clearly the biggest reason that I chose to work on it. The knowledge gained in the process of making it would be valuable to my career in the future, at least it should have some indirect impact on my work. It should allow me to have a much deeper understanding of how a programming language compiler works.
Having my own code base will allow me to change the library anyway I see fit. This alone offers me a lot more flexibility than OSL since I’m not familiar with their implementation.
Since Apple is currently in the transition from Intel chips to ARM, future MacOSs will be shipped on ARM architecture. Building OSL on ARM will require building all its dependencies on ARM too. If there is any implementation that is x86 specific in any of its dependencies, I will have to find a workaround implementation on ARM too. Without OSL on ARM, there is no way to port my renderer on Apple Silicon. Supporting Apple Silicon is in OSL’s roadmap, but it is unclear when it will be available by the time this blog was written.
OSL heavily uses this library called Open Image IO, which is another open-source project. While OIIO further depends on several other small libraries like, OpenExr, libpng, libtiff and a few others. Some of the basic data structures are only defined in OIIO, so decoupling OSL with OIIO would require quite some work. This was one of the options that I have considered, but after a second thought, it will also make updating OSL in my renderer very hard since it is locally modified.
Having too many dependencies does make OSL a bit ‘heavy’ than expected. My original expectation was just to have one OSL lib as dependencies, it certainly didn’t end up the way I planned. Eventually, there are other libs too and on some platforms, some of the libraries have to be dynamically linked too. Ideally, I could have spent much more time to make sure all of the dependencies are statically compiled, but it is way more sophisticated than it sounds. I wouldn’t want to spend too much time building those libraries from source code. Also, some of the pre-compiled libraries are OS-dependent on Ubuntu, which means that I will have to compile it another time on another version of Ubuntu. At the end of the day, after solving all these problems, features in OIIO are not even used in my renderer at all and I do not have a plan to use them in my renderer since doing it from scratch would be lots of fun, it is a dependency purely because OSL needs it.
There are other reasons that motivate me to implement my own shading language. However, those are mostly related to OSL. I would like to avoid having comments about OSL since at the time this blog was written, I was an employee of Sony (Naughty Dog).
With the above-mentioned reasons, I hope I have made it clear why I decided to implement my own shading language. Of course, I am fully aware that my own implementation will be way less robust than OSL since there is a team behind it and this tech has been involved for more than ten years. So my next question after finalizing my decision was whether this is doable by myself in a few months, I definitely didn’t want to deviate from my trail too much, I’m a graphics programmer anyway, no one will expect me to know too many details in designing a compiler.
Taking Advantage of Existed Work
The image below demonstrates some basic stages of compiling a programming language...