The inliner is yielding benefits for ZJIT

The inliner is yielding benefits for ZJIT | Rails at Scale

We recently enabled a really cool feature in ZJIT that makes it feel like a Real Compiler™: the inliner! We’ll write more about it soon. In this post, we’ll talk about one excellent concrete benefit we are already seeing and how it optimizes blocks in pretty much every Ruby program.

I’ll start off with a refresher on how blocks work in the Ruby interpreter, then show you how ZJIT understands and optimizes that bytecode, and then show you the impact of the inliner.

Ruby: a refresher

In the beginning, there were loops.

arr = [1, 2, 3] i = 0 sum = 0 while i arr.length sum += arr[i] i += 1 end

People used them to navigate and manipulate variable-length structures, like arrays and strings. This was fine.

Then, in the 1970s, a small group of computer scientists at Palo Alto Research Center invented a programming language called Smalltalk. One of the core features of Smalltalk was that everything was an object and computation was done by sending messages to objects.

This meant that iteration wouldn’t do at all. Instead, we would have to send the do: message to the array object and pass it a block object.

#(1 2 3) do: [:a | sum := sum + a].

Then, in the 1990s, Matz, inspired by Smalltalk and Perl, created Ruby. We still have “normal” loops but we also have a very Smalltalk-y way of doing it, too:

arr = [1, 2, 3] sum = 0 arr.each do |a| sum += a end

When this program gets compiled to Ruby bytecode, it ends up looking like a mostly normal method call to each except that we pass a special kind of argument to it: a block argument.

To see how this works inside CRuby, we’re going to look at a listing of YARV bytecode—CRuby bytecode. For more on YARV, I recommend Kevin Newton’s excellent Advent of YARV.

Ignore most of the bytecode dump below except for the instruction at 0009, the send instruction. We are send-ing (see? a message!) each with the block argument block in (passed a different way than “normal” arguments, hence the argc:0).

$ ruby --dump=insns /tmp/arrayeach.rb == disasm: #@/tmp/arrayeach.rb:1 (1,0)-(5,3)> local table (...) [ 2] arr@0 [ 1] sum@1 0000 duparray [1, 2, 3] ( 1)[Li] 0002 setlocal_WC_0 arr@0 0004 putobject_INT2FIX_0_ ( 2)[Li] 0005 setlocal_WC_0 sum@1 0007 getlocal_WC_0 arr@0 ( 3)[Li] 0009 send , block in 0012 leave ...continued below...

This block in from the bytecode listing above is the generated name of the instruction sequence (bytecode) corresponding to the block we passed to each. Its code, shown below, is the next thing in the bytecode dump. You can see the usage of local variables sum and a via getlocal and setlocal variants and also opt_plus, which represents addition (+). Again, the details are not terribly important:

...continued from above... == disasm: #@/tmp/arrayeach.rb:3 (3,9)-(5,3)> local table (...) [ 1] a@0 0000 getlocal_WC_1 sum@1 ( 4)[LiBc] 0002 getlocal_WC_0 a@0 0004 opt_plus [CcCr] 0006 dup 0007 setlocal_WC_1 sum@1 0009 leave ( 5)[Br]

To make this work, Array has an each method. This each method takes in its optional block argument and calls it once for every element in the array.

As with most of the core data structures, Array’s methods tend to be written in C and each is no different. Here is its nice and short definition with comments added by me:

VALUE rb_ary_each(VALUE ary) long i; // Some debug-mode-only checks ary_verify(ary); // If we didn't pass a block in, return an enumerator object RETURN_SIZED_ENUMERATOR(ary, 0, 0, ary_enum_length); // We passed a block in, so call it for every element for (i=0; iRARRAY_LEN(ary); i++) { rb_yield(RARRAY_AREF(ary, i)); return ary;

You can’t really see the block parameter to the C code because it’s passed in a special location on the Ruby VM’s own stack called the block handler. All you need to know is that rb_yield knows where to find that and how to call it.

There’s just one more thing, which is… hang on, weren’t we building a JIT to optimize Ruby code? How are we going to optimize this C code?

JITs: a refresher

Rewriting code from C to Ruby means that the JIT compiler gets a chance to introspect the run-time behavior and code. This means that, over time, as the compiler and the runtime system grow together, more and more code might get rewritten in Ruby. This started happening a couple of years ago with YJIT.

YJIT precipitated some interesting changes to Ruby VM internals. For example, in 2022, Array#each being written in C started to hurt: it was an opaque blob that the JIT couldn’t reason about. So Kokubun submitted a PR to rewrite it in Ruby. After some back and forth, in 2024, Kokubun landed a different PR that everyone was happy with.

class Array def each # :nodoc: Primitive.attr! :inline_block, :c_trace, :without_interrupts

unless defined?(yield) return Primitive.cexpr! 'SIZED_ENUMERATOR(self, 0, 0, ary_enum_length)' end i = 0 until Primitive.rb_jit_ary_at_end(i) yield Primitive.rb_jit_ary_at(i) i =...

The inliner is yielding benefits for ZJIT

Related Articles

(no title)

Scientists reverse brain aging, with a nasal spray

AI has torched the market for junior programmers

Al Vigier: Canada's AI strategy shouldn't include secret Palantir bills

The Anatomy of an AI-Native Org