offset_of! slices | arya dradjica
std::mem::offset_of! is a helpful little macro that lets you compute the offset of a particular field of a struct at compile time. I haven’t had much use for it myself, but a friend has been using it for a cool JIT’ed scripting language for Rust, so it’s come up occasionally.
It has a few quirks, though: you can’t use it for unsized fields (e.g. b in Foo { a: u8, b: [u8] }). This isn’t too common, but if you’re doing weird enough things that you need to compute byte offsets of fields, you are probably familiar with (and happy to (ab)use) dynamically sized types. Like all frustrating things, there’s a reason beneath the surface: in Rust, types that implement Sized have a fixed size and alignment, while others do not. With a type like Bar { a: u8, b: dyn Debug }, b could have an arbitrary type, thus an arbitrary alignment, thus an arbitrary offset.
Slices are a bit of an exception here: they do not implement Sized, and their size is not fixed at compile time, but their alignment is . The alignment of [T] is just the alignment of T. So the offset of a slice field in a struct can, in theory, be computed. std::mem::offset_of! supports this under the unstable offset_of_slice feature … but what if you wanted this behavior on stable Rust?
Behold (also on the Rust playground):
macro_rules! fun_offset_of {<br>($t:ty, $field:ident) => {<br>const {<br>// An empty space we can play within.<br>const EMPTY: &[u8] = &[0u8; 65536];<br>let empty: *const [u8] = &raw const *EMPTY;
// This cast only compiles if `$t` is `Sized` or has<br>// slice metadata.<br>let container = empty as *const $t;
// Now, extract the field. This is technically<br>// `unsafe`, but since we're in a `const` block,<br>// undefined behavior will be a compilation error.<br>let field = unsafe { &raw const (*container).$field };
// Compute the offset between `container` and<br>// `field`, in units of bytes. We satisfy the basic<br>// requirements of `offset_from`: both are derived<br>// from the same allocation, `*EMPTY`.<br>let container = container.cast::u8>();<br>let field = field.cast::u8>();<br>unsafe { field.offset_from(container) }<br>};<br>The idea is pretty simple: we can construct a fake instance of the containing type, get a pointer to the field within it, and return the offset of that pointer. All of these operations can be performed at compile-time! Due to the interesting semantics of pointer casts, the macro will only compile with containing types that are Sized or end with slice fields (the exact case we want to support).
Incidentally, I’ve relied on these pointer cast semantics before, in writing a flexible zero-copy (de)serialization mechanism for my day job. They are really handy!<br>Unfortunately, Rust requires (for &raw const (*container).$field) that the fake instance point to valid memory and the offset be in-bounds. We can’t just use a null pointer. It doesn’t care about the contents of that memory, so we do the simple thing and build a big byte array. The offset of the field must lie within this memory, so offsets larger than 64k won’t work, but I think that’s a manageable limitation. Undefined behavior at compile time will (always) be turned into a compilation error!
Before std::mem::offset_of! was stabilized, there were a few crates that provided similar functionality, e.g. offset, memoffset, repr_offset, etc. As far as I can tell, they mostly just use std::mem::offset_of! now, and never supported getting the offset of a slice field. Maybe somebody else has invented a similar hack before and hidden it away in an unrelated crate, but I couldn’t find anything obvious.
I didn’t need this functionality but it was fun to come up with. If this story needs a moral, it’s that you should have fun and commit Rust crimes :3
P.S. If you actually have a use case for this, let me know and I can make a crate; or feel free to copy the macro yourself, consider it licensed MIT or Apache-2.0.