Rambles around computer science
Rambles around computer science
Diverting trains of thought, wasting precious time
12 06 2026<br>--><br>Fri, 12 Jun 2026
System call stack alignment
[Quick new post... it's June and I'm emerging from my academic season,<br>so hopefully lots of new content will appear here over the rest of the year.]
Recently, in one of my exam-season “procrastination projects” (about which more anon),<br>I found myself with a simple enough question.<br>What stack pointer alignment is required when making a system call on Linux on x86-64?
You'd think this'd be documented somewhere by the kernel.<br>You'd think the kernel system call ABI in general would be documented.<br>I haven't found it yet... it's nowhere in the Documentation directory that I can find.<br>If you know better, please do send answers, on a postcard or otherwise.
I have found syscall_wrapper.h which<br>contains the basic information in the following rather obscure form.
/* Mapping of registers to parameters for syscalls on x86-64 and x32 */<br>#define SC_X86_64_REGS_TO_ARGS(x, ...) \<br>__MAP(x,__SC_ARGS \<br>,,regs->di,,regs->si,,regs->dx \<br>,,regs->r10,,regs->r8,,regs->r9) \
However, it doesn't cover stack alignment (nor clobber information).
Elsewhere, the same topic is covered in the System V psABI document<br>which suggests the stack alignment required<br>is 16 bytes, or rather, 16 bytes at the call site and<br>therefore 8-modulo-16 on entry.<br>(The return address takes the stack 8 bytes out of 16-byte alignment.<br>If the prologue pushes %rbp, 16-byte alignment<br>is quickly restored.)<br>But it only suggests this by omission... it documents the requirement<br>for ordinary calls and then has a section describing the kernel's ABI differences<br>which doesn't say anything about stack alignment being different.<br>However, empirically, from running some stuff under catch syscall in gdb,<br>I see only 8-byte alignment reliably: mostly it is 8-modulo-16 on entry to the syscall,<br>but it is fairly common to see exceptions to this.
I haven't found any other documentation. It probably exists somewhere. If you know,<br>let me know!
Just for fun, here's a related question, also not documented anywhere<br>that I'm aware of: what alignment does the Linux kernel give to fresh stack pointer<br>created on execve?<br>Why, 16 bytes of course, i.e. exactly wrong relative to the usual<br>8-modulo-16 convention.<br>This means if you want to hand-craft the ultimate entry point of the<br>program (_start in a statically linked binary or _dl_start in the dynamic linker)<br>you have to work around this<br>at the risk of running into<br>surprise errors later when your compiler has seen fit to touch the<br>stack using a strict-alignment instruction like movaps<br>based on the 8-modulo-16 assumption.<br>I know this has burned others, not just me,<br>who've tried to replace a program's entry<br>point, e.g. using the -e option to the linker.
Date: Tue Apr 20 17:40:45 2021 +0100
Fix handling of 6-argument syscalls on x86... the convention is different for sysenter than for int<br>0x80
-->
There is a weird quasi-tradition in Unix of<br>only documenting the roughly C-abstraction-level interfaces.<br>For example, POSIX says nothing about linking... at all!<br>When it comes to programming interfaces, it just documents the C APIs, not what<br>is happening at the object code level.<br>So although just about any Unix will have an ld command,<br>there's no standard about how do invoke it, or even what it does.
Meanwhile the Linux man pages, even in the system call section,<br>focus on the glibc wrappers.<br>They do, to their credit (and I really rate Michael Kerrisk's stewardship of this<br>information) also note when the kernel interface diverges.<br>But if you look at, say, syscalls(2)<br>or intro(2) (the introductory page of section 2 of the manual)<br>they don't attempt to go down to the assembly-language level.<br>Largely this is for portability of course, but as a side effect,<br>it leaves the question of this post unanswered.
(I don't know how far back this tradition goes... quite possibly<br>the entire Unix tradition since the C rewrite of c.1973 has taken a view of<br>emphasising the high-level language and de-emphasising the assembly level.<br>But sometimes, you really do want to know this stuff!)
Here's a final note on this topic<br>Until Linux 2.6.18, the kernel's supplied user header files defined a _syscall<br>family of macros that would generate a wrapper for any system call whose name, number and signature<br>were described by the arguments to the macro. Here is the 5-argument version,<br>from Debian hamm's /usr/include/asm/unistd.h (32-bit Intel only, naturally).
#define _syscall5(type,name,type1,arg1,type2,arg2,type3,arg3,type4,arg4, \<br>type5,arg5) \<br>type name (type1 arg1,type2 arg2,type3 arg3,type4 arstg4,type5 arg5) \<br>{ \<br>long __res; \<br>__asm__ volatile ("int $0x80" \<br>: "=a" (__res) \<br>: "0" (__NR_##name),"b" ((long)(arg1)),"c" ((long)(arg2)), \<br>"d" ((long)(arg3)),"S" ((long)(arg4)),"D" ((long)(arg5))); \<br>if (__res>=0) \<br>return (type) __res; \<br>errno=-__res; \<br>return -1; \
The current...