Poison, redzones and shadows: inside KASAN – Bootlin
Skip to content
Even if the presence of Rust is slowly increasing in the Linux kernel code base, the project largely remains written in C, and while this is the de facto language to write low level code, it unfortunately also comes with a significant ability of making mistakes, with the corresponding failures then coming in a wide variety of shapes: undefined behaviors, buffer overflows, segmentation faults… The kernel is not immune to such issues, and so kernel developers need some dedicated tooling to catch those issues early. As many problems have their roots in memory management, one tool that can legitimately sit in any kernel hacker toolbox is the Kernel Address Sanitizer, or KASAN for short. This blog does not aim to teach readers how to use KASAN: the kernel documentation is pretty explicit on this matter. We are rather going to explore KASAN internals, mostly to understand its impact (both at build time and at runtime), but up to some extent, to appreciate the elegant engineering involved in its implementation!
The Kernel Address Sanitizer
KASAN is a runtime checker implemented in the kernel, allowing to catch two kinds of memory manipulation mistakes: out-of-bounds accesses and use-after-free. This tooling has to be explicitly enabled at kernel build time, as it actually depends on two main parts:
some build time instrumentation inserted by the compiler: GCC or LLVM inserts some additional code in front of all memory-accessing instructions, thanks to the Address Sanitizer library.
But that’s not the only component: this additional instrumentation then needs a way to distinguish memory accesses that are legitimate from those that are invalid. To achieve this, KASAN relies on some special memory called the shadow memory, maintained by the kernel to track the whole memory it is able to use.
Enabling KASAN is pretty easy: we mostly need to enable CONFIG_KASAN and to rebuild our kernel. KASAN also comes with a few parameters that we can tune:
the inserted instrumentation can be inlined or outlined, depending on the trade-offs we can afford
some hardware support can be enabled (if our architecture supports it) to reduce KASAN overhead, like software tags and hardware tags
we can tune which kind of memory is tracked (vmalloc memory, stack memory, etc)
With the goal of learning and experimenting with KASAN support, we definitely want a basic setup allowing to run a test kernel in a virtualized environment: as we may end up performing wrong memory accesses, we don’t want to crash our host kernel. There are multiple solutions to bring up such a setup; using virtme-ng from Andrea Righi surely is one of the easiest solutions. Here is a very basic setup workflow to get familiar with KASAN reports:
$ git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git<br>$ cd linux && git checkout linux-7.1.y<br>$ vng -b \<br>--configitem CONFIG_KASAN=y \<br>--configitem CONFIG_KUNIT=y \<br>--configitem CONFIG_KASAN_KUNIT_TEST=m<br>$ vng --user root<br>With this basic list of commands, we are asking virtme-ng to configure and build a kernel with basic KASAN support and the corresponding kunit tests embedded. The final vng command starts this kernel in a QEMU environment emulating the same architecture as for the host used to build the kernel, and with a minimal rootfs. Once booted, we can generate and study KASAN reports by loading the corresponding test kernel module:
# modprobe kasan_test<br># dmesg<br>The kernel log buffer now contains plenty of KASAN reports related to faulty memory accesses, such as this one:
BUG: KASAN: slab-out-of-bounds in kmalloc_oob_right+0x705/0x7d0 [kasan_test]<br>Write of size 1 at addr ffff888005fa5f73 by task kunit_try_catch/249
CPU: 2 UID: 0 PID: 249 Comm: kunit_try_catch Tainted: G N 7.1.2-virtme #6 PREEMPT(lazy)<br>Tainted: [N]=TEST<br>Hardware name: Bochs Bochs, BIOS Bochs 01/01/2011<br>Call Trace:
dump_stack_lvl+0x4d/0x70<br>print_report+0x14b/0x4b0<br>? __pfx__raw_spin_lock_irqsave+0x10/0x10<br>kasan_report+0x119/0x140<br>? kmalloc_oob_right+0x705/0x7d0 [kasan_test]<br>? kmalloc_oob_right+0x705/0x7d0 [kasan_test]<br>kmalloc_oob_right+0x705/0x7d0 [kasan_test]<br>? __pfx_kmalloc_oob_right+0x10/0x10 [kasan_test]<br>? dequeue_task_fair+0x14d/0x6c0<br>? kvm_clock_get_cycles+0x18/0x30<br>? ktime_get_ts64+0x1ce/0x380<br>kunit_try_run_case+0x1a0/0x2c0<br>? __pfx_kunit_try_run_case+0x10/0x10<br>? kthread_affine_node+0x1e6/0x2d0<br>? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10<br>kunit_generic_run_threadfn_adapter+0x85/0xf0<br>kthread+0x329/0x410<br>? recalc_sigpending+0x162/0x210<br>? __pfx_kthread+0x10/0x10<br>ret_from_fork+0x289/0x580<br>? __pfx_ret_from_fork+0x10/0x10<br>? __switch_to+0x585/0xf70<br>? __pfx_kthread+0x10/0x10<br>ret_from_fork_asm+0x1a/0x30
Allocated by task 249:<br>kasan_save_stack+0x2f/0x50<br>kasan_save_track+0x14/0x30<br>__kasan_kmalloc+0x7f/0x90<br>kmalloc_oob_right+0xae/0x7d0...