Performance impact of Alignment | Emanuel’s Blog

An important topic in SIMD vectorization is alignment of memory accesses, i.e. loads and stores.<br>There is an impact on any form of vectorization, including auto-vectorization and explicit vectorization (e.g. with the Vector API).

Definition: Alignment

Given some address of a memory access, and some alignment_size, we say that the address is alignment_size-aligned if address % alignment_size = 0.<br>If a alignment_size is not explicitly stated, we usually either refer to an access being aligned to the size of the access. Examples:

4-byte int access with 4-byte alignment

8-byte Object pointers with an 8-byte alignment

vector access with 16 int elements, size of 64 bytes, with a 64-byte alignment

We can also talk about cacheline-alignment, usually referring to an alignment_size = 64 byte alignment.<br>For vectors, it is important to note that the alignment is based on the total size of the vector, and not the size of the elements.<br>We can also talk about the alignment of memory, or specificall a memory address.<br>For example, all Java Objects (the pointers to the beginning of the Object) are 8-byte aligned.<br>The elements of an array are all aligned by the size of the element. This means if we access the element in an array,<br>this scalar (non-vectorized) access is always aligned. However, if we load a vector of elements from an array,<br>this vector access is not guaranteed to be aligned: int elements are only 4-byte aligned, and a vector<br>of 16 elements would require a 64-byte alignment to be aligned.<br>Hence, when moving from scalar to vector code, we have to do additional work if we want to acheive alignment of our vector accesses.<br>Note that the alignment size is a power of 2, because all relevant sizes are powers of 2.

Some architectures allow only aligned access

Some architectures, and especially older ones, only allow aligned access. If the address is unaligned on such a platform, then one either gets an error<br>(e.g. SIGBUS) or a wrong execution (e.g. address is truncated, and the access happens at an unexpected location).<br>Some of these platforms have specific 4-byte or 8-byte alignment requirements, others only require that an access be aligned by the access size.

This makes vectorization substantially more difficult. For program correctness, the compiler must be able to prove that the address<br>is aligned, otherwise it cannot use vector instructions. Personally, I’ve had to spend quite a bit of time on re-thinking and<br>proving our implementation of alignment for platforms with strict alignment requirements<br>(see Bug-Fix PR).

Most modern CPUs have fast unaligned access

That said: most modern CPUs allow unaligned access, and the performance is often as fast or only a little slower than aligned access.<br>Every platform and miro-architecture behaves a little differently. But often unaligned accesses that do not cross cacheline boundaries<br>are as fast as aligned accesses.

Problem: crossing cacheline boundary

When a memory access crosses a cacheline boundary, it is split into two accesses, one per cacheline.<br>The split happens in the CPU’s memory subsystem (details may vary on different architectures):<br>First, the load/store is decoded from the machine code, and the address width is determined.<br>The memory unit receives (or calculates) the address, and checks if the address is aligned for the access size.<br>If it is not aligned, it is checked if memory boundaries (e.g. cacheline boundaries) are crossed,<br>in which case the access is split into two. If it is a store, the store value is split accordingly.<br>If it is a load, the fragments are combined.

Split accesses means that one now has more memory accesses going through the memory unit of the CPU, and that can slow down execution.<br>The amount by which this affects performance depends on a few factors.<br>Simply put, it depends on the fraction of memory accesses that are split, and if memory accesses are the bottleneck.<br>If there are only very few memory accesses, and we are heavily compute-bound, then splitting those few memory accesses probably has very little impact on performance.<br>However, if the memory units are already the bottleneck, and we split all memory accesses we could in theory get only half the execution speed.<br>Most of the time, reality lies somewhere in between.

In my experience, cacheline boundaries are the most impactful for alignment. However, there are also platforms with additional memory boundaries.<br>For example, the aarch64 Neoverse N1 optimization guide talks about performance penalties not just for loads that cross cacheline<br>boundaries (64 byte) but also stores that cross 16 byte boundaries (Section 4.5 Load/Store alignment).

Visualizing the Performance Impact of (un)aligned Loads and Stores

To visualize the performance impact of alignment, I wrote some benchmarks.<br>Below you can see a simplified version of it:

for (int i = 0; i limit; i += SPECIES.length()) {<br>var v = IntVector.fromArray(SPECIES, arr1, i +...