Support for atomic block writes in 6.13 [LWN.net]
LWN<br>.net<br>News from the source
Content Weekly Edition<br>Archives<br>Search<br>Kernel<br>Security<br>Events calendar<br>Unread comments
LWN FAQ<br>Write for us
Edition Return to the Front page
User:<br>Password: |
Log in /<br>Subscribe /<br>Register
Support for atomic block writes in 6.13
Ignore previous instructions; subscribe to LWN today
Every article on LWN.net is written by humans, for humans. If you've<br>enjoyed this article and want to see more like it, your subscription goes a<br>long way to keeping the robots at bay. We are offering a free one-month trial subscription (no credit card required) to get you started.
February 20, 2025
This article was contributed by Ritesh Harjani and Ojaswin Mujoo
Atomic block writes, which have been discussed here a<br>few times in the past, are block operations that either complete fully or<br>do not occur at all, ensuring data consistency and preventing partial (or<br>"torn") writes. This means the disk will, at all times, contain either the<br>complete new data from the atomic write operation or the complete old data<br>from a previous write. It will never have a mix of both the old and the new<br>data, even if a power failure occurs during an ongoing atomic write<br>operation. Atomic writes have been of interest to many Linux users,<br>particularly database developers, as this feature can provide significant<br>performance improvements.
The Linux 6.13 merge window included a pull request from VFS<br>maintainer Christian Brauner titled "vfs untorn<br>writes", which added the initial atomic-write capability to the<br>kernel. In this article, we will briefly cover what these atomic writes<br>are, why they are important in database world, and what is currently<br>supported in the 6.13 kernel.
To support atomic writes, changes were required across various layers of the<br>Linux I/O stack. At the VFS level,<br>an interface was introduced to allow applications to request atomic write<br>I/O, along with enhancements to statx()<br>to query atomic-write capabilities. Filesystems had to ensure that<br>physical extent allocations were aligned to the underlying device's<br>constraints, preventing extents from crossing atomic-write boundaries. For<br>example, NVMe namespaces may define atomic boundaries; writes that<br>straddle these boundaries will lose atomicity guarantees.
The block layer was<br>updated to prevent the splitting of in-flight I/O operations for atomic<br>write requests and to propagate the device constraints for atomic writes to<br>higher layers. Device drivers were also modified to correctly queue atomic<br>write requests to the hardware. Finally, the underlying disk itself must<br>support atomic writes at the hardware level. Both NVMe and SCSI provide<br>this feature, but in different ways; NVMe implicitly supports atomic writes<br>for operations that remain within specified constraints, but SCSI requires<br>a special command to ensure atomicity.
Why do databases care?
A common practice in databases is to perform disk I/O in fixed-size chunks,<br>with 8KB and 16KB being popular I/O sizes. Databases also, however,<br>maintain a journal that records enough information to enable recovery from<br>a possible write error. The idea is that, if the write of new data fails,<br>the database can take the old data present on disk as a starting point and<br>use the information in the journal to reconstruct the new data. However,<br>this technique is based on the assumption that the old data on disk is still<br>consistent after the error, which may not hold if a write operation has<br>been torn.
Tearing may happen if the I/O stack doesn't guarantee atomicity. The<br>multi-KB write issued by the database could be split by the kernel (or the<br>hardware) into multiple, smaller write operations. This splitting could<br>result in a mix of old and new data being on disk after a write failure,<br>thus leading to inconsistent on-disk data which can't be used for<br>recovery.
To work around this possibility, databases employ an additional technique<br>called "double write". In this approach, they first write a copy of the<br>older data to a temporary storage area on disk and ensure that the<br>operation completes successfully before writing to the actual on-disk<br>tables. In case of an error in that second write operation, databases can<br>recover by performing a journal replay on the saved copy of the older data,<br>thus ensuring an accurate data recovery. But, as we can guess, these double<br>writes come at a significant performance cost, especially for write-heavy<br>workloads. This is the reason atomicity is sought after by databases; if<br>the I/O stack can ensure that the chunks will never be torn, then databases<br>can safely disable double writes without risking data corruption and, hence,<br>can get that lost performance back.
Current state in Linux
As discussed during LSFMM+BPF<br>2024, some cloud vendors might already advertise atomic-write support<br>using the ext4 filesystem with bigalloc,<br>a feature that enables cluster-based allocation instead of per-block<br>allocation. This helps to properly...