Reconsidering O_CREAT|O_DIRECTORY [LWN.net]
LWN<br>.net<br>News from the source
Content Weekly Edition<br>Archives<br>Search<br>Kernel<br>Security<br>Events calendar<br>Unread comments
LWN FAQ<br>Write for us
Edition Return to the Front page
User:<br>Password: |
Log in /<br>Subscribe /<br>Register
Reconsidering O_CREAT|O_DIRECTORY
Benefits for LWN subscribers
The primary benefit from subscribing to LWN<br>is helping to keep us publishing, but, beyond that, subscribers get<br>immediate access to all site content and access to a number of extra<br>site features. Please sign up today!
By Jonathan Corbet<br>July 30, 2026
Linux provides a system call (mkdir())<br>to create a directory, and a few variants of<br>open() that can open a directory. There is, however, no<br>system call in Linux that can create and open a directory in a single,<br>race-free call. Jori Koolstra has been working on remedying that<br>situation, most recently by repurposing a set of open() flags that currently<br>return an error. There are, however, concerns that show just how hard it<br>can be to create user-space interfaces that do not present traps for<br>application developers.
Creating and opening a directory in a single system call is simpler and<br>more efficient than using two, of course. It also can guard against the<br>possibility that some other process will, between the creation and open<br>steps, replace a directory with something else. Detecting that case<br>is possible on Linux now, but it requires some defensive programming<br>of the type that application developers are not always good at. Thus the<br>desire for a more straightforward way to accomplish that pair of operations.
In March, Koolstra attempted to address this problem with a patch<br>series adding a new system call, mkdirat_fd(), that would<br>return a file descriptor for the newly created directory. That system call<br>was changed to mkdirat2() in a subsequent<br>patch. There were a number of concerns about the implementation, but<br>also about creating a new system call in the first place. Christian<br>Brauner, in particular, thought<br>that this problem was better solved with a modification to how<br>open() handles a couple of existing flags.
Specifically, all of the open() variants support the<br>O_DIRECTORY flag, which is necessary if the application is trying<br>to open a directory rather than a normal file. Also supported is<br>O_CREAT, which instructs open() to create the named file<br>if it does not already exist. It would make sense to interpret the<br>combination of those two flags as a request to create a directory and open<br>it at the same time. Adding this capability to open(), Brauner<br>said, would also make many of the other features of the system call, such<br>as the ability to place restrictions on how the name is resolved, available<br>for free. So, he concluded: "I think here it is pretty clear that<br>O_DIRECTORY|O_CREAT is the right thing to do".
Koolstra duly implemented the new API as an RFC patch<br>set, followed by three more RFC and three non-RFC versions; it was only<br>after the<br>last of those was posted that other developers started to take a<br>serious look at the proposed API, and not all of them were convinced that<br>it was the right approach. Pedro Falcato, in particular, argued that this API<br>would be nearly impossible for applications to use in a portable way, given<br>the number of different ways that O_CREAT|O_DIRECTORY has been<br>interpreted in the past.
That story is, indeed, a bit complicated; LWN covered it in 2023. The ways in which Linux has handled<br>an open() call with that flag combination include:
Older kernels would fail if the named file existed, returning<br>ENOTDIR if it is a regular file and EISDIR if it is<br>a directory. If, instead, nothing existed by the given name,<br>open() would create a regular file, which seems unlikely to<br>be what the caller wanted.<br>As of the 5.7 release in 2020, the kernel started returning an error<br>in the last case, while still creating the file, which seemed<br>even less likely to be what the developer was hoping for.<br>Brauner added a<br>patch to 6.4 causing the kernel to fail with EINVAL for<br>that flag combination in all cases.
Add in the fact that other systems supporting POSIX system calls have their<br>own interpretation of that flag combination, Falcato said, and the result<br>is going to be difficult to use properly:
If you're writing something that wants to be portable, or that<br>intends to standardize on something, you have some 5 different<br>behaviors all across the FOSS UNIX landscape, not considering<br>everything else. It's also something that will, FWIW, probably<br>never be included in POSIX because no one can agree on the<br>semantics here.
TL;DR I don't see, given the reasons above, how users are supposed to use<br>this without it being a total minefield.
He suggested that limiting the change to openat2() might be a<br>better approach.
Brauner, though, dismissed<br>the concern, saying: "I don't think any of this is really an argument<br>worth considering". The behavior of that flag combination has been<br>consistent on Linux for years, he said, so it...