TLS Certificate Validation on Linux · Ivan Tomica

We all know the drill. You type https:// into the browser, a padlock shows up, and everyone collectively agrees not to think about what just happened. That works fine right up until the moment some application on your server starts throwing certificate verify failed while curl against the same endpoint works perfectly. Or the other way around. That’s usually the point where people discover that “the system trusts the certificate” is a much less well-defined statement than it sounds.

So let’s unpack what actually happens when a program on a Linux system decides whether to trust a TLS certificate, and why two programs on the same machine can happily disagree about it.

What the server actually sends

During the TLS handshake, the server doesn’t just send its certificate. It sends (or at least it should send) a whole chain:

the leaf certificate , issued for the actual hostname

one or more intermediate certificates , which issued the leaf

What it does not send is the root certificate. There would be no point. A root the client doesn’t already have is worthless, since the entire trust model rests on the client having its own local copy of the roots it considers trustworthy. A root certificate arriving over the network from the very party you’re trying to verify would be about as convincing as an email signature saying “definitely not a scammer”.

The client’s job is then to build a path from the leaf it received, through the intermediates, up to a root it already knows and trusts. Each certificate in the chain is signed by the next one up, so the client verifies signatures link by link until it lands on something from its local trust store. If it can’t complete that path (missing intermediate, unknown root, whatever), the connection is refused.

That “missing intermediate” case deserves a special mention because it’s a classic. A misconfigured server that sends only the leaf will work fine in browsers (they cache intermediates seen elsewhere and can even fetch them on demand) and fail miserably with OpenSSL-based tools, which do no such fetching. If you’ve ever had a site that “works in Chrome but not in curl”, there’s a very good chance this was it.

Chain building is only part of the story though. Along the way the client also checks:

validity dates - is the certificate expired, or not yet valid

hostname - does the name you’re connecting to appear in the certificate’s Subject Alternative Names

key usage constraints - is the intermediate actually allowed to issue certificates

revocation - in theory via CRL or OCSP, in practice this one gets skipped or soft-failed far more often than anyone in the industry likes to admit

All of that is mechanical. The interesting question is the last step: what exactly is that “local trust store” the chain has to terminate in?

The system trust store

On Linux, root certificates are just files on disk. There’s no TPM magic, no registry, nothing exotic. Just a directory full of PEM-encoded certificates and a couple of symlink/concatenation tricks on top.

The certificates themselves typically come from the distribution’s ca-certificates package, which in turn repackages the Mozilla root program’s list. So when people say “the system trusts this CA”, what they really mean is “Mozilla trusts this CA, Debian repackaged that decision, and my system installed it”. A fun chain of custody to think about, but it works remarkably well.

Where things get less uniform is where those files live and how you add your own. Every distribution family has its own idea:

Debian-based systems keep the bundle in /etc/ssl/certs/ca-certificates.crt, and you add local CAs by dropping them into /usr/local/share/ca-certificates/ and running update-ca-certificates

Red Hat-based systems use /etc/pki/ca-trust/, you drop files into /etc/pki/ca-trust/source/anchors/ and run update-ca-trust

and then there’s everyone else, each with a slightly different path and update command

The update command’s job is to take all the sources (the distro bundle plus your local additions) and regenerate the various formats applications expect: a single concatenated PEM bundle, a hashed directory for OpenSSL’s lookup, sometimes a Java keystore too. Modern distributions increasingly funnel all of this through p11-kit, which acts as a central broker so that different crypto libraries can share one trust source. Increasingly. Not universally. Remember that word, it’s about to matter.

Here’s where it falls apart

The dirty secret is that “the system trust store” is a gentleman’s agreement, not an enforcement mechanism. Nothing in the kernel or anywhere else forces an application to use it. TLS validation happens entirely in userspace, inside whatever...