. -->

Why strict memory overcommit matters for Postgres | ClickHouse<br>Skip to content

Open searchOpen region selectorEnglish<br>Japanese<br>Korean<br>Chinese<br>French<br>Spanish<br>Portuguese<br>Arabic

48.9kSign inGet Started

->Scroll to top<br>BackBlog<br>Engineering<br>Copy pageCopied!More actionsView as Markdown Open this page in Markdown<br>Open in ChatGPT Ask questions about this page<br>Open in Claude Ask questions about this page<br>Open in v0 Ask questions about this page

Why strict memory overcommit matters for Postgres

Kaushik Iska<br>Jul 28, 2026 · 9 minutes read

What is memory overcommit, and why does Postgres like for it to be strict? When a process calls malloc, Linux grants address space and attaches physical memory only when each page is first touched. The kernel's term for the granted space is a commitment, and overcommit is its policy of extending more commitments than the machine can back, since most allocations are never fully used. When touched pages exceed physical memory, the kernel invokes the OOM killer, which SIGKILLs a process to reclaim memory.

Postgres is unusually exposed to that policy. Every backend shares one memory segment holding shared buffers, WAL buffers, and lock tables, and a backend that dies by SIGKILL can leave that segment in an unknown state. The postmaster's only safe response is to assume corruption: it terminates every backend, drops every connection, and replays the WAL through crash recovery.

A single backend killed by the OOM killer mid-query therefore restarts the entire instance.

In ClickHouse Managed Postgres every server runs strict overcommit, vm.overcommit_memory = 2. The kernel tracks committed memory and refuses allocations past a commit limit, so the failure surfaces as ENOMEM from malloc before physical memory is exhausted. Postgres handles ENOMEM like any other error: the query fails with out of memory, the transaction rolls back, and the remaining connections keep working. This is the setting the Postgres documentation recommends in its chapter on kernel resources.

Setting the commit limit #

Strict overcommit is only useful if the limit is in the right place. Too high and the kernel still hands out more than it can back; too low and queries fail while memory sits free. Every server gets an explicit limit in kilobytes rather than the kernel's default ratio:

1# /etc/sysctl.d/99-overcommit.conf<br>2vm.overcommit_memory=2<br>3vm.overcommit_kbytes=Copy command

The limit is 80% of the memory that is available for ordinary allocations, plus 2GB. Two details set the size of that first term.

The first is that a quarter of the machine is already reserved as huge pages for shared_buffers, and huge pages do not count toward the commit limit. Postgres maps that segment once at startup, outside the accounting the commit limit governs, so the limit is computed against the remaining 75% of RAM rather than the whole machine.

The second is the 80%: what remains has to cover the kernel itself, page tables, the page cache, and every per-backend allocation Postgres makes at runtime, so the limit deliberately stops short of the memory that is actually there. The 2GB on top is headroom for the sidecar processes that share the box with Postgres, the backup agent and the metrics exporters.

The result is a commit limit of roughly 60% of total RAM plus 2GB, and a kernel that starts returning ENOMEM while several gigabytes are still physically free. That gap is the point. The error arrives while there is still memory available to handle the error.

Get started with ClickHouse Managed Postgres today<br>Interested in seeing how ClickHouse Managed Postgres works on your data? Get started with ClickHouse Cloud in minutes and receive $300 in free credits.<br>Sign up<br>Seeing it on real hardware #

We put both policies on a single EC2 box: an m7i.2xlarge (8 vCPU, 30.8GiB), Postgres 16 with a quarter of memory reserved as 2MB huge pages and shared_buffers = 6864MB mapped onto that pool, and a 3GB pgbench dataset. Of the 30.8GiB, the huge page pool takes 7.7GiB, which leaves 23.1GiB for everything else. The production formula puts the strict commit limit at 20.5GiB, about 2.6GiB below what the machine can physically hand out.

The workload is the same in both arms. Twenty idle sessions stay connected as bystanders, a client opens a fresh connection every 0.3 seconds to see whether the server is reachable, and then sessions arrive one at a time, each building about 2GB of in-memory arrays and holding it. Nothing about the query is exotic: it is an ordinary aggregate over generate_series that happens to be large, the shape of a query that sorts or aggregates more than anyone expected. Sessions keep arriving until the run ends itself. The only variable between the two arms is vm.overcommit_memory.

Under the kernel's default policy, committed memory climbs straight through the limit the kernel itself reports. CommitLimit on this box reads 11.6GiB, and committed memory passed it at the sixth session and kept going to...