pgrun.dev
← All posts

Why we run Postgres on local NVMe

· Alex Shapalov

Most managed Postgres runs on network-attached storage. It is a reasonable default: the disk is replicated, it survives the instance, and you can grow it whenever you like. But every page your database reads that is not already in shared buffers costs you a network round trip.

The round trip is the whole story

Local NVMe answers a random read in roughly 100 microseconds. Network storage answers the same read in 500 to 1000 microseconds, and that gap is not something you tune away — it is the speed of the network. On a workload that misses shared buffers, that difference shows up directly in your p99.

You can watch it happen. This is the query we reach for first when a database feels slow:

SELECT queryid,
       calls,
       mean_exec_time,
       shared_blks_read,
       shared_blks_hit
FROM pg_stat_statements
WHERE shared_blks_read > 0
ORDER BY mean_exec_time DESC
LIMIT 10;

A high shared_blks_read next to a slow mean_exec_time means you are paying for storage latency. On network storage the fix is usually "buy more RAM and hope it all fits in cache". On local NVMe the read was never expensive enough to matter.

The trade-off

Local NVMe is physically attached to one machine, which means two things you have to design around:

  • The disk is a fixed size. You cannot grow it on a whim. You pick a size up front and you watch it.
  • The disk does not outlive the machine. If the host dies, that copy is gone.

Neither is a reason to avoid it — they are reasons to take replication and backups seriously. We run a standby with streaming replication and continuous backups to object storage, so a host failure costs you a failover, not your data.

The honest summary: network storage makes durability easy and latency hard. Local NVMe makes latency easy and durability your job. We would rather do that work once, carefully, than hand every one of your queries a network round trip forever.

What this means for you

Nothing, ideally. You get a connection string, your reads come back in microseconds, and the failover and backups are ours to worry about. That is the entire point of managed Postgres.