Postgresql Shared Memory Segment Docker Compose

Legacy context

Legacy context. This archive preserves technical notes and operational references from a software development environment. The material reflects an educational focus on open-source tools, system architecture, and engineering practices—particularly around data synchronization, containerized services, and interactive application design.

Key point 1. The notes are offered as a neutral, independent reference for those studying database administration, deployment workflows, or related infrastructure topics. No current organization, product, or service is represented here. The content is historical and may not reflect modern best practices or software versions.

Key point 2. Readers are encouraged to verify all technical details against current official documentation before applying them in production. The archive does not provide certification, support, or legal or medical guidance. It exists solely to preserve and share past engineering notes for educational purposes.

PostgreSQL Shared Memory Segments in Docker Compose: A Practical Guide. When running PostgreSQL inside Docker Compose, one of the most common runtime failures is the `could not resize shared memory segment` error, often accompanied by `No space left on device` or `Invalid argument`. This is not a disk-space issue—it is a kernel-level shared memory (`/dev/shm`) limitation. This guide explains how to diagnose, configure, and avoid these failures, with concrete decision criteria and a compact reference.

PostgreSQL uses two types of shared memory:

• Dynamic shared memory (DSM): Used for parallel queries, parallel index builds, and some replication features. This is allocated in `/dev/shm` (POSIX shared memory) or via anonymous shared memory.
• Main shared memory: The `shared_buffers` parameter, which is allocated at server start. In modern PostgreSQL (9.3+), this is typically allocated using `mmap` with `MAP_SHARED` on `/dev/shm` when available.

Key point 5. Inside a Docker container, `/dev/shm` defaults to 64 MB (set by Docker’s `--shm-size` flag). PostgreSQL’s default `shared_buffers` is 128 MB, and parallel workers can request additional DSM segments. When the combined demand exceeds 64 MB, the kernel refuses to create new segments, and PostgreSQL crashes or fails to start.

Key point 6. Key diagnostic command (inside the container):

```bash

df -h /dev/shm

```

If the output shows `64M` and your `shared_buffers` is larger, that is the root cause.

1. Docker Compose Configuration: The `shm_size` Directive. Docker Compose (v2.x and v3.x) supports a top-level `shm_size` key for each service. This sets the size of `/dev/shm` for that container.

Key point 8. Minimal working example (`docker-compose.yml`):

```yaml

version: "3.9"

services:

db:

image: postgres:16

environment:

POSTGRES_PASSWORD: example

shm_size: '1gb'

volumes:

• pgdata:/var/lib/postgresql/data

volumes:

pgdata:

```.

Decision criteria for sizing `shm_size`:

• Base value: Start with `shared_buffers` + 25% headroom. If `shared_buffers=128MB`, set `shm_size=256mb`.
• Parallel queries: If you set `max_parallel_workers_per_gather` > 0, each worker can use up to `min_parallel_table_scan_size` worth of DSM. A safe rule: add `max_parallel_workers * 32MB` to your base.
• Heavy analytical workloads: For ETL or reporting with many concurrent sessions, use `2gb` or more. PostgreSQL does not pre-allocate this; it is only used on demand.
• Memory-constrained hosts: If the host has limited RAM, prefer reducing `shared_buffers` (e.g., to 256MB) rather than increasing `shm_size` beyond physical memory.

Key point 10

Alternative: `tmpfs` mount

You can also mount a custom tmpfs at `/dev/shm`:

```yaml

services:

db:

image: postgres:16

tmpfs:

• /dev/shm:rw,size=1g

```

This is functionally equivalent to `shm_size` but gives you explicit mount options. Use `shm_size` for simplicity; use `tmpfs` if you need `noexec` or `nosuid` flags.

1. Common Mistakes and Failure Modes

Mistake 1: Setting `shm_size` on the wrong level.

`shm_size` must be under the service definition, not under `volumes` or `environment`. A common typo is placing it under `deploy` (which is for Swarm) or under `build`. Compose will silently ignore unknown keys, so the container still gets 64 MB.

Key point 12

Mistake 2: Confusing `shm_size` with `mem_limit`.

`mem_limit` caps total container memory (RAM + swap). `shm_size` only affects `/dev/shm`. If you set `mem_limit: 512m` and `shm_size: 1g`, the kernel may still fail because the tmpfs counts against the container’s memory cgroup. Always ensure `shm_size` ≤ `mem_limit` if you set both.

Key point 13

Mistake 3: Using `postgres` image with `--ipc=host`.

Some guides suggest `ipc: host` to share the host’s `/dev/shm`. This is dangerous: it removes isolation and can cause the container to see the host’s shared memory, leading to unpredictable behavior. Avoid this unless you fully understand the security implications.

Key point 14

Mistake 4: Ignoring `dynamic_shared_memory_type`.

PostgreSQL has a `dynamic_shared_memory_type` parameter (default `posix`). If you set it to `mmap`, it uses files in `$PGDATA` instead of `/dev/shm`. This avoids the `shm_size` issue but can slow down parallel queries due to disk I/O. Only change this if you cannot increase `shm_size` (e.g., on a managed host).

Key point 15

Mistake 5: Not restarting after changing `shm_size`.

Changing `shm_size` in Compose requires `docker compose down` and `docker compose up -d`—a simple `restart` does not recreate the container’s mount namespace. Use `down` to remove the container, then `up`.

1. Verifying and Tuning Inside the Container

After starting the container, verify the setting:

```bash

docker compose exec db df -h /dev/shm

```.

Key point 17

To see actual DSM usage:

```sql

SELECT name, size FROM pg_shmem_allocations ORDER BY size DESC;

```

This shows the main shared memory segments. For dynamic segments, use:

```sql

SELECT * FROM pg_stat_database WHERE datname = 'postgres';

```

(Note: DSM segments are transient; they appear only during parallel operations.)