Course outline · 0% complete

0/29 lessons0%

Course overview →

Resource limits: memory, CPU, and exit code 137

lesson 9-3 · ~10 min · 27/29

One container can starve the whole machine

Lesson 1-3 established that containers share the host's kernel, and that has a consequence this unit has to face: by default a container may use all of the machine's memory and CPU. One app with a memory leak can push the host into thrashing and take down every other container on it, including the database. Platforms therefore make you declare limits, and every serious compose file or deploy config carries them.

The flags:

docker run -d --memory 512m --cpus 1.5 myapp:v1

The two resources are enforced differently, and the difference is what you feel in production:

  • CPU over the limit → the container is throttled. The kernel simply schedules it less, so the app gets slow but keeps running
  • Memory over the limit → the container is killed. Memory cannot be politely taken back from a process, so the kernel's OOM killer (out-of-memory killer) terminates it on the spot

Reading the symptoms

An OOM-killed container leaves a specific fingerprint. docker ps -a shows:

STATUS: Exited (137) 2 minutes ago

137 is worth memorizing. Unix reports "killed by signal N" as exit code 128 + N, and the OOM killer uses SIGKILL, signal 9, so 128 + 9 = 137. When a container dies repeatedly with 137 and docker logs ends mid-sentence with no error message, that is a memory limit being hit, not a bug in the final log line.

Live usage per container comes from:

docker stats

which shows a live table of each container's memory and CPU against its limits. In compose, limits go under the service:

  app:
    build: .
    deploy:
      resources:
        limits:
          memory: 512M

With restart: unless-stopped from lesson 9-2, an OOM-killed app restarts automatically, which keeps the service up while you hunt the leak, but a restart loop (up for a minute, killed, up, killed) is itself the classic sign the limit is too low or the leak is fast.

Code exercise · bash

Run this to burn the exit-code arithmetic in. Unix encodes "killed by signal N" as 128 + N, and the OOM killer sends signal 9 (SIGKILL).

Quiz

A container keeps dying every few minutes with Exited (137), and its logs just stop with no error. What is the most likely cause?

Problem

Fill in the arithmetic: an OOM-killed container exits with code 137 because Unix reports kills as 128 plus the signal number, and SIGKILL is signal number ____.