Docker changed how we ship software. Before containers, "it works on my machine" was the most dreaded phrase in operations. Environment drift between development, staging, and production caused countless incidents. Docker solved this by packaging applications with their entire runtime environment into portable, reproducible units.
But Docker is more than just a convenience tool. For DevOps engineers, it is foundational infrastructure. It underpins CI/CD pipelines, microservice architectures, and every major orchestration platform. Whether you are containerizing a legacy monolith or running hundreds of microservices, a deep understanding of Docker internals, best practices, and production patterns will save you from 3 AM pages.
This guide covers everything from container fundamentals through production deployment -- with real commands and configurations you can use today.
Before diving into Docker commands, it is worth understanding what containers actually are at the OS level.
An image is a read-only template -- a filesystem snapshot plus metadata (environment variables, default command, exposed ports). A container is a running instance of an image. You can run many containers from the same image, each with its own writable layer on top.
Think of it like classes and objects in programming: the image is the class definition, and containers are instances.
Image (read-only layers)
├── Layer 4: COPY app.js /app/ (your code)
├── Layer 3: RUN npm install (dependencies)
├── Layer 2: RUN apt-get install nodejs (runtime)
└── Layer 1: Ubuntu 22.04 base (OS)
Container (image + writable layer)
├── Writable Layer (logs, temp files, runtime state)
└── Image Layers (shared, read-only)
Containers are not virtual machines. They share the host kernel but are isolated using two Linux kernel features:
Namespaces provide isolation:
pid -- process isolation (container sees only its own processes)net -- network stack isolation (own interfaces, IP addresses, routing tables)mnt -- filesystem isolation (own mount points)uts -- hostname isolationipc -- inter-process communication isolationuser -- user/group ID mapping (rootless containers)
Cgroups (Control Groups) limit resources:
- CPU shares and pinning
- Memory limits (with OOM killer integration)
- I/O bandwidth
- Number of processes
This is why containers start in milliseconds (no OS to boot) and use far less memory than VMs (no duplicate kernel).
The Open Container Initiative (OCI) defines industry standards for container formats and runtimes. This means images built with Docker work in Podman, containerd, CRI-O, and any OCI-compliant runtime. The two key specs are:
- Image Spec: How container images are structured (layers, manifests, config)
- Runtime Spec: How containers are created and run (lifecycle, configuration)
Understanding Docker's components helps you debug issues and make better architectural decisions.
docker CLI (client)
|
v (REST API over Unix socket)
dockerd (Docker daemon)
|
v
containerd (container runtime)
|
v
runc (OCI runtime -- actually creates containers)
|
v
Linux kernel (namespaces, cgroups)
- Docker CLI (
docker): The command-line client you interact with. Sends API calls to the daemon.
- Docker Daemon (
dockerd): Manages images, containers, networks, and volumes. Listens on /var/run/docker.sock.
- containerd: Lower-level container runtime that manages the container lifecycle (pull, create, start, stop).
- runc: The actual OCI runtime binary that creates Linux containers using namespaces and cgroups.
- Registry: Stores and distributes images (Docker Hub, GitHub Container Registry, private registries).
When you run docker run nginx, the CLI talks to dockerd, which tells containerd to pull the image and create a container, which delegates to runc to set up namespaces and start the process.
A well-built Docker image is small, secure, and cache-friendly. A poorly built one is a 2 GB blob that rebuilds from scratch on every code change.
Multi-stage builds are the single most impactful optimization. They separate the build environment from the runtime environment:
# Stage 1: Build
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build
# Stage 2: Run
FROM node:20-alpine AS runner
WORKDIR /app
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./
USER appuser
EXPOSE 3000
CMD ["node", "dist/index.js"]
The builder stage has all build tools (compilers, dev dependencies). The runner stage only has what is needed at runtime. This can reduce image sizes by 80% or more.
Tip: Use the Dockerfile Generator to create production-ready multi-stage Dockerfiles for Node.js, Python, Go, Java, and static sites.
Docker caches each layer. If a layer's input has not changed, Docker reuses the cached version. This means ordering matters:
# Bad: code changes invalidate npm install cache
COPY . .
RUN npm install
# Good: dependencies cached separately from code
COPY package*.json ./
RUN npm install
COPY . .
The rule: copy things that change less frequently first. Dependencies change less often than source code, so copy and install them before copying application code.
Your base image choice has massive implications for image size and security:
| Base Image | Size | Packages | Use Case |
|---|
ubuntu:22.04 | ~77 MB | Full OS | Legacy apps needing system libraries |
debian:bookworm-slim | ~74 MB | Minimal | When you need apt but want smaller |
alpine:3.19 | ~7 MB | musl + apk | Most production workloads |
distroless | ~2 MB | Nothing | Static binaries (Go, Rust) |
scratch | 0 MB | Nothing | Absolute minimum (static binaries only) |
Alpine is the sweet spot for most use cases. It is tiny, has a package manager, and uses musl libc (which occasionally causes compatibility issues with software expecting glibc -- watch out for Python packages with native extensions).
Always create a .dockerignore file. Without it, COPY . . sends everything to the Docker daemon, including node_modules, .git, test files, and secrets:
.git
.gitignore
node_modules
npm-debug.log
Dockerfile
docker-compose*.yml
.env
.env.*
*.md
tests/
coverage/
.vscode/
.idea/
Use build arguments for values that change between builds but should not be in the image:
ARG NODE_ENV=production
ARG APP_VERSION=unknown
ENV NODE_ENV=$NODE_ENV
LABEL org.opencontainers.image.version=$APP_VERSION
LABEL org.opencontainers.image.source="https://github.com/your-org/your-app"
docker build --build-arg APP_VERSION=1.2.3 -t myapp:1.2.3 .
OCI labels make images traceable -- you can always find which commit built which image.
Tip: Run your Dockerfiles through the Dockerfile Linter to catch security issues, inefficient layer ordering, and missing best practices before building.
Docker Compose defines and manages multi-container applications. It is how most teams run local development environments and many production deployments.
A docker-compose.yml file defines services (containers), networks, and volumes:
services:
web:
build:
context: .
dockerfile: Dockerfile
ports:
- "3000:3000"
environment:
- DATABASE_URL=postgres://app:secret@db:5432/myapp
- REDIS_URL=redis://cache:6379
depends_on:
db:
condition: service_healthy
cache:
condition: service_started
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
restart: unless-stopped
deploy:
resources:
limits:
memory: 512M
cpus: "0.5"
db:
image: postgres:16-alpine
volumes:
- pgdata:/var/lib/postgresql/data
environment:
POSTGRES_DB: myapp
POSTGRES_USER: app
POSTGRES_PASSWORD: secret
healthcheck:
test: ["CMD-SHELL", "pg_isready -U app -d myapp"]
interval: 10s
timeout: 5s
retries: 5
restart: unless-stopped
cache:
image: redis:7-alpine
command: redis-server --maxmemory 128mb --maxmemory-policy allkeys-lru
restart: unless-stopped
volumes:
pgdata:
Health checks with depends_on: Use condition: service_healthy so your app does not start before the database is actually ready to accept connections (not just running).
Named volumes: Always use named volumes for persistent data. Bind mounts are for development. Named volumes survive docker compose down (but not docker compose down -v).
Environment variables: Use .env files for local development and environment-specific overrides. Never commit secrets to docker-compose.yml:
POSTGRES_PASSWORD=my-secret-password
services:
db:
environment:
POSTGRES_PASSWORD: $POSTGRES_PASSWORD
Profiles for optional services:
services:
debug-tools:
image: nicolaka/netshoot
profiles:
- debug
docker compose --profile debug up
Tip: Use the Docker Run to Compose converter to turn complex docker run commands into clean Compose files. Read more in our in-depth guide on Docker Compose best practices.
Docker networking determines how containers communicate with each other and the outside world.
| Driver | Scope | Use Case |
|---|
bridge | Single host | Default for standalone containers |
host | Single host | Container uses host network stack directly |
overlay | Multi-host | Docker Swarm or cross-host communication |
macvlan | Single host | Container gets its own MAC address on the LAN |
none | Single host | No networking (isolated) |
The default bridge network has limitations -- containers can only communicate by IP address, not by name. Always create user-defined bridge networks:
docker network create myapp
docker run -d --name web --network myapp nginx
docker run -d --name api --network myapp myapi:latest
On user-defined bridges, containers can resolve each other by name using Docker's built-in DNS. The web container can reach api at http://api:8080.
Docker runs an embedded DNS server at 127.0.0.11 for user-defined networks. Service discovery is automatic -- every container's name becomes a DNS entry. In Compose, the service name is the DNS name:
services:
web:
environment:
- DB_HOST=db
- CACHE_HOST=cache
db:
image: postgres:16
cache:
image: redis:7
Port mapping publishes container ports to the host:
docker run -p 8080:80 nginx
docker run -p 127.0.0.1:8080:80 nginx
docker run -p 80 nginx
Security tip: Always bind to 127.0.0.1 for services that should not be directly accessible from the network. Docker's port mapping bypasses iptables firewall rules by default.
Deep dive: Read our full article on Docker networking covering overlay networks, DNS troubleshooting, and firewall gotchas.
Containers are ephemeral -- when they stop, all data written inside them is lost. Volumes provide persistent storage.
docker run -v pgdata:/var/lib/postgresql/data postgres:16
docker run -v /host/path:/container/path nginx
docker run --tmpfs /tmp:rw,noexec,nosuid nginx
docker run -v /host/config:/etc/nginx/conf.d:ro nginx
| Type | Persistence | Management | Use Case |
|---|
| Named volume | Survives container removal | Docker CLI | Databases, application data |
| Bind mount | Depends on host path | Manual | Development, config files |
| tmpfs | Lost on container stop | Automatic | Secrets, temp data, caches |
Always use named volumes for databases. Bind mounts depend on the host path existing and having correct permissions. Named volumes are portable and managed by Docker:
docker volume create pgdata
docker volume inspect pgdata
docker run --rm -v pgdata:/data -v $(pwd):/backup alpine \
tar czf /backup/pgdata-backup.tar.gz -C /data .
docker run --rm -v pgdata:/data -v $(pwd):/backup alpine \
tar xzf /backup/pgdata-backup.tar.gz -C /data
Clean up unused volumes regularly. Docker does not automatically remove volumes when containers are removed:
docker volume ls
docker volume prune
docker volume prune -a
Deep dive: See our complete article on Docker volume management for advanced patterns including volume drivers, NFS mounts, and backup strategies.
Docker's default configuration is not secure for production. Every container runs as root, has write access to its filesystem, and shares the host kernel. Here is how to harden it.
The most important security measure. If an attacker compromises your app, they should not have root access:
# Create a non-root user
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
# Copy files with correct ownership
COPY --chown=appuser:appgroup . /app
# Switch to non-root user
USER appuser
docker exec mycontainer whoami
docker run --user 1000:1000 myimage
Prevent attackers from writing malicious files:
docker run --read-only \
--tmpfs /tmp:rw,noexec,nosuid \
--tmpfs /var/run:rw,noexec,nosuid \
myapp
In Compose:
services:
web:
read_only: true
tmpfs:
- /tmp
- /var/run
Linux capabilities grant fine-grained privileges. Docker gives containers a default set that is too permissive:
docker run --cap-drop=ALL --cap-add=NET_BIND_SERVICE myapp
services:
web:
cap_drop:
- ALL
cap_add:
- NET_BIND_SERVICE
Seccomp restricts which system calls a container can make. AppArmor restricts file access, network access, and capabilities:
docker run myapp
docker run --security-opt seccomp=unconfined myapp
Scan images for known vulnerabilities before deployment:
docker scout cves myapp:latest
trivy image myapp:latest
grype myapp:latest
Integrate scanning into your CI pipeline. Block deployments if critical vulnerabilities are found.
- Run as non-root user (
USER directive)
- Use minimal base images (Alpine or distroless)
- Read-only filesystem with tmpfs for writable paths
- Drop all capabilities, add back only what is needed
- Pin image versions (
FROM node:20.11.1-alpine, not FROM node:latest)
- Scan images in CI/CD pipeline
- Never store secrets in images (use runtime injection)
- Use
.dockerignore to exclude sensitive files from build context
Tip: Use the Dockerfile Linter to automatically check your Dockerfiles against these security best practices and get a security score. For a complete walkthrough, see our guide on Dockerfile security.
Running containers in development is easy. Running them reliably in production requires attention to health checks, resource limits, logging, and graceful shutdown.
Health checks tell Docker (and orchestrators) whether your container is actually working:
HEALTHCHECK --interval=30s --timeout=10s --retries=3 --start-period=60s \
CMD curl -f http://localhost:3000/health || exit 1
docker inspect --format='{{.State.Health.Status}}' mycontainer
start_period is critical -- it gives your app time to initialize before Docker starts checking health. Without it, slow-starting apps (JVM, large Node.js apps) get killed during startup.
Always set resource limits in production. Without them, a memory leak in one container can take down the entire host:
docker run -d \
--memory=512m \
--memory-swap=512m \
--cpus=0.5 \
--pids-limit=100 \
myapp
In Compose:
services:
web:
deploy:
resources:
limits:
memory: 512M
cpus: "0.5"
reservations:
memory: 256M
cpus: "0.25"
Setting --memory-swap equal to --memory disables swap, which is what you want -- swapping inside containers destroys performance and makes OOM issues harder to diagnose.
Docker captures stdout and stderr from your container. The default json-file driver writes unlimited logs that will fill your disk:
{
"log-driver": "json-file",
"log-opts": {
"max-size": "10m",
"max-file": "3"
}
}
Per container:
docker run --log-opt max-size=10m --log-opt max-file=3 myapp
For production, consider forwarding logs to a centralized system (Loki, ELK, Datadog) instead of relying on local files.
docker run -d --restart=always myapp
docker run -d --restart=on-failure:5 myapp
docker run -d --restart=unless-stopped myapp
In Compose, restart: unless-stopped is the best default for production services. It restarts crashed containers but respects manual docker compose stop.
When Docker stops a container, it sends SIGTERM and waits 10 seconds before sending SIGKILL. Your app must handle SIGTERM to shut down gracefully:
process.on('SIGTERM', async () => {
console.log('SIGTERM received, shutting down gracefully');
server.close(() => {
db.close();
process.exit(0);
});
});
docker stop --time=30 mycontainer
services:
web:
stop_grace_period: 30s
Common pitfall: If your Dockerfile uses CMD ["sh", "-c", "node app.js"], SIGTERM goes to sh, not your Node.js process. Use exec form instead: CMD ["node", "app.js"].
When something goes wrong in production, you need the right tools to diagnose quickly.
docker stats
docker stats --no-stream --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"
docker logs -f mycontainer
docker logs --tail 100 mycontainer
docker logs --since 2026-02-19T10:00:00 mycontainer
docker logs -t mycontainer
docker compose logs -f
docker compose logs -f web
docker exec -it mycontainer /bin/sh
docker exec -u 0 -it mycontainer /bin/sh
docker run -it --entrypoint /bin/sh myimage
docker cp mycontainer:/app/logs/error.log ./error.log
docker inspect mycontainer
docker top mycontainer
docker events
docker events --filter event=die
docker events --filter event=oom
docker events --filter container=mycontainer
Watch for oom events -- they mean your container hit its memory limit and was killed. Increase the memory limit or fix the leak.
Container exits immediately:
docker inspect mycontainer --format='{{.State.ExitCode}}'
docker logs mycontainer
docker run -it myimage
Container is running but not responding:
docker inspect --format='{{.State.Health.Status}}' mycontainer
docker exec -it mycontainer curl -s localhost:3000/health
docker exec -it mycontainer netstat -tlnp
Disk space issues:
docker system df
docker system df -v
docker system prune -a --volumes
docker builder prune -af
Network connectivity issues:
docker network ls
docker network inspect mynetwork
docker exec mycontainer nslookup other-service
docker run -it --network mynetwork nicolaka/netshoot
Building Docker configurations by hand is error-prone -- one missing flag or a bad Dockerfile pattern can cost hours of debugging. Here are the RawOps tools built specifically for Docker workflows:
-
Dockerfile Generator — Generate production-ready, multi-stage Dockerfiles for Node.js, Python, Go, Java, and static sites. Includes .dockerignore, build arguments, and security best practices baked in.
-
Docker Run to Compose — Convert complex docker run commands (with all their flags) into clean docker-compose.yml files. Also works in reverse -- convert Compose back to docker run commands.
-
Dockerfile Linter — Analyze your Dockerfiles against 21 rules covering security, performance, and best practices. Get a security score and auto-fix suggestions.
-
Docker CLI Builder — Build Docker CLI commands interactively. Covers 18 Docker actions with all their flags -- no more digging through docker --help pages.
Docker is the foundation, but modern DevOps builds on top of it. Here are the natural next steps:
- Docker Compose in depth — Read our guide on Docker Compose best practices for production-grade multi-service setups
- Dockerfile security — Dive deeper with Dockerfile security best practices covering supply chain attacks, secret management, and minimal images
- Docker networking — Understand the full networking model in Docker networking deep dive
- Persistent storage — Master volumes and bind mounts with Docker volume management
- Container orchestration — When you outgrow single-host Docker, read our Kubernetes troubleshooting guide
Docker is not just a tool -- it is a fundamental shift in how we think about deploying software. The patterns in this guide will serve you whether you are running a single container or orchestrating thousands. Start with the basics, enforce security from day one, and automate everything you can.