Docker: multi-stage builds for tiny images

Docker multi-stage builds significantly reduce the size of container images by separating build-time dependencies from runtime requirements. This approach…

Docker multi-stage builds significantly reduce the size of container images by separating build-time dependencies from runtime requirements. This approach leads to smaller images, faster deployments, improved security by reducing the attack surface, and lower storage costs. This article details the implementation of multi-stage builds, focusing on practical examples for various development stacks and best practices for optimization.

Understanding the Multi-Stage Build Concept

A multi-stage build involves defining multiple FROM instructions in a single Dockerfile. Each FROM instruction starts a new build stage. Critically, files can be copied from one stage to another using the COPY --from=<stage_name_or_number> instruction. This mechanism allows build artifacts (like compiled binaries, static assets, or bundled application code) to be transferred to a much smaller, production-ready base image, leaving behind all the tools and intermediate files used during compilation or packaging.

The Problem Multi-Stage Builds Solve

Without multi-stage builds (prior to Docker 17.05), developers often employed two common patterns:

• Single-stage build: A single Dockerfile that included all build tools and runtime dependencies, leading to bloated images (e.g., a golang:alpine image for compiling, then shipping the binary in the same image).
• Builder pattern: Using separate Dockerfiles or scripts to build artifacts, then manually copying them into a final production image. This was cumbersome and less integrated.

Multi-stage builds streamline the builder pattern directly within a single Dockerfile, making the process more efficient and easier to maintain.

Implementing Multi-Stage Builds: Practical Examples

Example 1: Go Application

Go applications compile into static binaries, making them ideal candidates for multi-stage builds using a scratch or alpine base image for the final stage.

# Stage 1: Build the Go application
FROM golang:1.22-alpine AS builder

WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download

COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -a -ldflags '-extldflags "-static"' -o /app/server .

# Stage 2: Create the final lean image
FROM alpine:3.19

WORKDIR /app
COPY --from=builder /app/server .
EXPOSE 8080
CMD ["./server"]

In this example:

• golang:1.22-alpine is used for compilation. This stage is named builder.
• go mod download fetches dependencies, and go build compiles the application. CGO_ENABLED=0 ensures a statically linked binary, crucial for compatibility with alpine.
• The second stage starts with a fresh alpine:3.19 image.
• COPY --from=builder /app/server . transfers only the compiled binary from the builder stage to the final image.
• The resulting image will typically be a few MBs, significantly smaller than the golang:1.22-alpine base image (around 300MB).

Example 2: Node.js Application (React/Vue/Angular Frontend)

Frontend applications often involve many development dependencies (e.g., Webpack, Babel, TypeScript) that are not needed at runtime. Multi-stage builds can drastically reduce image size by only shipping the compiled static assets and a web server.

# Stage 1: Build the frontend application
FROM node:20-alpine AS builder

WORKDIR /app
COPY package.json package-lock.json ./
RUN npm install

COPY . .
RUN npm run build # This command generates static assets in a 'dist' or 'build' folder

# Stage 2: Serve the static assets with Nginx
FROM nginx:1.25-alpine

# Remove default Nginx config to replace it with our own
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d/default.conf

COPY --from=builder /app/build /usr/share/nginx/html # Adjust '/app/build' as per your project's build output
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Here:

• node:20-alpine serves as the build environment (named builder).
• npm install and npm run build execute the build process.
• The second stage uses a tiny nginx:1.25-alpine image.
• A custom nginx.conf is copied to configure the web server.
• COPY --from=builder /app/build /usr/share/nginx/html copies the static HTML, CSS, and JS files generated by npm run build into the Nginx serving directory.

Example 3: Python Application with Dependencies

Python applications often have many C-extension dependencies that require build tools like gcc and Python development headers. These are not needed at runtime.

# Stage 1: Build dependencies with build tools
FROM python:3.11-slim-bullseye AS builder

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Stage 2: Create a minimal runtime image
FROM python:3.11-slim-bullseye

WORKDIR /app
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY . .

EXPOSE 5000
CMD ["python", "app.py"]

This Python example shows a slight variation. Instead of just copying a binary, we copy the installed Python packages from the builder stage. This is useful when the slim image already contains the necessary runtime Python interpreter and system libraries, but we want to avoid installing build dependencies in the final image.

For more complex C-extension heavy packages, you might use python:3.11-buster as the builder and then switch to a python:3.11-slim-buster for the final image, ensuring the ABI compatibility of the installed modules.

Best Practices for Image Optimization

Leverage .dockerignore

Always use a .dockerignore file in your project's root directory. This prevents unnecessary files (like .git, node_modules in the root, IDE files, local dev data) from being sent to the Docker daemon build context, speeding up builds and reducing image layer size.

# .dockerignore example
**/.git
**/.vscode
**/node_modules
**/tmp
**/logs
Dockerfile
README.md

Pin Base Image Versions

Always use specific version tags for your base images (e.g., alpine:3.19, golang:1.22-alpine) rather than floating tags like latest or alpine. This ensures reproducible builds.

Order Layers Effectively

Place instructions that change less frequently (e.g., dependency installation) earlier in the Dockerfile. Docker caches layers, so changes to later instructions won't invalidate earlier, stable layers, leading to faster rebuilds.

# Good: Copy dependencies first, then application code
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .

# Bad: Copy everything then install, forcing re-install on every code change
COPY . .
RUN pip install -r requirements.txt

Minimize Layers

Combine multiple RUN commands using && \ to reduce the number of layers in your image. Each RUN instruction creates a new layer, and fewer layers generally mean smaller images and better performance.

RUN apt-get update && \
    apt-get install -y --no-install-recommends some-package && \
    rm -rf /var/lib/apt/lists/*

Clean Up During Build

Remove build caches and unnecessary files immediately after they are used within the same RUN instruction. This ensures they don't persist in the image layer.

• For apt: rm -rf /var/lib/apt/lists/*
• For yum: yum clean all && rm -rf /var/cache/yum
• For npm: npm cache clean --force (though npm install is often followed by a npm prune --production if only production dependencies are needed, or simply not copied in the final stage).
• For pip: pip install --no-cache-dir

Use Minimal Base Images

Whenever possible, use minimal base images like alpine, slim variants (e.g., python:3.11-slim-bullseye), or even scratch for statically linked binaries. These images contain only the essentials, drastically reducing the attack surface and size.

Common Pitfalls and Troubleshooting

Missing Runtime Dependencies

One of the most common issues with minimal base images is missing shared libraries that your application or its dependencies require at runtime. For example, a Python package might depend on libpq-dev (PostgreSQL client libraries) which isn't present in python:slim. When this happens, your container will likely crash with errors like "No such file or directory" for a dynamic library. The solution is to identify the missing libraries (e.g., using ldd your_binary or strace in a debugging container) and install them in the final stage's Dockerfile:

# Example: Adding PostgreSQL client libraries to a python:slim image
FROM python:3.11-slim-bullseye

RUN apt-get update && \
    apt-get install -y --no-install-recommends libpq-dev && \
    rm -rf /var/lib/apt/lists/*
# ... rest of your Dockerfile

Incorrect Copy Paths

Ensure that the source path in COPY --from=<stage> <source_path> <destination_path> precisely matches the location of the artifacts in the builder stage. Relative paths can be tricky; always use absolute paths for clarity (e.g., /app/build, not ./build).

Permissions Issues

If your application fails to start or interact with files, it might be a permission problem. Files copied from a builder stage often retain the user and group IDs of the builder user. If your final image runs as a non-root user (which is a good security practice), you might need to adjust permissions:

# Example: Ensure files are owned by a non-root user
FROM alpine:3.19

RUN addgroup -S appgroup && adduser -S appuser -G appgroup
WORKDIR /app
COPY --from=builder --chown=appuser:appgroup /app/server .
USER appuser
CMD ["./server"]

The --chown flag (Docker Engine 18.09+) is very useful for this. Otherwise, a separate RUN chown -R appuser:appgroup /app command would be necessary.