Use docker compose for Local Dev Stacks

Developing modern applications often involves a complex interplay of services: databases, caching layers, message queues, and custom APIs. Replicating…

Developing modern applications often involves a complex interplay of services: databases, caching layers, message queues, and custom APIs. Replicating this multi-service environment consistently across developer workstations can be challenging. Docker Compose provides a powerful, declarative way to define and run multi-container Docker applications, making it an indispensable tool for local development stacks. This article details how to effectively leverage Docker Compose for robust, reproducible, and efficient local development.

Structuring Your compose.yaml for Local Development

The core of your Docker Compose setup is the compose.yaml file (or docker-compose.yml for older versions). This file defines the services, networks, and volumes that make up your application stack. For local development, prioritize readability and explicit configuration.

Defining Services

Each service in your stack gets its own entry. Best practice for local development often involves binding container ports to host ports, mounting local source code, and setting environment variables specific to development.

version: '3.8' # Always specify a version for compatibility and features

services:
  webapp:
    build:
      context: ./app # Path to your application's Dockerfile
      dockerfile: Dockerfile.dev # Use a specific Dockerfile for development if needed
    ports:
      - "8000:8000" # Map container port 8000 to host port 8000
    volumes:
      - ./app:/app # Mount local source code into the container
      - webapp_node_modules:/app/node_modules # Example: use a named volume for dependencies
    environment:
      NODE_ENV: development
      DATABASE_URL: postgres://user:password@db:5432/myapp_dev
    depends_on:
      db:
        condition: service_healthy # Ensure DB is healthy before starting webapp
    healthcheck: # Define a health check for the webapp itself
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 20s # Give the app time to start up

  db:
    image: postgres:14-alpine # Use a specific, stable version
    ports:
      - "5432:5432"
    volumes:
      - db_data:/var/lib/postgresql/data # Persist database data
    environment:
      POSTGRES_DB: myapp_dev
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
    healthcheck: # Critical for `depends_on: service_healthy`
      test: ["CMD-SHELL", "pg_isready -U $POSTGRES_USER -d $POSTGRES_DB"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  db_data:
  webapp_node_modules: # Define named volumes

• build vs. image: For local development, build is common to incorporate recent code changes. For external dependencies (like databases), image is preferred for specific versions.
• Port Mapping: "HOST_PORT:CONTAINER_PORT" allows access to services from your host machine.
• Volume Mounting: ./app:/app mounts your local application directory directly into the container, enabling live code changes without rebuilding the image. Named volumes (e.g., db_data, webapp_node_modules) are crucial for persistent data and avoiding re-downloading dependencies on every container start.
• Environment Variables: Use these to pass configuration like database credentials or API keys. Avoid hardcoding sensitive information; use .env files or environment variables on the host for production, but for local, explicit declarations are fine.
• depends_on: Specifies dependencies between services. Crucially, the condition: service_healthy ensures that a service is fully functional (not just started) before its dependents attempt to connect. This prevents "connection refused" errors during startup.
• healthcheck: Defines a command to check if a container is still working. Essential for depends_on: service_healthy and for identifying failing services quickly.

Overriding Configuration with compose.override.yaml

Each developer's local setup might have unique requirements (e.g., different host ports, specific resource limits, or local-only tools). Rather than modifying the main compose.yaml, which should be committed to version control, use a compose.override.yaml file. This file is automatically merged by Docker Compose and should be gitignored.

# compose.override.yaml (ignored by Git)
version: '3.8'

services:
  webapp:
    ports:
      - "8080:8000" # Developer A prefers port 8080
    environment:
      DEBUG_MODE: "true" # Enable debug mode locally
    volumes:
      - /Users/developer_a/my_custom_log_path:/app/logs # Mount a custom log path

  db:
    ports:
      - "5433:5432" # Developer A prefers to run two Postgres instances

Docker Compose merges files in alphabetical order by default, so compose.override.yaml takes precedence over compose.yaml for conflicting keys. You can also specify multiple override files or a custom override file using the -f flag (e.g., docker compose -f compose.yaml -f compose.dev.yaml up).

Efficient Development with docker compose watch

A common pain point in containerized development is the need to rebuild and restart containers after every code change. docker compose watch (introduced in Docker Compose v2.17.0) provides a hot-reloading mechanism that streamlines this process significantly. Instead of rebuilding images, it monitors your local filesystem and performs predefined actions on changes.

Configuring watch in compose.yaml

Add a develop section to your service definition:

version: '3.8'

services:
  webapp:
    build:
      context: ./app
      dockerfile: Dockerfile.dev
    ports:
      - "8000:8000"
    volumes:
      - ./app:/app
      - webapp_node_modules:/app/node_modules
    environment:
      NODE_ENV: development
      DATABASE_URL: postgres://user:password@db:5432/myapp_dev
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 20s

    develop: # New section for watch
      watch:
        - path: ./app # Watch the entire app directory
          action: sync # Sync changes directly into the container
          target: /app
        - path: ./app/package.json # Watch specific files for different actions
          action: rebuild # Rebuild if package.json changes (e.g., new dependencies)
        - path: ./app/src/**/*.js # Or watch specific file patterns
          action: sync
          target: /app/src

  db:
    image: postgres:14-alpine
    ports:
      - "5432:5432"
    volumes:
      - db_data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: myapp_dev
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U $POSTGRES_USER -d $POSTGRES_DB"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  db_data:
  webapp_node_modules:

watch Actions Explained

• sync: The most common action for local development. When files in the specified path change on the host, they are automatically copied into the container at the target path. This is significantly faster than rebuilding.
• rebuild: If a change in the path necessitates a full image rebuild (e.g., changes to Dockerfile or dependency manifest files like package.json or requirements.txt), this action triggers a rebuild and restart of the service.
• sync+restart: (Not shown above) Synchronizes files and then restarts the service. Useful for applications that don't have built-in hot-reloading but can benefit from a quick restart.

To start your development stack with watch enabled, simply run: docker compose watch. This will start all services and then monitor the specified paths for changes.

Practical Commands and Workflows

• Start services: docker compose up -d (-d for detached mode)
• Start services with watch: docker compose watch
• Stop services: docker compose down (removes containers and default networks)
• Stop services and remove volumes: docker compose down -v (useful for a clean slate)
• Rebuild specific service: docker compose build webapp
• View logs: docker compose logs -f webapp (-f for follow mode)
• Execute a command in a service: docker compose exec webapp bash (opens a shell)
• Run a one-off command: docker compose run --rm webapp npm test (--rm removes the container after exit)

Common Pitfalls and Troubleshooting

• Port Conflicts: If you get an error like Error starting userland proxy: listen tcp 0.0.0.0:8000: bind: address already in use, another process on your host is using the specified port. Either stop the conflicting process or change the host port in your compose.yaml/compose.override.yaml.
• Volume Permission Issues: Especially on Linux, the user inside the container might not have permissions to write to mounted host volumes. This can manifest as "permission denied" errors when the application tries to create files or directories. Solutions include:
  • Setting the user directive in compose.yaml to match your host user's UID/GID.
  • Adjusting permissions on the host directory (e.g., sudo chown -R $USER:$USER ./app).
  • Using named volumes for directories requiring writes by the container user.
• depends_on and Health Checks: Without proper healthcheck definitions, depends_on only waits for the container to start, not for the application inside to be ready. This leads to "connection refused" errors from dependent services. Always define health checks for critical services like databases and backend APIs.
• Caching Issues with build: If your Dockerfile caches dependencies and you update your package.json or requirements.txt without triggering a rebuild, your service might run with outdated dependencies. Ensure your develop.watch configuration or manual rebuilds address this. For example, explicitly trigger a rebuild when dependency files change.
• docker compose watch Performance: Watching large directories with many files can sometimes consume significant CPU resources, especially on macOS. Be judicious with your watch paths; specify only the directories that contain actively developed code.