Advanced Docker Compose Features
09 Jul 2024I recently set up a Prefect environment using Docker Compose and wanted to document some of the slightly more advanced features I used. This setup deploys a PostgreSQL database, Prefect server (API and front end), and two workers - one for running flows as subprocesses and another for running them as Docker containers.
While this configuration is specific to Prefect, the overall structure and many of the techniques are widely applicable. The pattern of bringing up a database alongside one or more application services is common in many Docker Compose stacks. As such, the advanced features explored here can be valuable in a variety of containerized deployments, from web applications to microservices architectures.
The full Docker Compose file, along with the rest of the code, is available in this repo.
Here are the key Docker Compose features I found particularly useful:
1. YAML Anchors and Aliases
YAML anchors allow reusing common configuration across multiple services:
x-prefect-common: &prefect-common
build:
context: .
networks:
- prefect
restart: always
services:
server:
<<: *prefect-common
command: prefect server start --host server --port 4200
# Additional server-specific config
The &prefect-common
anchor defines a set of common configurations. The <<: *prefect-common
syntax then merges these common configs into each service that needs them. This keeps the compose file DRY and easier to maintain.
2. Health Checks
The compose file defines detailed health checks for services:
healthcheck:
test: ["CMD", "curl", "-f", "http://server:4200/api/health"]
start_interval: 5s
start_period: 15s
The start_interval
specifies more frequent health checks during startup to make everything come up quicker, but only for the initial startup period. The start_period
defines a grace period before starting these health checks. This allows for more nuanced control over service initialization, balancing quick startup with allowing sufficient time for services to initialize.
3. Startup Order
Health checks are used in combination with depends_on
to ensure proper service startup order:
docker-worker:
<<: *prefect-common
depends_on:
server:
condition: service_healthy
This configuration ensures that the docker-worker
service only starts after the server
service is not just running, but actually healthy according to its defined health check.
4. Docker-in-Docker
The Docker worker service includes a specific mount to enable interaction with the host’s Docker daemon:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
This mount allows the worker to call the Docker daemon on the host system, enabling it to spin up new containers for flow runs. This is crucial for the Docker worker’s ability to create and manage containers for Prefect flows.