Docker Compose deployment#

The bundled deploy/docker/docker-compose.yml is the canonical reference shape for a Strata stack on one host. Bare docker compose up -d brings up the TiKV-default 2-replica lab (PD + TiKV + ceph + ceph-b + strata-a + strata-b + nginx LB + prometheus + grafana). The Cassandra-backed regression lab lives under --profile cassandra so an operator can make up-cassandra to validate the Cassandra meta backend side by side.

The compose file itself is the source of truth. When it changes, this page may lag by one PR; cross-check ports + env before depending on them.

Prerequisites#

• Docker Engine ≥24 (or Docker Desktop ≥4.20). macOS via Lima also works (DOCKER_HOST=unix:///Users/.../.lima/.../sock/docker.sock).
• ≥4 vCPU + 8 GiB free RSS for the full default stack (PD + TiKV + two RADOS clusters + two gateway replicas + Prometheus + Grafana).
• make (and the Makefile targets the compose file from the repo root).

Install#

The repo’s Makefile wraps the canonical bring-up flow:

make up        # docker compose up -d (TiKV-default lab)
make wait-tikv && make wait-ceph && make wait-strata-lab

make dev runs the same sequence plus a tail of gateway logs — that is the convenience target documented in Get Started.

For the Cassandra-backed regression lab:

make up-cassandra        # docker compose --profile cassandra up -d
make wait-cassandra && make wait-ceph

For tracing (OTel collector + Jaeger UI on :16686):

docker compose --profile tracing up -d otel-collector jaeger
# then set OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318 on the gateway

Profiles compose: docker compose --profile cassandra --profile tracing up -d brings up the bare default + Cassandra-backed gateway + OTel stack in one shot.

Configure#

Service map#

The default docker compose up -d brings up nine services (pd + tikv + ceph + ceph-b + strata-a + strata-b + strata-lb-nginx + prometheus + grafana). Every profile-gated service stays silent until requested.

Profiles#

Env vars (top knobs)#

The runtime config is a TOML file mounted at /etc/strata/strata.toml (see deploy/strata.toml). Env vars override the file. Full table at Reference — environment variables.

Volumes + mounts#

The compose file uses named volumes for state so docker compose down doesn’t lose data, and bind-mounts for config so edits take effect on next up.

Tear down + drop state: make down (removes containers across every known profile; named volumes persist). docker compose -f deploy/docker/docker-compose.yml down -v nukes the volumes too.

Verify#

curl http://127.0.0.1:9999/healthz   # nginx LB → strata-a or strata-b
curl http://127.0.0.1:9999/readyz    # both replicas + RADOS + TiKV
curl http://127.0.0.1:10001/readyz   # strata-a direct
curl http://127.0.0.1:10002/readyz   # strata-b direct
aws --endpoint-url http://127.0.0.1:9999 --no-sign-request s3 ls

The operator console: http://127.0.0.1:9999/console/ (trailing slash). Prometheus UI: :9090. Grafana: :3000 (admin/adminpass). Jaeger (tracing profile): :16686.

Monitor#

• Gateway metrics: Prometheus auto-scrapes strata-a:9000/metrics and strata-b:9000/metrics. The provisioned Grafana dashboard surfaces request rate, GC backlog, replication lag, worker panic counters. See Best Practices — monitoring.
• Traces: the tracing profile spins an OTel collector + Jaeger all-in-one. Set OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318 on the gateway containers and inspect spans in the Jaeger UI.
• Logs: JSON to stdout; docker compose logs -f strata-a strata-b for live tail. Each line carries request_id + node_id.

Troubleshoot#

• make up hangs on wait-tikv. PD took longer than the timeout to elect a leader. docker compose logs pd should show PD leader within ~30 s; if not, restart PD.
• wait-strata-lab fails. Either RADOS isn’t ready (run make wait-ceph) or the gateway is crash-looping (check docker compose logs strata-a).
• aws-cli against :9999 returns 404 on the wrong bucket. Path-style URLs route by /<bucket>/<key>. For virtual-hosted-style, set STRATA_VHOST_PATTERN=*.s3.local and use --endpoint-url http://bucket.s3.local:9999.
• JWT-bootstrap clash: the strata-jwt-shared volume is shared between strata-a + strata-b. If you start strata-cassandra on top, it joins the same volume — session JWTs validate across all three replicas.
• Cassandra profile won’t reach strata-cassandra:9998. Either Cassandra isn’t ready (make wait-cassandra) or you forgot the --profile cassandra flag on docker compose up.

Production checklist#

For any compose-managed deployment that survives past the lab phase:

• Pin every image tag to a SHA, not the latest / vN.Y.Z mutable form.
• Replace STRATA_AUTH_MODE=optional with required; rotate STRATA_STATIC_CREDENTIALS out of the host env into a secret store + STRATA_CONFIG_FILE.
• Move cassandra / pd / tikv / ceph to dedicated hosts — bundled-on-the-same-box is for labs only. PD ≥3 + TiKV ≥3 for raft majority in production.
• Configure backups for every data-bearing volume — see Operate — backup & restore.
• Wire Prometheus alertmanager + log shipping (the bundled compose ships scrape + dashboard, not alerts).
• Front the gateway with a real LB if you run the bare-default 2-replica shape outside the lab (the bundled nginx is unauthenticated + no TLS).
• Size STRATA_OTEL_RINGBUF_BYTES for expected traffic (default 4 MiB ≈ a few thousand spans). Wire OTLP collector for retention beyond the ring.

Cross-references#

• Single-node deployment — when you don’t need the full compose stack.
• Multi-replica cluster — operator guide for the bare-default 2-replica TiKV lab.
• Operate — day-2 workflows (drain, scale, back up).
• Reference — environment variables — full env knob table.
• Architecture — Storage — backend selection + chunking.
• Architecture — Backends — TiKV — why TiKV is the lab default.
• Architecture — Migrations — TiKV-default lab — what changed when the lab default flipped.