move

2026-06-18 22:06:40 -05:00
parent 6c5fda9c3a
commit e78383be1f
24 changed files with 0 additions and 0 deletions
@@ -0,0 +1,345 @@
+# Docker and Cloud-Native Patterns
+
+Source: https://docs.docker.com/build/building/best-practices/ | https://docs.astral.sh/uv/guides/integration/docker/ | https://fastapi.tiangolo.com/deployment/docker/ | https://uvicorn.dev/deployment/
+
+---
+
+## Canonical Multi-Stage Dockerfile (uv + FastAPI + uvicorn)
+
+```dockerfile
+# syntax=docker/dockerfile:1
+
+# ── builder stage ─────────────────────────────────────────────────────────────
+FROM python:3.12-slim-bookworm AS builder
+
+# Pin uv version — never use :latest in production
+COPY --from=ghcr.io/astral-sh/uv:0.5.27 /uv /uvx /bin/
+
+WORKDIR /app
+
+# Layer: install dependencies only (cached until pyproject.toml or uv.lock changes)
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=bind,source=uv.lock,target=uv.lock \
+    --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
+    uv sync --locked --no-install-project --no-dev --no-editable
+
+# Layer: copy source and install project
+COPY src/ /app/src/
+COPY pyproject.toml uv.lock /app/
+
+RUN --mount=type=cache,target=/root/.cache/uv \
+    uv sync --locked --no-dev --no-editable
+
+# ── runtime stage ─────────────────────────────────────────────────────────────
+FROM python:3.12-slim-bookworm AS runtime
+
+# Non-root user with explicit UID/GID
+RUN groupadd --system --gid 1001 appgroup && \
+    useradd --system --uid 1001 --gid appgroup --no-log-init --home /app appuser
+
+WORKDIR /app
+
+# Copy only the venv (uv binary stays in builder)
+COPY --from=builder --chown=appuser:appgroup /app/.venv /app/.venv
+
+# Copy app source
+COPY --from=builder --chown=appuser:appgroup /app/src /app/src
+
+# Activate the venv via PATH — do not rely on `uv run` at runtime
+ENV PATH="/app/.venv/bin:$PATH" \
+    PYTHONUNBUFFERED=1 \
+    PYTHONDONTWRITEBYTECODE=1 \
+    UV_PYTHON_DOWNLOADS=0
+
+USER appuser
+
+EXPOSE 8000
+
+# Liveness probe — no extra tools needed; uses stdlib urllib
+HEALTHCHECK --interval=30s --timeout=10s --start-period=15s --retries=3 \
+    CMD python -c \
+        "import urllib.request; urllib.request.urlopen('http://localhost:8000/healthz')" \
+        || exit 1
+
+# Exec form (NOT shell form) — required for SIGTERM → graceful shutdown
+CMD ["uvicorn", "my_app.main:app", \
+     "--host", "0.0.0.0", \
+     "--port", "8000", \
+     "--workers", "1", \
+     "--proxy-headers", \
+     "--forwarded-allow-ips", "*"]
+```
+
+---
+
+## .dockerignore
+
+```
+# Python artifacts
+.venv/
+__pycache__/
+*.pyc
+*.pyo
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+dist/
+*.egg-info/
+
+# Environment and secrets
+.env
+.env.*
+!.env.example
+
+# Git
+.git/
+.gitignore
+
+# Docs and local tooling
+README.md
+docs/
+*.md
+```
+
+**Always add `.venv/` to `.dockerignore`** — it is platform-specific and will cause subtle failures if copied into the image.
+
+---
+
+## docker-compose.yml (Local Development)
+
+```yaml
+services:
+  app:
+    build:
+      context: .
+      target: runtime     # build only up to the runtime stage
+    ports:
+      - "8000:8000"
+    environment:
+      DEBUG: "true"
+      LOG_LEVEL: "debug"
+    env_file:
+      - .env
+    restart: unless-stopped
+
+    # Docker Compose Watch — hot reload without rebuilding image
+    develop:
+      watch:
+        - action: sync
+          path: ./src
+          target: /app/src
+          ignore:
+            - .venv/
+            - __pycache__/
+        - action: rebuild
+          path: ./pyproject.toml
+        - action: rebuild
+          path: ./uv.lock
+```
+
+Run with:
+
+```bash
+docker compose up               # start normally
+docker compose watch            # start with live sync
+docker compose up --build       # force rebuild
+```
+
+---
+
+## Key Dockerfile Rules
+
+### Multi-Stage Builds
+
+- **Stage 1 (`builder`)**: Has uv, build tools, compiles `.pyc` files if needed.
+- **Stage 2 (`runtime`)**: Minimal; contains only the venv and source. No uv, no build tools.
+- Final image size: typically 150–200 MB for a FastAPI app (vs 500+ MB with a single stage).
+
+### Layer Caching Strategy
+
+Copy files in order of change frequency (least → most):
+
+```
+1. uv.lock + pyproject.toml  → install deps (cached for days)
+2. Source code               → install project (invalidated on every code change)
+```
+
+This means dependency installation is only re-run when `uv.lock` or `pyproject.toml` changes.
+
+### CMD Exec Form (Critical)
+
+```dockerfile
+# ✅ Exec form — process receives SIGTERM directly → graceful shutdown
+CMD ["uvicorn", "my_app.main:app", "--host", "0.0.0.0", "--port", "8000"]
+
+# ❌ Shell form — process is a child of /bin/sh → SIGTERM goes to shell, not uvicorn
+CMD uvicorn my_app.main:app --host 0.0.0.0 --port 8000
+```
+
+Shell form breaks graceful shutdown and FastAPI lifespan shutdown events.
+
+### Non-Root User
+
+```dockerfile
+RUN groupadd --system --gid 1001 appgroup && \
+    useradd --system --uid 1001 --gid appgroup --no-log-init --home /app appuser
+USER appuser
+```
+
+- Use `--system` for service accounts (no shell, no home by default).
+- Use `--no-log-init` to avoid `/var/log/faillog` disk exhaustion (Go runtime bug in older kernels).
+- Use explicit UID/GID (1001) — deterministic, scanners can reason about it.
+- Never run as UID 0 (root) in production.
+
+### HEALTHCHECK
+
+```dockerfile
+HEALTHCHECK --interval=30s --timeout=10s --start-period=15s --retries=3 \
+    CMD python -c \
+        "import urllib.request; urllib.request.urlopen('http://localhost:8000/healthz')" \
+        || exit 1
+```
+
+- `--start-period`: grace period on startup before health checks begin.
+- `--retries`: mark as unhealthy only after N consecutive failures.
+- Uses stdlib `urllib` — no extra tools needed, not even `curl`.
+
+---
+
+## Cloud-Native Twelve-Factor Principles Applied
+
+### Factor III — Config (Environment Variables)
+
+- All config comes from environment variables, never from code.
+- `pydantic-settings` reads from env automatically.
+- Provide `.env.example` with documentation; never commit `.env`.
+
+```bash
+# Runtime injection
+docker run -e DATABASE_URL="postgresql://..." -e SECRET_KEY="..." my-app
+# or via env_file in compose / Kubernetes Secret
+```
+
+### Factor XI — Logs (Treat as Event Streams)
+
+- Set `PYTHONUNBUFFERED=1` — logs are flushed immediately to stdout/stderr.
+- Never write logs to files inside the container.
+- Configure uvicorn to log JSON in production:
+
+```python
+# In create_app() lifespan or a logging setup module
+import logging
+import json
+
+class JSONFormatter(logging.Formatter):
+    def format(self, record: logging.LogRecord) -> str:
+        return json.dumps({
+            "level": record.levelname,
+            "message": record.getMessage(),
+            "logger": record.name,
+        })
+```
+
+Or use `structlog` with the JSON renderer for production.
+
+### Factor IX — Disposability (Fast Startup, Graceful Shutdown)
+
+- FastAPI lifespan handles startup/shutdown.
+- Uvicorn forwards `SIGTERM` to the Python process when using exec form `CMD`.
+- The process should be fully ready to serve requests within 10 seconds.
+- Kubernetes `terminationGracePeriodSeconds` should be >= your timeout.
+
+---
+
+## Kubernetes Readiness / Liveness Probes
+
+Expose two endpoints:
+
+```python
+@router.get("/healthz", include_in_schema=False)  # liveness
+async def health():
+    return {"status": "ok"}
+
+@router.get("/readyz", include_in_schema=False)   # readiness
+async def readiness(request: Request):
+    # Check DB, cache, etc.
+    return {"status": "ready"}
+```
+
+Kubernetes manifest snippet:
+
+```yaml
+livenessProbe:
+  httpGet:
+    path: /healthz
+    port: 8000
+  initialDelaySeconds: 10
+  periodSeconds: 30
+
+readinessProbe:
+  httpGet:
+    path: /readyz
+    port: 8000
+  initialDelaySeconds: 5
+  periodSeconds: 10
+```
+
+---
+
+## Scaling: Workers vs. Replicas
+
+| Deployment | Workers setting | Reasoning |
+|---|---|---|
+| Kubernetes / Cloud Run | `--workers 1` | Orchestrator handles replication; 1 process per container for predictable memory |
+| Docker Compose (single host) | `--workers 4` | No external replication; use CPU cores |
+| Local dev | `--reload` (single process) | Hot reload only works with single process |
+
+**Never use `--reload` in production.**
+
+---
+
+## Base Image Selection
+
+| Use case | Recommended base |
+|---|---|
+| Standard production | `python:3.12-slim-bookworm` |
+| Smallest possible image | `python:3.12-alpine3.20` (musl; watch for C extension compat) |
+| uv-managed Python | `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` |
+| Security-hardened | `cgr.dev/chainguard/python:latest` (distroless) |
+
+**Do not use:**
+- `python:latest` — unpinned, breaks reproducibility
+- `tiangolo/uvicorn-gunicorn-fastapi` — deprecated by FastAPI team
+- Full `python:3.x` (non-slim) — 300+ MB unnecessary overhead
+
+---
+
+## Build Optimizations
+
+```bash
+# Enable BuildKit (default in Docker >= 23)
+export DOCKER_BUILDKIT=1
+
+# Build with cache mount (fastest for repeated local builds)
+docker build -t my-app .
+
+# CI: force fresh base image + no layer cache
+docker build --pull --no-cache -t my-app .
+
+# Multi-platform build (for ARM deployment from x86 CI)
+docker buildx build --platform linux/amd64,linux/arm64 -t my-app .
+```
+
+---
+
+## Security Hardening Checklist
+
+- [ ] Non-root user with explicit UID/GID
+- [ ] Read-only filesystem where possible (`--read-only` docker run flag or `securityContext.readOnlyRootFilesystem: true` in K8s)
+- [ ] No secrets in `ENV` Dockerfile instructions
+- [ ] Minimal base image (slim/alpine)
+- [ ] Multi-stage build (no build tools in runtime image)
+- [ ] Pin base image and uv version (not `:latest`)
+- [ ] `.dockerignore` excludes `.env`, `.git`, `.venv`
+- [ ] `EXPOSE` only the port actually used
+- [ ] Regular base image updates in CI (`docker build --pull`)