V6.1 backups: once more into the breach.
Quality Gate / gate (push) Successful in 2m29s

This commit is contained in:
Jim Lancaster
2026-09-02 10:59:28 -05:00
parent 494f378e48
commit 96bb80d91f
17 changed files with 120 additions and 353 deletions
+29 -67
View File
@@ -1,75 +1,46 @@
# Backup and Restore (V6.0 Phase 4)
# Backup and Restore (V6.1)
This guide defines operational backup/restore for clean-slate recovery of the Docker runtime with host-level backups that Synology Drive can replicate.
This guide defines operational backup/restore for clean-slate recovery of the Docker runtime using a host-visible backup folder.
## 1. Backup artifacts (clean-slate recovery set)
## 1. Backup artifacts
- Primary local backup location: `DATABASE_BACKUP_DIR` (recommended production value: `/backup`)
- Local files created per run:
- Backup target root: `BACKUP_DIR` (recommended production value: `/backup`)
- Database artifact per run:
- `postgres-YYYYMMDD-HHMMSS.dump` (PostgreSQL custom dump via `pg_dump -Fc`)
- `backup-YYYYMMDD-HHMMSS.manifest` (artifact index)
- Optional Synology files created per run (when `SYNOLOGY_BACKUP_DIR` is set):
- `postgres-YYYYMMDD-HHMMSS.dump` (copied from local)
- `config-YYYYMMDD-HHMMSS.tar.gz` (deployment config snapshot: `.env.production`, `docker-compose.production.yml`, `deploy/cloudflared/config.yml` when present)
- `logs-YYYYMMDD-HHMMSS.tar.gz` (snapshot of `/app/data/logs`)
- `backup-YYYYMMDD-HHMMSS.manifest`
- Local app-data mirror (updated each run):
- `data/**`
- Local incremental media mirror (copied only if missing):
- `uploads/homepage.md`
- `uploads/documents/**`
- `uploads/photos/**`
- `backup-YYYYMMDD-HHMMSS.manifest` (run manifest)
- Media/config mirrors under `BACKUP_DIR`:
- `uploads/**` (incremental copy: new files only)
- `prompts/**` (prompt directory mirror)
Retention:
- `BACKUP_RETENTION_DAYS` applies to `postgres-*.dump` and `backup-*.manifest` files.
## 2. Creating backups
Use the scripted command from the repository root:
Run from repository root:
```bash
sh deploy/backup/create_postgres_backup.sh
```
Environment overrides:
Environment variables used by the backup script:
- `BACKUP_DIR` (default `DATABASE_BACKUP_DIR`, then `./data/backups`)
- `APP_DATA_BACKUP_DIR` (default `${BACKUP_DIR}/data`)
- `UPLOADS_BACKUP_DIR` (default `${BACKUP_DIR}/uploads`)
- `BACKUP_DIR` (default `./data/backups`)
- `BACKUP_RETENTION_DAYS` (default `14`)
- `SYNOLOGY_BACKUP_DIR` (if set, dump/config/log/manifest are copied to this mounted path and media is synced incrementally)
- `ENV_FILE` (default `.env.production`)
- `COMPOSE_FILE` (default `docker-compose.production.yml`)
- `UPLOAD_DIR` (default `/app/uploads`)
- `PROMPT_DIR` (default `/app/prompts`)
- `POSTGRES_HOST` (default `postgres`)
- `POSTGRES_PORT` (default `5432`)
- `POSTGRES_DB` (required)
- `POSTGRES_USER` (required)
- `POSTGRES_PASSWORD` (required)
Recommended production setup:
- Mount a host-visible folder into `/backup` for both `app` and `worker` services.
- Set `DATABASE_BACKUP_DIR=/backup` in `.env.production`.
- Leave `SYNOLOGY_BACKUP_DIR` empty when Synology Drive Client handles replication from the host folder.
Example with Synology mount:
```bash
SYNOLOGY_BACKUP_DIR=/mnt/synology-backups sh deploy/backup/create_postgres_backup.sh
```
## 2.1 Optional direct Synology mount (LXC)
This section is only needed when using direct container-side Synology copy via `SYNOLOGY_BACKUP_DIR`.
Manual `mount` commands are lost after reboot unless persisted.
Use the template (copy, edit placeholders, then run):
```bash
cp deploy/backup/mount_synology_cifs.example.sh /root/mount_synology_cifs.sh
nano /root/mount_synology_cifs.sh
sh /root/mount_synology_cifs.sh
```
Then add the printed `/etc/fstab` line (with your real values) so the mount survives reboot.
Recommended pattern:
- Keep NAS credentials in a local file like `/etc/samba/credentials/transcription-synology` with `chmod 600`.
- Keep `SYNOLOGY_BACKUP_DIR` in `.env.production` aligned to that mount point (for example `/mnt/synology-backups`).
- The script will also read `SYNOLOGY_BACKUP_DIR` from `ENV_FILE` when not exported in the shell.
- Mount a host-visible folder into `/backup` for both `app` and `worker`.
- Set `BACKUP_DIR=/backup` in `.env.production`.
- Use host-level tooling (for example Synology Drive Client on the host) to replicate that folder externally.
## 3. Restoring from backup
@@ -77,21 +48,12 @@ Restore requires downtime for app + worker writes.
1. Stop app and worker:
- `docker compose --env-file .env.production -f docker-compose.production.yml stop app worker`
2. Restore:
- `sh deploy/backup/restore_postgres_backup.sh ./data/backups/postgres-YYYYMMDD-HHMMSS.dump`
- if `SYNOLOGY_BACKUP_DIR/uploads` exists, media is restored from the Synology mirror.
- if `SYNOLOGY_BACKUP_DIR/config-YYYYMMDD-HHMMSS.tar.gz` exists, config is restored from that archive.
2. Restore database:
- `sh deploy/backup/restore_postgres_backup.sh /backup/postgres-YYYYMMDD-HHMMSS.dump`
3. Start app and worker:
- `docker compose --env-file .env.production -f docker-compose.production.yml start app worker`
4. Validate `/healthz` and run one smoke workflow.
Notes:
- Config restore extracts the archived files back into the current repository path.
- Legacy full-archive restores (`uploads-*.tar.gz`, `config-*.tar.gz` beside the dump) are still supported for older backups.
## 4. Retention and recovery targets
- Retention baseline: keep at least 14 days of backups locally.
- Synology copy: replicate each backup artifact set to DS420j mounted path.
- Periodic restore drill: run at least once per release cycle to verify recovery.
- `restore_postgres_backup.sh` still supports legacy archive restore paths for older backup sets.
+1 -1
View File
@@ -37,7 +37,7 @@ This runbook is the operational checklist for releasing and monitoring the trans
- stdout aggregation receives events
- file logs are written under `./data/logs`
5. Create a fresh PostgreSQL backup after successful deployment:
- `sh deploy/backup/create_postgres_backup.sh` (creates local DB dump + local incremental uploads mirror under `DATABASE_BACKUP_DIR`)
- `sh deploy/backup/create_postgres_backup.sh` (creates DB dump plus uploads/prompts backups under `BACKUP_DIR`)
## 3. Rollback triggers and actions
+1 -1
View File
@@ -30,7 +30,7 @@ Settings manages installation-local registries, safe runtime .env settings, and
- Runtime Settings includes an explicit "Other settings not shown here" markdown table listing:
- secrets (`OPENROUTER_API_KEY`, `DATABASE__PASSWORD`)
- high-risk database connection settings (`DATABASE__DRIVER`, `DATABASE__PATH`, `DATABASE__HOST`, `DATABASE__PORT`, `DATABASE__DATABASE`, `DATABASE__USER`)
and deployment/helper keys (`POSTGRES_*`, `CLOUDFLARE_TUNNEL_TOKEN`, `BACKUP_DIR`, `APP_DATA_BACKUP_DIR`, `UPLOADS_BACKUP_DIR`, `BACKUP_RETENTION_DAYS`, `SYNOLOGY_BACKUP_DIR`, `RUNTIME_SETTINGS_ENV_FILE`, `ENV_FILE`, `COMPOSE_FILE`), and directs edits for those keys to the resolved runtime env file path.
and deployment/helper keys (`POSTGRES_*`, `CLOUDFLARE_TUNNEL_TOKEN`, `BACKUP_DIR`, `BACKUP_RETENTION_DAYS`, `RUNTIME_SETTINGS_ENV_FILE`, `ENV_FILE`, `COMPOSE_FILE`) plus legacy/deprecated keys (`DATABASE_BACKUP_DIR`, `APP_DATA_BACKUP_DIR`, `UPLOADS_BACKUP_DIR`, `SYNOLOGY_BACKUP_DIR`), and directs edits for those keys to the resolved runtime env file path.
- Document Types, Person Roles, and Tags support Add/Edit/Delete with existing guardrails.
- Prompts exposes only `transcribe_document.md` for editing and restore-from-backup.
- Home Page Text edits the same Markdown content rendered on `/homepage`.