generated from john/python-template
This commit is contained in:
+29
-67
@@ -1,75 +1,46 @@
|
||||
# Backup and Restore (V6.0 Phase 4)
|
||||
# Backup and Restore (V6.1)
|
||||
|
||||
This guide defines operational backup/restore for clean-slate recovery of the Docker runtime with host-level backups that Synology Drive can replicate.
|
||||
This guide defines operational backup/restore for clean-slate recovery of the Docker runtime using a host-visible backup folder.
|
||||
|
||||
## 1. Backup artifacts (clean-slate recovery set)
|
||||
## 1. Backup artifacts
|
||||
|
||||
- Primary local backup location: `DATABASE_BACKUP_DIR` (recommended production value: `/backup`)
|
||||
- Local files created per run:
|
||||
- Backup target root: `BACKUP_DIR` (recommended production value: `/backup`)
|
||||
- Database artifact per run:
|
||||
- `postgres-YYYYMMDD-HHMMSS.dump` (PostgreSQL custom dump via `pg_dump -Fc`)
|
||||
- `backup-YYYYMMDD-HHMMSS.manifest` (artifact index)
|
||||
- Optional Synology files created per run (when `SYNOLOGY_BACKUP_DIR` is set):
|
||||
- `postgres-YYYYMMDD-HHMMSS.dump` (copied from local)
|
||||
- `config-YYYYMMDD-HHMMSS.tar.gz` (deployment config snapshot: `.env.production`, `docker-compose.production.yml`, `deploy/cloudflared/config.yml` when present)
|
||||
- `logs-YYYYMMDD-HHMMSS.tar.gz` (snapshot of `/app/data/logs`)
|
||||
- `backup-YYYYMMDD-HHMMSS.manifest`
|
||||
- Local app-data mirror (updated each run):
|
||||
- `data/**`
|
||||
- Local incremental media mirror (copied only if missing):
|
||||
- `uploads/homepage.md`
|
||||
- `uploads/documents/**`
|
||||
- `uploads/photos/**`
|
||||
- `backup-YYYYMMDD-HHMMSS.manifest` (run manifest)
|
||||
- Media/config mirrors under `BACKUP_DIR`:
|
||||
- `uploads/**` (incremental copy: new files only)
|
||||
- `prompts/**` (prompt directory mirror)
|
||||
|
||||
Retention:
|
||||
|
||||
- `BACKUP_RETENTION_DAYS` applies to `postgres-*.dump` and `backup-*.manifest` files.
|
||||
|
||||
## 2. Creating backups
|
||||
|
||||
Use the scripted command from the repository root:
|
||||
Run from repository root:
|
||||
|
||||
```bash
|
||||
sh deploy/backup/create_postgres_backup.sh
|
||||
```
|
||||
|
||||
Environment overrides:
|
||||
Environment variables used by the backup script:
|
||||
|
||||
- `BACKUP_DIR` (default `DATABASE_BACKUP_DIR`, then `./data/backups`)
|
||||
- `APP_DATA_BACKUP_DIR` (default `${BACKUP_DIR}/data`)
|
||||
- `UPLOADS_BACKUP_DIR` (default `${BACKUP_DIR}/uploads`)
|
||||
- `BACKUP_DIR` (default `./data/backups`)
|
||||
- `BACKUP_RETENTION_DAYS` (default `14`)
|
||||
- `SYNOLOGY_BACKUP_DIR` (if set, dump/config/log/manifest are copied to this mounted path and media is synced incrementally)
|
||||
- `ENV_FILE` (default `.env.production`)
|
||||
- `COMPOSE_FILE` (default `docker-compose.production.yml`)
|
||||
- `UPLOAD_DIR` (default `/app/uploads`)
|
||||
- `PROMPT_DIR` (default `/app/prompts`)
|
||||
- `POSTGRES_HOST` (default `postgres`)
|
||||
- `POSTGRES_PORT` (default `5432`)
|
||||
- `POSTGRES_DB` (required)
|
||||
- `POSTGRES_USER` (required)
|
||||
- `POSTGRES_PASSWORD` (required)
|
||||
|
||||
Recommended production setup:
|
||||
|
||||
- Mount a host-visible folder into `/backup` for both `app` and `worker` services.
|
||||
- Set `DATABASE_BACKUP_DIR=/backup` in `.env.production`.
|
||||
- Leave `SYNOLOGY_BACKUP_DIR` empty when Synology Drive Client handles replication from the host folder.
|
||||
|
||||
Example with Synology mount:
|
||||
|
||||
```bash
|
||||
SYNOLOGY_BACKUP_DIR=/mnt/synology-backups sh deploy/backup/create_postgres_backup.sh
|
||||
```
|
||||
|
||||
## 2.1 Optional direct Synology mount (LXC)
|
||||
|
||||
This section is only needed when using direct container-side Synology copy via `SYNOLOGY_BACKUP_DIR`.
|
||||
Manual `mount` commands are lost after reboot unless persisted.
|
||||
|
||||
Use the template (copy, edit placeholders, then run):
|
||||
|
||||
```bash
|
||||
cp deploy/backup/mount_synology_cifs.example.sh /root/mount_synology_cifs.sh
|
||||
nano /root/mount_synology_cifs.sh
|
||||
sh /root/mount_synology_cifs.sh
|
||||
```
|
||||
|
||||
Then add the printed `/etc/fstab` line (with your real values) so the mount survives reboot.
|
||||
|
||||
Recommended pattern:
|
||||
|
||||
- Keep NAS credentials in a local file like `/etc/samba/credentials/transcription-synology` with `chmod 600`.
|
||||
- Keep `SYNOLOGY_BACKUP_DIR` in `.env.production` aligned to that mount point (for example `/mnt/synology-backups`).
|
||||
- The script will also read `SYNOLOGY_BACKUP_DIR` from `ENV_FILE` when not exported in the shell.
|
||||
- Mount a host-visible folder into `/backup` for both `app` and `worker`.
|
||||
- Set `BACKUP_DIR=/backup` in `.env.production`.
|
||||
- Use host-level tooling (for example Synology Drive Client on the host) to replicate that folder externally.
|
||||
|
||||
## 3. Restoring from backup
|
||||
|
||||
@@ -77,21 +48,12 @@ Restore requires downtime for app + worker writes.
|
||||
|
||||
1. Stop app and worker:
|
||||
- `docker compose --env-file .env.production -f docker-compose.production.yml stop app worker`
|
||||
2. Restore:
|
||||
- `sh deploy/backup/restore_postgres_backup.sh ./data/backups/postgres-YYYYMMDD-HHMMSS.dump`
|
||||
- if `SYNOLOGY_BACKUP_DIR/uploads` exists, media is restored from the Synology mirror.
|
||||
- if `SYNOLOGY_BACKUP_DIR/config-YYYYMMDD-HHMMSS.tar.gz` exists, config is restored from that archive.
|
||||
2. Restore database:
|
||||
- `sh deploy/backup/restore_postgres_backup.sh /backup/postgres-YYYYMMDD-HHMMSS.dump`
|
||||
3. Start app and worker:
|
||||
- `docker compose --env-file .env.production -f docker-compose.production.yml start app worker`
|
||||
4. Validate `/healthz` and run one smoke workflow.
|
||||
|
||||
Notes:
|
||||
|
||||
- Config restore extracts the archived files back into the current repository path.
|
||||
- Legacy full-archive restores (`uploads-*.tar.gz`, `config-*.tar.gz` beside the dump) are still supported for older backups.
|
||||
|
||||
## 4. Retention and recovery targets
|
||||
|
||||
- Retention baseline: keep at least 14 days of backups locally.
|
||||
- Synology copy: replicate each backup artifact set to DS420j mounted path.
|
||||
- Periodic restore drill: run at least once per release cycle to verify recovery.
|
||||
- `restore_postgres_backup.sh` still supports legacy archive restore paths for older backup sets.
|
||||
|
||||
@@ -37,7 +37,7 @@ This runbook is the operational checklist for releasing and monitoring the trans
|
||||
- stdout aggregation receives events
|
||||
- file logs are written under `./data/logs`
|
||||
5. Create a fresh PostgreSQL backup after successful deployment:
|
||||
- `sh deploy/backup/create_postgres_backup.sh` (creates local DB dump + local incremental uploads mirror under `DATABASE_BACKUP_DIR`)
|
||||
- `sh deploy/backup/create_postgres_backup.sh` (creates DB dump plus uploads/prompts backups under `BACKUP_DIR`)
|
||||
|
||||
## 3. Rollback triggers and actions
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ Settings manages installation-local registries, safe runtime .env settings, and
|
||||
- Runtime Settings includes an explicit "Other settings not shown here" markdown table listing:
|
||||
- secrets (`OPENROUTER_API_KEY`, `DATABASE__PASSWORD`)
|
||||
- high-risk database connection settings (`DATABASE__DRIVER`, `DATABASE__PATH`, `DATABASE__HOST`, `DATABASE__PORT`, `DATABASE__DATABASE`, `DATABASE__USER`)
|
||||
and deployment/helper keys (`POSTGRES_*`, `CLOUDFLARE_TUNNEL_TOKEN`, `BACKUP_DIR`, `APP_DATA_BACKUP_DIR`, `UPLOADS_BACKUP_DIR`, `BACKUP_RETENTION_DAYS`, `SYNOLOGY_BACKUP_DIR`, `RUNTIME_SETTINGS_ENV_FILE`, `ENV_FILE`, `COMPOSE_FILE`), and directs edits for those keys to the resolved runtime env file path.
|
||||
and deployment/helper keys (`POSTGRES_*`, `CLOUDFLARE_TUNNEL_TOKEN`, `BACKUP_DIR`, `BACKUP_RETENTION_DAYS`, `RUNTIME_SETTINGS_ENV_FILE`, `ENV_FILE`, `COMPOSE_FILE`) plus legacy/deprecated keys (`DATABASE_BACKUP_DIR`, `APP_DATA_BACKUP_DIR`, `UPLOADS_BACKUP_DIR`, `SYNOLOGY_BACKUP_DIR`), and directs edits for those keys to the resolved runtime env file path.
|
||||
- Document Types, Person Roles, and Tags support Add/Edit/Delete with existing guardrails.
|
||||
- Prompts exposes only `transcribe_document.md` for editing and restore-from-backup.
|
||||
- Home Page Text edits the same Markdown content rendered on `/homepage`.
|
||||
|
||||
Reference in New Issue
Block a user