# Database Rebuild Migration Workflow

This project uses an explicit **export/import rebuild workflow** for schema migration.

Policy:
- Do not add runtime legacy-compatibility write paths.
- Rebuild a fresh target database from current models.
- Export current data/media, then import into the fresh target.

## Commands

### 1) Export current DB + uploads into a bundle

```bash
uv run python tools/export_import_migration.py export --bundle-dir .migration-bundle
```

Optional source overrides:
- `--source-db <path-or-sqlalchemy-url>`
- `--source-upload-dir <path>`

### 2) Import bundle into a fresh target (SQLite or PostgreSQL)

```bash
uv run python tools/export_import_migration.py import --bundle-dir .migration-bundle --target-db .\data\transcription-new.db --target-upload-dir .\data-new
```

PostgreSQL target example:

```bash
uv run python tools/export_import_migration.py import --bundle-dir .migration-bundle --target-db postgresql://transcription:change-me@localhost:5432/transcription --target-upload-dir .\data-new
```

### 3) Verify migration parity and integrity

```bash
uv run python tools/export_import_migration.py verify --source-db .\data\transcription.db --target-db postgresql://transcription:change-me@localhost:5432/transcription
```

The verify command checks:

- row-count parity across migration tables
- orphan-reference checks for `source`, `job`, `job_source`, and `execution_attempt`
- duplicate `(job_id, source_id, attempt_number)` in `execution_attempt`

Exit code:

- `0` when counts and integrity checks pass
- `1` when mismatches or integrity violations are detected

### 4) One-shot export+import

```bash
uv run python tools/export_import_migration.py migrate --bundle-dir .migration-bundle --target-db .\data\transcription-new.db --target-upload-dir .\data-new
```

## What gets migrated

- Tables (in dependency order): `document_type`, `person_role`, `tag`, `document`, `person`, `photo`, `document_person`, `document_tag`, `person_tag`, `job`, `source`, `job_source`, `execution_attempt`.
- Media tree under `UPLOAD_DIR`.

The bundle contains:
- `database.json` (row export)
- `uploads/` (copied media files)

Path normalization during export/import:
- `source.file_path` is normalized to `documents/...` (upload-root-relative POSIX).
- `photo.path` is normalized to `photos/...` (upload-root-relative POSIX).

Legacy V4.x portrait/homepage backfill in the export step:
- If the source DB has no `photo` table, the exporter synthesizes `photo` rows from legacy `person.portrait_path` values and from legacy homepage image files under `UPLOAD_DIR/homepage`.
- Legacy portrait and homepage image files are copied into the unified `UPLOAD_DIR/photos/{photo_id}{suffix}` layout in the migration bundle.
- Legacy homepage markdown is relocated from `UPLOAD_DIR/homepage/homepage.md` to `UPLOAD_DIR/homepage.md`.
- Legacy `person.full_name` values are split into `given_names` + `last_name` for V5.1 schema compatibility.

## Cutover (SQLite -> PostgreSQL)

After importing to a fresh target:
1. Stop app and worker services to freeze writes.
2. Export a migration bundle from the last SQLite state.
3. Import bundle to PostgreSQL target.
4. Run `verify` against source and target before switching runtime.
5. Switch runtime config to PostgreSQL (`DATABASE__DRIVER=postgres` and related `DATABASE__*` values).
6. Start app and worker services.
7. Run smoke checks (`/healthz`, create/upload/process one job).

## Rollback

If verify or smoke checks fail:

1. Stop app and worker services.
2. Revert runtime config to SQLite.
3. Start app and worker against pre-cutover SQLite database.
4. Preserve failed migration bundle and logs for analysis.