transcription/docs/roadmap_plan.md

Roadmap Plan (Starting at V6.0)

This roadmap starts at V6.0 and tracks forward-looking work only.

V6.0 - Hosting Migration

Objective: move from local-only operation to secure, stable remote hosting. Status: Completed

Detailed plan: v6_0_hosting_migration_plan.md

Scope

1. Containerize app runtime for production deployment.
2. Run PostgreSQL in Docker and migrate from SQLite.
3. Add Cloudflare Tunnel exposure with Access protection.
4. Add operational safeguards (health checks, restart policies, backups).

Deliverables

• Production-ready docker-compose deployment for app + database + tunnel.
• Environment-based configuration for DB, uploads, prompts, and logging.
• Verified data migration path into PostgreSQL.
• Runbook updates for deploy, rollback, and backup/restore.

Exit Criteria

• /healthz reports healthy app and worker in deployed environment.
• One end-to-end document -> source -> job workflow succeeds remotely.
• Backup and restore procedure is tested.

Accomplished

1. Delivered production Docker deployment with split app/worker, postgres, and cloudflared.
2. Landed SQLite -> PostgreSQL migration tooling and runbook coverage.
3. Added production health/reliability wiring and operational runbooks for deploy/rollback/recovery.
4. Established host-visible backup workflow and restore path for PostgreSQL plus media/config assets.

V6.1 - Testing and Refinement

Objective: improve navigation and operational workflows after user feedback. Status: Completed

Scope

1. Make Document Detail the primary source-page workspace:
   • Use Source-style pan/zoom + previous/next page controls.
   • Move editable revision controls into Document Detail.
   • Move archival/system metadata to dedicated Document Info route.
2. Simplify top navigation:
   • Remove top-level Tags and Sources entries.
   • Retire the Tags page and the global Source Asset Records entry flow.
3. Improve list/detail clarity:
   • Add Document transcription status to Archival Documents list.
   • Add Document Date in People Detail -> Linked Documents table.
4. Add worker-backed Settings maintenance runs:
   • Add maintenance_run persistence (id, job_type, status, started_at, finished_at, triggered_by, summary, log_path, error_detail).
   • Add Run Backup and Run Storage Reconciliation actions that enqueue runs and execute in the worker.
   • Add run history with status, duration, summary, and log view/download.
   • Defer daily/weekly scheduling controls to V6.2.

Accomplished

1. Refactored Document Detail into the primary source-page workspace (pan/zoom viewer, previous/next page navigation, editable revision flow) and moved archival/system metadata to Document Info.
2. Simplified top navigation by removing Tags/Sources entries and retiring the Tags page/global Source Asset Records flow.
3. Improved data clarity with document transcription status in Archival Documents and Document Date in People Detail linked documents.
4. Implemented queue-backed maintenance operations (maintenance_run model/service/worker/UI) with run history and log view/download.
5. Hardened runtime settings operations in production:
   • runtime settings writes target mounted .env.production,
   • fallback write path for single-file bind mounts,
   • explicit hidden/deployment-key disclosure in Settings UI.
6. Simplified backup configuration and behavior:
   • standardized on BACKUP_DIR + BACKUP_RETENTION_DAYS,
   • backup script uses DATABASE__* persistence keys,
   • compose maps Postgres container init values from DATABASE__*,
   • env contract drift tests now guard .env.production.example.

V6.2 - Reporting Features

[ More thoughts on additional features:

• Ability to search within a document to find potential users to add to the People table.
• Ability to search within a document to find dates to associate with people (e.g., John Smith was born on July 7, 1899; or "In February 1942, Martha and I drove to Wisconsin", "We took a vacation to Alaska in the summer of 1987", etc ]

Objective: improve research value with person-centric outputs.

Scope

1. Person timeline views using document dates and linked records.
2. AI-assisted biography/family-history generation from curated sources.
3. Exportable report views (human-readable, print-oriented).

Deliverables

• Timeline UI and service queries with clear ordering/filters.
• Prompted narrative generation workflow using existing evidence-safe patterns.
• Saved/printable report presentation for review and sharing.

Exit Criteria

• Timelines are reproducible from persisted records.
• Narrative generation is traceable to source records and prompts.
• Reports can be reviewed without modifying archival source data.

V6.3 - Access Control and Multi-User Readiness

[ More thoughts on user accounts:

• Create a generic "view only" user that does not have the rights to alter any of the data
• Limit user accounts access to data by Tag. I have distant family members that I would want to share the transcribed data with, but they would only be interested in a subset of it. For example my Cochran cousins would have no interest in Lancaster documents, so limit the Cochra Clan cousins to view-only access to documents tagged "cochran clan" ]

Objective: prepare for managed collaboration beyond single-user operation.

Scope

1. Introduce application-level authentication.
2. Add role-based authorization (admin/editor/contributor/viewer).
3. Add audit visibility for user-attributed write actions.

Deliverables

• User identity model and login/session flow.
• Route/page/service authorization enforcement.
• Audit metadata for sensitive create/update/delete workflows.

Exit Criteria

• Unauthorized operations are blocked consistently across UI/API.
• Role policies are enforced by deterministic tests.
• User-attributed changes are visible for audit/review.

V7 or later - Scalability and Multi-Tenant Direction (Optional)

Objective: keep architecture ready for broader deployment footprints.

Scope

1. Evaluate per-tenant or per-user data partitioning strategy.
2. Formalize connection/runtime strategy for tenant-aware DB selection.
3. Expand operational telemetry for throughput and cost monitoring.

Deliverables

• Decision document for tenancy model and migration strategy.
• Prototype-safe runtime boundary for selecting data targets.
• Monitoring baseline for queue depth, job latency, and provider cost trends.

Exit Criteria

• Selected tenancy strategy is documented and testable.
• Operational metrics support capacity planning.

Planning Notes

• Keep architecture, schema, and UI contracts synchronized in docs/ as each version lands.
• Prefer explicit schema migration over runtime compatibility write paths.
• Preserve evidence/provenance guarantees when adding new AI-powered features.