## MVP Definition: Historical Document Transcription System

### 1. MVP Objective
Deliver the thinnest possible end-to-end vertical slice — a user uploads an image of a document, the system transcribes it via the OpenRouter Python SDK, and the user reads the resulting transcript — with just enough persistence and structure to validate the core value proposition: *can AI-driven transcription, guided by curated prompts, produce useful verbatim transcripts of historical family documents?*  

The MVP deliberately defers full-text search, export, revision history, MongoDB, and timeline assembly. These are additive features that don't need validation before the core transcription loop is proven.

---

### 2. Core User Story
*As a family historian, I can upload a photo of a historical document, wait for it to be transcribed, and read the verbatim transcript — so I can evaluate whether this system will work for my thousands of documents.*

---

### 3. In-Scope Requirements (from ```requirements.md```)  

| Requirement | ID | MVP Rationale |
| --- |  --- |  --- |
| End-to-end transcription with lifecycle state | REQ-0 | This is the MVP. |
| Upload one or more images from the web UI | REQ-1 | Core entry point. MVP supports single-image upload (multi-image is a stretch goal). |
| Asynchronous processing → transcription or failure | REQ-2 | Validates the AI transcription pipeline. |
| Persist and expose job states (queued → processing → transcribed/failed) | 	REQ-3 | Minimum feedback loop for the user. |
| Persist transcription output and failure details | REQ-4 | User must be able to read the result. |
| UI views for status and transcript reading | REQ-5 | The user needs to see what happened. |
| Background processing to keep UI responsive | REQ-6 | Essential for usability during long AI calls. |
| Centralized config and logging at startup | REQ-8 | Small effort, high payoff for debugging. |
| Store transcription prompts as Markdown files | REQ-12 | Core to the Prompt Curation Policy in intent.md. Start with a single prompt file. |


### Deferred to Post-MVP
| Requirement | ID | Why Deferred |
| ---  | --- | --- |
| Lifespan-owned runtime resources (engine, session factory, etc.) | REQ-7 | Important for production robustness, but a simple global or module-level setup is adequate for MVP validation. |
| Docker Compose (app + PostgreSQL + optional MongoDB) | REQ-9 | MVP runs locally with SQLite to eliminate container overhead during rapid iteration. PostgreSQL migration is Stage 1 hardening. |
| Explicit, opt-in schema bootstrap | REQ-10 | MVP uses auto-create-tables at startup (SQLModel create_all). Production schema discipline comes after the model stabilizes. |
| Service-backed persistence for core data | REQ-11 | MVP uses a thin repository layer over SQLite. Full service abstraction follows once the domain model is proven. |

---

### 4. MVP Feature Set
#### Feature 1: Document Upload (UI)
* A single NiceGUI page with a file-upload widget (accepts .jpg, .png, .tiff, .pdf).
* On upload: save the file to a local uploads/ directory, create a Document record, create a Job record with status queued.
* Minimal metadata capture: original filename, upload timestamp.

#### Feature 2: Asynchronous Transcription Worker
* An in-process background worker (Python asyncio task or BackgroundTasks) that:
    1. Picks up queued jobs.
    2. Transitions status to processing.
    3. Sends the image + the curated Markdown prompt to an AI vision model via OpenRouter.
    4. On success: saves the transcript text, transitions to transcribed.
    5. On failure: saves the error detail, transitions to failed.

#### Feature 3: Transcription Prompt (Markdown Asset)
* A single Markdown file (prompts/transcribe_document.md) encoding the verbatim transcription rules from intent.md (the Document Issues table, scholarly guidelines, etc.).
* The worker reads this file at invocation time and injects it as the system/user prompt.

#### Feature 4: Job Status & Transcript Viewer (UI)
* A job list page showing all jobs with their current status (queued / processing / transcribed / failed).
* A transcript detail page showing:
    * The original uploaded image (rendered inline).
    * The transcription text (or the failure reason).
    * Timestamp metadata.

#### Feature 5: Minimal Persistence (SQLite + SQLModel)
* Three tables/models:
    * Document: id, filename, file_path, uploaded_at.
    * Job: id, document_id (FK), status, created_at, updated_at.
    * Transcript: id, job_id (FK), text, error_detail, created_at.
* SQLite database file stored locally. Auto-created on first startup.

#### Feature 6: Centralized Configuration
* A single config.py (or Pydantic BaseSettings) loading:
    * PROVIDER (fixed to openrouter for MVP)
    * OPENROUTER_API_KEY (required)
    * PROVIDER_MODEL (default: OpenRouter model slug for vision transcription)
    * OPENROUTER_HTTP_REFERER (optional; app attribution)
    * OPENROUTER_APP_TITLE (optional; app attribution)
    * DATABASE_URL (default: sqlite:///./transcription.db)
    * UPLOAD_DIR (default: ./uploads)
    * PROMPT_DIR (default: ./prompts)

#### Feature 7: MVP Dependency Baseline (OpenRouter-Centric)
* Runtime dependencies:
    * openrouter (official OpenRouter Python SDK)
    * pydantic
    * pydantic-settings
    * sqlmodel
* Explicitly out of MVP runtime dependencies:
    * google-genai (deferred until/if Gemini is introduced post-MVP)

---

### 5. MVP Architecture (Simplified)

```Apply
┌─────────────────────────────────────────────┐
│              NiceGUI Web UI                 │
│  ┌──────────────┐   ┌───────────────────┐   │
│  │ Upload Page  │   │ Jobs / Transcript │   │
│  └──────┬───────┘   └───────┬───────────┘   │
│         │                   │               │
│         ▼                   ▼               │
│     ┌───────────────────────────┐           │
│     │   Application Service     │           │
│     │  (upload, job lifecycle)  │           │
│     └─────┬─────────────┬───────┘           │
│           │             │                   │
│     ┌─────▼─────┐ ┌─────▼───────────────┐   │
│     │ SQLite DB │ │ Background Worker   │   │
│     │ (SQLModel)│ │ → AI Vision Provider│   │
│     └───────────┘ └─────────────────────┘   │
│                          │                  │
│                    ┌─────▼──────┐           │
│                    │ prompts/   │           │
│                    │ *.md files │           │
│                    └────────────┘           │
└─────────────────────────────────────────────┘
```

---

#### 6. Proposed File Structure

```Apply
project-root/
├── docs/                          # (existing)
├── prompts/
│   └── transcribe_document.md     # curated transcription prompt
├── src/
│   └── transcription/
│       ├── __init__.py
│       ├── app.py                 # FastAPI + NiceGUI app entrypoint
│       ├── config.py              # Pydantic BaseSettings
│       ├── models.py              # SQLModel: Document, Job, Transcript
│       ├── db.py                  # engine, session, create_all
│       ├── providers/
│       │   ├── __init__.py
│       │   ├── base.py            # provider interface (transcribe contract)
│       │   ├── openrouter.py      # OpenRouter via official Python SDK
│       ├── services/
│       │   ├── __init__.py
│       │   ├── upload.py          # save file + create records
│       │   └── transcription.py   # call provider, update job
│       ├── worker.py              # background job loop
│       └── ui/
│           ├── __init__.py
│           ├── upload_page.py     # NiceGUI upload page
│           └── jobs_page.py       # NiceGUI job list + detail
├── tests/
│   ├── test_models.py
│   ├── test_upload.py
│   └── test_transcription.py
├── pyproject.toml
└── README.md
```

---

#### 7. MVP Validation Criteria
The MVP is considered validated when:

1. ✅ A user can upload an image of a document through the browser.
2. ✅ The system asynchronously sends the image to the configured AI vision model with the curated prompt.
3. ✅ The transcript (or failure reason) is persisted and visible in the UI.
4. ✅ The transcription follows verbatim scholarly rules defined in intent.md (spot-checked by the user on real family documents).
5. ✅ The transcription prompt is stored as a standalone Markdown file and can be edited without code changes.
6. ✅ Job status transitions are visible: queued → processing → transcribed/failed.

---

### 8. Key Feedback Questions the MVP Should Answer
These are the real unknowns this MVP exists to resolve:

| #	| Question | How We Learn |
| --- | --- | --- |
| 1	| Is AI transcription quality good enough for this document corpus?	| User reviews 20–50 real transcriptions against originals. |
| 2	| Does the verbatim prompt produce scholarly-quality output, or does it need major rework?	| Compare output to the Document Issues table rules in intent.md. |
| 3	| What document types are hardest (old cursive, faded ink, pencil, postcards)?	| Track which uploads produce failed or low-quality results. |
| 4	| Is single-image upload sufficient, or is batch upload needed early?	| User friction during real scanning sessions. |
| 5	| What metadata is missing that the user wishes they could capture at upload time?	| User feedback after processing real batches. |

---

#### 9. What Comes After MVP (Immediate Post-MVP)
Once the core transcription loop is validated, the next priorities (aligned to Architecture Stage 1) are:

1. **Multi-image upload** — process a batch from a scanning session.
2. **PostgreSQL migration** — swap SQLite for containerized PostgreSQL (REQ-9, REQ-10).
3. **Revision history** — allow the user to edit/correct transcripts with immutable version tracking.
4. **Full-text search** — search across all accepted transcripts.
5. **Repository/service layer formalization** — proper ports/adapters as the domain model stabilizes.
6. **Docker Compose deployment** — containerize the app for reproducible operation.

---

#### 10. Implementation Approach
Recommended build order for the MVP (each step produces a testable increment):

| Step | Deliverable | Validates |
| --- | --- | --- |
| 1 | config.py + models.py + db.py — data layer with SQLite | Schema and config foundation |
| 2 | prompts/transcribe_document.md — curated prompt from intent.md | Prompt asset pattern |
| 3 | services/transcription.py + providers/ — call AI vision provider with prompt + image | Core AI integration |
| 4 | services/upload.py + worker.py — upload handling + background job loop | End-to-end pipeline (CLI-testable) |
| 5 | ui/upload_page.py + ui/jobs_page.py — NiceGUI pages | User-facing interface |
| 6 | tests/ — unit + integration tests	Automated verification |

This MVP is deliberately narrow: **one prompt, one provider (OpenRouter), one user, one image at a time, SQLite, no containers**. Every omission is intentional — the goal is to get real family documents through the transcription pipeline as fast as possible and let the quality of the output guide every subsequent decision.