diff --git a/README.md b/README.md index e69de29..0447cad 100644 --- a/README.md +++ b/README.md @@ -0,0 +1,8 @@ +# Transcription + +Historical document transcription system. + +## Prompt Artifacts + +Prompt files are stored in `prompts/` and loaded from `PROMPT_DIR` (default: `./prompts`). +The canonical MVP prompt is `prompts/transcribe_document.md`. diff --git a/docs/step2.md b/docs/step2.md index 99d6ffa..4694b1e 100644 --- a/docs/step2.md +++ b/docs/step2.md @@ -1,6 +1,6 @@ -Great—here’s a **detailed implementation plan for Step 2** based on your docs and current project state. +## Step 2: prompts/transcribe_document.md -## Step 2 Goal +### Goal Implement the MVP prompt artifact system by creating a curated transcription prompt file: @@ -121,6 +121,129 @@ In Step 2, ensure docs reflect this and that Step 3 will resolve: --- +## Task-by-Task Execution Checklist + +## Phase A — Scaffold files + +- [ ] **A1. Create prompt directory** + - Path: `prompts/` + - Verify: directory exists at repo root + +- [ ] **A2. Create canonical prompt file** + - Path: `prompts/transcribe_document.md` + - Verify: file exists and is non-empty + +- [ ] **A3. (Recommended) Create prompt library README** + - Path: `prompts/README.md` + - Verify: includes naming + revision conventions + +--- + +## Phase B — Author prompt content (core work) + +- [ ] **B1. Add Purpose section** + - States verbatim historical transcription objective + - Explicitly disallows summarization/paraphrase + +- [ ] **B2. Add Output Contract section** + - Plain text output expectation + - Preserve meaningful structure and reading order + - No fabricated text + +- [ ] **B3. Add Rule Set from `docs/Intent.md`** + - Misspellings/errors: `[sic]` + - Missing words: `[word]` + - Uncertain readings: `[guess?]` + - Illegible regions: `[illegible]` / reason labels + - Crossed-out text: `[deleted: ...]` + - Squeezed-in text: `[inserted: ...]` + - Superscripts/abbrev handling guidance + - Non-text visuals: bracketed descriptive labels + - Marginalia formatting cue + - Rejoin line-break hyphenated words silently + - Ambiguous capitalization policy + - Hierarchical outline numbering preservation + +- [ ] **B4. Add Ambiguity and Confidence policy** + - “Mark uncertainty instead of guessing” + - “Never silently normalize uncertain passages” + +- [ ] **B5. Add Final Self-Check section** + - Checklist for fidelity, uncertainty labeling, and format compliance + +--- + +## Phase C — Add validations (tests) + +- [ ] **C1. Create prompt tests file** + - Path: `tests/test_prompts.py` + +- [ ] **C2. Add existence/health checks** + - Prompt file exists + - Prompt file has content (non-whitespace) + +- [ ] **C3. Add semantic anchor checks** + - Mentions verbatim behavior + - Mentions uncertainty marker pattern (`?` in brackets conceptually) + - Mentions illegible handling + - Mentions deleted/inserted conventions + +- [ ] **C4. Keep tests resilient** + - Avoid exact full-file snapshot assertions + - Assert required concepts, not precise phrasing + +--- + +## Phase D — Documentation alignment + +- [ ] **D1. Update top-level docs/README reference** + - Mention that prompts live in `prompts/` + - Mention Step 3 loads from `PROMPT_DIR` + +- [ ] **D2. Confirm config compatibility** + - `src/transcription/config.py` already uses `prompt_dir = Path("./prompts")` + - No code change needed unless naming/path mismatch appears + +--- + +## Phase E — Verification + +- [ ] **E1. Run targeted test file** + - `uv run pytest tests/test_prompts.py -q` + +- [ ] **E2. Run full suite** + - `uv run pytest -q` + +- [ ] **E3. Confirm no regressions** + - All existing tests still green (expected: previous 20 + new prompt tests) + +--- + +## Phase F — Commit plan (recommended granularity) + +- [ ] **F1. Commit 1: scaffold** + - `prompts/transcribe_document.md` (initial structure) + - `prompts/README.md` (if included) + +- [ ] **F2. Commit 2: finalized prompt content** + - full rule-complete prompt text + +- [ ] **F3. Commit 3: tests + docs alignment** + - `tests/test_prompts.py` + - README/docs mention of prompt artifact pattern + +--- + +## Done Criteria (quick gate) + +- [ ] Canonical prompt exists and is curated for verbatim transcription. +- [ ] Prompt encodes all high-value handling rules from `docs/Intent.md`. +- [ ] Prompt tests pass. +- [ ] Full project tests pass with `uv`. +- [ ] Ready for Step 3 provider integration. + +--- + ## Acceptance Criteria (Definition of Done) Step 2 is complete when all are true: diff --git a/docs/step3.md b/docs/step3.md new file mode 100644 index 0000000..c7097af --- /dev/null +++ b/docs/step3.md @@ -0,0 +1,188 @@ +## Step 3: services/transcription.py + providers/ + +### Objective + +Implement the **AI transcription integration layer** so the app can: + +1. Read the curated prompt from `PROMPT_DIR` +2. Send prompt + image to the configured provider (OpenRouter) +3. Return normalized transcription output (or structured failure) + +This corresponds to MVP Step 3 from `docs/mvp.md`: +- `services/transcription.py` +- `providers/` adapter(s) + +--- + +## Scope for Step 3 + +### In scope +- Provider abstraction and OpenRouter adapter +- Prompt file loading utility in service layer +- Image payload preparation +- One high-level transcription service function usable by Step 4 worker +- Unit tests (mocked provider SDK, no external calls) + +### Out of scope +- Job polling/background loop (Step 4) +- DB status transition orchestration in worker loop (Step 4) +- UI invocation/wiring (Step 5) + +--- + +## Planned Deliverables + +### Source files +- `src/transcription/providers/base.py` +- `src/transcription/providers/openrouter.py` +- `src/transcription/providers/__init__.py` (exports + factory) +- `src/transcription/services/transcription.py` +- `src/transcription/services/__init__.py` (optional export) + +### Tests +- `tests/test_providers_openrouter.py` +- `tests/test_transcription_service.py` + +--- + +## Design Decisions (before coding) + +1. **Provider interface first** + - Define a stable contract independent of SDK specifics. + - Prevent Step 4 from depending on raw SDK response shapes. + +2. **Service returns normalized result object** + - Include: `text`, `provider`, `model`, `raw_error`/exception metadata. + - Worker can map this cleanly to `Transcript` and `JobStatus`. + +3. **Prompt loaded from file at call time** + - Uses `get_settings().prompt_dir / "transcribe_document.md"`. + - Keeps prompt edits hot-swappable without code changes. + +4. **Clear exception boundary** + - SDK/network/model failures become predictable domain exceptions: + - `ProviderError` + - `PromptLoadError` + - `TranscriptionError` (optional top-level wrapper) + +5. **Model resolution policy** + - Use `settings.provider_model` if set + - Otherwise use adapter default constant (e.g., vision-capable model slug) + +--- + +## Task-by-Task Execution Checklist + +## Phase A — Provider contract + +- [ ] Create `src/transcription/providers/base.py` +- [ ] Define protocol/ABC for transcription providers: + - [ ] method signature accepts prompt text + image bytes (or data URL) + mime type + - [ ] returns normalized text result (and optional metadata) +- [ ] Define shared provider exceptions: + - [ ] `ProviderError` + - [ ] optional subclasses (`ProviderAuthError`, `ProviderResponseError`) + +--- + +## Phase B — OpenRouter adapter + +- [ ] Create `src/transcription/providers/openrouter.py` +- [ ] Implement `OpenRouterTranscriptionProvider` with: + - [ ] config-driven API key usage + - [ ] optional referer/title attribution headers + - [ ] model resolution fallback when `provider_model` is unset +- [ ] Implement request building: + - [ ] prompt included as instruction content + - [ ] image included in supported format for vision call +- [ ] Implement response parsing: + - [ ] extract final transcript text from SDK response + - [ ] validate non-empty text +- [ ] Wrap SDK failures into `ProviderError` with clean message + +--- + +## Phase C — Provider factory + +- [ ] Update `src/transcription/providers/__init__.py` +- [ ] Add `get_transcription_provider()` factory: + - [ ] reads `settings.provider` + - [ ] returns OpenRouter adapter for `openrouter` + - [ ] raises explicit error for unsupported provider values + +--- + +## Phase D — Transcription service (Step 3 core) + +- [ ] Create `src/transcription/services/transcription.py` +- [ ] Add prompt loader function: + - [ ] default file: `transcribe_document.md` + - [ ] raises `PromptLoadError` on missing/empty file +- [ ] Add image loader/validator: + - [ ] path existence check + - [ ] allowed mime detection (`.jpg/.jpeg/.png/.tiff/.pdf` policy aligned to MVP) +- [ ] Add high-level function (name example): + - [ ] `transcribe_document_image(image_path, prompt_name="transcribe_document.md")` + - [ ] loads prompt + image + - [ ] calls provider from factory + - [ ] returns normalized transcription result object +- [ ] Add structured logging at key boundaries: + - [ ] prompt loaded + - [ ] provider invoked + - [ ] success/failure outcome (no sensitive data in logs) + +--- + +## Phase E — Tests (mocked, deterministic) + +## `tests/test_providers_openrouter.py` +- [ ] test adapter initializes from settings +- [ ] test model fallback when `provider_model is None` +- [ ] test referer/title options are included when set +- [ ] test successful SDK response parses transcript text +- [ ] test SDK exception maps to `ProviderError` +- [ ] test empty/invalid response maps to `ProviderError` + +## `tests/test_transcription_service.py` +- [ ] test prompt loader reads canonical prompt file +- [ ] test missing prompt raises `PromptLoadError` +- [ ] test transcription function loads file and calls provider once +- [ ] test image path missing raises clear error +- [ ] test provider error is propagated/wrapped predictably +- [ ] test returned result includes transcript text and metadata + +> Keep these unit tests mocked (no real OpenRouter calls in default suite). + +--- + +## Phase F — Verification commands + +- [ ] `uv run pytest tests/test_providers_openrouter.py -q` +- [ ] `uv run pytest tests/test_transcription_service.py -q` +- [ ] `uv run pytest -q` + +--- + +## Implementation Notes / Guardrails + +- Avoid coupling Step 3 service to DB models directly (that belongs in Step 4 orchestration). +- Do not silently swallow provider errors. +- Keep prompt filename stable (`transcribe_document.md`) unless explicitly parameterized. +- Keep request/response normalization inside provider adapter, not worker/UI layers. + +--- + +## Definition of Done (Step 3) + +Step 3 is done when: + +1. Provider abstraction exists and OpenRouter adapter is implemented. +2. Service can transcribe a local image using prompt file content. +3. Failures are returned as structured exceptions, not raw SDK traceback noise. +4. Unit tests for provider and service pass. +5. Full suite remains green under `uv run pytest -q`. +6. Step 4 can call a single service function to process queued jobs. + +--- + +If you want, I can now convert this into a **PR-ready markdown checklist** (same format as Step 2) and then implement it once you confirm. \ No newline at end of file diff --git a/prompts/README.md b/prompts/README.md new file mode 100644 index 0000000..0b3e454 --- /dev/null +++ b/prompts/README.md @@ -0,0 +1,13 @@ +# Prompt Artifacts + +This directory stores transcription prompts as individual Markdown artifacts. + +## Conventions +- Keep one prompt per file. +- Use stable, descriptive snake_case file names. +- Prefer incremental edits to a single prompt per change for clean history. +- Keep prompts human-readable and policy-focused. +- Do not store secrets in prompt files. + +## Current Prompt +- `transcribe_document.md`: baseline verbatim transcription policy for historical documents. diff --git a/prompts/transcribe_document.md b/prompts/transcribe_document.md new file mode 100644 index 0000000..c136e57 --- /dev/null +++ b/prompts/transcribe_document.md @@ -0,0 +1,72 @@ +# Historical Document Verbatim Transcription Prompt + +## Purpose +Transcribe the provided historical document image as a faithful **verbatim** transcript. +Do not summarize. Do not paraphrase. Do not modernize style. + +## Output Contract +- Return only the transcription text. +- Preserve original wording, punctuation, and meaningful structure. +- Keep line/section flow readable while preserving intent and document organization. +- Never invent missing content. + +## Rules for Ambiguous or Damaged Text + +### Misspellings and original errors +- Preserve original spelling. +- Add `[sic]` immediately after an evident original error. + +### Missing words or clear omissions +- If a single missing word is obvious from context, insert it in square brackets. +- Example form: `[to]` + +### Uncertain readings +- If best-effort interpretation is uncertain, use bracketed guess with question mark. +- Example form: `[Boston?]` + +### Completely illegible text +- Use a clear bracketed label. +- Preferred forms: `[illegible]`, `[torn]`, `[ink blot]`, `[remainder of page torn]` + +### Crossed-out text +- Preserve it using: `[deleted: ...]` + +### Squeezed-in or above-line insertions +- Preserve it using: `[inserted: ...]` + +### Superscripts and abbreviations +- Bring superscript letters down to baseline text. +- Expand only when clearly intended; if expanded, place added letters in brackets. + +### Non-text visual elements +- Describe briefly in square brackets. +- Example forms: `[wax notary seal attached here]`, `[sketch of a fort layout]` + +### Marginalia and side notes +- Signal location before the note text. +- Example form: `[written in left margin: ...]` + +### Line-break hyphenation +- Rejoin words split across line breaks when they are clearly one word. +- Remove only line-break hyphens used for wrapping. + +### Ambiguous capitalization +- Prefer modern capitalization only when uncertainty is high. +- Preserve clearly intentional archaic capitalization. + +### Hierarchical outlines and numbering +- Preserve original numbering characters exactly (including roman numerals and unusual suffixes). +- Preserve indentation levels. +- Do not silently correct sequence mistakes; if clearly erroneous, preserve and use `[sic]` where appropriate. + +## Confidence and Integrity Policy +- When uncertain, mark uncertainty explicitly rather than guessing silently. +- If text cannot be read, use a bracketed illegibility label instead of fabrication. +- Do not add commentary outside the transcription. + +## Final Self-Check +Before finalizing, ensure: +1. The transcript is verbatim and not summarized. +2. Uncertain/illegible areas are explicitly marked. +3. Crossed-out and inserted text are preserved with required tags. +4. Structure/ordering is preserved as faithfully as possible. diff --git a/tests/test_prompts.py b/tests/test_prompts.py new file mode 100644 index 0000000..fdf2231 --- /dev/null +++ b/tests/test_prompts.py @@ -0,0 +1,42 @@ +"""Tests for prompt artifacts in prompts/.""" + +from pathlib import Path + + +PROMPT_PATH = Path("prompts/transcribe_document.md") + + +def _prompt_text() -> str: + """Read prompt text from the canonical prompt file.""" + return PROMPT_PATH.read_text(encoding="utf-8") + + +class TestPromptArtifact: + """Verify prompt artifact presence and baseline semantics.""" + + def test_prompt_file_exists(self): + """Canonical transcription prompt file exists.""" + assert PROMPT_PATH.exists() + + def test_prompt_file_is_not_empty(self): + """Canonical prompt file has non-whitespace content.""" + text = _prompt_text() + assert text.strip() + + def test_prompt_mentions_verbatim_behavior(self): + """Prompt explicitly enforces verbatim transcription behavior.""" + text = _prompt_text().lower() + assert "verbatim" in text + assert "do not summarize" in text + + def test_prompt_includes_uncertainty_and_illegible_markers(self): + """Prompt contains conventions for uncertainty and illegible text.""" + text = _prompt_text().lower() + assert "[boston?]" in text + assert "[illegible]" in text + + def test_prompt_includes_deleted_and_inserted_conventions(self): + """Prompt contains conventions for deleted and inserted text.""" + text = _prompt_text().lower() + assert "[deleted:" in text + assert "[inserted:" in text