transcription/docs/step2.md

Step 2: prompts/transcribe_document.md

Goal

Implement the MVP prompt artifact system by creating a curated transcription prompt file:

• prompts/transcribe_document.md

This step primarily satisfies:

• REQ-12: prompts stored as individual Markdown artifacts
• MVP Feature 3: prompt-driven verbatim transcription behavior grounded in docs/Intent.md

---

Scope for Step 2

In scope

1. Create prompt artifact directory and first prompt file.
2. Encode transcription rules from docs/Intent.md into a model-facing prompt.
3. Define stable prompt structure so future revisions are easy to diff/review.
4. Add lightweight tests that validate artifact presence and baseline quality constraints.
5. Update docs/README references so Step 3 can consume prompt file directly.

Out of scope

• Provider integration logic (Step 3)
• Worker/job orchestration (Step 4)
• UI behavior (Step 5)

---

Proposed Deliverables

1. prompts/transcribe_document.md

   • production prompt text for historical document transcription

2. prompts/README.md (recommended)

   • conventions for prompt files, revision policy, naming

3. tests/test_prompts.py (recommended)

   • artifact existence + structure checks

4. Small docs update (README or docs reference)

   • indicate that prompts are file-based and loaded from PROMPT_DIR

---

Detailed Work Breakdown

1) Create prompt artifact folder and canonical file

• Add prompts/ at repo root.
• Add transcribe_document.md as the first curated artifact.
• Keep filename stable; this becomes the default in Step 3 unless overridden.

2) Author prompt content using a strict, sectioned format

Use section headers so future diffs are clean and policy changes are isolated.

Suggested sections:

1. Purpose

   • verbatim scholarly transcription of historical documents

2. Output requirements

   • plain text only
   • no summaries, no paraphrasing
   • preserve reading order and meaningful structure

3. Core fidelity rules

   • preserve original wording and punctuation
   • don’t silently normalize grammar/spelling
   • no invented content

4. Issue-handling rules (mapped from Intent table)

   • misspellings with [sic]
   • missing words with [word]
   • uncertainty with [guess?]
   • illegible with [illegible] / reason tags
   • crossed-out text as [deleted: ...]
   • inserted text as [inserted: ...]
   • superscripts handling guidance
   • non-text elements as [description]
   • marginalia format [written in left margin: ...]
   • line-break hyphen rejoin behavior
   • capitalization policy
   • hierarchical outline preservation (including unusual numbering)

5. Confidence/ambiguity policy

   • prefer explicit uncertainty markers over hallucination

6. Final self-checklist for model

   • did I preserve structure?
   • did I mark uncertain text?
   • did I avoid silent corrections?

3) Add prompt-library conventions (prompts/README.md)

Recommended conventions:

• one prompt per file
• snake_case names
• each file starts with purpose + behavior contract
• iterative edits, one prompt per PR where possible
• no secrets in prompt files

4) Add tests for prompt assets (tests/test_prompts.py)

Keep tests robust but not brittle.

Recommended tests:

1. test_prompt_file_exists
2. test_prompt_file_is_not_empty
3. test_prompt_mentions_verbatim_behavior
4. test_prompt_includes_uncertainty_and_illegible_markers
5. test_prompt_includes_deleted_and_inserted_conventions

Avoid exact full-text matching; verify key semantic anchors only.

5) Optional config alignment check

Current config already has:

• prompt_dir: Path = Path("./prompts")

In Step 2, ensure docs reflect this and that Step 3 will resolve:

• PROMPT_DIR / "transcribe_document.md"

---

Task-by-Task Execution Checklist

Phase A — Scaffold files

• A1. Create prompt directory

  • Path: prompts/
  • Verify: directory exists at repo root

• A2. Create canonical prompt file

  • Path: prompts/transcribe_document.md
  • Verify: file exists and is non-empty

• A3. (Recommended) Create prompt library README

  • Path: prompts/README.md
  • Verify: includes naming + revision conventions

---

Phase B — Author prompt content (core work)

• B1. Add Purpose section

  • States verbatim historical transcription objective
  • Explicitly disallows summarization/paraphrase

• B2. Add Output Contract section

  • Plain text output expectation
  • Preserve meaningful structure and reading order
  • No fabricated text

• B3. Add Rule Set from docs/Intent.md

  • Misspellings/errors: [sic]
  • Missing words: [word]
  • Uncertain readings: [guess?]
  • Illegible regions: [illegible] / reason labels
  • Crossed-out text: [deleted: ...]
  • Squeezed-in text: [inserted: ...]
  • Superscripts/abbrev handling guidance
  • Non-text visuals: bracketed descriptive labels
  • Marginalia formatting cue
  • Rejoin line-break hyphenated words silently
  • Ambiguous capitalization policy
  • Hierarchical outline numbering preservation

• B4. Add Ambiguity and Confidence policy

  • “Mark uncertainty instead of guessing”
  • “Never silently normalize uncertain passages”

• B5. Add Final Self-Check section

  • Checklist for fidelity, uncertainty labeling, and format compliance

---

Phase C — Add validations (tests)

• C1. Create prompt tests file

  • Path: tests/test_prompts.py

• C2. Add existence/health checks

  • Prompt file exists
  • Prompt file has content (non-whitespace)

• C3. Add semantic anchor checks

  • Mentions verbatim behavior
  • Mentions uncertainty marker pattern (? in brackets conceptually)
  • Mentions illegible handling
  • Mentions deleted/inserted conventions

• C4. Keep tests resilient

  • Avoid exact full-file snapshot assertions
  • Assert required concepts, not precise phrasing

---

Phase D — Documentation alignment

• D1. Update top-level docs/README reference

  • Mention that prompts live in prompts/
  • Mention Step 3 loads from PROMPT_DIR

• D2. Confirm config compatibility

  • src/transcription/config.py already uses prompt_dir = Path("./prompts")
  • No code change needed unless naming/path mismatch appears

---

Phase E — Verification

• E1. Run targeted test file

  • uv run pytest tests/test_prompts.py -q

• E2. Run full suite

  • uv run pytest -q

• E3. Confirm no regressions

  • All existing tests still green (expected: previous 20 + new prompt tests)

---

Phase F — Commit plan (recommended granularity)

• F1. Commit 1: scaffold

  • prompts/transcribe_document.md (initial structure)
  • prompts/README.md (if included)

• F2. Commit 2: finalized prompt content

  • full rule-complete prompt text

• F3. Commit 3: tests + docs alignment

  • tests/test_prompts.py
  • README/docs mention of prompt artifact pattern

---

Done Criteria (quick gate)

• Canonical prompt exists and is curated for verbatim transcription.
• Prompt encodes all high-value handling rules from docs/Intent.md.
• Prompt tests pass.
• Full project tests pass with uv.
• Ready for Step 3 provider integration.

---

Acceptance Criteria (Definition of Done)

Step 2 is complete when all are true:

1. prompts/transcribe_document.md exists and is committed.
2. Prompt includes all critical handling rules from docs/Intent.md.
3. Prompt is structured with stable section headings for future curation.
4. Prompt tests pass under uv run pytest -q.
5. Existing tests remain green (total suite still passes).
6. Docs indicate prompt artifact location and curation policy.

---

Risks and Mitigations

1. Risk: prompt too vague → hallucinated reconstructions

   • Mitigation: explicit uncertainty/illegible conventions and “no invention” rule.

2. Risk: prompt too rigid for mixed document types

   • Mitigation: include neutral defaults + clear annotation formats.

3. Risk: brittle tests block iterative prompt tuning

   • Mitigation: test semantic anchors, not exact wording.

---

Handoff to Step 3

After Step 2, Step 3 can immediately:

1. Load transcribe_document.md from PROMPT_DIR
2. Inject prompt into OpenRouter request
3. Start validating real transcription behavior with minimal glue code