generated from john/python-template
156 lines
4.9 KiB
Markdown
156 lines
4.9 KiB
Markdown
Great—here’s a **detailed implementation plan for Step 2** based on your docs and current project state.
|
||
|
||
## Step 2 Goal
|
||
|
||
Implement the MVP prompt artifact system by creating a curated transcription prompt file:
|
||
|
||
- `prompts/transcribe_document.md`
|
||
|
||
This step primarily satisfies:
|
||
|
||
- **REQ-12**: prompts stored as individual Markdown artifacts
|
||
- MVP Feature 3: prompt-driven verbatim transcription behavior grounded in `docs/Intent.md`
|
||
|
||
---
|
||
|
||
## Scope for Step 2
|
||
|
||
### In scope
|
||
1. Create prompt artifact directory and first prompt file.
|
||
2. Encode transcription rules from `docs/Intent.md` into a model-facing prompt.
|
||
3. Define stable prompt structure so future revisions are easy to diff/review.
|
||
4. Add lightweight tests that validate artifact presence and baseline quality constraints.
|
||
5. Update docs/README references so Step 3 can consume prompt file directly.
|
||
|
||
### Out of scope
|
||
- Provider integration logic (Step 3)
|
||
- Worker/job orchestration (Step 4)
|
||
- UI behavior (Step 5)
|
||
|
||
---
|
||
|
||
## Proposed Deliverables
|
||
|
||
1. **`prompts/transcribe_document.md`**
|
||
- production prompt text for historical document transcription
|
||
|
||
2. **`prompts/README.md`** (recommended)
|
||
- conventions for prompt files, revision policy, naming
|
||
|
||
3. **`tests/test_prompts.py`** (recommended)
|
||
- artifact existence + structure checks
|
||
|
||
4. **Small docs update** (README or docs reference)
|
||
- indicate that prompts are file-based and loaded from `PROMPT_DIR`
|
||
|
||
---
|
||
|
||
## Detailed Work Breakdown
|
||
|
||
### 1) Create prompt artifact folder and canonical file
|
||
- Add `prompts/` at repo root.
|
||
- Add `transcribe_document.md` as the first curated artifact.
|
||
- Keep filename stable; this becomes the default in Step 3 unless overridden.
|
||
|
||
### 2) Author prompt content using a strict, sectioned format
|
||
Use section headers so future diffs are clean and policy changes are isolated.
|
||
|
||
Suggested sections:
|
||
|
||
1. **Purpose**
|
||
- verbatim scholarly transcription of historical documents
|
||
|
||
2. **Output requirements**
|
||
- plain text only
|
||
- no summaries, no paraphrasing
|
||
- preserve reading order and meaningful structure
|
||
|
||
3. **Core fidelity rules**
|
||
- preserve original wording and punctuation
|
||
- don’t silently normalize grammar/spelling
|
||
- no invented content
|
||
|
||
4. **Issue-handling rules (mapped from Intent table)**
|
||
- misspellings with `[sic]`
|
||
- missing words with `[word]`
|
||
- uncertainty with `[guess?]`
|
||
- illegible with `[illegible]` / reason tags
|
||
- crossed-out text as `[deleted: ...]`
|
||
- inserted text as `[inserted: ...]`
|
||
- superscripts handling guidance
|
||
- non-text elements as `[description]`
|
||
- marginalia format `[written in left margin: ...]`
|
||
- line-break hyphen rejoin behavior
|
||
- capitalization policy
|
||
- hierarchical outline preservation (including unusual numbering)
|
||
|
||
5. **Confidence/ambiguity policy**
|
||
- prefer explicit uncertainty markers over hallucination
|
||
|
||
6. **Final self-checklist for model**
|
||
- did I preserve structure?
|
||
- did I mark uncertain text?
|
||
- did I avoid silent corrections?
|
||
|
||
### 3) Add prompt-library conventions (`prompts/README.md`)
|
||
Recommended conventions:
|
||
- one prompt per file
|
||
- snake_case names
|
||
- each file starts with purpose + behavior contract
|
||
- iterative edits, one prompt per PR where possible
|
||
- no secrets in prompt files
|
||
|
||
### 4) Add tests for prompt assets (`tests/test_prompts.py`)
|
||
Keep tests robust but not brittle.
|
||
|
||
Recommended tests:
|
||
1. `test_prompt_file_exists`
|
||
2. `test_prompt_file_is_not_empty`
|
||
3. `test_prompt_mentions_verbatim_behavior`
|
||
4. `test_prompt_includes_uncertainty_and_illegible_markers`
|
||
5. `test_prompt_includes_deleted_and_inserted_conventions`
|
||
|
||
Avoid exact full-text matching; verify key semantic anchors only.
|
||
|
||
### 5) Optional config alignment check
|
||
Current config already has:
|
||
- `prompt_dir: Path = Path("./prompts")`
|
||
|
||
In Step 2, ensure docs reflect this and that Step 3 will resolve:
|
||
- `PROMPT_DIR / "transcribe_document.md"`
|
||
|
||
---
|
||
|
||
## Acceptance Criteria (Definition of Done)
|
||
|
||
Step 2 is complete when all are true:
|
||
|
||
1. `prompts/transcribe_document.md` exists and is committed.
|
||
2. Prompt includes all critical handling rules from `docs/Intent.md`.
|
||
3. Prompt is structured with stable section headings for future curation.
|
||
4. Prompt tests pass under `uv run pytest -q`.
|
||
5. Existing tests remain green (total suite still passes).
|
||
6. Docs indicate prompt artifact location and curation policy.
|
||
|
||
---
|
||
|
||
## Risks and Mitigations
|
||
|
||
1. **Risk: prompt too vague → hallucinated reconstructions**
|
||
- Mitigation: explicit uncertainty/illegible conventions and “no invention” rule.
|
||
|
||
2. **Risk: prompt too rigid for mixed document types**
|
||
- Mitigation: include neutral defaults + clear annotation formats.
|
||
|
||
3. **Risk: brittle tests block iterative prompt tuning**
|
||
- Mitigation: test semantic anchors, not exact wording.
|
||
|
||
---
|
||
|
||
## Handoff to Step 3
|
||
|
||
After Step 2, Step 3 can immediately:
|
||
1. Load `transcribe_document.md` from `PROMPT_DIR`
|
||
2. Inject prompt into OpenRouter request
|
||
3. Start validating real transcription behavior with minimal glue code
|