# Document Transcription System Overview (Version 3)

This project is a personal-scale application for transcribing, indexing, and preserving historical family documents, letters, postcards, and journals.

## Start Here

Read [architecture_v3.md](https://www.google.com/search?q=architecture_v3.md) first for technical overview and system design.

## Core V3 Capabilities

* **Folder & Multi-Image Ingestion:** Upload one or more images that map sequentially (`page_number`) under a single `Document`.
* **Parallel Async AI Vision Engine:** Concurrently process single-page image transcriptions using Python `asyncio` bounded by rate limiters.
* **Portable Relational Storage:** SQLModel and SQLAlchemy preserve a portable relational model across the supported backends, with SQLite for local development/testing and PostgreSQL as the production database target.
* **Cross-Platform Operations:** Canonical developer and recovery workflows run through Python-based, OS-independent tooling rather than platform-specific shell scripts.
* **Complete Auditability & Provenance:** Capture frozen submission-time input prompts (`system_prompt`, `user_prompt`) and hyperparameters (`temperature`, `top_p`) on `Job`, plus per-page operational metrics (`ai_metadata`) and full provider response envelopes (`raw_api_response`) on `JobSource`.
* **Asset Integrity Tracking:** Calculate and store cryptographic hashes (SHA-256) and file sizes on `Source` image records while preserving clean filesystem storage.
* **Pydantic V2 Validation:** End-to-end type safety, DB row mapping, and JSON payload validation.
* **Historical Person Management:** Track authors and recipients across documents with rich biographical entities (`Person`).
* **Page-Level Execution Auditing & Revisions:** Store immutable point-in-time machine output per run while enabling inline human corrections (`revised_text`).
* **Partial Failure Recovery:** Bounded batch execution that isolates single-page API errors (`partial_success`) for simple retries.

## Technical Stack

* **Application Web Framework:** FastAPI + NiceGUI
* **Persistence Engine:** SQLModel / SQLAlchemy (SQLite for development/testing, PostgreSQL for production)
* **Data Validation & Schemas:** Pydantic V2
* **Concurrency & Workers:** Python `asyncio` worker pool with `asyncio.Semaphore`
* **Vision Providers:** OpenAI, Anthropic, and OpenRouter Vision models via native SDK adapters

---

## Technology References

* [FastAPI documentation](https://fastapi.tiangolo.com/)
* [NiceGUI documentation](https://nicegui.io/documentation)
* [SQLModel documentation](https://sqlmodel.tiangolo.com/)
* [Python asyncio](https://www.google.com/search?q=https://docs.python.org/3/library/asyncio.html%23module-asyncio)
* [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)

## Documentation Index

- System Overview (this document)
- [System Design Intent](invariant/intent.md)
- [Transcription Methodology](invariant/transcription_methodology.md)
- [System Architecture](architecture_v3.md)
- [System Requirements](requirements_v3.md)
- [Data model](schema_v3.md)
- [Error Handling Policy](error_handling_v3.md)
- [Implementation Plan](implementation_plan_v3.md)