# Document Transcription System Overview (Version 2)

This project is a personal-scale application for transcribing, indexing, and preserving historical family documents, letters, postcards, and journals.

## Start Here

Read [architecture_v2.md](architecture_v2.md) first for technical overview and system design.

## Core V2 Capabilities

* **Folder & Multi-Image Ingestion:** Upload whole folders or image batches that map sequentially (`page_number`) under a single `Document`.
* **Parallel Async AI Vision Engine:** Concurrently process single-page image transcriptions using Python `asyncio` bounded by rate limiters.
* **Robust PostgreSQL Storage:** Relational storage for entities with native `UUID`, `TIMESTAMPTZ`, and `JSONB` for deep AI spatial metadata and raw envelopes.
* **Pydantic V2 Validation:** End-to-end type safety, DB row mapping, and JSONB payload validation.
* **Historical Person Management:** Track authors and recipients across documents with rich biographical entities (`Person`).
* **Page-Level Execution Auditing & Revisions:** Store immutable point-in-time machine output per run while enabling inline human corrections (`revised_text`).
* **Partial Failure Recovery:** Bounded batch execution that isolates single-page API errors (`partial_success`) for simple retries.

## Technical Stack

* **Application Web Framework:** FastAPI + NiceGUI
* **Persistence Engine:** PostgreSQL 18+
* **Data Validation & Schemas:** Pydantic V2
* **Concurrency & Workers:** Python `asyncio` worker pool with `asyncio.Semaphore`
* **Vision Providers:** OpenAI (GPT-4o) and Anthropic (Claude 3.5 Sonnet) via native SDKs

---

## Technology References

- [FastAPI documentation](https://fastapi.tiangolo.com/)
- [NiceGUI documentation](https://nicegui.io/documentation)
- [PostgreSQL documentation](https://www.postgresql.org/docs/)
- [Python asyncio](https://docs.python.org/3/library/asyncio.html#module-asyncio)
- [Pydantic Validation](https://pydantic.dev/docs/validation/latest/get-started/)
- [Pydantic AI](https://pydantic.dev/docs/ai/overview/)
 
## Documentation Index

- System Overview (this document)
- [System Design Intent](intent.md)
- [Transcription Methodology](transcription_methodology.md)
- [System Architecture](architecture_v2.md)
- [System Requirements](requirements_v2.md)
- [Data model](schema_v2.md)
- [Error Handling Policy](error_handling_v2.md)
- [Implementation Plan](implementation_plan_v2.md)