Update docs

This commit is contained in:
Jim Lancaster
2026-06-23 10:49:11 -05:00
parent 2bb74cd1cc
commit 187324d903
5 changed files with 522 additions and 77 deletions
+33 -77
View File
@@ -1,95 +1,51 @@
## Handwriting Transcription System
This project is a starter for transcribing historical documents with an LLM-powered, graph-based backend. It combines a NiceGUI web interface, a FastAPI + LangGraph application, and PostgreSQL persistence behind an Nginx entrypoint.
This project is a production application for transcribing and preserving historical family documents. It is intentionally designed for personal-scale use, with a simplicity-first architecture that is easy to operate and easy to extend.
The system is designed to be easy to run locally with Docker Compose and easy to extend for more advanced orchestration, scaling, and model strategies.
## Start Here
## What The System Does
Read [architecture.md](architecture.md) first.
At a high level, users upload one or more document images from the web UI. Each upload is tracked as a job in the database, processed through a LangGraph workflow, transcribed by an LLM, and stored with status history and events.
The architecture page is the primary technical reference and defines:
Core outcomes:
- Upload handwritten images through a minimal web interface.
- Persist image data, job state, events, errors, and transcription text in PostgreSQL.
- Track lifecycle states from upload through completion or failure.
- Inspect job status and results through API endpoints and UI pages.
- deployed topology and infrastructure limits
- module boundaries and dependency flow
- processing life cycle and data ownership
- test strategy, risk controls, and extension path
## Architecture Overview
## What The Application Does
The project uses a single Python backend that serves both API endpoints and NiceGUI pages.
At a high level, users upload images of handwritten, typed, or typeset documents, run asynchronous transcription jobs, review and edit transcript revisions, and search across accepted text.
### Front End
- NiceGUI pages mounted on FastAPI with periodic refresh for live job status updates.
- Supports uploading images, viewing current job states, and reading completed transcriptions.
- Displays failure states and error messages for troubleshooting.
Core capabilities:
### Backend
- FastAPI app for HTTP endpoints and page rendering.
- LangGraph workflow for multi-step transcription execution.
- Background task execution for asynchronous processing after upload.
- Centralized app configuration through pydantic-settings.
- Centralized logging initialization via one logging.config setup call at startup.
- Lifespan-owned runtime resources for the SQLAlchemy engine, async session factory, PostgreSQL checkpoint connection, and compiled graph.
- document upload and metadata capture
- asynchronous transcription with visible job status
- transcription prompt management with one Markdown file per prompt for human refinement over time
- revision history for transcript edits
- full-text search over accepted transcripts
- export of transcript data
### Database
- PostgreSQL is the only persistent store.
- SQLModel defines schema and data access.
- SQLAlchemy async access uses one engine per process and one async session per request or concurrent background task.
- Persists:
- image records
- transcription jobs
- processing events/history
- transcription output
- error details
- LangGraph checkpointing is stored in PostgreSQL for resumable workflow state.
- Schema bootstrap is explicit and opt-in; normal startup does not mutate production schema automatically.
## Production Operating Model
### Infrastructure
- Docker Compose runs exactly three containers:
- backend (FastAPI + LangGraph)
- frontend (Nginx reverse proxy)
- db (PostgreSQL)
- Nginx acts as the public entrypoint and proxies requests to the backend.
The system runs with minimal operational overhead:
## Processing Lifecycle
- PostgreSQL in a dedicated Docker container is considered extremely lightweight and simple for this system
- MongoDB in a dedicated Docker container is also considered extremely lightweight and simple for document-centric persistence
- a three-container deployment (app, PostgreSQL, MongoDB) is a simple and acceptable baseline
- no required queue or search-engine containers in the baseline setup
Each uploaded image moves through explicit statuses:
- upload
- queued
- processing
- transcribed
- failed
- completed
This operating model keeps deployment and maintenance simple while preserving clean boundaries for future scale.
Typical flow:
1. Image is uploaded and validated.
2. Image and job metadata are stored in PostgreSQL.
3. Job is queued and processed through LangGraph nodes.
4. LLM transcription is generated.
5. Result and processing events are saved.
6. Job ends as completed or failed with error details.
## Documentation Map
This state-driven model enables reliable inspection, retries, and recovery.
- Architecture and technical design: [architecture.md](architecture.md)
- Testing strategy and guidance: [tests.md](tests.md)
- Runtime and deployment requirements: [requirements.md](requirements.md)
- Domain context and transcription policy: [intent.md](intent.md)
## Configuration And Observability
## Glossary
Configuration is loaded once at startup using a pydantic-settings class and can be propagated through request/workflow execution via context variables where scoped access is needed.
Logging is initialized once through a centralized logging.config call, and modules use named loggers for consistent observability across API, workflow, and persistence layers.
Readiness checks validate both SQLAlchemy connectivity and graph runtime initialization so operational status reflects the actual owned runtime resources.
## Why This Starter Exists
This project intentionally balances practicality and extensibility:
- Minimal UI and straightforward APIs for fast iteration.
- Durable workflow state and clear job history for operational visibility.
- Clean separation of concerns across API, graph nodes, data models, and infrastructure.
- Local-first developer experience with uv and Docker Compose.
It is suitable as a baseline for production systems that need better queueing, multi-worker scaling, richer auth, or additional document processing features.
## Related Documentation
- Project description: [docs/Historical_Document_Transcription.md](docs/Historical_Document_Transcription.md)
- Project build prompt:
- Document-oriented persistence: Storing data as flexible records instead of fixed relational rows.
- Prompt artifact: A single Markdown file that defines one transcription prompt and is edited independently.
- System of record: The authoritative persistent store for canonical data.