Add starting docs

2026-06-21 18:54:42 -05:00
parent 974e328150
commit d0c5e3cb7a
2 changed files with 128 additions and 0 deletions
@@ -0,0 +1,95 @@
+## Handwriting Transcription System
+
+This project is a starter for transcribing historical documents with an LLM-powered, graph-based backend. It combines a NiceGUI web interface, a FastAPI + LangGraph application, and PostgreSQL persistence behind an Nginx entrypoint.
+
+The system is designed to be easy to run locally with Docker Compose and easy to extend for more advanced orchestration, scaling, and model strategies.
+
+## What The System Does
+
+At a high level, users upload one or more document images from the web UI. Each upload is tracked as a job in the database, processed through a LangGraph workflow, transcribed by an LLM, and stored with status history and events.
+
+Core outcomes:
+- Upload handwritten images through a minimal web interface.
+- Persist image data, job state, events, errors, and transcription text in PostgreSQL.
+- Track lifecycle states from upload through completion or failure.
+- Inspect job status and results through API endpoints and UI pages.
+
+## Architecture Overview
+
+The project uses a single Python backend that serves both API endpoints and NiceGUI pages.
+
+### Front End
+- NiceGUI pages mounted on FastAPI with periodic refresh for live job status updates.
+- Supports uploading images, viewing current job states, and reading completed transcriptions.
+- Displays failure states and error messages for troubleshooting.
+
+### Backend
+- FastAPI app for HTTP endpoints and page rendering.
+- LangGraph workflow for multi-step transcription execution.
+- Background task execution for asynchronous processing after upload.
+- Centralized app configuration through pydantic-settings.
+- Centralized logging initialization via one logging.config setup call at startup.
+- Lifespan-owned runtime resources for the SQLAlchemy engine, async session factory, PostgreSQL checkpoint connection, and compiled graph.
+
+### Database
+- PostgreSQL is the only persistent store.
+- SQLModel defines schema and data access.
+- SQLAlchemy async access uses one engine per process and one async session per request or concurrent background task.
+- Persists:
+	- image records
+	- transcription jobs
+	- processing events/history
+	- transcription output
+	- error details
+- LangGraph checkpointing is stored in PostgreSQL for resumable workflow state.
+- Schema bootstrap is explicit and opt-in; normal startup does not mutate production schema automatically.
+
+### Infrastructure
+- Docker Compose runs exactly three containers:
+	- backend (FastAPI + LangGraph)
+	- frontend (Nginx reverse proxy)
+	- db (PostgreSQL)
+- Nginx acts as the public entrypoint and proxies requests to the backend.
+
+## Processing Lifecycle
+
+Each uploaded image moves through explicit statuses:
+- upload
+- queued
+- processing
+- transcribed
+- failed
+- completed
+
+Typical flow:
+1. Image is uploaded and validated.
+2. Image and job metadata are stored in PostgreSQL.
+3. Job is queued and processed through LangGraph nodes.
+4. LLM transcription is generated.
+5. Result and processing events are saved.
+6. Job ends as completed or failed with error details.
+
+This state-driven model enables reliable inspection, retries, and recovery.
+
+## Configuration And Observability
+
+Configuration is loaded once at startup using a pydantic-settings class and can be propagated through request/workflow execution via context variables where scoped access is needed.
+
+Logging is initialized once through a centralized logging.config call, and modules use named loggers for consistent observability across API, workflow, and persistence layers.
+
+Readiness checks validate both SQLAlchemy connectivity and graph runtime initialization so operational status reflects the actual owned runtime resources.
+
+## Why This Starter Exists
+
+This project intentionally balances practicality and extensibility:
+- Minimal UI and straightforward APIs for fast iteration.
+- Durable workflow state and clear job history for operational visibility.
+- Clean separation of concerns across API, graph nodes, data models, and infrastructure.
+- Local-first developer experience with uv and Docker Compose.
+
+It is suitable as a baseline for production systems that need better queueing, multi-worker scaling, richer auth, or additional document processing features.
+
+## Related Documentation
+
+- Project description: [docs/Historical_Document_Transcription.md](docs/Historical_Document_Transcription.md)
+- Project build prompt: