generated from john/python-template
35 lines
2.5 KiB
Markdown
35 lines
2.5 KiB
Markdown
# AI Coding Assistant Project Briefing & Context
|
|
|
|
## Project Mission
|
|
This application is a family history archival and transcription platform. Its primary goal is to accept scanned document images (letters, postcards, logbooks, diaries), execute OCR and structured transcription via AI vision models (OpenAI GPT-4o, Anthropic Claude 3.5 Sonnet), and manage historical metadata (authors, recipients, dates, and locations).
|
|
|
|
---
|
|
|
|
## Technical Stack & Architecture
|
|
* **Database:** PostgreSQL 13+ with native `UUID` (`gen_random_uuid()`) and `JSONB` columns.
|
|
* **Backend Runtime / Concurrency:** Python utilizing `asyncio` for concurrent HTTP API calls to AI providers, with strict rate-limiting via `asyncio.Semaphore`.
|
|
* **Validation & Types:** Python with **Pydantic** model definitions. Incoming AI responses must be parsed and validated with Pydantic models *before* database insertion.
|
|
* **ORM / Database Access:** SQLModel and SQLAlchemy, using parameterized statements and PostgreSQL-native types.
|
|
|
|
---
|
|
|
|
## Core System Directives for AI Code Generation
|
|
|
|
### 1. Data Immutability vs. Human Corrections
|
|
* `job_source.raw_transcription` and `source.raw_transcription` represent original, point-in-time machine outputs and are **immutable**.
|
|
* Human corrections occur on `source.revised_text`.
|
|
* When fetching text for the UI, always display `COALESCE(source.revised_text, source.raw_transcription)`.
|
|
|
|
### 2. Async Execution & Batching Rules
|
|
* A `job` represents an overarching execution run for a folder/group of images belonging to a single `document`.
|
|
* Images are submitted to AI APIs **one at a time in rapid succession** using `asyncio` worker pools.
|
|
* Each single-image API call populates a row in `job_source` with its own `status`, `raw_transcription`, `ai_metadata`, and `raw_api_response`.
|
|
* If 9 of 10 pages succeed and 1 fails, `job_source.status` for the failed image becomes `'failed'`, while `job.status` becomes `'partial_success'`. Do not mark the entire batch as failed if partial results exist.
|
|
|
|
### 3. Entity Relationships
|
|
* **Authors/Recipients:** A `document` can have multiple authors and recipients. Do NOT put direct `author_id` foreign keys on `document`. Query authors/recipients via `document_person` where `role = 'author'` or `role = 'recipient'`.
|
|
* **Page Ordering:** Multi-page documents must always be queried using `ORDER BY page_number ASC`.
|
|
|
|
### 4. Database Mutations
|
|
* Always use parameterized SQL queries (`$1`, `$2`) to prevent SQL injection.
|
|
* Store datetimes using UTC ISO 8601 strings or native PostgreSQL `TIMESTAMPTZ`. |