AI Coding Assistant Project Briefing & Context

Project Mission

This application is a family history archival and transcription platform. Its primary goal is to accept scanned document images (letters, postcards, logbooks, diaries), execute OCR and structured transcription via AI vision models (OpenAI GPT-4o, Anthropic Claude 3.5 Sonnet), and manage historical metadata (authors, recipients, dates, and locations).

Technical Stack & Architecture

Database: PostgreSQL 13+ with native UUID (gen_random_uuid()) and JSONB columns.
Backend Runtime / Concurrency: Python utilizing asyncio for concurrent HTTP API calls to AI providers, with strict rate-limiting via asyncio.Semaphore.
Validation & Types: Python with Pydantic model definitions. Incoming AI responses must be parsed and validated with Pydantic models before database insertion.
ORM / Database Access: SQLModel and SQLAlchemy, using parameterized statements and PostgreSQL-native types.

Core System Directives for AI Code Generation

1. Data Immutability vs. Human Corrections

job_source.raw_transcription and source.raw_transcription represent original, point-in-time machine outputs and are immutable.
Human corrections occur on source.revised_text.
When fetching text for the UI, always display COALESCE(source.revised_text, source.raw_transcription).

2. Async Execution & Batching Rules

A job represents an overarching execution run for a folder/group of images belonging to a single document.
Images are submitted to AI APIs one at a time in rapid succession using asyncio worker pools.
Each single-image API call populates a row in job_source with its own status, raw_transcription, ai_metadata, and raw_api_response.
If 9 of 10 pages succeed and 1 fails, job_source.status for the failed image becomes 'failed', while job.status becomes 'partial_success'. Do not mark the entire batch as failed if partial results exist.

3. Entity Relationships

Authors/Recipients: A document can have multiple authors and recipients. Do NOT put direct author_id foreign keys on document. Query authors/recipients via document_person where role = 'author' or role = 'recipient'.
Page Ordering: Multi-page documents must always be queried using ORDER BY page_number ASC.

4. Database Mutations

Always use parameterized SQL queries ($1, $2) to prevent SQL injection.
Store datetimes using UTC ISO 8601 strings or native PostgreSQL TIMESTAMPTZ.

2.5 KiB Raw Blame History