transcription/docs/step1.md

I now have a complete understanding of all the constraints. Here is the detailed implementation plan for Step 1:

---

Step 1 Implementation Plan: config.py + models.py + db.py

Purpose

Establish the foundational data layer and configuration system that every subsequent MVP step builds on. At the end of this step, the project has a runnable Python package with a validated schema, typed configuration, and a test suite proving the data layer works — before any UI, worker, or AI provider code exists.

---

1. Prerequisite: Project Structure Scaffolding

Before writing any logic, create the package skeleton so imports work correctly.

Files to create (empty __init__.py stubs)

src/
└── transcription/
    ├── __init__.py
    ├── providers/
    │   └── __init__.py
    ├── services/
    │   └── __init__.py
    └── ui/
        └── __init__.py

Files to create (with logic — the Step 1 deliverables)

src/transcription/config.py
src/transcription/models.py
src/transcription/db.py

Test files to create

tests/
├── __init__.py
├── conftest.py
├── test_config.py
├── test_models.py
└── test_db.py

Update pyproject.toml

Add the dependencies that Step 1 requires and won't change later:

[project]
name = "transcription"
version = "0.1.0"
description = "Historical document transcription system"
readme = "README.md"
requires-python = ">=3.12"
dependencies = [
    "google-genai>=1.0.0",
    "openai>=2.43.0",
    "pydantic>=2.13.4",
    "pydantic-settings>=2.9.1",
    "sqlmodel>=0.0.25",
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "pytest-asyncio>=0.25",
]

Key additions:

• pydantic-settings — for BaseSettings with env-var loading (this was split out of pydantic core in v2)
• sqlmodel — provides SQLModel (which bundles SQLAlchemy + Pydantic model integration) and the SQLite driver
• pytest + pytest-asyncio — in dev extras for test execution

Delete hello.py

The placeholder file is no longer needed.

---

2. config.py — Centralized Configuration

Satisfies: REQ-8 (centralized config and logging at startup)

Design Decisions

Decision	Rationale
Use pydantic-settings BaseSettings	Type-safe, validates on construction, loads from env vars and .env files automatically
PROVIDER as a string enum (openrouter, gemini)	Drives provider factory in Step 3; validated at startup, not at first API call
PROVIDER_MODEL defaults to None	Each provider adapter (Step 3) supplies its own sensible default when None; avoids config knowing about provider-specific model names
PROVIDER_BASE_URL defaults to None	Only needed to override OpenRouter's base URL; Gemini ignores it. None means "use provider default"
DATABASE_URL defaults to SQLite	Zero-setup local development; PostgreSQL swap is a single env-var change post-MVP
UPLOAD_DIR and PROMPT_DIR as Path objects	Enables .mkdir(parents=True, exist_ok=True) and path validation at startup
Logging configured in a setup_logging() function	Called once at startup; uses stdlib logging with a simple format. No third-party logging library needed for MVP

Proposed Implementation

"""Centralized application configuration.

All settings are loaded from environment variables (or a .env file)
once at startup. Provider-specific defaults (model names, base URLs)
are resolved by the provider adapters, not here.
"""

from enum import StrEnum
from functools import lru_cache
from pathlib import Path
import logging

from pydantic_settings import BaseSettings, SettingsConfigDict


class Provider(StrEnum):
    OPENROUTER = "openrouter"
    GEMINI = "gemini"


class Settings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",
        env_file_encoding="utf-8",
        extra="ignore",
    )

    # --- AI provider ---
    provider: Provider = Provider.OPENROUTER
    provider_api_key: str
    provider_model: str | None = None
    provider_base_url: str | None = None

    # --- persistence ---
    database_url: str = "sqlite:///./transcription.db"

    # --- filesystem paths ---
    upload_dir: Path = Path("./uploads")
    prompt_dir: Path = Path("./prompts")


@lru_cache(maxsize=1)
def get_settings() -> Settings:
    """Return the singleton Settings instance.

    Cached so the entire application shares one validated config.
    """
    return Settings()


def setup_logging() -> None:
    """Configure root logging once at startup."""
    logging.basicConfig(
        level=logging.INFO,
        format="%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
        datefmt="%Y-%m-%d %H:%M:%S",
    )

Key Behaviors

• Startup validation: If PROVIDER_API_KEY is missing from the environment, Settings() raises a ValidationError immediately — the app won't start with a missing key.
• .env support: Developers can create a .env file in the project root for local keys; it's never committed (already covered by the existing .gitignore pattern or a new entry).
• extra="ignore": Unknown env vars don't cause errors, keeping the config resilient to unrelated environment variables.
• lru_cache: get_settings() is the single access point. All modules import and call this function rather than constructing Settings directly.

.env template (not committed — add to .gitignore)

PROVIDER=openrouter
PROVIDER_API_KEY=sk-or-...
# PROVIDER_MODEL=          # optional: provider adapter supplies default
# PROVIDER_BASE_URL=       # optional: override provider endpoint
# DATABASE_URL=sqlite:///./transcription.db
# UPLOAD_DIR=./uploads
# PROMPT_DIR=./prompts

.gitignore addition

# ... existing entries ...

# Environment secrets
.env

---

3. models.py — SQLModel Domain Models

Satisfies: REQ-3 (persist and expose job states), REQ-4 (persist transcription output and failure details)

Design Decisions

Decision	Rationale
Three models: Document, Job, Transcript	Minimal set from MVP Feature 5. One-to-many from Document→Job and one-to-one from Job→Transcript
JobStatus as a StrEnum	Readable in the database ("queued" not 1), type-safe in Python, trivially serializable to JSON for the UI
Status values: queued, processing, transcribed, failed	Matches MVP Feature 2 lifecycle. REQ-3 also lists upload and completed — these are deferred to post-MVP when revision/review workflows exist
UUIDs for primary keys	Avoids auto-increment collision concerns if we later move to PostgreSQL; safe for distributed ID generation; uuid4 is simple
uploaded_at, created_at, updated_at as UTC datetime	Timezone-naive UTC by convention for MVP. Sufficient for single-user, single-timezone operation
Transcript.text is nullable	A failed job creates a Transcript with text=None and error_detail populated, keeping the query model uniform
Relationships via SQLModel Relationship	Enables document.jobs and job.transcript navigation in service code without manual joins

Proposed Implementation

"""SQLModel domain models for the transcription system.

Three models capture the MVP lifecycle:
  Document  →  one-to-many  →  Job  →  one-to-one  →  Transcript
"""

from datetime import datetime, timezone
from enum import StrEnum
from uuid import UUID, uuid4

from sqlmodel import Field, Relationship, SQLModel


class JobStatus(StrEnum):
    QUEUED = "queued"
    PROCESSING = "processing"
    TRANSCRIBED = "transcribed"
    FAILED = "failed"


class Document(SQLModel, table=True):
    """An uploaded document image."""

    id: UUID = Field(default_factory=uuid4, primary_key=True)
    filename: str
    file_path: str
    uploaded_at: datetime = Field(
        default_factory=lambda: datetime.now(timezone.utc),
    )

    # --- relationships ---
    jobs: list["Job"] = Relationship(back_populates="document")


class Job(SQLModel, table=True):
    """A transcription job tied to a single document."""

    id: UUID = Field(default_factory=uuid4, primary_key=True)
    document_id: UUID = Field(foreign_key="document.id")
    status: JobStatus = Field(default=JobStatus.QUEUED)
    created_at: datetime = Field(
        default_factory=lambda: datetime.now(timezone.utc),
    )
    updated_at: datetime = Field(
        default_factory=lambda: datetime.now(timezone.utc),
    )

    # --- relationships ---
    document: Document = Relationship(back_populates="jobs")
    transcript: "Transcript | None" = Relationship(back_populates="job")


class Transcript(SQLModel, table=True):
    """The output of a transcription job."""

    id: UUID = Field(default_factory=uuid4, primary_key=True)
    job_id: UUID = Field(foreign_key="job.id", unique=True)
    text: str | None = None
    error_detail: str | None = None
    created_at: datetime = Field(
        default_factory=lambda: datetime.now(timezone.utc),
    )

    # --- relationships ---
    job: Job = Relationship(back_populates="transcript")

Entity-Relationship Summary

┌──────────┐       ┌──────────┐       ┌─────────────┐
│ Document │ 1───* │   Job    │ 1───1 │ Transcript  │
├──────────┤       ├──────────┤       ├─────────────┤
│ id (PK)  │       │ id (PK)  │       │ id (PK)     │
│ filename │       │ doc_id   │──FK──▶│ job_id (FK) │
│ file_path│       │ status   │       │ text        │
│ uploaded │       │ created  │       │ error_detail│
│          │       │ updated  │       │ created     │
└──────────┘       └──────────┘       └─────────────┘

Why Only Four Status Values

REQ-3 lists six states: upload, queued, processing, transcribed, failed, completed. The MVP simplifies this:

REQ-3 State	MVP Treatment
upload	Implicit — the Document record exists before a Job is created. No separate job state needed.
queued	✅ Included — job created, waiting for worker pickup
processing	✅ Included — worker is actively transcribing
transcribed	✅ Included — AI output received and stored
failed	✅ Included — error captured
completed	Deferred — implies human review/acceptance. In MVP, transcribed is the terminal success state.

---

4. db.py — Database Engine and Session Management

Satisfies: MVP Feature 5 (SQLite auto-created on first startup)

Design Decisions

Decision	Rationale
Module-level create_engine + Session factory	REQ-7 (lifespan-owned resources) is deferred. A module-level engine is adequate for MVP's single-process, single-user operation
create_all() as an explicit function	Called at app startup. MVP auto-creates tables (REQ-10 deferred), but the function is isolated so it's easy to gate behind a flag later
get_session() as a generator	Standard FastAPI/SQLModel pattern — yields a session, ensures cleanup. Compatible with Depends() when the API layer arrives in Step 5
echo=False default	Keeps logs clean. Can be toggled for debugging

Proposed Implementation

"""Database engine, session factory, and schema bootstrap.

MVP uses SQLite with auto-create-tables at startup.
PostgreSQL migration is a post-MVP configuration change.
"""

from collections.abc import Generator

from sqlmodel import Session, SQLModel, create_engine

from transcription.config import get_settings


def _build_engine():
    settings = get_settings()
    connect_args = {}
    if settings.database_url.startswith("sqlite"):
        connect_args["check_same_thread"] = False
    return create_engine(
        settings.database_url,
        echo=False,
        connect_args=connect_args,
    )


engine = _build_engine()


def create_all() -> None:
    """Create all tables. Called once at application startup."""
    SQLModel.metadata.create_all(engine)


def get_session() -> Generator[Session, None, None]:
    """Yield a database session and ensure cleanup."""
    with Session(engine) as session:
        yield session

SQLite-Specific Note

check_same_thread=False is required for SQLite when the session may be accessed from different threads (e.g., a background worker on a different thread than the request handler). This setting is harmless and ignored for PostgreSQL connection strings.

---

5. Test Plan

tests/conftest.py — Shared Fixtures

"""Shared test fixtures.

Every test gets a fresh in-memory SQLite database so tests are
isolated, fast, and leave no artifacts on disk.
"""

import pytest
from sqlmodel import Session, SQLModel, create_engine


@pytest.fixture
def session():
    """Provide a clean database session for each test."""
    engine = create_engine(
        "sqlite://",  # in-memory
        connect_args={"check_same_thread": False},
    )
    SQLModel.metadata.create_all(engine)
    with Session(engine) as session:
        yield session

tests/test_config.py — Configuration Tests

Test	What It Verifies
test_settings_loads_from_env	Settings constructs successfully when PROVIDER_API_KEY is set via env var
test_settings_requires_api_key	Settings() raises ValidationError when PROVIDER_API_KEY is missing
test_provider_defaults_to_openrouter	Default provider is openrouter when not explicitly set
test_provider_rejects_invalid_value	Setting PROVIDER=invalid raises ValidationError
test_optional_fields_default_to_none	provider_model and provider_base_url are None when unset
test_path_fields_are_path_objects	upload_dir and prompt_dir are Path instances

tests/test_models.py — Model & Relationship Tests

Test	What It Verifies
test_create_document	A Document can be persisted and read back with correct fields
test_document_defaults	id is auto-generated UUID, uploaded_at is populated
test_create_job_with_document	A Job linked to a Document via FK persists correctly
test_job_defaults	Default status is queued, created_at and updated_at are populated
test_job_status_transitions	Status can be updated from queued → processing → transcribed
test_job_status_to_failed	Status can be updated from processing → failed
test_create_transcript_success	A Transcript with text set and error_detail=None persists correctly
test_create_transcript_failure	A Transcript with text=None and error_detail set persists correctly
test_document_jobs_relationship	document.jobs returns the linked Job list
test_job_transcript_relationship	job.transcript returns the linked Transcript
test_transcript_job_id_unique	Inserting two transcripts with the same job_id raises an integrity error

tests/test_db.py — Database Bootstrap Tests

Test	What It Verifies
test_create_all_creates_tables	After create_all(), the expected tables (document, job, transcript) exist in the database
test_get_session_yields_session	get_session() yields a usable Session object
test_session_cleanup_on_exit	After the generator is exhausted, the session is closed

---

6. Step 1 Completion Checklist

When all of the following are true, Step 1 is done and Step 2 can begin:

#	Criterion	How to Verify
1	src/transcription/ package exists with config.py, models.py, db.py	ls / file inspection
2	Empty __init__.py stubs exist for providers/, services/, ui/	ls / file inspection
3	Settings loads from environment and validates PROVIDER_API_KEY is present	test_config.py passes
4	Document, Job, Transcript models create tables in SQLite	test_models.py passes
5	JobStatus enum has exactly four values: queued, processing, transcribed, failed	test_models.py passes
6	Foreign key relationships work: Document→Job→Transcript	test_models.py passes
7	create_all() bootstraps the schema; get_session() yields a working session	test_db.py passes
8	All tests pass: uv run pytest tests/	CI / local run
9	hello.py is deleted	File inspection
10	pyproject.toml includes sqlmodel, pydantic-settings, pytest, pytest-asyncio	File inspection
11	.env.example documents all config vars; .env is in .gitignore	File inspection

---

7. What This Step Does NOT Include

Explicitly out of scope to prevent scope creep:

Excluded	Reason
FastAPI / NiceGUI app entrypoint	Step 5
Provider adapters (openrouter.py, gemini.py)	Step 3
Upload service logic	Step 4
Worker / background processing	Step 4
Transcription prompt files	Step 2
Alembic or migration tooling	Post-MVP (REQ-10 deferred)
Async session factory	Post-MVP (REQ-7 deferred)

---

This plan produces a fully tested, importable data foundation. Every subsequent step imports from transcription.config, transcription.models, and transcription.db without modification. When you're ready, switch to Agent mode and I'll implement it.