V4.6 Phase 7: drive ty check to zero and add a blocking quality gate [HIGH-06]

Baseline was 207 diagnostics. Two real bugs were hiding in the noise:

- tools/run_destructive_tests.py imported ctypes.wintypes at module scope,
  which raises on non-Windows, and called fcntl unconditionally. The Windows
  and POSIX implementations now live under a module-level sys.platform split.
- tests/ui/test_sources_page.py constructed Source(...) without document_id.

Structural fixes, not suppressions:

- New src/transcription/db/loading.py owns the SQLModel-field to
  QueryableAttribute reinterpretation via orm_attribute()/selectinload()/
  defer(). This removed 42 "# pyright: ignore[reportArgumentType]" comments
  across documents/jobs/people/sources. Its docstring records that
  selectinload(A.b, B.c) is NOT equivalent to the chained form: varargs
  applies the selectin strategy only to the last path element, which under
  lazy="raise" raises InvalidRequestError at render time.
- db/session.py transaction_scope no longer accepts or yields
  AsyncSessionTransaction. No caller ever passed one, sessionmaker.begin()
  yields an AsyncSession, and the dead branch was latently buggy because
  services call .exec(). Cleared 7 workflows.py diagnostics.
- services/registry.py RegistryService is bound by a new RegistryEntry
  Protocol instead of bare SQLModel, so the shared implementation can read
  id/label/normalized_label/is_active. Cleared 9 diagnostics.
- Column expressions in sources.py/jobs.py/test_store.py wrap in sqlmodel
  col(), the idiom already used in registry.py.
- read_source_navigation wraps its literal tuple bounds in literal().
- normalization.py narrows with isinstance(image, TiffImageFile) rather than
  comparing image.format, since tag_v2 is TIFF-only.
- linked_people.render uses @ui.refreshable_method, the NiceGUI API for bound
  methods.
- The OpenRouter capturing client re-raises ResponseNotRead when the response
  stream is not async rather than mis-wrapping it.

Tooling gate:

- New .pre-commit-config.yaml runs ruff check and ty check as blocking hooks.
  No pre-commit config previously existed. Negative-tested: injecting a type
  error fails both hooks.
- The last two "# pyright: ignore" comments (config.py) are removed; ty does
  not honor pyright directives. One "# ty: ignore" remains, in
  tests/test_prompts.py, where the test deliberately assigns to a frozen
  field to assert ValidationError.
- asyncio_default_fixture_loop_scope is pinned to "function" so
  pytest-asyncio behavior does not shift on upgrade.

Verification: ruff check clean, ty check reports 0 diagnostics, 292 passed
and 4 skipped, pre-commit passes and demonstrably fails on a regression, and
tools/run_destructive_tests.py runs on Windows.

Co-authored-by: Copilot App <[email protected]>
This commit is contained in:
zoltan57
2026-08-17 19:57:23 -05:00
co-authored by Copilot App
parent 597be2691c
commit 66e2dce465
28 changed files with 316 additions and 184 deletions
+18
View File
@@ -0,0 +1,18 @@
# Quality gate for V4.6 [HIGH-06]. Both hooks are blocking: a regression in
# `ruff check` or `ty check` fails the commit.
repos:
- repo: local
hooks:
- id: ruff
name: ruff check
entry: ruff check
language: system
types_or: [python, pyi]
require_serial: true
- id: ty
name: ty check
entry: ty check
language: system
types_or: [python, pyi]
pass_filenames: false
require_serial: true
+1
View File
@@ -41,6 +41,7 @@ dev = [
[tool.pytest.ini_options] [tool.pytest.ini_options]
addopts = "--strict-markers -q" addopts = "--strict-markers -q"
asyncio_mode = "strict" asyncio_mode = "strict"
asyncio_default_fixture_loop_scope = "function"
filterwarnings = [ filterwarnings = [
"error:coroutine .* was never awaited:RuntimeWarning", "error:coroutine .* was never awaited:RuntimeWarning",
] ]
+2 -2
View File
@@ -172,13 +172,13 @@ class Settings(BaseSettings):
@cache @cache
def get_settings(**kwargs: Any) -> Settings: def get_settings(**kwargs: Any) -> Settings:
"""Load cached settings without reading process CLI arguments.""" """Load cached settings without reading process CLI arguments."""
return Settings(_cli_parse_args=False, **kwargs) # pyright: ignore[reportCallIssue] return Settings(_cli_parse_args=False, **kwargs)
def parse_cli_settings(args: Sequence[str] | None = None) -> Settings: def parse_cli_settings(args: Sequence[str] | None = None) -> Settings:
"""Load settings with CLI arguments at the executable boundary.""" """Load settings with CLI arguments at the executable boundary."""
cli_args = True if args is None else list(args) cli_args = True if args is None else list(args)
return Settings(_cli_parse_args=cli_args) # pyright: ignore[reportCallIssue] return Settings(_cli_parse_args=cli_args)
LOGGING_CONFIG: dict[str, Any] = { LOGGING_CONFIG: dict[str, Any] = {
+45
View File
@@ -0,0 +1,45 @@
"""Typed loader-option wrappers for SQLModel relationship attributes.
SQLModel declares relationships with their runtime Python type, so
``Document.jobs`` is annotated ``list[Job]`` even though at runtime it is an
``InstrumentedAttribute``. SQLAlchemy's loader options are typed against
``QueryableAttribute``, so every eager-load call site reads as a type error to a
static checker even though the code is correct.
These wrappers put that reinterpretation in one documented place instead of
scattering a suppression comment across every eager-load call. Import
``selectinload`` and ``defer`` from here rather than from ``sqlalchemy.orm``.
Multi-level eager loads must keep using the chained form --
``selectinload(A.b).selectinload(orm_attribute(B.c))`` -- and not the varargs
form ``selectinload(A.b, B.c)``. The two produce the same loader path, but
varargs applies the selectin strategy only to the last element while the
intermediate falls back to its default strategy. Every relationship here
declares ``lazy="raise"``, so the varargs form raises at render time.
"""
from __future__ import annotations
from typing import Any
from typing import cast
from sqlalchemy.orm import defer as _defer
from sqlalchemy.orm import selectinload as _selectinload
from sqlalchemy.orm.attributes import QueryableAttribute
from sqlalchemy.orm.strategy_options import _AbstractLoad
def orm_attribute(attribute: object) -> QueryableAttribute[Any]:
"""Reinterpret a SQLModel relationship or field as its ORM descriptor."""
return cast("QueryableAttribute[Any]", attribute)
def selectinload(*keys: object) -> _AbstractLoad:
"""``sqlalchemy.orm.selectinload`` accepting SQLModel-annotated attributes."""
return _selectinload(*(orm_attribute(key) for key in keys))
def defer(*keys: object, raiseload: bool = False) -> _AbstractLoad:
"""``sqlalchemy.orm.defer`` accepting SQLModel-annotated attributes."""
first, *rest = (orm_attribute(key) for key in keys)
return _defer(first, *rest, raiseload=raiseload)
+2 -1
View File
@@ -23,6 +23,7 @@ from sqlalchemy import Uuid
from sqlalchemy import inspect as sqlalchemy_inspect from sqlalchemy import inspect as sqlalchemy_inspect
from sqlalchemy.dialects.postgresql import JSONB from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.exc import NoInspectionAvailable from sqlalchemy.exc import NoInspectionAvailable
from sqlalchemy.orm.state import InstanceState
from sqlalchemy.types import TypeDecorator from sqlalchemy.types import TypeDecorator
from sqlmodel import Field from sqlmodel import Field
from sqlmodel import Relationship from sqlmodel import Relationship
@@ -38,7 +39,7 @@ def _loaded_attribute(instance: object, attribute: str) -> Any | None:
without catching exceptions indiscriminately. without catching exceptions indiscriminately.
""" """
try: try:
state = sqlalchemy_inspect(instance) state: InstanceState[Any] = sqlalchemy_inspect(instance, raiseerr=True)
except NoInspectionAvailable: except NoInspectionAvailable:
return None return None
if attribute in state.unloaded: if attribute in state.unloaded:
+6 -14
View File
@@ -3,7 +3,6 @@ from contextlib import asynccontextmanager
from typing import Annotated from typing import Annotated
from fastapi import Depends from fastapi import Depends
from sqlalchemy.ext.asyncio import AsyncSessionTransaction
from sqlalchemy.ext.asyncio import async_sessionmaker from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
@@ -86,16 +85,12 @@ async def transaction_scope(
settings: Settings | None = None, settings: Settings | None = None,
database_url: str | None = None, database_url: str | None = None,
session_factory: SessionFactory | None = None, session_factory: SessionFactory | None = None,
session: AsyncSession | AsyncSessionTransaction | None = None, session: AsyncSession | None = None,
) -> AsyncGenerator[AsyncSession | AsyncSessionTransaction]: ) -> AsyncGenerator[AsyncSession]:
match session: if session is not None:
case AsyncSession() as async_session: if not session.in_transaction():
if not async_session.in_transaction():
raise RuntimeError("A supplied session must have an active transaction") raise RuntimeError("A supplied session must have an active transaction")
yield async_session yield session
return
case AsyncSessionTransaction() as async_transaction:
yield async_transaction
return return
active_session_factory = session_factory or resolve_session_factory( active_session_factory = session_factory or resolve_session_factory(
@@ -106,7 +101,4 @@ async def transaction_scope(
yield owned_session yield owned_session
type TransactionScopeDep = Annotated[ type TransactionScopeDep = Annotated[AsyncSession, Depends(transaction_scope)]
AsyncSession | AsyncSessionTransaction,
Depends(transaction_scope),
]
+4 -1
View File
@@ -74,7 +74,10 @@ class _CapturingAsyncClient:
try: try:
self.last_body = response.content self.last_body = response.content
except httpx.ResponseNotRead: except httpx.ResponseNotRead:
response.stream = _CapturingAsyncByteStream(response.stream, self._capture_body) stream = response.stream
if not isinstance(stream, httpx.AsyncByteStream):
raise
response.stream = _CapturingAsyncByteStream(stream, self._capture_body)
return response return response
def build_request(self, *args: Any, **kwargs: Any) -> httpx.Request: def build_request(self, *args: Any, **kwargs: Any) -> httpx.Request:
+15 -14
View File
@@ -10,13 +10,14 @@ from uuid import UUID
from sqlalchemy.exc import IntegrityError from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import async_sessionmaker from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.orm import selectinload
from sqlmodel import SQLModel from sqlmodel import SQLModel
from sqlmodel import col from sqlmodel import col
from sqlmodel import select from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from ..config import Settings from ..config import Settings
from ..db.loading import orm_attribute
from ..db.loading import selectinload
from ..db.models import Document from ..db.models import Document
from ..db.models import DocumentPerson from ..db.models import DocumentPerson
from ..db.models import DocumentType from ..db.models import DocumentType
@@ -189,8 +190,8 @@ class DocumentService(ServiceBase):
session=_session, session=_session,
document_id=document_id, document_id=document_id,
options=( options=(
selectinload(Document.jobs), # pyright: ignore[reportArgumentType] selectinload(Document.jobs),
selectinload(Document.sources), # pyright: ignore[reportArgumentType] selectinload(Document.sources),
), ),
suggestion="Re-upload the source document and retry.", suggestion="Re-upload the source document and retry.",
) )
@@ -218,9 +219,9 @@ class DocumentService(ServiceBase):
session=_session, session=_session,
document_id=document.id, document_id=document.id,
options=( options=(
selectinload(Document.jobs), # pyright: ignore[reportArgumentType] selectinload(Document.jobs),
selectinload(Document.sources), # pyright: ignore[reportArgumentType] selectinload(Document.sources),
selectinload(Document.document_people), # pyright: ignore[reportArgumentType] selectinload(Document.document_people),
), ),
) )
@@ -275,9 +276,9 @@ class DocumentService(ServiceBase):
"""List documents with relations needed by the archival table.""" """List documents with relations needed by the archival table."""
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
query = select(Document).options( query = select(Document).options(
selectinload(Document.document_people).selectinload(DocumentPerson.person), # pyright: ignore[reportArgumentType] selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.person)),
selectinload(Document.document_people).selectinload(DocumentPerson.role_ref), # pyright: ignore[reportArgumentType] selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
selectinload(Document.document_type_ref), # pyright: ignore[reportArgumentType] selectinload(Document.document_type_ref),
) )
result = await _session.exec(query) result = await _session.exec(query)
return result.all() return result.all()
@@ -288,11 +289,11 @@ class DocumentService(ServiceBase):
query = ( query = (
select(Document) select(Document)
.options( .options(
selectinload(Document.jobs), # pyright: ignore[reportArgumentType] selectinload(Document.jobs),
selectinload(Document.sources), # pyright: ignore[reportArgumentType] selectinload(Document.sources),
selectinload(Document.document_people).selectinload(DocumentPerson.person), # pyright: ignore[reportArgumentType] selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.person)),
selectinload(Document.document_people).selectinload(DocumentPerson.role_ref), # pyright: ignore[reportArgumentType] selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
selectinload(Document.document_type_ref), # pyright: ignore[reportArgumentType] selectinload(Document.document_type_ref),
) )
.where(Document.id == document_id) .where(Document.id == document_id)
.execution_options(populate_existing=True) .execution_options(populate_existing=True)
+18 -14
View File
@@ -6,10 +6,12 @@ from pathlib import Path
from uuid import UUID from uuid import UUID
from sqlalchemy import func from sqlalchemy import func
from sqlalchemy.orm import selectinload from sqlmodel import col
from sqlmodel import select from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from ..db.loading import orm_attribute
from ..db.loading import selectinload
from ..db.models import ExecutionAttempt from ..db.models import ExecutionAttempt
from ..db.models import Job from ..db.models import Job
from ..db.models import JobSource from ..db.models import JobSource
@@ -64,8 +66,8 @@ class JobService(ServiceBase):
query = ( query = (
select(Job) select(Job)
.options( .options(
selectinload(Job.document), # pyright: ignore[reportArgumentType] selectinload(Job.document),
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType] selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
) )
.where(Job.id == job_id) .where(Job.id == job_id)
.execution_options(populate_existing=True) .execution_options(populate_existing=True)
@@ -100,13 +102,15 @@ class JobService(ServiceBase):
"""Query jobs from the database based on provided filters.""" """Query jobs from the database based on provided filters."""
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
query = select(Job).options( query = select(Job).options(
selectinload(Job.document), # pyright: ignore[reportArgumentType] selectinload(Job.document),
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType] selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
) )
if status is not None: if status is not None:
query = query.where(Job.status == status) query = query.where(Job.status == status)
if filename is not None: if filename is not None:
query = query.where(Job.job_sources.any(JobSource.source.has(Source.filename == filename))) query = query.where(
col(Job.job_sources).any(col(JobSource.source).has(col(Source.filename) == filename))
)
result = await _session.exec(query) result = await _session.exec(query)
return result.all() return result.all()
@@ -118,8 +122,8 @@ class JobService(ServiceBase):
"""List all jobs in the database with eagerly loaded documents.""" """List all jobs in the database with eagerly loaded documents."""
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
query = select(Job).options( query = select(Job).options(
selectinload(Job.document), # pyright: ignore[reportArgumentType] selectinload(Job.document),
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType] selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
) )
result = await _session.exec(query) result = await _session.exec(query)
return result.all() return result.all()
@@ -152,8 +156,8 @@ class JobService(ServiceBase):
query = ( query = (
select(Job) select(Job)
.options( .options(
selectinload(Job.document), # pyright: ignore[reportArgumentType] selectinload(Job.document),
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType] selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
) )
.where(Job.id == job_id) .where(Job.id == job_id)
.execution_options(populate_existing=True) .execution_options(populate_existing=True)
@@ -184,7 +188,7 @@ class JobService(ServiceBase):
select(Job) select(Job)
.where(Job.status == JobStatus.QUEUED) .where(Job.status == JobStatus.QUEUED)
# Break ties by id so "next" is stable when two rows share close timestamps. # Break ties by id so "next" is stable when two rows share close timestamps.
.order_by(Job.date_created, Job.id) # pyright: ignore[reportArgumentType] .order_by(col(Job.date_created), col(Job.id))
.limit(1) .limit(1)
) )
if _session.get_bind().dialect.name == "postgresql": if _session.get_bind().dialect.name == "postgresql":
@@ -304,7 +308,7 @@ class JobService(ServiceBase):
( (
await session.exec( await session.exec(
select(ProcessingArtifact).where( select(ProcessingArtifact).where(
ProcessingArtifact.execution_attempt_id.in_(attempt_ids) col(ProcessingArtifact.execution_attempt_id).in_(attempt_ids)
) )
) )
).all() ).all()
@@ -351,7 +355,7 @@ class JobService(ServiceBase):
query = ( query = (
select(Job) select(Job)
.options( .options(
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType] selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
) )
.where(Job.id == job_id) .where(Job.id == job_id)
.execution_options(populate_existing=True) .execution_options(populate_existing=True)
@@ -388,7 +392,7 @@ class JobService(ServiceBase):
query = ( query = (
select(Job) select(Job)
.options( .options(
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType] selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
) )
.where(Job.id == job_id) .where(Job.id == job_id)
.execution_options(populate_existing=True) .execution_options(populate_existing=True)
+2 -1
View File
@@ -10,6 +10,7 @@ from pathlib import Path
from PIL import Image from PIL import Image
from PIL import UnidentifiedImageError from PIL import UnidentifiedImageError
from PIL.TiffImagePlugin import TiffImageFile
from transcription.errors import AppError from transcription.errors import AppError
from transcription.errors import ErrorCategory from transcription.errors import ErrorCategory
@@ -65,7 +66,7 @@ def normalize_orientation(path: str | Path, *, media_type: str) -> OrientationNo
return None return None
transpose, rotation = transformation transpose, rotation = transformation
if image.format == "TIFF": if isinstance(image, TiffImageFile):
original_width = int(image.tag_v2.get(256, image.width)) original_width = int(image.tag_v2.get(256, image.width))
original_height = int(image.tag_v2.get(257, image.height)) original_height = int(image.tag_v2.get(257, image.height))
# Pillow applies TIFF orientation while decoding; copying freezes those upright pixels. # Pillow applies TIFF orientation while decoding; copying freezes those upright pixels.
+8 -7
View File
@@ -12,7 +12,6 @@ from uuid import UUID
from sqlalchemy.exc import IntegrityError from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import async_sessionmaker from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.orm import selectinload
from sqlmodel import SQLModel from sqlmodel import SQLModel
from sqlmodel import col from sqlmodel import col
from sqlmodel import select from sqlmodel import select
@@ -20,6 +19,8 @@ from sqlmodel.ext.asyncio.session import AsyncSession
from ..config import Settings from ..config import Settings
from ..config import get_settings from ..config import get_settings
from ..db.loading import orm_attribute
from ..db.loading import selectinload
from ..db.models import Document from ..db.models import Document
from ..db.models import DocumentPerson from ..db.models import DocumentPerson
from ..db.models import Person from ..db.models import Person
@@ -144,7 +145,7 @@ class PeopleService(ServiceBase):
existing = await _session.get( existing = await _session.get(
Person, Person,
person.id, person.id,
options=(selectinload(Person.document_people),), # pyright: ignore[reportArgumentType] options=(selectinload(Person.document_people),),
) )
if existing is None: if existing is None:
raise self._not_found(f"Person with id {person.id} not found") raise self._not_found(f"Person with id {person.id} not found")
@@ -209,8 +210,8 @@ class PeopleService(ServiceBase):
query = ( query = (
select(Person) select(Person)
.options( .options(
selectinload(Person.document_people).selectinload(DocumentPerson.document), # pyright: ignore[reportArgumentType] selectinload(Person.document_people).selectinload(orm_attribute(DocumentPerson.document)),
selectinload(Person.document_people).selectinload(DocumentPerson.role_ref), # pyright: ignore[reportArgumentType] selectinload(Person.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
) )
.where(Person.id == person_id) .where(Person.id == person_id)
.execution_options(populate_existing=True) .execution_options(populate_existing=True)
@@ -329,9 +330,9 @@ class PeopleService(ServiceBase):
) -> Sequence[DocumentPerson]: ) -> Sequence[DocumentPerson]:
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
query = select(DocumentPerson).options( query = select(DocumentPerson).options(
selectinload(DocumentPerson.document), # pyright: ignore[reportArgumentType] selectinload(DocumentPerson.document),
selectinload(DocumentPerson.person), # pyright: ignore[reportArgumentType] selectinload(DocumentPerson.person),
selectinload(DocumentPerson.role_ref), # pyright: ignore[reportArgumentType] selectinload(DocumentPerson.role_ref),
) )
if document_id is not None: if document_id is not None:
query = query.where(DocumentPerson.document_id == document_id) query = query.where(DocumentPerson.document_id == document_id)
+18 -1
View File
@@ -12,6 +12,7 @@ from __future__ import annotations
from abc import abstractmethod from abc import abstractmethod
from collections.abc import Sequence from collections.abc import Sequence
from typing import Any from typing import Any
from typing import Protocol
from uuid import UUID from uuid import UUID
from sqlalchemy import func from sqlalchemy import func
@@ -26,7 +27,23 @@ from ..errors import ErrorCategory
from .base import ServiceBase from .base import ServiceBase
class RegistryService[ModelT: SQLModel](ServiceBase): class RegistryEntry(Protocol):
"""Structural contract every registry table row satisfies.
Bounding ``RegistryService`` by this protocol rather than by bare ``SQLModel``
lets the shared implementation read ``id``/``label``/``normalized_label``/
``is_active`` off the model class without suppressions.
"""
id: UUID
label: str
normalized_label: str
is_active: bool
def __init__(self, /, **data: Any) -> None: ...
class RegistryService[ModelT: RegistryEntry](ServiceBase):
"""Generic create/read/update/delete behavior for a registry table. """Generic create/read/update/delete behavior for a registry table.
Subclasses declare the model, the error type, the user-facing noun, and the Subclasses declare the model, the error type, the user-facing noun, and the
+28 -26
View File
@@ -25,10 +25,9 @@ from pydantic import TypeAdapter
from pydantic import ValidationError from pydantic import ValidationError
from sqlalchemy import func from sqlalchemy import func
from sqlalchemy import inspect as sqlalchemy_inspect from sqlalchemy import inspect as sqlalchemy_inspect
from sqlalchemy import literal
from sqlalchemy import tuple_ from sqlalchemy import tuple_
from sqlalchemy.ext.asyncio import async_sessionmaker from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.orm import defer
from sqlalchemy.orm import selectinload
from sqlmodel import col from sqlmodel import col
from sqlmodel import select from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
@@ -55,6 +54,9 @@ from transcription.providers import TransportEvidence
from transcription.providers import get_transcription_provider from transcription.providers import get_transcription_provider
from transcription.providers.evidence import canonical_json_bytes from transcription.providers.evidence import canonical_json_bytes
from ..db.loading import defer
from ..db.loading import orm_attribute
from ..db.loading import selectinload
from .base import ServiceBase from .base import ServiceBase
from .normalization import ORIENTATION_PRODUCER from .normalization import ORIENTATION_PRODUCER
from .normalization import ORIENTATION_PRODUCER_VERSION from .normalization import ORIENTATION_PRODUCER_VERSION
@@ -195,8 +197,8 @@ class SourceService(ServiceBase):
query = ( query = (
select(Source) select(Source)
.options( .options(
selectinload(Source.document), # pyright: ignore[reportArgumentType] selectinload(Source.document),
selectinload(Source.job_sources).selectinload(JobSource.job), # pyright: ignore[reportArgumentType] selectinload(Source.job_sources).selectinload(orm_attribute(JobSource.job)),
) )
.where(Source.id == source_id) .where(Source.id == source_id)
.execution_options(populate_existing=True) .execution_options(populate_existing=True)
@@ -226,11 +228,11 @@ class SourceService(ServiceBase):
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
query = ( query = (
select(ExecutionAttempt) select(ExecutionAttempt)
.options(defer(ExecutionAttempt.transport_body)) # pyright: ignore[reportArgumentType] .options(defer(ExecutionAttempt.transport_body))
.where(ExecutionAttempt.job_source_id == job_source_id) .where(ExecutionAttempt.job_source_id == job_source_id)
.order_by( .order_by(
ExecutionAttempt.attempt_number.desc(), # pyright: ignore[reportAttributeAccessIssue] col(ExecutionAttempt.attempt_number).desc(),
ExecutionAttempt.id.desc(), # pyright: ignore[reportAttributeAccessIssue] col(ExecutionAttempt.id).desc(),
) )
.limit(1) .limit(1)
) )
@@ -250,7 +252,7 @@ class SourceService(ServiceBase):
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
source = await self._read_source(session=_session, source_id=source_id) source = await self._read_source(session=_session, source_id=source_id)
position = (col(Source.page_number), col(Source.id)) position = (col(Source.page_number), col(Source.id))
current = (source.page_number, source_id) current = (literal(source.page_number), literal(source_id))
previous_query = ( previous_query = (
select(col(Source.id)) select(col(Source.id))
@@ -290,8 +292,8 @@ class SourceService(ServiceBase):
session=_session, session=_session,
source_id=source_id, source_id=source_id,
options=( options=(
selectinload(Source.job_sources), # pyright: ignore[reportArgumentType] selectinload(Source.job_sources),
selectinload(Source.processing_artifacts), # pyright: ignore[reportArgumentType] selectinload(Source.processing_artifacts),
), ),
) )
@@ -379,7 +381,7 @@ class SourceService(ServiceBase):
job_source = await _session.get( job_source = await _session.get(
JobSource, JobSource,
job_source_id, job_source_id,
options=(selectinload(JobSource.source),), # pyright: ignore[reportArgumentType] options=(selectinload(JobSource.source),),
) )
if job_source is None: if job_source is None:
raise TranscriptionNotFoundError( raise TranscriptionNotFoundError(
@@ -444,8 +446,8 @@ class SourceService(ServiceBase):
session=_session, session=_session,
source_id=source_id, source_id=source_id,
options=( options=(
selectinload(Source.job_sources), # pyright: ignore[reportArgumentType] selectinload(Source.job_sources),
selectinload(Source.processing_artifacts), # pyright: ignore[reportArgumentType] selectinload(Source.processing_artifacts),
), ),
) )
@@ -511,8 +513,8 @@ class SourceService(ServiceBase):
"""List job-source records, optionally filtered by job.""" """List job-source records, optionally filtered by job."""
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
query = select(JobSource).options( query = select(JobSource).options(
selectinload(JobSource.job), # pyright: ignore[reportArgumentType] selectinload(JobSource.job),
selectinload(JobSource.source), # pyright: ignore[reportArgumentType] selectinload(JobSource.source),
) )
if job_id is not None: if job_id is not None:
query = query.where(JobSource.job_id == job_id) query = query.where(JobSource.job_id == job_id)
@@ -720,10 +722,10 @@ class SourceService(ServiceBase):
if job_id is not None: if job_id is not None:
query = query.where(ExecutionAttempt.job_id == job_id) query = query.where(ExecutionAttempt.job_id == job_id)
query = query.order_by( query = query.order_by(
ExecutionAttempt.job_id, col(ExecutionAttempt.job_id),
ExecutionAttempt.source_id, col(ExecutionAttempt.source_id),
ExecutionAttempt.attempt_number, col(ExecutionAttempt.attempt_number),
ExecutionAttempt.id, col(ExecutionAttempt.id),
) )
return (await _session.exec(query)).all() return (await _session.exec(query)).all()
@@ -987,7 +989,7 @@ class SourceService(ServiceBase):
query = ( query = (
select(ProcessingArtifact) select(ProcessingArtifact)
.where(ProcessingArtifact.source_id == source_id) .where(ProcessingArtifact.source_id == source_id)
.order_by(ProcessingArtifact.created_at, ProcessingArtifact.id) .order_by(col(ProcessingArtifact.created_at), col(ProcessingArtifact.id))
.limit(limit) .limit(limit)
) )
return (await _session.exec(query)).all() return (await _session.exec(query)).all()
@@ -1003,9 +1005,9 @@ class SourceService(ServiceBase):
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
query = ( query = (
select(ProcessingArtifact) select(ProcessingArtifact)
.options(defer(ProcessingArtifact.inline_payload)) # pyright: ignore[reportArgumentType] .options(defer(ProcessingArtifact.inline_payload))
.where(ProcessingArtifact.source_id == source_id) .where(ProcessingArtifact.source_id == source_id)
.order_by(ProcessingArtifact.created_at, ProcessingArtifact.id) .order_by(col(ProcessingArtifact.created_at), col(ProcessingArtifact.id))
.limit(limit) .limit(limit)
) )
return (await _session.exec(query)).all() return (await _session.exec(query)).all()
@@ -1138,10 +1140,10 @@ class SourceService(ServiceBase):
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
query = ( query = (
select(Source) select(Source)
.join(JobSource, JobSource.source_id == Source.id) .join(JobSource, col(JobSource.source_id) == col(Source.id))
.where(JobSource.job_id == job_id) .where(col(JobSource.job_id) == job_id)
.where(Source.revised_text.is_not(None)) .where(col(Source.revised_text).is_not(None))
.order_by(Source.date_revised) # pyright: ignore[reportArgumentType] .order_by(col(Source.date_revised))
) )
result = await _session.exec(query) result = await _session.exec(query)
return result.all() return result.all()
@@ -50,7 +50,7 @@ class LinkedPeopleEditor:
"""Return the complete staged link set for persistence.""" """Return the complete staged link set for persistence."""
return [DocumentPersonInput(person_id=link.person_id, role_id=link.role_id) for link in self.links] return [DocumentPersonInput(person_id=link.person_id, role_id=link.role_id) for link in self.links]
@ui.refreshable @ui.refreshable_method
def render(self) -> None: def render(self) -> None:
ui.label("Linked People").classes("text-sm font-semibold ui-text-primary mt-2") ui.label("Linked People").classes("text-sm font-semibold ui-text-primary mt-2")
ui.button("Create new person", on_click=lambda: ui.navigate.to("/people/new"), icon="person_add").props( ui.button("Create new person", on_click=lambda: ui.navigate.to("/people/new"), icon="person_add").props(
+1 -1
View File
@@ -407,7 +407,7 @@ def _render_provider_evidence(
def _artifact_display(artifacts: list[ProcessingArtifact]) -> list[dict[str, object]] | None: def _artifact_display(artifacts: list[ProcessingArtifact]) -> list[dict[str, object]] | None:
payload = [ payload: list[dict[str, object]] = [
{ {
"id": str(artifact.id), "id": str(artifact.id),
"type": artifact.artifact_type, "type": artifact.artifact_type,
+22 -12
View File
@@ -2,8 +2,10 @@
import asyncio import asyncio
from types import SimpleNamespace from types import SimpleNamespace
from typing import cast
import pytest import pytest
from openrouter import OpenRouter
from transcription.config import Settings from transcription.config import Settings
from transcription.providers.base import ProviderError from transcription.providers.base import ProviderError
@@ -13,7 +15,7 @@ from transcription.providers.openrouter import OpenRouterTranscriptionProvider
class _FakeChat: class _FakeChat:
def __init__(self, response=None, error: Exception | None = None): def __init__(self, response=None, error: BaseException | None = None):
self._response = response self._response = response
self._error = error self._error = error
self.calls = [] self.calls = []
@@ -26,10 +28,15 @@ class _FakeChat:
class _FakeClient: class _FakeClient:
def __init__(self, response=None, error: Exception | None = None): def __init__(self, response=None, error: BaseException | None = None):
self.chat = _FakeChat(response=response, error=error) self.chat = _FakeChat(response=response, error=error)
def _fake_client(response=None, error: BaseException | None = None) -> OpenRouter:
"""Return a stub typed as the SDK client the provider declares."""
return cast("OpenRouter", _FakeClient(response=response, error=error))
@pytest.mark.unit @pytest.mark.unit
class TestOpenRouterProviderInit: class TestOpenRouterProviderInit:
"""Verify OpenRouter provider initialization behavior.""" """Verify OpenRouter provider initialization behavior."""
@@ -37,13 +44,13 @@ class TestOpenRouterProviderInit:
def test_model_falls_back_to_default_when_unset(self): def test_model_falls_back_to_default_when_unset(self):
"""Provider uses adapter default model when provider_model is None.""" """Provider uses adapter default model when provider_model is None."""
settings = Settings(openrouter_api_key="test-key", provider_model=None) settings = Settings(openrouter_api_key="test-key", provider_model=None)
provider = OpenRouterTranscriptionProvider(settings=settings, client=_FakeClient()) provider = OpenRouterTranscriptionProvider(settings=settings, client=_fake_client())
assert provider.model == DEFAULT_OPENROUTER_MODEL assert provider.model == DEFAULT_OPENROUTER_MODEL
def test_model_uses_configured_value(self): def test_model_uses_configured_value(self):
"""Provider uses configured provider_model when present.""" """Provider uses configured provider_model when present."""
settings = Settings(openrouter_api_key="test-key", provider_model="vendor/custom-model") settings = Settings(openrouter_api_key="test-key", provider_model="vendor/custom-model")
provider = OpenRouterTranscriptionProvider(settings=settings, client=_FakeClient()) provider = OpenRouterTranscriptionProvider(settings=settings, client=_fake_client())
assert provider.model == "vendor/custom-model" assert provider.model == "vendor/custom-model"
@@ -61,7 +68,7 @@ class TestOpenRouterProviderTranscribe:
openrouter_http_referer="https://example.test", openrouter_http_referer="https://example.test",
openrouter_app_title="Transcription App", openrouter_app_title="Transcription App",
) )
provider = OpenRouterTranscriptionProvider(settings=settings, client=client) provider = OpenRouterTranscriptionProvider(settings=settings, client=cast("OpenRouter", client))
result = await provider.transcribe( result = await provider.transcribe(
prompt_text="Prompt body", prompt_text="Prompt body",
@@ -79,7 +86,10 @@ class TestOpenRouterProviderTranscribe:
"""Transcribe passes configured sampling parameters through to OpenRouter.""" """Transcribe passes configured sampling parameters through to OpenRouter."""
response = {"model": "vendor/model-a", "choices": [{"message": {"content": "Transcript text"}}]} response = {"model": "vendor/model-a", "choices": [{"message": {"content": "Transcript text"}}]}
client = _FakeClient(response=response) client = _FakeClient(response=response)
provider = OpenRouterTranscriptionProvider(settings=Settings(openrouter_api_key="test-key"), client=client) provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"),
client=cast("OpenRouter", client),
)
result = await provider.transcribe( result = await provider.transcribe(
prompt_text="Prompt body", prompt_text="Prompt body",
@@ -110,7 +120,7 @@ class TestOpenRouterProviderTranscribe:
} }
provider = OpenRouterTranscriptionProvider( provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"), settings=Settings(openrouter_api_key="test-key"),
client=_FakeClient(response=response), client=_fake_client(response=response),
) )
result = await provider.transcribe( result = await provider.transcribe(
@@ -134,7 +144,7 @@ class TestOpenRouterProviderTranscribe:
"""Transcribe converts SDK failures to ProviderError.""" """Transcribe converts SDK failures to ProviderError."""
provider = OpenRouterTranscriptionProvider( provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"), settings=Settings(openrouter_api_key="test-key"),
client=_FakeClient(error=RuntimeError("network down")), client=_fake_client(error=RuntimeError("network down")),
) )
with pytest.raises(ProviderError): with pytest.raises(ProviderError):
@@ -149,7 +159,7 @@ class TestOpenRouterProviderTranscribe:
"""Caller and shutdown cancellation must not be relabeled as a timeout.""" """Caller and shutdown cancellation must not be relabeled as a timeout."""
provider = OpenRouterTranscriptionProvider( provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"), settings=Settings(openrouter_api_key="test-key"),
client=_FakeClient(error=asyncio.CancelledError()), client=_fake_client(error=asyncio.CancelledError()),
) )
with pytest.raises(asyncio.CancelledError): with pytest.raises(asyncio.CancelledError):
@@ -166,7 +176,7 @@ class TestOpenRouterProviderTranscribe:
client = _FakeClient(response=response) client = _FakeClient(response=response)
provider = OpenRouterTranscriptionProvider( provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"), settings=Settings(openrouter_api_key="test-key"),
client=client, client=cast("OpenRouter", client),
) )
await provider.transcribe( await provider.transcribe(
@@ -185,7 +195,7 @@ class TestOpenRouterProviderTranscribe:
"""Transcribe raises ProviderResponseError for missing completion text.""" """Transcribe raises ProviderResponseError for missing completion text."""
provider = OpenRouterTranscriptionProvider( provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"), settings=Settings(openrouter_api_key="test-key"),
client=_FakeClient(response=SimpleNamespace(choices=[])), client=_fake_client(response=SimpleNamespace(choices=[])),
) )
with pytest.raises(ProviderResponseError): with pytest.raises(ProviderResponseError):
@@ -204,7 +214,7 @@ class TestOpenRouterProviderTranscribe:
} }
provider = OpenRouterTranscriptionProvider( provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"), settings=Settings(openrouter_api_key="test-key"),
client=_FakeClient(response=response), client=_fake_client(response=response),
) )
result = await provider.transcribe( result = await provider.transcribe(
+11 -3
View File
@@ -14,6 +14,7 @@ from transcription.db.models import JobSource
from transcription.db.models import Source from transcription.db.models import Source
from transcription.providers import RequestManifest from transcription.providers import RequestManifest
from transcription.providers import TranscriptionResult from transcription.providers import TranscriptionResult
from transcription.providers.evidence import SourceEvidenceReference
from transcription.providers.evidence import build_software_context from transcription.providers.evidence import build_software_context
from transcription.services import ServiceBundle from transcription.services import ServiceBundle
from transcription.services.documents import DocumentService from transcription.services.documents import DocumentService
@@ -62,7 +63,9 @@ def test_orientation_three_is_physically_rotated_and_metadata_removed(tmp_path):
with Image.open(path) as source_image, Image.open(io.BytesIO(result.content)) as derivative: with Image.open(path) as source_image, Image.open(io.BytesIO(result.content)) as derivative:
assert source_image.getexif()[274] == 3 assert source_image.getexif()[274] == 3
assert derivative.getexif().get(274, 1) == 1 assert derivative.getexif().get(274, 1) == 1
assert derivative.getpixel((0, 0))[2] > derivative.getpixel((0, 0))[0] pixel = derivative.getpixel((0, 0))
assert isinstance(pixel, tuple)
assert pixel[2] > pixel[0]
@pytest.mark.unit @pytest.mark.unit
@@ -130,7 +133,9 @@ async def test_resolve_provider_input_persists_exact_derivative(default_session_
assert provider_input.derivative_id == artifacts[0].id assert provider_input.derivative_id == artifacts[0].id
assert provider_input.path.read_bytes() != original assert provider_input.path.read_bytes() != original
assert hashlib.sha256(provider_input.path.read_bytes()).hexdigest() == provider_input.digest_sha256 assert hashlib.sha256(provider_input.path.read_bytes()).hexdigest() == provider_input.digest_sha256
assert artifacts[0].coordinate_metadata["original_orientation"] == 3 coordinate_metadata = artifacts[0].coordinate_metadata
assert coordinate_metadata is not None
assert coordinate_metadata["original_orientation"] == 3
@pytest.mark.integration @pytest.mark.integration
@@ -219,8 +224,11 @@ async def test_worker_sends_exact_derivative_and_links_attempt_evidence(
attempts = await services.sources.list_execution_attempts(source_id=source.id) attempts = await services.sources.list_execution_attempts(source_id=source.id)
artifacts = await services.sources.list_processing_artifacts(source_id=source.id) artifacts = await services.sources.list_processing_artifacts(source_id=source.id)
source_reference = captured["source_reference"] source_reference = captured["source_reference"]
assert isinstance(source_reference, SourceEvidenceReference)
captured_bytes = captured["bytes"]
assert isinstance(captured_bytes, bytes)
assert source_path.read_bytes() == original assert source_path.read_bytes() == original
assert hashlib.sha256(captured["bytes"]).hexdigest() == source_reference.digest_sha256 assert hashlib.sha256(captured_bytes).hexdigest() == source_reference.digest_sha256
assert source_reference.derivative_id is not None assert source_reference.derivative_id is not None
assert {artifact.artifact_type for artifact in artifacts} == { assert {artifact.artifact_type for artifact in artifacts} == {
"orientation_normalized_model_input", "orientation_normalized_model_input",
+3 -2
View File
@@ -2,6 +2,7 @@ from pathlib import Path
from uuid import uuid4 from uuid import uuid4
import pytest import pytest
from sqlmodel import col
from sqlmodel import select from sqlmodel import select
from transcription.config import Settings from transcription.config import Settings
@@ -63,7 +64,7 @@ async def test_create_job_for_document_sorts_sources_and_creates_links(async_ses
sources = ( sources = (
await async_session.exec( await async_session.exec(
select(Source).where(Source.document_id == document.id).order_by(Source.page_number) # pyright: ignore[reportArgumentType] select(Source).where(Source.document_id == document.id).order_by(col(Source.page_number))
) )
).all() ).all()
assert [source.upload_name for source in sources] == ["A_page.pdf", "b_page.pdf"] assert [source.upload_name for source in sources] == ["A_page.pdf", "b_page.pdf"]
@@ -99,7 +100,7 @@ async def test_create_document_job_stores_source_under_document_id_directory(asy
source = ( source = (
await async_session.exec( await async_session.exec(
select(Source).where(Source.document_id == result.document_id).order_by(Source.page_number) # pyright: ignore[reportArgumentType] select(Source).where(Source.document_id == result.document_id).order_by(col(Source.page_number))
) )
).first() ).first()
assert source is not None assert source is not None
+6 -2
View File
@@ -1,6 +1,7 @@
from uuid import uuid4 from uuid import uuid4
import pytest import pytest
from pydantic import JsonValue
from transcription.db.models import Document from transcription.db.models import Document
from transcription.db.models import DocumentPerson from transcription.db.models import DocumentPerson
@@ -303,8 +304,11 @@ async def test_update_job_source_transcription_persists_provider_json_payloads(d
JobSource(job_id=job.id, source_id=source.id, status=JobSourceStatus.PENDING) JobSource(job_id=job.id, source_id=source.id, status=JobSourceStatus.PENDING)
) )
metadata = {"finish_reason": "stop", "usage": {"input_tokens": 11, "output_tokens": 22, "total_tokens": 33}} metadata: dict[str, JsonValue] = {
raw_payload = {"id": "resp_xyz", "choices": [{"message": {"content": "provider transcript"}}]} "finish_reason": "stop",
"usage": {"input_tokens": 11, "output_tokens": 22, "total_tokens": 33},
}
raw_payload: dict[str, JsonValue] = {"id": "resp_xyz", "choices": [{"message": {"content": "provider transcript"}}]}
await transcriptions.update_job_source_transcription( await transcriptions.update_job_source_transcription(
job_id=job.id, job_id=job.id,
+4 -2
View File
@@ -1,6 +1,7 @@
"""Tests for transcription.config — settings loading, provider defaults, and paths.""" """Tests for transcription.config — settings loading, provider defaults, and paths."""
from pathlib import Path from pathlib import Path
from typing import Any
import pytest import pytest
from pydantic import ValidationError from pydantic import ValidationError
@@ -10,9 +11,9 @@ from transcription.config import Settings
from transcription.config import parse_cli_settings from transcription.config import parse_cli_settings
def _make_settings(**overrides) -> Settings: def _make_settings(**overrides: Any) -> Settings:
"""Build a Settings instance with a dummy API key unless overridden.""" """Build a Settings instance with a dummy API key unless overridden."""
defaults = {"openrouter_api_key": "test-key-abc123", "provider_models": None} defaults: dict[str, Any] = {"openrouter_api_key": "test-key-abc123", "provider_models": None}
defaults.update(overrides) defaults.update(overrides)
return Settings(**defaults) return Settings(**defaults)
@@ -168,6 +169,7 @@ def test_openrouter_client_timeout_tracks_the_configured_budget():
settings = Settings(openrouter_api_key="test-key", worker_provider_timeout_seconds=123.0) settings = Settings(openrouter_api_key="test-key", worker_provider_timeout_seconds=123.0)
provider = OpenRouterTranscriptionProvider(settings=settings) provider = OpenRouterTranscriptionProvider(settings=settings)
assert provider._capturing_client is not None
timeout = provider._capturing_client._client.timeout timeout = provider._capturing_client._client.timeout
assert timeout.read == 123.0 assert timeout.read == 123.0
+2 -1
View File
@@ -6,6 +6,7 @@ import pytest
import sqlalchemy as sa import sqlalchemy as sa
from sqlalchemy import inspect from sqlalchemy import inspect
from sqlalchemy.dialects import postgresql from sqlalchemy.dialects import postgresql
from sqlalchemy.exc import SAWarning
from sqlmodel import SQLModel from sqlmodel import SQLModel
from sqlmodel import select from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
@@ -148,7 +149,7 @@ async def test_create_all_declares_hot_path_indexes(tmp_path):
def test_metadata_has_no_unresolvable_table_cycle(): def test_metadata_has_no_unresolvable_table_cycle():
"""create_all must be able to order every table, including on PostgreSQL.""" """create_all must be able to order every table, including on PostgreSQL."""
with warnings.catch_warnings(): with warnings.catch_warnings():
warnings.simplefilter("error", sa.exc.SAWarning) warnings.simplefilter("error", SAWarning)
ordered = [table.name for table in SQLModel.metadata.sorted_tables] ordered = [table.name for table in SQLModel.metadata.sorted_tables]
assert ordered.index("source") < ordered.index("execution_attempt") assert ordered.index("source") < ordered.index("execution_attempt")
+9 -8
View File
@@ -1,5 +1,6 @@
"""Tests for the V2 SQLModel persistence layer and relationships.""" """Tests for the V2 SQLModel persistence layer and relationships."""
from typing import Any
from uuid import UUID from uuid import UUID
import pytest import pytest
@@ -17,8 +18,8 @@ from transcription.db.models import PersonRole
from transcription.db.models import Source from transcription.db.models import Source
def _make_document(**overrides) -> Document: def _make_document(**overrides: Any) -> Document:
defaults = { defaults: dict[str, Any] = {
"name": "letter bundle", "name": "letter bundle",
"notes": "Family correspondence", "notes": "Family correspondence",
} }
@@ -51,8 +52,8 @@ def _persist_document(session) -> Document:
return document return document
def _persist_person(session, **overrides) -> Person: def _persist_person(session, **overrides: Any) -> Person:
defaults = {"full_name": "Ada Lovelace"} defaults: dict[str, Any] = {"full_name": "Ada Lovelace"}
defaults.update(overrides) defaults.update(overrides)
person = Person(**defaults) person = Person(**defaults)
session.add(person) session.add(person)
@@ -69,8 +70,8 @@ def _persist_job(session, document: Document) -> Job:
return job return job
def _persist_source(session, document: Document, *, page_number: int = 1, **overrides) -> Source: def _persist_source(session, document: Document, *, page_number: int = 1, **overrides: Any) -> Source:
defaults = { defaults: dict[str, Any] = {
"document_id": document.id, "document_id": document.id,
"page_number": page_number, "page_number": page_number,
"upload_name": "letter.jpg", "upload_name": "letter.jpg",
@@ -88,8 +89,8 @@ def _persist_source(session, document: Document, *, page_number: int = 1, **over
return source return source
def _persist_job_source(session, job: Job, source: Source, **overrides) -> JobSource: def _persist_job_source(session, job: Job, source: Source, **overrides: Any) -> JobSource:
defaults = { defaults: dict[str, Any] = {
"job_id": job.id, "job_id": job.id,
"source_id": source.id, "source_id": source.id,
"status": JobSourceStatus.PENDING, "status": JobSourceStatus.PENDING,
+1 -1
View File
@@ -86,7 +86,7 @@ class TestPromptConfiguration:
assert execution.temperature == 0.2 assert execution.temperature == 0.2
assert execution.top_p == 0.9 assert execution.top_p == 0.9
with pytest.raises(ValidationError): with pytest.raises(ValidationError):
execution.prompt_name = "changed.md" execution.prompt_name = "changed.md" # ty: ignore[invalid-assignment]
def test_rejects_prompt_path_traversal_even_with_direct_loader_call(self, tmp_path): def test_rejects_prompt_path_traversal_even_with_direct_loader_call(self, tmp_path):
outside_prompt = tmp_path / "outside.md" outside_prompt = tmp_path / "outside.md"
+16 -3
View File
@@ -10,6 +10,7 @@ from uuid import uuid4
import httpx import httpx
import pytest import pytest
from pydantic import JsonValue
from transcription.benchmarking import EditorialAssessment from transcription.benchmarking import EditorialAssessment
from transcription.benchmarking import score_transcription from transcription.benchmarking import score_transcription
@@ -33,6 +34,18 @@ from transcription.services.sources import TranscriptionError
from transcription.services.sources import transcribe_document_image from transcription.services.sources import transcribe_document_image
def _json_object(value: JsonValue) -> dict[str, JsonValue]:
"""Narrow a JSON export member to an object, asserting the export shape."""
assert isinstance(value, dict)
return value
def _json_array(value: JsonValue) -> list[JsonValue]:
"""Narrow a JSON export member to an array, asserting the export shape."""
assert isinstance(value, list)
return value
class _ChunkedAsyncStream(httpx.AsyncByteStream): class _ChunkedAsyncStream(httpx.AsyncByteStream):
def __init__(self, chunks: list[bytes]): def __init__(self, chunks: list[bytes]):
self._chunks = chunks self._chunks = chunks
@@ -261,9 +274,9 @@ async def test_attempts_are_append_only_and_exported_with_integrity(default_sess
) )
) )
export = await sources.build_evidence_export(source_id=source.id) export = await sources.build_evidence_export(source_id=source.id)
assert export["source"]["digest_sha256"] == "a" * 64 assert _json_object(export["source"])["digest_sha256"] == "a" * 64
assert [item["attempt_number"] for item in export["attempts"]] == [1, 2] assert [_json_object(item)["attempt_number"] for item in _json_array(export["attempts"])] == [1, 2]
assert export["artifacts"][0]["id"] == str(artifact.id) assert _json_object(_json_array(export["artifacts"])[0])["id"] == str(artifact.id)
assert "file_path" not in json.dumps(export) assert "file_path" not in json.dumps(export)
with pytest.raises(JobDeleteBlockedError): with pytest.raises(JobDeleteBlockedError):
await jobs.delete_job_with_guardrails(job_id=job.id) await jobs.delete_job_with_guardrails(job_id=job.id)
+5 -3
View File
@@ -1,9 +1,11 @@
import asyncio import asyncio
import logging import logging
from typing import cast
import pytest import pytest
from transcription.services import ServiceBundle from transcription.services import ServiceBundle
from transcription.services.sources import SourceService
from transcription.worker import process_next_queued_job from transcription.worker import process_next_queued_job
from transcription.worker import run_worker_loop from transcription.worker import run_worker_loop
@@ -43,7 +45,7 @@ async def test_run_worker_loop_reuses_one_bundle_across_jobs(monkeypatch):
nonlocal closed nonlocal closed
closed = True closed = True
bundle = ServiceBundle(sources=_Sources()) # type: ignore[arg-type] bundle = ServiceBundle(sources=cast("SourceService", _Sources()))
monkeypatch.setattr( monkeypatch.setattr(
"transcription.worker.ServiceBundle.from_session_factory", "transcription.worker.ServiceBundle.from_session_factory",
classmethod(lambda _cls, _factory=None, **_kwargs: bundle), classmethod(lambda _cls, _factory=None, **_kwargs: bundle),
@@ -75,7 +77,7 @@ async def test_process_next_closes_provider_for_the_bundle_it_owns(monkeypatch):
nonlocal closed nonlocal closed
closed = True closed = True
bundle = ServiceBundle(sources=_Sources()) # type: ignore[arg-type] bundle = ServiceBundle(sources=cast("SourceService", _Sources()))
monkeypatch.setattr( monkeypatch.setattr(
"transcription.worker.ServiceBundle.from_session_factory", "transcription.worker.ServiceBundle.from_session_factory",
classmethod(lambda _cls, _factory=None, **_kwargs: bundle), classmethod(lambda _cls, _factory=None, **_kwargs: bundle),
@@ -101,7 +103,7 @@ async def test_process_next_leaves_a_caller_owned_bundle_open(monkeypatch):
nonlocal closed nonlocal closed
closed = True closed = True
bundle = ServiceBundle(sources=_Sources()) # type: ignore[arg-type] bundle = ServiceBundle(sources=cast("SourceService", _Sources()))
async def _no_job(*, services, session): async def _no_job(*, services, session):
_ = (services, session) _ = (services, session)
+5 -4
View File
@@ -2,9 +2,9 @@
from __future__ import annotations from __future__ import annotations
from collections.abc import AsyncGenerator
from collections.abc import Awaitable from collections.abc import Awaitable
from collections.abc import Callable from collections.abc import Callable
from collections.abc import Generator
from datetime import UTC from datetime import UTC
from datetime import datetime from datetime import datetime
from pathlib import Path from pathlib import Path
@@ -32,12 +32,13 @@ from transcription.db.models import Source
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
def app_client(tmp_path_factory: pytest.TempPathFactory) -> AsyncGenerator[tuple[FastAPI, TestClient]]: def app_client(tmp_path_factory: pytest.TempPathFactory) -> Generator[tuple[FastAPI, TestClient]]:
"""Provide a real application and test client backed by in-memory SQLite.""" """Provide a real application and test client backed by in-memory SQLite."""
tmp_path = tmp_path_factory.mktemp("ui") tmp_path = tmp_path_factory.mktemp("ui")
database = SqliteSettings(path=str(tmp_path / "ui-tests.db"))
settings = Settings( settings = Settings(
openrouter_api_key="test-key", openrouter_api_key="test-key",
database=SqliteSettings(path=str(tmp_path / "ui-tests.db")), database=database,
environment="test", environment="test",
bootstrap_schema_on_startup=True, bootstrap_schema_on_startup=True,
upload_dir=tmp_path / "uploads", upload_dir=tmp_path / "uploads",
@@ -48,7 +49,7 @@ def app_client(tmp_path_factory: pytest.TempPathFactory) -> AsyncGenerator[tuple
with TestClient(app) as client: with TestClient(app) as client:
runtime_path = Path(str(app.state.runtime.engine.url.database)).resolve() runtime_path = Path(str(app.state.runtime.engine.url.database)).resolve()
expected_path = Path(settings.database.path).resolve() expected_path = Path(database.path).resolve()
if runtime_path != expected_path: if runtime_path != expected_path:
raise RuntimeError( raise RuntimeError(
"Refusing to initialize destructive UI fixtures against " "Refusing to initialize destructive UI fixtures against "
+3 -1
View File
@@ -1,12 +1,13 @@
"""Tests for the sources page routes and Source model properties.""" """Tests for the sources page routes and Source model properties."""
from pathlib import Path from pathlib import Path
from uuid import uuid4
import pytest import pytest
from sqlalchemy.orm import selectinload
from sqlmodel import select from sqlmodel import select
from transcription.db import session_scope from transcription.db import session_scope
from transcription.db.loading import selectinload
from transcription.db.models import Document from transcription.db.models import Document
from transcription.db.models import Job from transcription.db.models import Job
from transcription.db.models import JobSourceStatus from transcription.db.models import JobSourceStatus
@@ -24,6 +25,7 @@ class TestSourceModelProperties:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_source_properties_with_no_job_sources(self): async def test_source_properties_with_no_job_sources(self):
source = Source( source = Source(
document_id=uuid4(),
page_number=1, page_number=1,
upload_name="page_one.png", upload_name="page_one.png",
filename="stored_page_one.png", filename="stored_page_one.png",
+12 -11
View File
@@ -7,7 +7,6 @@ import shlex
import shutil import shutil
import subprocess import subprocess
import sys import sys
from ctypes import wintypes
from datetime import datetime from datetime import datetime
from pathlib import Path from pathlib import Path
@@ -20,17 +19,16 @@ def show_phase(title: str) -> None:
print(f"========== {title} ==========") print(f"========== {title} ==========")
def test_file_unlocked(path: Path) -> bool: if sys.platform == "win32":
# `ctypes.wintypes` raises on import off Windows, and `fcntl` does not exist
# on Windows, so the two implementations are selected at module level where a
# type checker can narrow `sys.platform` and analyze only the live branch.
from ctypes import wintypes
def test_file_unlocked(path: Path) -> bool:
if not path.exists(): if not path.exists():
return True return True
if os.name == "nt":
return _test_file_unlocked_windows(path)
return _test_file_unlocked_posix(path)
def _test_file_unlocked_windows(path: Path) -> bool:
generic_read = 0x80000000 generic_read = 0x80000000
generic_write = 0x40000000 generic_write = 0x40000000
open_existing = 3 open_existing = 3
@@ -66,10 +64,13 @@ def _test_file_unlocked_windows(path: Path) -> bool:
kernel32.CloseHandle(handle) kernel32.CloseHandle(handle)
return True return True
else:
def _test_file_unlocked_posix(path: Path) -> bool:
import fcntl import fcntl
def test_file_unlocked(path: Path) -> bool:
if not path.exists():
return True
fd = os.open(path, os.O_RDWR) fd = os.open(path, os.O_RDWR)
try: try:
try: try: