V4.6 Phase 7: drive ty check to zero and add a blocking quality gate [HIGH-06]

Baseline was 207 diagnostics. Two real bugs were hiding in the noise:

- tools/run_destructive_tests.py imported ctypes.wintypes at module scope,
  which raises on non-Windows, and called fcntl unconditionally. The Windows
  and POSIX implementations now live under a module-level sys.platform split.
- tests/ui/test_sources_page.py constructed Source(...) without document_id.

Structural fixes, not suppressions:

- New src/transcription/db/loading.py owns the SQLModel-field to
  QueryableAttribute reinterpretation via orm_attribute()/selectinload()/
  defer(). This removed 42 "# pyright: ignore[reportArgumentType]" comments
  across documents/jobs/people/sources. Its docstring records that
  selectinload(A.b, B.c) is NOT equivalent to the chained form: varargs
  applies the selectin strategy only to the last path element, which under
  lazy="raise" raises InvalidRequestError at render time.
- db/session.py transaction_scope no longer accepts or yields
  AsyncSessionTransaction. No caller ever passed one, sessionmaker.begin()
  yields an AsyncSession, and the dead branch was latently buggy because
  services call .exec(). Cleared 7 workflows.py diagnostics.
- services/registry.py RegistryService is bound by a new RegistryEntry
  Protocol instead of bare SQLModel, so the shared implementation can read
  id/label/normalized_label/is_active. Cleared 9 diagnostics.
- Column expressions in sources.py/jobs.py/test_store.py wrap in sqlmodel
  col(), the idiom already used in registry.py.
- read_source_navigation wraps its literal tuple bounds in literal().
- normalization.py narrows with isinstance(image, TiffImageFile) rather than
  comparing image.format, since tag_v2 is TIFF-only.
- linked_people.render uses @ui.refreshable_method, the NiceGUI API for bound
  methods.
- The OpenRouter capturing client re-raises ResponseNotRead when the response
  stream is not async rather than mis-wrapping it.

Tooling gate:

- New .pre-commit-config.yaml runs ruff check and ty check as blocking hooks.
  No pre-commit config previously existed. Negative-tested: injecting a type
  error fails both hooks.
- The last two "# pyright: ignore" comments (config.py) are removed; ty does
  not honor pyright directives. One "# ty: ignore" remains, in
  tests/test_prompts.py, where the test deliberately assigns to a frozen
  field to assert ValidationError.
- asyncio_default_fixture_loop_scope is pinned to "function" so
  pytest-asyncio behavior does not shift on upgrade.

Verification: ruff check clean, ty check reports 0 diagnostics, 292 passed
and 4 skipped, pre-commit passes and demonstrably fails on a regression, and
tools/run_destructive_tests.py runs on Windows.

Co-authored-by: Copilot App <[email protected]>
This commit is contained in:
zoltan57
2026-08-17 19:57:23 -05:00
co-authored by Copilot App
parent 597be2691c
commit 66e2dce465
28 changed files with 316 additions and 184 deletions
+18
View File
@@ -0,0 +1,18 @@
# Quality gate for V4.6 [HIGH-06]. Both hooks are blocking: a regression in
# `ruff check` or `ty check` fails the commit.
repos:
- repo: local
hooks:
- id: ruff
name: ruff check
entry: ruff check
language: system
types_or: [python, pyi]
require_serial: true
- id: ty
name: ty check
entry: ty check
language: system
types_or: [python, pyi]
pass_filenames: false
require_serial: true
+1
View File
@@ -41,6 +41,7 @@ dev = [
[tool.pytest.ini_options]
addopts = "--strict-markers -q"
asyncio_mode = "strict"
asyncio_default_fixture_loop_scope = "function"
filterwarnings = [
"error:coroutine .* was never awaited:RuntimeWarning",
]
+2 -2
View File
@@ -172,13 +172,13 @@ class Settings(BaseSettings):
@cache
def get_settings(**kwargs: Any) -> Settings:
"""Load cached settings without reading process CLI arguments."""
return Settings(_cli_parse_args=False, **kwargs) # pyright: ignore[reportCallIssue]
return Settings(_cli_parse_args=False, **kwargs)
def parse_cli_settings(args: Sequence[str] | None = None) -> Settings:
"""Load settings with CLI arguments at the executable boundary."""
cli_args = True if args is None else list(args)
return Settings(_cli_parse_args=cli_args) # pyright: ignore[reportCallIssue]
return Settings(_cli_parse_args=cli_args)
LOGGING_CONFIG: dict[str, Any] = {
+45
View File
@@ -0,0 +1,45 @@
"""Typed loader-option wrappers for SQLModel relationship attributes.
SQLModel declares relationships with their runtime Python type, so
``Document.jobs`` is annotated ``list[Job]`` even though at runtime it is an
``InstrumentedAttribute``. SQLAlchemy's loader options are typed against
``QueryableAttribute``, so every eager-load call site reads as a type error to a
static checker even though the code is correct.
These wrappers put that reinterpretation in one documented place instead of
scattering a suppression comment across every eager-load call. Import
``selectinload`` and ``defer`` from here rather than from ``sqlalchemy.orm``.
Multi-level eager loads must keep using the chained form --
``selectinload(A.b).selectinload(orm_attribute(B.c))`` -- and not the varargs
form ``selectinload(A.b, B.c)``. The two produce the same loader path, but
varargs applies the selectin strategy only to the last element while the
intermediate falls back to its default strategy. Every relationship here
declares ``lazy="raise"``, so the varargs form raises at render time.
"""
from __future__ import annotations
from typing import Any
from typing import cast
from sqlalchemy.orm import defer as _defer
from sqlalchemy.orm import selectinload as _selectinload
from sqlalchemy.orm.attributes import QueryableAttribute
from sqlalchemy.orm.strategy_options import _AbstractLoad
def orm_attribute(attribute: object) -> QueryableAttribute[Any]:
"""Reinterpret a SQLModel relationship or field as its ORM descriptor."""
return cast("QueryableAttribute[Any]", attribute)
def selectinload(*keys: object) -> _AbstractLoad:
"""``sqlalchemy.orm.selectinload`` accepting SQLModel-annotated attributes."""
return _selectinload(*(orm_attribute(key) for key in keys))
def defer(*keys: object, raiseload: bool = False) -> _AbstractLoad:
"""``sqlalchemy.orm.defer`` accepting SQLModel-annotated attributes."""
first, *rest = (orm_attribute(key) for key in keys)
return _defer(first, *rest, raiseload=raiseload)
+2 -1
View File
@@ -23,6 +23,7 @@ from sqlalchemy import Uuid
from sqlalchemy import inspect as sqlalchemy_inspect
from sqlalchemy.dialects.postgresql import JSONB
from sqlalchemy.exc import NoInspectionAvailable
from sqlalchemy.orm.state import InstanceState
from sqlalchemy.types import TypeDecorator
from sqlmodel import Field
from sqlmodel import Relationship
@@ -38,7 +39,7 @@ def _loaded_attribute(instance: object, attribute: str) -> Any | None:
without catching exceptions indiscriminately.
"""
try:
state = sqlalchemy_inspect(instance)
state: InstanceState[Any] = sqlalchemy_inspect(instance, raiseerr=True)
except NoInspectionAvailable:
return None
if attribute in state.unloaded:
+8 -16
View File
@@ -3,7 +3,6 @@ from contextlib import asynccontextmanager
from typing import Annotated
from fastapi import Depends
from sqlalchemy.ext.asyncio import AsyncSessionTransaction
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession
@@ -86,17 +85,13 @@ async def transaction_scope(
settings: Settings | None = None,
database_url: str | None = None,
session_factory: SessionFactory | None = None,
session: AsyncSession | AsyncSessionTransaction | None = None,
) -> AsyncGenerator[AsyncSession | AsyncSessionTransaction]:
match session:
case AsyncSession() as async_session:
if not async_session.in_transaction():
raise RuntimeError("A supplied session must have an active transaction")
yield async_session
return
case AsyncSessionTransaction() as async_transaction:
yield async_transaction
return
session: AsyncSession | None = None,
) -> AsyncGenerator[AsyncSession]:
if session is not None:
if not session.in_transaction():
raise RuntimeError("A supplied session must have an active transaction")
yield session
return
active_session_factory = session_factory or resolve_session_factory(
database_url,
@@ -106,7 +101,4 @@ async def transaction_scope(
yield owned_session
type TransactionScopeDep = Annotated[
AsyncSession | AsyncSessionTransaction,
Depends(transaction_scope),
]
type TransactionScopeDep = Annotated[AsyncSession, Depends(transaction_scope)]
+4 -1
View File
@@ -74,7 +74,10 @@ class _CapturingAsyncClient:
try:
self.last_body = response.content
except httpx.ResponseNotRead:
response.stream = _CapturingAsyncByteStream(response.stream, self._capture_body)
stream = response.stream
if not isinstance(stream, httpx.AsyncByteStream):
raise
response.stream = _CapturingAsyncByteStream(stream, self._capture_body)
return response
def build_request(self, *args: Any, **kwargs: Any) -> httpx.Request:
+15 -14
View File
@@ -10,13 +10,14 @@ from uuid import UUID
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.orm import selectinload
from sqlmodel import SQLModel
from sqlmodel import col
from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
from ..config import Settings
from ..db.loading import orm_attribute
from ..db.loading import selectinload
from ..db.models import Document
from ..db.models import DocumentPerson
from ..db.models import DocumentType
@@ -189,8 +190,8 @@ class DocumentService(ServiceBase):
session=_session,
document_id=document_id,
options=(
selectinload(Document.jobs), # pyright: ignore[reportArgumentType]
selectinload(Document.sources), # pyright: ignore[reportArgumentType]
selectinload(Document.jobs),
selectinload(Document.sources),
),
suggestion="Re-upload the source document and retry.",
)
@@ -218,9 +219,9 @@ class DocumentService(ServiceBase):
session=_session,
document_id=document.id,
options=(
selectinload(Document.jobs), # pyright: ignore[reportArgumentType]
selectinload(Document.sources), # pyright: ignore[reportArgumentType]
selectinload(Document.document_people), # pyright: ignore[reportArgumentType]
selectinload(Document.jobs),
selectinload(Document.sources),
selectinload(Document.document_people),
),
)
@@ -275,9 +276,9 @@ class DocumentService(ServiceBase):
"""List documents with relations needed by the archival table."""
async with self._session_scope(session) as _session:
query = select(Document).options(
selectinload(Document.document_people).selectinload(DocumentPerson.person), # pyright: ignore[reportArgumentType]
selectinload(Document.document_people).selectinload(DocumentPerson.role_ref), # pyright: ignore[reportArgumentType]
selectinload(Document.document_type_ref), # pyright: ignore[reportArgumentType]
selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.person)),
selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
selectinload(Document.document_type_ref),
)
result = await _session.exec(query)
return result.all()
@@ -288,11 +289,11 @@ class DocumentService(ServiceBase):
query = (
select(Document)
.options(
selectinload(Document.jobs), # pyright: ignore[reportArgumentType]
selectinload(Document.sources), # pyright: ignore[reportArgumentType]
selectinload(Document.document_people).selectinload(DocumentPerson.person), # pyright: ignore[reportArgumentType]
selectinload(Document.document_people).selectinload(DocumentPerson.role_ref), # pyright: ignore[reportArgumentType]
selectinload(Document.document_type_ref), # pyright: ignore[reportArgumentType]
selectinload(Document.jobs),
selectinload(Document.sources),
selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.person)),
selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
selectinload(Document.document_type_ref),
)
.where(Document.id == document_id)
.execution_options(populate_existing=True)
+18 -14
View File
@@ -6,10 +6,12 @@ from pathlib import Path
from uuid import UUID
from sqlalchemy import func
from sqlalchemy.orm import selectinload
from sqlmodel import col
from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
from ..db.loading import orm_attribute
from ..db.loading import selectinload
from ..db.models import ExecutionAttempt
from ..db.models import Job
from ..db.models import JobSource
@@ -64,8 +66,8 @@ class JobService(ServiceBase):
query = (
select(Job)
.options(
selectinload(Job.document), # pyright: ignore[reportArgumentType]
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType]
selectinload(Job.document),
selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
)
.where(Job.id == job_id)
.execution_options(populate_existing=True)
@@ -100,13 +102,15 @@ class JobService(ServiceBase):
"""Query jobs from the database based on provided filters."""
async with self._session_scope(session) as _session:
query = select(Job).options(
selectinload(Job.document), # pyright: ignore[reportArgumentType]
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType]
selectinload(Job.document),
selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
)
if status is not None:
query = query.where(Job.status == status)
if filename is not None:
query = query.where(Job.job_sources.any(JobSource.source.has(Source.filename == filename)))
query = query.where(
col(Job.job_sources).any(col(JobSource.source).has(col(Source.filename) == filename))
)
result = await _session.exec(query)
return result.all()
@@ -118,8 +122,8 @@ class JobService(ServiceBase):
"""List all jobs in the database with eagerly loaded documents."""
async with self._session_scope(session) as _session:
query = select(Job).options(
selectinload(Job.document), # pyright: ignore[reportArgumentType]
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType]
selectinload(Job.document),
selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
)
result = await _session.exec(query)
return result.all()
@@ -152,8 +156,8 @@ class JobService(ServiceBase):
query = (
select(Job)
.options(
selectinload(Job.document), # pyright: ignore[reportArgumentType]
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType]
selectinload(Job.document),
selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
)
.where(Job.id == job_id)
.execution_options(populate_existing=True)
@@ -184,7 +188,7 @@ class JobService(ServiceBase):
select(Job)
.where(Job.status == JobStatus.QUEUED)
# Break ties by id so "next" is stable when two rows share close timestamps.
.order_by(Job.date_created, Job.id) # pyright: ignore[reportArgumentType]
.order_by(col(Job.date_created), col(Job.id))
.limit(1)
)
if _session.get_bind().dialect.name == "postgresql":
@@ -304,7 +308,7 @@ class JobService(ServiceBase):
(
await session.exec(
select(ProcessingArtifact).where(
ProcessingArtifact.execution_attempt_id.in_(attempt_ids)
col(ProcessingArtifact.execution_attempt_id).in_(attempt_ids)
)
)
).all()
@@ -351,7 +355,7 @@ class JobService(ServiceBase):
query = (
select(Job)
.options(
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType]
selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
)
.where(Job.id == job_id)
.execution_options(populate_existing=True)
@@ -388,7 +392,7 @@ class JobService(ServiceBase):
query = (
select(Job)
.options(
selectinload(Job.job_sources).selectinload(JobSource.source), # pyright: ignore[reportArgumentType]
selectinload(Job.job_sources).selectinload(orm_attribute(JobSource.source)),
)
.where(Job.id == job_id)
.execution_options(populate_existing=True)
+2 -1
View File
@@ -10,6 +10,7 @@ from pathlib import Path
from PIL import Image
from PIL import UnidentifiedImageError
from PIL.TiffImagePlugin import TiffImageFile
from transcription.errors import AppError
from transcription.errors import ErrorCategory
@@ -65,7 +66,7 @@ def normalize_orientation(path: str | Path, *, media_type: str) -> OrientationNo
return None
transpose, rotation = transformation
if image.format == "TIFF":
if isinstance(image, TiffImageFile):
original_width = int(image.tag_v2.get(256, image.width))
original_height = int(image.tag_v2.get(257, image.height))
# Pillow applies TIFF orientation while decoding; copying freezes those upright pixels.
+8 -7
View File
@@ -12,7 +12,6 @@ from uuid import UUID
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.orm import selectinload
from sqlmodel import SQLModel
from sqlmodel import col
from sqlmodel import select
@@ -20,6 +19,8 @@ from sqlmodel.ext.asyncio.session import AsyncSession
from ..config import Settings
from ..config import get_settings
from ..db.loading import orm_attribute
from ..db.loading import selectinload
from ..db.models import Document
from ..db.models import DocumentPerson
from ..db.models import Person
@@ -144,7 +145,7 @@ class PeopleService(ServiceBase):
existing = await _session.get(
Person,
person.id,
options=(selectinload(Person.document_people),), # pyright: ignore[reportArgumentType]
options=(selectinload(Person.document_people),),
)
if existing is None:
raise self._not_found(f"Person with id {person.id} not found")
@@ -209,8 +210,8 @@ class PeopleService(ServiceBase):
query = (
select(Person)
.options(
selectinload(Person.document_people).selectinload(DocumentPerson.document), # pyright: ignore[reportArgumentType]
selectinload(Person.document_people).selectinload(DocumentPerson.role_ref), # pyright: ignore[reportArgumentType]
selectinload(Person.document_people).selectinload(orm_attribute(DocumentPerson.document)),
selectinload(Person.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
)
.where(Person.id == person_id)
.execution_options(populate_existing=True)
@@ -329,9 +330,9 @@ class PeopleService(ServiceBase):
) -> Sequence[DocumentPerson]:
async with self._session_scope(session) as _session:
query = select(DocumentPerson).options(
selectinload(DocumentPerson.document), # pyright: ignore[reportArgumentType]
selectinload(DocumentPerson.person), # pyright: ignore[reportArgumentType]
selectinload(DocumentPerson.role_ref), # pyright: ignore[reportArgumentType]
selectinload(DocumentPerson.document),
selectinload(DocumentPerson.person),
selectinload(DocumentPerson.role_ref),
)
if document_id is not None:
query = query.where(DocumentPerson.document_id == document_id)
+18 -1
View File
@@ -12,6 +12,7 @@ from __future__ import annotations
from abc import abstractmethod
from collections.abc import Sequence
from typing import Any
from typing import Protocol
from uuid import UUID
from sqlalchemy import func
@@ -26,7 +27,23 @@ from ..errors import ErrorCategory
from .base import ServiceBase
class RegistryService[ModelT: SQLModel](ServiceBase):
class RegistryEntry(Protocol):
"""Structural contract every registry table row satisfies.
Bounding ``RegistryService`` by this protocol rather than by bare ``SQLModel``
lets the shared implementation read ``id``/``label``/``normalized_label``/
``is_active`` off the model class without suppressions.
"""
id: UUID
label: str
normalized_label: str
is_active: bool
def __init__(self, /, **data: Any) -> None: ...
class RegistryService[ModelT: RegistryEntry](ServiceBase):
"""Generic create/read/update/delete behavior for a registry table.
Subclasses declare the model, the error type, the user-facing noun, and the
+28 -26
View File
@@ -25,10 +25,9 @@ from pydantic import TypeAdapter
from pydantic import ValidationError
from sqlalchemy import func
from sqlalchemy import inspect as sqlalchemy_inspect
from sqlalchemy import literal
from sqlalchemy import tuple_
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.orm import defer
from sqlalchemy.orm import selectinload
from sqlmodel import col
from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
@@ -55,6 +54,9 @@ from transcription.providers import TransportEvidence
from transcription.providers import get_transcription_provider
from transcription.providers.evidence import canonical_json_bytes
from ..db.loading import defer
from ..db.loading import orm_attribute
from ..db.loading import selectinload
from .base import ServiceBase
from .normalization import ORIENTATION_PRODUCER
from .normalization import ORIENTATION_PRODUCER_VERSION
@@ -195,8 +197,8 @@ class SourceService(ServiceBase):
query = (
select(Source)
.options(
selectinload(Source.document), # pyright: ignore[reportArgumentType]
selectinload(Source.job_sources).selectinload(JobSource.job), # pyright: ignore[reportArgumentType]
selectinload(Source.document),
selectinload(Source.job_sources).selectinload(orm_attribute(JobSource.job)),
)
.where(Source.id == source_id)
.execution_options(populate_existing=True)
@@ -226,11 +228,11 @@ class SourceService(ServiceBase):
async with self._session_scope(session) as _session:
query = (
select(ExecutionAttempt)
.options(defer(ExecutionAttempt.transport_body)) # pyright: ignore[reportArgumentType]
.options(defer(ExecutionAttempt.transport_body))
.where(ExecutionAttempt.job_source_id == job_source_id)
.order_by(
ExecutionAttempt.attempt_number.desc(), # pyright: ignore[reportAttributeAccessIssue]
ExecutionAttempt.id.desc(), # pyright: ignore[reportAttributeAccessIssue]
col(ExecutionAttempt.attempt_number).desc(),
col(ExecutionAttempt.id).desc(),
)
.limit(1)
)
@@ -250,7 +252,7 @@ class SourceService(ServiceBase):
async with self._session_scope(session) as _session:
source = await self._read_source(session=_session, source_id=source_id)
position = (col(Source.page_number), col(Source.id))
current = (source.page_number, source_id)
current = (literal(source.page_number), literal(source_id))
previous_query = (
select(col(Source.id))
@@ -290,8 +292,8 @@ class SourceService(ServiceBase):
session=_session,
source_id=source_id,
options=(
selectinload(Source.job_sources), # pyright: ignore[reportArgumentType]
selectinload(Source.processing_artifacts), # pyright: ignore[reportArgumentType]
selectinload(Source.job_sources),
selectinload(Source.processing_artifacts),
),
)
@@ -379,7 +381,7 @@ class SourceService(ServiceBase):
job_source = await _session.get(
JobSource,
job_source_id,
options=(selectinload(JobSource.source),), # pyright: ignore[reportArgumentType]
options=(selectinload(JobSource.source),),
)
if job_source is None:
raise TranscriptionNotFoundError(
@@ -444,8 +446,8 @@ class SourceService(ServiceBase):
session=_session,
source_id=source_id,
options=(
selectinload(Source.job_sources), # pyright: ignore[reportArgumentType]
selectinload(Source.processing_artifacts), # pyright: ignore[reportArgumentType]
selectinload(Source.job_sources),
selectinload(Source.processing_artifacts),
),
)
@@ -511,8 +513,8 @@ class SourceService(ServiceBase):
"""List job-source records, optionally filtered by job."""
async with self._session_scope(session) as _session:
query = select(JobSource).options(
selectinload(JobSource.job), # pyright: ignore[reportArgumentType]
selectinload(JobSource.source), # pyright: ignore[reportArgumentType]
selectinload(JobSource.job),
selectinload(JobSource.source),
)
if job_id is not None:
query = query.where(JobSource.job_id == job_id)
@@ -720,10 +722,10 @@ class SourceService(ServiceBase):
if job_id is not None:
query = query.where(ExecutionAttempt.job_id == job_id)
query = query.order_by(
ExecutionAttempt.job_id,
ExecutionAttempt.source_id,
ExecutionAttempt.attempt_number,
ExecutionAttempt.id,
col(ExecutionAttempt.job_id),
col(ExecutionAttempt.source_id),
col(ExecutionAttempt.attempt_number),
col(ExecutionAttempt.id),
)
return (await _session.exec(query)).all()
@@ -987,7 +989,7 @@ class SourceService(ServiceBase):
query = (
select(ProcessingArtifact)
.where(ProcessingArtifact.source_id == source_id)
.order_by(ProcessingArtifact.created_at, ProcessingArtifact.id)
.order_by(col(ProcessingArtifact.created_at), col(ProcessingArtifact.id))
.limit(limit)
)
return (await _session.exec(query)).all()
@@ -1003,9 +1005,9 @@ class SourceService(ServiceBase):
async with self._session_scope(session) as _session:
query = (
select(ProcessingArtifact)
.options(defer(ProcessingArtifact.inline_payload)) # pyright: ignore[reportArgumentType]
.options(defer(ProcessingArtifact.inline_payload))
.where(ProcessingArtifact.source_id == source_id)
.order_by(ProcessingArtifact.created_at, ProcessingArtifact.id)
.order_by(col(ProcessingArtifact.created_at), col(ProcessingArtifact.id))
.limit(limit)
)
return (await _session.exec(query)).all()
@@ -1138,10 +1140,10 @@ class SourceService(ServiceBase):
async with self._session_scope(session) as _session:
query = (
select(Source)
.join(JobSource, JobSource.source_id == Source.id)
.where(JobSource.job_id == job_id)
.where(Source.revised_text.is_not(None))
.order_by(Source.date_revised) # pyright: ignore[reportArgumentType]
.join(JobSource, col(JobSource.source_id) == col(Source.id))
.where(col(JobSource.job_id) == job_id)
.where(col(Source.revised_text).is_not(None))
.order_by(col(Source.date_revised))
)
result = await _session.exec(query)
return result.all()
@@ -50,7 +50,7 @@ class LinkedPeopleEditor:
"""Return the complete staged link set for persistence."""
return [DocumentPersonInput(person_id=link.person_id, role_id=link.role_id) for link in self.links]
@ui.refreshable
@ui.refreshable_method
def render(self) -> None:
ui.label("Linked People").classes("text-sm font-semibold ui-text-primary mt-2")
ui.button("Create new person", on_click=lambda: ui.navigate.to("/people/new"), icon="person_add").props(
+1 -1
View File
@@ -407,7 +407,7 @@ def _render_provider_evidence(
def _artifact_display(artifacts: list[ProcessingArtifact]) -> list[dict[str, object]] | None:
payload = [
payload: list[dict[str, object]] = [
{
"id": str(artifact.id),
"type": artifact.artifact_type,
+22 -12
View File
@@ -2,8 +2,10 @@
import asyncio
from types import SimpleNamespace
from typing import cast
import pytest
from openrouter import OpenRouter
from transcription.config import Settings
from transcription.providers.base import ProviderError
@@ -13,7 +15,7 @@ from transcription.providers.openrouter import OpenRouterTranscriptionProvider
class _FakeChat:
def __init__(self, response=None, error: Exception | None = None):
def __init__(self, response=None, error: BaseException | None = None):
self._response = response
self._error = error
self.calls = []
@@ -26,10 +28,15 @@ class _FakeChat:
class _FakeClient:
def __init__(self, response=None, error: Exception | None = None):
def __init__(self, response=None, error: BaseException | None = None):
self.chat = _FakeChat(response=response, error=error)
def _fake_client(response=None, error: BaseException | None = None) -> OpenRouter:
"""Return a stub typed as the SDK client the provider declares."""
return cast("OpenRouter", _FakeClient(response=response, error=error))
@pytest.mark.unit
class TestOpenRouterProviderInit:
"""Verify OpenRouter provider initialization behavior."""
@@ -37,13 +44,13 @@ class TestOpenRouterProviderInit:
def test_model_falls_back_to_default_when_unset(self):
"""Provider uses adapter default model when provider_model is None."""
settings = Settings(openrouter_api_key="test-key", provider_model=None)
provider = OpenRouterTranscriptionProvider(settings=settings, client=_FakeClient())
provider = OpenRouterTranscriptionProvider(settings=settings, client=_fake_client())
assert provider.model == DEFAULT_OPENROUTER_MODEL
def test_model_uses_configured_value(self):
"""Provider uses configured provider_model when present."""
settings = Settings(openrouter_api_key="test-key", provider_model="vendor/custom-model")
provider = OpenRouterTranscriptionProvider(settings=settings, client=_FakeClient())
provider = OpenRouterTranscriptionProvider(settings=settings, client=_fake_client())
assert provider.model == "vendor/custom-model"
@@ -61,7 +68,7 @@ class TestOpenRouterProviderTranscribe:
openrouter_http_referer="https://example.test",
openrouter_app_title="Transcription App",
)
provider = OpenRouterTranscriptionProvider(settings=settings, client=client)
provider = OpenRouterTranscriptionProvider(settings=settings, client=cast("OpenRouter", client))
result = await provider.transcribe(
prompt_text="Prompt body",
@@ -79,7 +86,10 @@ class TestOpenRouterProviderTranscribe:
"""Transcribe passes configured sampling parameters through to OpenRouter."""
response = {"model": "vendor/model-a", "choices": [{"message": {"content": "Transcript text"}}]}
client = _FakeClient(response=response)
provider = OpenRouterTranscriptionProvider(settings=Settings(openrouter_api_key="test-key"), client=client)
provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"),
client=cast("OpenRouter", client),
)
result = await provider.transcribe(
prompt_text="Prompt body",
@@ -110,7 +120,7 @@ class TestOpenRouterProviderTranscribe:
}
provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"),
client=_FakeClient(response=response),
client=_fake_client(response=response),
)
result = await provider.transcribe(
@@ -134,7 +144,7 @@ class TestOpenRouterProviderTranscribe:
"""Transcribe converts SDK failures to ProviderError."""
provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"),
client=_FakeClient(error=RuntimeError("network down")),
client=_fake_client(error=RuntimeError("network down")),
)
with pytest.raises(ProviderError):
@@ -149,7 +159,7 @@ class TestOpenRouterProviderTranscribe:
"""Caller and shutdown cancellation must not be relabeled as a timeout."""
provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"),
client=_FakeClient(error=asyncio.CancelledError()),
client=_fake_client(error=asyncio.CancelledError()),
)
with pytest.raises(asyncio.CancelledError):
@@ -166,7 +176,7 @@ class TestOpenRouterProviderTranscribe:
client = _FakeClient(response=response)
provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"),
client=client,
client=cast("OpenRouter", client),
)
await provider.transcribe(
@@ -185,7 +195,7 @@ class TestOpenRouterProviderTranscribe:
"""Transcribe raises ProviderResponseError for missing completion text."""
provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"),
client=_FakeClient(response=SimpleNamespace(choices=[])),
client=_fake_client(response=SimpleNamespace(choices=[])),
)
with pytest.raises(ProviderResponseError):
@@ -204,7 +214,7 @@ class TestOpenRouterProviderTranscribe:
}
provider = OpenRouterTranscriptionProvider(
settings=Settings(openrouter_api_key="test-key"),
client=_FakeClient(response=response),
client=_fake_client(response=response),
)
result = await provider.transcribe(
+11 -3
View File
@@ -14,6 +14,7 @@ from transcription.db.models import JobSource
from transcription.db.models import Source
from transcription.providers import RequestManifest
from transcription.providers import TranscriptionResult
from transcription.providers.evidence import SourceEvidenceReference
from transcription.providers.evidence import build_software_context
from transcription.services import ServiceBundle
from transcription.services.documents import DocumentService
@@ -62,7 +63,9 @@ def test_orientation_three_is_physically_rotated_and_metadata_removed(tmp_path):
with Image.open(path) as source_image, Image.open(io.BytesIO(result.content)) as derivative:
assert source_image.getexif()[274] == 3
assert derivative.getexif().get(274, 1) == 1
assert derivative.getpixel((0, 0))[2] > derivative.getpixel((0, 0))[0]
pixel = derivative.getpixel((0, 0))
assert isinstance(pixel, tuple)
assert pixel[2] > pixel[0]
@pytest.mark.unit
@@ -130,7 +133,9 @@ async def test_resolve_provider_input_persists_exact_derivative(default_session_
assert provider_input.derivative_id == artifacts[0].id
assert provider_input.path.read_bytes() != original
assert hashlib.sha256(provider_input.path.read_bytes()).hexdigest() == provider_input.digest_sha256
assert artifacts[0].coordinate_metadata["original_orientation"] == 3
coordinate_metadata = artifacts[0].coordinate_metadata
assert coordinate_metadata is not None
assert coordinate_metadata["original_orientation"] == 3
@pytest.mark.integration
@@ -219,8 +224,11 @@ async def test_worker_sends_exact_derivative_and_links_attempt_evidence(
attempts = await services.sources.list_execution_attempts(source_id=source.id)
artifacts = await services.sources.list_processing_artifacts(source_id=source.id)
source_reference = captured["source_reference"]
assert isinstance(source_reference, SourceEvidenceReference)
captured_bytes = captured["bytes"]
assert isinstance(captured_bytes, bytes)
assert source_path.read_bytes() == original
assert hashlib.sha256(captured["bytes"]).hexdigest() == source_reference.digest_sha256
assert hashlib.sha256(captured_bytes).hexdigest() == source_reference.digest_sha256
assert source_reference.derivative_id is not None
assert {artifact.artifact_type for artifact in artifacts} == {
"orientation_normalized_model_input",
+3 -2
View File
@@ -2,6 +2,7 @@ from pathlib import Path
from uuid import uuid4
import pytest
from sqlmodel import col
from sqlmodel import select
from transcription.config import Settings
@@ -63,7 +64,7 @@ async def test_create_job_for_document_sorts_sources_and_creates_links(async_ses
sources = (
await async_session.exec(
select(Source).where(Source.document_id == document.id).order_by(Source.page_number) # pyright: ignore[reportArgumentType]
select(Source).where(Source.document_id == document.id).order_by(col(Source.page_number))
)
).all()
assert [source.upload_name for source in sources] == ["A_page.pdf", "b_page.pdf"]
@@ -99,7 +100,7 @@ async def test_create_document_job_stores_source_under_document_id_directory(asy
source = (
await async_session.exec(
select(Source).where(Source.document_id == result.document_id).order_by(Source.page_number) # pyright: ignore[reportArgumentType]
select(Source).where(Source.document_id == result.document_id).order_by(col(Source.page_number))
)
).first()
assert source is not None
+6 -2
View File
@@ -1,6 +1,7 @@
from uuid import uuid4
import pytest
from pydantic import JsonValue
from transcription.db.models import Document
from transcription.db.models import DocumentPerson
@@ -303,8 +304,11 @@ async def test_update_job_source_transcription_persists_provider_json_payloads(d
JobSource(job_id=job.id, source_id=source.id, status=JobSourceStatus.PENDING)
)
metadata = {"finish_reason": "stop", "usage": {"input_tokens": 11, "output_tokens": 22, "total_tokens": 33}}
raw_payload = {"id": "resp_xyz", "choices": [{"message": {"content": "provider transcript"}}]}
metadata: dict[str, JsonValue] = {
"finish_reason": "stop",
"usage": {"input_tokens": 11, "output_tokens": 22, "total_tokens": 33},
}
raw_payload: dict[str, JsonValue] = {"id": "resp_xyz", "choices": [{"message": {"content": "provider transcript"}}]}
await transcriptions.update_job_source_transcription(
job_id=job.id,
+4 -2
View File
@@ -1,6 +1,7 @@
"""Tests for transcription.config — settings loading, provider defaults, and paths."""
from pathlib import Path
from typing import Any
import pytest
from pydantic import ValidationError
@@ -10,9 +11,9 @@ from transcription.config import Settings
from transcription.config import parse_cli_settings
def _make_settings(**overrides) -> Settings:
def _make_settings(**overrides: Any) -> Settings:
"""Build a Settings instance with a dummy API key unless overridden."""
defaults = {"openrouter_api_key": "test-key-abc123", "provider_models": None}
defaults: dict[str, Any] = {"openrouter_api_key": "test-key-abc123", "provider_models": None}
defaults.update(overrides)
return Settings(**defaults)
@@ -168,6 +169,7 @@ def test_openrouter_client_timeout_tracks_the_configured_budget():
settings = Settings(openrouter_api_key="test-key", worker_provider_timeout_seconds=123.0)
provider = OpenRouterTranscriptionProvider(settings=settings)
assert provider._capturing_client is not None
timeout = provider._capturing_client._client.timeout
assert timeout.read == 123.0
+2 -1
View File
@@ -6,6 +6,7 @@ import pytest
import sqlalchemy as sa
from sqlalchemy import inspect
from sqlalchemy.dialects import postgresql
from sqlalchemy.exc import SAWarning
from sqlmodel import SQLModel
from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
@@ -148,7 +149,7 @@ async def test_create_all_declares_hot_path_indexes(tmp_path):
def test_metadata_has_no_unresolvable_table_cycle():
"""create_all must be able to order every table, including on PostgreSQL."""
with warnings.catch_warnings():
warnings.simplefilter("error", sa.exc.SAWarning)
warnings.simplefilter("error", SAWarning)
ordered = [table.name for table in SQLModel.metadata.sorted_tables]
assert ordered.index("source") < ordered.index("execution_attempt")
+9 -8
View File
@@ -1,5 +1,6 @@
"""Tests for the V2 SQLModel persistence layer and relationships."""
from typing import Any
from uuid import UUID
import pytest
@@ -17,8 +18,8 @@ from transcription.db.models import PersonRole
from transcription.db.models import Source
def _make_document(**overrides) -> Document:
defaults = {
def _make_document(**overrides: Any) -> Document:
defaults: dict[str, Any] = {
"name": "letter bundle",
"notes": "Family correspondence",
}
@@ -51,8 +52,8 @@ def _persist_document(session) -> Document:
return document
def _persist_person(session, **overrides) -> Person:
defaults = {"full_name": "Ada Lovelace"}
def _persist_person(session, **overrides: Any) -> Person:
defaults: dict[str, Any] = {"full_name": "Ada Lovelace"}
defaults.update(overrides)
person = Person(**defaults)
session.add(person)
@@ -69,8 +70,8 @@ def _persist_job(session, document: Document) -> Job:
return job
def _persist_source(session, document: Document, *, page_number: int = 1, **overrides) -> Source:
defaults = {
def _persist_source(session, document: Document, *, page_number: int = 1, **overrides: Any) -> Source:
defaults: dict[str, Any] = {
"document_id": document.id,
"page_number": page_number,
"upload_name": "letter.jpg",
@@ -88,8 +89,8 @@ def _persist_source(session, document: Document, *, page_number: int = 1, **over
return source
def _persist_job_source(session, job: Job, source: Source, **overrides) -> JobSource:
defaults = {
def _persist_job_source(session, job: Job, source: Source, **overrides: Any) -> JobSource:
defaults: dict[str, Any] = {
"job_id": job.id,
"source_id": source.id,
"status": JobSourceStatus.PENDING,
+1 -1
View File
@@ -86,7 +86,7 @@ class TestPromptConfiguration:
assert execution.temperature == 0.2
assert execution.top_p == 0.9
with pytest.raises(ValidationError):
execution.prompt_name = "changed.md"
execution.prompt_name = "changed.md" # ty: ignore[invalid-assignment]
def test_rejects_prompt_path_traversal_even_with_direct_loader_call(self, tmp_path):
outside_prompt = tmp_path / "outside.md"
+16 -3
View File
@@ -10,6 +10,7 @@ from uuid import uuid4
import httpx
import pytest
from pydantic import JsonValue
from transcription.benchmarking import EditorialAssessment
from transcription.benchmarking import score_transcription
@@ -33,6 +34,18 @@ from transcription.services.sources import TranscriptionError
from transcription.services.sources import transcribe_document_image
def _json_object(value: JsonValue) -> dict[str, JsonValue]:
"""Narrow a JSON export member to an object, asserting the export shape."""
assert isinstance(value, dict)
return value
def _json_array(value: JsonValue) -> list[JsonValue]:
"""Narrow a JSON export member to an array, asserting the export shape."""
assert isinstance(value, list)
return value
class _ChunkedAsyncStream(httpx.AsyncByteStream):
def __init__(self, chunks: list[bytes]):
self._chunks = chunks
@@ -261,9 +274,9 @@ async def test_attempts_are_append_only_and_exported_with_integrity(default_sess
)
)
export = await sources.build_evidence_export(source_id=source.id)
assert export["source"]["digest_sha256"] == "a" * 64
assert [item["attempt_number"] for item in export["attempts"]] == [1, 2]
assert export["artifacts"][0]["id"] == str(artifact.id)
assert _json_object(export["source"])["digest_sha256"] == "a" * 64
assert [_json_object(item)["attempt_number"] for item in _json_array(export["attempts"])] == [1, 2]
assert _json_object(_json_array(export["artifacts"])[0])["id"] == str(artifact.id)
assert "file_path" not in json.dumps(export)
with pytest.raises(JobDeleteBlockedError):
await jobs.delete_job_with_guardrails(job_id=job.id)
+5 -3
View File
@@ -1,9 +1,11 @@
import asyncio
import logging
from typing import cast
import pytest
from transcription.services import ServiceBundle
from transcription.services.sources import SourceService
from transcription.worker import process_next_queued_job
from transcription.worker import run_worker_loop
@@ -43,7 +45,7 @@ async def test_run_worker_loop_reuses_one_bundle_across_jobs(monkeypatch):
nonlocal closed
closed = True
bundle = ServiceBundle(sources=_Sources()) # type: ignore[arg-type]
bundle = ServiceBundle(sources=cast("SourceService", _Sources()))
monkeypatch.setattr(
"transcription.worker.ServiceBundle.from_session_factory",
classmethod(lambda _cls, _factory=None, **_kwargs: bundle),
@@ -75,7 +77,7 @@ async def test_process_next_closes_provider_for_the_bundle_it_owns(monkeypatch):
nonlocal closed
closed = True
bundle = ServiceBundle(sources=_Sources()) # type: ignore[arg-type]
bundle = ServiceBundle(sources=cast("SourceService", _Sources()))
monkeypatch.setattr(
"transcription.worker.ServiceBundle.from_session_factory",
classmethod(lambda _cls, _factory=None, **_kwargs: bundle),
@@ -101,7 +103,7 @@ async def test_process_next_leaves_a_caller_owned_bundle_open(monkeypatch):
nonlocal closed
closed = True
bundle = ServiceBundle(sources=_Sources()) # type: ignore[arg-type]
bundle = ServiceBundle(sources=cast("SourceService", _Sources()))
async def _no_job(*, services, session):
_ = (services, session)
+5 -4
View File
@@ -2,9 +2,9 @@
from __future__ import annotations
from collections.abc import AsyncGenerator
from collections.abc import Awaitable
from collections.abc import Callable
from collections.abc import Generator
from datetime import UTC
from datetime import datetime
from pathlib import Path
@@ -32,12 +32,13 @@ from transcription.db.models import Source
@pytest.fixture(scope="session")
def app_client(tmp_path_factory: pytest.TempPathFactory) -> AsyncGenerator[tuple[FastAPI, TestClient]]:
def app_client(tmp_path_factory: pytest.TempPathFactory) -> Generator[tuple[FastAPI, TestClient]]:
"""Provide a real application and test client backed by in-memory SQLite."""
tmp_path = tmp_path_factory.mktemp("ui")
database = SqliteSettings(path=str(tmp_path / "ui-tests.db"))
settings = Settings(
openrouter_api_key="test-key",
database=SqliteSettings(path=str(tmp_path / "ui-tests.db")),
database=database,
environment="test",
bootstrap_schema_on_startup=True,
upload_dir=tmp_path / "uploads",
@@ -48,7 +49,7 @@ def app_client(tmp_path_factory: pytest.TempPathFactory) -> AsyncGenerator[tuple
with TestClient(app) as client:
runtime_path = Path(str(app.state.runtime.engine.url.database)).resolve()
expected_path = Path(settings.database.path).resolve()
expected_path = Path(database.path).resolve()
if runtime_path != expected_path:
raise RuntimeError(
"Refusing to initialize destructive UI fixtures against "
+3 -1
View File
@@ -1,12 +1,13 @@
"""Tests for the sources page routes and Source model properties."""
from pathlib import Path
from uuid import uuid4
import pytest
from sqlalchemy.orm import selectinload
from sqlmodel import select
from transcription.db import session_scope
from transcription.db.loading import selectinload
from transcription.db.models import Document
from transcription.db.models import Job
from transcription.db.models import JobSourceStatus
@@ -24,6 +25,7 @@ class TestSourceModelProperties:
@pytest.mark.asyncio
async def test_source_properties_with_no_job_sources(self):
source = Source(
document_id=uuid4(),
page_number=1,
upload_name="page_one.png",
filename="stored_page_one.png",
+58 -57
View File
@@ -7,7 +7,6 @@ import shlex
import shutil
import subprocess
import sys
from ctypes import wintypes
from datetime import datetime
from pathlib import Path
@@ -20,67 +19,69 @@ def show_phase(title: str) -> None:
print(f"========== {title} ==========")
def test_file_unlocked(path: Path) -> bool:
if not path.exists():
if sys.platform == "win32":
# `ctypes.wintypes` raises on import off Windows, and `fcntl` does not exist
# on Windows, so the two implementations are selected at module level where a
# type checker can narrow `sys.platform` and analyze only the live branch.
from ctypes import wintypes
def test_file_unlocked(path: Path) -> bool:
if not path.exists():
return True
generic_read = 0x80000000
generic_write = 0x40000000
open_existing = 3
file_attribute_normal = 0x80
invalid_handle_value = wintypes.HANDLE(-1).value
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
kernel32.CreateFileW.argtypes = [
wintypes.LPCWSTR,
wintypes.DWORD,
wintypes.DWORD,
wintypes.LPVOID,
wintypes.DWORD,
wintypes.DWORD,
wintypes.HANDLE,
]
kernel32.CreateFileW.restype = wintypes.HANDLE
kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
kernel32.CloseHandle.restype = wintypes.BOOL
handle = kernel32.CreateFileW(
str(path),
generic_read | generic_write,
0,
None,
open_existing,
file_attribute_normal,
None,
)
if handle == invalid_handle_value:
return False
kernel32.CloseHandle(handle)
return True
if os.name == "nt":
return _test_file_unlocked_windows(path)
return _test_file_unlocked_posix(path)
def _test_file_unlocked_windows(path: Path) -> bool:
generic_read = 0x80000000
generic_write = 0x40000000
open_existing = 3
file_attribute_normal = 0x80
invalid_handle_value = wintypes.HANDLE(-1).value
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
kernel32.CreateFileW.argtypes = [
wintypes.LPCWSTR,
wintypes.DWORD,
wintypes.DWORD,
wintypes.LPVOID,
wintypes.DWORD,
wintypes.DWORD,
wintypes.HANDLE,
]
kernel32.CreateFileW.restype = wintypes.HANDLE
kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
kernel32.CloseHandle.restype = wintypes.BOOL
handle = kernel32.CreateFileW(
str(path),
generic_read | generic_write,
0,
None,
open_existing,
file_attribute_normal,
None,
)
if handle == invalid_handle_value:
return False
kernel32.CloseHandle(handle)
return True
def _test_file_unlocked_posix(path: Path) -> bool:
else:
import fcntl
fd = os.open(path, os.O_RDWR)
try:
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
return False
else:
fcntl.flock(fd, fcntl.LOCK_UN)
def test_file_unlocked(path: Path) -> bool:
if not path.exists():
return True
finally:
os.close(fd)
fd = os.open(path, os.O_RDWR)
try:
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
return False
else:
fcntl.flock(fd, fcntl.LOCK_UN)
return True
finally:
os.close(fd)
def wait_for_restore_preflight(db_file_path: Path) -> bool: