Move orientation normalization to the Source-ingest boundary and delete the
ProcessingArtifact subsystem it was built to serve.
Stored pages are now already upright, so nothing downstream derives a rotated
copy: every stored byte is the byte a provider is later sent. Rotation runs in
store_source_file ahead of hashing, so source.file_hash and file_size_bytes
describe exactly what is on disk. normalize_orientation becomes bytes-in /
bytes-out, and JPEG output reuses the source quantization tables and chroma
subsampling instead of re-quantizing at a fixed quality - measured at 50.3-56.1
dB PSNR at -6% size, against 50.0-53.5 dB at +38% for quality=95.
ProcessingArtifact held 2 rows against 77 successful transcriptions; the
subsystem effectively never ran. Deleting it removes the artifact cluster from
sources.py, the derivative resolution in workflows.py, the pre-provider commit
that only existed to make an artifact row durable, and the artifact evidence
dump from the Source detail page. The transcription_quality_warnings payload
folds into execution_attempt.normalized_metadata, so that feature keeps working
without the table.
tools/migrate_v46_to_v47.py carries steps 1 and 2: it rotated the 58 stored
images carrying EXIF orientation 3 in place, updated their recorded hash and
size, dropped processing_artifact and removed its one external file. It is
idempotent, keyed on state rather than a version marker.
tools/migrate_v45_to_v46.py is deleted. That migration is complete, and after
V4.7 it would restore a V4.5 backup into a schema that no longer matches.
Also fixes tests/test_config.py, which read the developer's local .env and
failed whenever WORKER_MAX_RETRIES was set.
Co-authored-by: Copilot App <[email protected]>
Baseline was 207 diagnostics. Two real bugs were hiding in the noise:
- tools/run_destructive_tests.py imported ctypes.wintypes at module scope,
which raises on non-Windows, and called fcntl unconditionally. The Windows
and POSIX implementations now live under a module-level sys.platform split.
- tests/ui/test_sources_page.py constructed Source(...) without document_id.
Structural fixes, not suppressions:
- New src/transcription/db/loading.py owns the SQLModel-field to
QueryableAttribute reinterpretation via orm_attribute()/selectinload()/
defer(). This removed 42 "# pyright: ignore[reportArgumentType]" comments
across documents/jobs/people/sources. Its docstring records that
selectinload(A.b, B.c) is NOT equivalent to the chained form: varargs
applies the selectin strategy only to the last path element, which under
lazy="raise" raises InvalidRequestError at render time.
- db/session.py transaction_scope no longer accepts or yields
AsyncSessionTransaction. No caller ever passed one, sessionmaker.begin()
yields an AsyncSession, and the dead branch was latently buggy because
services call .exec(). Cleared 7 workflows.py diagnostics.
- services/registry.py RegistryService is bound by a new RegistryEntry
Protocol instead of bare SQLModel, so the shared implementation can read
id/label/normalized_label/is_active. Cleared 9 diagnostics.
- Column expressions in sources.py/jobs.py/test_store.py wrap in sqlmodel
col(), the idiom already used in registry.py.
- read_source_navigation wraps its literal tuple bounds in literal().
- normalization.py narrows with isinstance(image, TiffImageFile) rather than
comparing image.format, since tag_v2 is TIFF-only.
- linked_people.render uses @ui.refreshable_method, the NiceGUI API for bound
methods.
- The OpenRouter capturing client re-raises ResponseNotRead when the response
stream is not async rather than mis-wrapping it.
Tooling gate:
- New .pre-commit-config.yaml runs ruff check and ty check as blocking hooks.
No pre-commit config previously existed. Negative-tested: injecting a type
error fails both hooks.
- The last two "# pyright: ignore" comments (config.py) are removed; ty does
not honor pyright directives. One "# ty: ignore" remains, in
tests/test_prompts.py, where the test deliberately assigns to a frozen
field to assert ValidationError.
- asyncio_default_fixture_loop_scope is pinned to "function" so
pytest-asyncio behavior does not shift on upgrade.
Verification: ruff check clean, ty check reports 0 diagnostics, 292 passed
and 4 skipped, pre-commit passes and demonstrably fails on a regression, and
tools/run_destructive_tests.py runs on Windows.
Co-authored-by: Copilot App <[email protected]>
Claim jobs atomically [CRIT-01]
- Replace JobService.read_next_queued_job with claim_next_queued_job, which
selects and transitions QUEUED -> PROCESSING inside one transaction. The old
read-then-write sequence left a window in which two workers could observe the
same QUEUED row.
- Add the missing .limit(1). The poll previously ordered the entire queued set
and discarded all but the first row.
- Drop the eager loads from the hot poll entirely. They were pure waste:
process_queued_job immediately re-reads the job through read_job with the
relationships it actually needs.
- Guard the row with with_for_update(skip_locked=True) on PostgreSQL so the
claim stays correct once more than one worker exists. On SQLite the claim is a
bounded single-writer transaction.
- Correct the comment at the remaining direct-call claim site, which described
the hazard rather than the guarantee.
Reuse the provider connection [HIGH-02]
- Build the ServiceBundle once per worker loop instead of once per job, and
close it at loop shutdown. Every job previously constructed a new
SourceService, and with it a new provider adapter and a new httpx.AsyncClient,
paying a full TLS handshake per page and discarding the connection pool.
- process_next_queued_job now accepts an optional caller-owned bundle and only
closes bundles it created itself.
Uncap the provider timeout [HIGH-03]
- Remove le=20.0 from worker_provider_timeout_seconds. The cap equalled the
default, so the ceiling could never be raised, and dense-page vision
transcription routinely needs longer. Default raised to 180s.
- Pass an explicit httpx.Timeout to the OpenRouter AsyncClient. httpx defaults
every phase to 5 seconds, so the real read budget was 5s regardless of the
configured value; the outer asyncio.wait_for could never be the binding
constraint. Connect stays at 10s.
Tighten the provider boundary [MED-03]
- Declare model, current_request_manifest, current_transport_evidence, and
aclose on the TranscriptionProvider Protocol.
- Delete the per-call inspect.signature(adapter.transcribe).parameters
reflection and the untyped kwargs dict it fed. The Protocol had declared
requested_model all along, so the reflection was dead defensive weight on the
hot path.
- Replace the three getattr probes for aclose and the evidence attributes with
direct typed access.
Deduplicate bundle construction [MED-06]
- Add ServiceBundle.from_session_factory and ServiceBundle.aclose, replacing the
duplicated four-service instantiation blocks in app.py and worker.py.
- _recover_stale_processing_jobs now uses the bundle built moments earlier
instead of constructing a second JobService.
Tests
- Claiming returns the oldest job, marks it PROCESSING, never hands the same job
out twice, and emits exactly one unadorned SELECT carrying LIMIT and no JOIN.
- The worker loop threads one bundle through consecutive jobs and closes it once
at shutdown; a caller-owned bundle is left open.
- Settings accepts a timeout above 20 seconds and still rejects zero.
- The OpenRouter client's read, write, and pool timeouts track the configured
budget rather than the httpx default.
Note: .env in this checkout still pins WORKER_PROVIDER_TIMEOUT_SECONDS=20 and
should be raised to pick up this fix.
Verified: 268 passed, 4 skipped; ruff check clean.
Co-authored-by: Copilot App <[email protected]>
Pure remediation; no behavior change. Every item traces to a finding in
docs/architecture_code_review_2026-08-17.md.
Deletions
- Delete app_state.py, which had zero importers and whose get_session_factory
raised TypeError at runtime [HIGH-01].
- Delete services/transcription.py and point build_prompt_execution imports at
services/sources.py; drop the store.py compatibility aliases [MED-05].
- Delete ServiceBase.queue and its unparameterized asyncio.Queue [MED-07].
- Delete db/operations.get_next_queued_job, a divergent duplicate [CRIT-01].
- Drop the discarded load_docs parameter from list_jobs [LOW-03].
Config
- Delete worker_retry_backoff_seconds; no backoff behavior existed anywhere, so
wiring it would have been a new feature [MED-02].
- Wire sqlite_check_same_thread through get_engine. The engine hardcoded the
setting's own default, so this preserves behavior exactly [MED-02].
- Replace DATABASE_URL in docker-compose.yml with the nested DATABASE__DRIVER /
DATABASE__PATH names. Settings uses env_nested_delimiter with extra="ignore",
so DATABASE_URL was silently discarded [MED-10].
UI
- Move the 23KB inline VIBESCRIBE_LOGO_SVG to ui/static/vibescribe_logo.svg and
load it through a cached read_svg sibling of read_css [MED-09].
- Route the portrait upload failure through error_presenter.show_error [LOW-07].
- Cancel the job detail auto-refresh timer instead of only deactivating it, and
name its interval constant [LOW-06].
Worker
- Make WorkerNotifier runtime_checkable and validate the resolved object in
resolve_worker_notifier, which previously returned any non-None attribute
unchecked [LOW-04].
Docs and lint
- Fix two stale paths in services.instructions.md, one of which pointed at the
module deleted here [LOW-02].
- ruff check --fix to zero [LOW-01].
Verified: 264 passed, 4 skipped; ruff check clean.
Co-authored-by: Copilot App <[email protected]>
Jim Lancaster
John Lancaster