bbchops/transcription
1
0
Fork 0
generated from john/python-template
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

Phase 3: extract EvidenceService and rewrite the service ownership rule

Browse Source 
Decompose SourceService along the aggregate boundary and then correct the
instruction file that caused it to grow, in that order. The refactor is the
empirical test of the rule.

services/evidence.py (new)
  EvidenceService owns ExecutionAttempt: read_latest_execution_attempt,
  list_execution_attempts, promote_machine_attempt, build_evidence_export,
  plus the LatestExecutionAttempt projection. Moved verbatim from sources.py.

services/errors.py (new)
  The five-class error hierarchy (PromptLoadError, TranscriptionError,
  TranscriptionNotFoundError, SourceDeleteBlockedError,
  CandidatePromotionError) moved out of sources.py. evidence.py needs
  TranscriptionNotFoundError, and test_service_boundaries.py correctly
  rejected the sibling import. errors.py defines no *Service class, so it is
  a legal shared home. This was the boundary test doing its job, not an
  obstacle to route around.

sources.py 1,389 -> 885 lines (1,063 after Phase 2).

services/__init__.py
  ServiceBundle and from_session_factory register evidence. Note that
  field-by-field ServiceBundle construction silently binds services to the
  process-global session factory via default_factory; from_session_factory is
  the only safe constructor. Two test bundles were fixed for this.

.github/instructions/services.instructions.md
  Rewritten to describe the boundaries the decomposition actually produced,
  per plan Phase 3 task 7 and review log [59].

  - "1 service class per data model" -> one service class per aggregate.
    The table-shaped rule is the measured cause of sources.py reaching
    1,389 lines; DocumentType has no lifecycle without Document.
  - New Model Ownership section. Junctions are owned by their lifecycle
    owner, the service that creates and deletes the rows: document_person
    to PeopleService (sole writer, measured), job_source to SourceService.
    Two carve-outs are stated rather than left as silent violations:
    cascade deletion when a service deletes its own aggregate root, and
    status transitions that create and delete nothing (cancel_job,
    resubmit_failed_sources), which are Job lifecycle events on the work
    queue. EvidenceService.promote_machine_attempt's two-field write to
    Source is named and scoped.
  - Mandatory CRUD softened to intent. It was already false: five modules
    define no service class, EvidenceService has no create/delete because
    ExecutionAttempt is append-only, RegistryService uses <op>_entry.
  - Separated reading across models via eager loads from the owning root,
    which is allowed, from importing another service, which is not. The old
    line 13 and lines 75-77 read as contradictory.
  - Typo: picutre.

  No code was moved to satisfy the rule.

tests/test_service_boundaries.py
  Docstring no longer cites the instruction file by line number; that anchor
  would desynchronise silently. errors.py added to the neutral-module list.

Verified: 292 passed, 4 skipped, 0 ruff, 0 ty. All 25 /ui/* routes walked
against the live app; 24x 200. /ui/documents/{id}/sources 404s via a 307 that
drops the /ui prefix, confirmed pre-existing (last touched in 6a3ee26) and
left alone as out of scope.

Co-authored-by: Copilot App <[email protected]>
This commit is contained in:
zoltan57
2026-08-18 15:54:27 -05:00
co-authored by Copilot App
parent 11097b9cfe
commit 7dd0d2c9bf
15 changed files with 381 additions and 242 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/transcription/services/__init__.py
+12 -1
View File
@@ -9,12 +9,21 @@ from sqlmodel.ext.asyncio.session import AsyncSession
from ..config import Settings
from .documents import DocumentService
from .evidence import EvidenceService
from .jobs import JobService
from .people import PeopleService
from .prompts import PromptStore
from .sources import SourceService
__all__ = ["DocumentService", "JobService", "PeopleService", "PromptStore", "ServiceBundle", "SourceService"]
__all__ = [
"DocumentService",
"EvidenceService",
"JobService",
"PeopleService",
"PromptStore",
"ServiceBundle",
"SourceService",
]
@dataclass(frozen=True, slots=True)
@@ -25,6 +34,7 @@ class ServiceBundle:
sources: SourceService = field(default_factory=SourceService)
jobs: JobService = field(default_factory=JobService)
people: PeopleService = field(default_factory=PeopleService)
evidence: EvidenceService = field(default_factory=EvidenceService)
@classmethod
def from_session_factory(
@@ -41,6 +51,7 @@ class ServiceBundle:
sources=SourceService(session_factory=session_factory, settings=settings),
jobs=JobService(session_factory=session_factory, settings=settings),
people=PeopleService(session_factory=session_factory, settings=settings),
evidence=EvidenceService(session_factory=session_factory, settings=settings),
)
async def aclose(self) -> None:
src/transcription/services/errors.py
+32
View File
@@ -0,0 +1,32 @@
"""Error vocabulary shared across the source, evidence, and prompt services.
These live in a neutral module rather than in the service that raises them
because more than one service raises them, and ``services.instructions.md``
forbids a service module from importing a sibling. Orchestration modules and
the UI import from here, so the exception a caller catches does not change when
an operation moves between services.
"""
from __future__ import annotations
from transcription.errors import AppError
class PromptLoadError(AppError):
"""Raised when prompt artifacts cannot be loaded safely."""
class TranscriptionError(AppError):
"""Raised when transcription execution fails."""
class TranscriptionNotFoundError(TranscriptionError):
"""Raised when a transcription-related resource is not found."""
class SourceDeleteBlockedError(TranscriptionError):
"""Raised when source deletion is blocked by dependency policy."""
class CandidatePromotionError(TranscriptionError):
"""Raised when a machine attempt cannot be selected for its Source."""
src/transcription/services/evidence.py
+210
View File
@@ -0,0 +1,210 @@
"""Read and export the immutable execution evidence trail.
``ExecutionAttempt`` is append-only: one row per provider call, written once by
the transcription workflow and never updated. Everything here is therefore a
read, a projection, or an export, with one exception - ``promote_machine_attempt``
selects which attempt a ``Source`` presents, which is an evidence decision even
though the write lands on ``Source``.
"""
from __future__ import annotations
import base64
import hashlib
from collections.abc import Sequence
from dataclasses import dataclass
from uuid import UUID
from pydantic import JsonValue
from sqlalchemy import inspect as sqlalchemy_inspect
from sqlmodel import col
from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
from transcription.db.models import ExecutionAttempt
from transcription.db.models import JobSourceStatus
from transcription.db.models import Source
from transcription.errors import ErrorCategory
from ..db.loading import defer
from .base import ServiceBase
from .errors import CandidatePromotionError
from .errors import TranscriptionNotFoundError
@dataclass(frozen=True, slots=True)
class LatestExecutionAttempt:
"""One execution attempt plus the loader facts a caller needs to render it."""
attempt: ExecutionAttempt
transport_body_deferred: bool
class EvidenceService(ServiceBase):
"""Read, project, and export execution attempt evidence."""
async def read_latest_execution_attempt(
self,
*,
job_source_id: UUID,
session: AsyncSession | None = None,
) -> LatestExecutionAttempt | None:
"""Read only the latest immutable attempt for one compatibility projection.
The transport body is deferred because it can be arbitrarily large; the
returned read model reports that as a plain flag so callers never have to
inspect ORM loader state.
"""
async with self._session_scope(session) as _session:
query = (
select(ExecutionAttempt)
.options(defer(ExecutionAttempt.transport_body))
.where(ExecutionAttempt.job_source_id == job_source_id)
.order_by(
col(ExecutionAttempt.attempt_number).desc(),
col(ExecutionAttempt.id).desc(),
)
.limit(1)
)
attempt = (await _session.exec(query)).first()
if attempt is None:
return None
deferred = "transport_body" in sqlalchemy_inspect(attempt).unloaded
return LatestExecutionAttempt(attempt=attempt, transport_body_deferred=deferred)
async def list_execution_attempts(
self,
*,
source_id: UUID | None = None,
job_id: UUID | None = None,
session: AsyncSession | None = None,
) -> Sequence[ExecutionAttempt]:
"""List immutable execution evidence in stable attempt order."""
async with self._session_scope(session) as _session:
query = select(ExecutionAttempt)
if source_id is not None:
query = query.where(ExecutionAttempt.source_id == source_id)
if job_id is not None:
query = query.where(ExecutionAttempt.job_id == job_id)
query = query.order_by(
col(ExecutionAttempt.job_id),
col(ExecutionAttempt.source_id),
col(ExecutionAttempt.attempt_number),
col(ExecutionAttempt.id),
)
return (await _session.exec(query)).all()
async def promote_machine_attempt(
self,
*,
source_id: UUID,
execution_attempt_id: UUID,
session: AsyncSession | None = None,
) -> Source:
"""Atomically select one successful machine attempt as the Source projection."""
async with self._session_scope(session) as _session:
source = await self._read_source(
session=_session,
source_id=source_id,
suggestion="Refresh Source Detail and retry.",
)
attempt = await _session.get(ExecutionAttempt, execution_attempt_id)
if (
attempt is None
or attempt.source_id != source_id
or attempt.status != JobSourceStatus.TRANSCRIBED
or not attempt.raw_transcription
):
raise CandidatePromotionError(
"Only a successful transcription attempt belonging to this Source can be selected",
category=ErrorCategory.VALIDATION,
suggestion="Select an available successful candidate from Source Detail.",
)
source.preferred_execution_attempt_id = attempt.id
source.raw_transcription = attempt.raw_transcription
await self._finalize(session=_session, caller_session=session, refresh=(source,))
return source
async def build_evidence_export(
self,
*,
source_id: UUID,
session: AsyncSession | None = None,
) -> dict[str, JsonValue]:
"""Build a versioned, source-reference-only evidence export."""
async with self._session_scope(session) as _session:
source = await self._read_source(session=_session, source_id=source_id)
attempts = list(await self.list_execution_attempts(source_id=source_id, session=_session))
attempt_payloads = [
{
"id": str(attempt.id),
"job_id": str(attempt.job_id),
"source_id": str(attempt.source_id),
"attempt_number": attempt.attempt_number,
"status": attempt.status.value,
"provider": attempt.provider,
"model": attempt.model,
"request_manifest": attempt.request_manifest,
"request_manifest_sha256": attempt.request_manifest_sha256,
"request_manifest_schema_version": attempt.request_manifest_schema_version,
"transport": {
"response_received": attempt.response_received,
"status_code": attempt.transport_status_code,
"body_base64": (
base64.b64encode(attempt.transport_body).decode("ascii")
if attempt.transport_body is not None
else None
),
"body_sha256": (
hashlib.sha256(attempt.transport_body).hexdigest()
if attempt.transport_body is not None
else None
),
"content_type": attempt.transport_content_type,
"content_encoding": attempt.transport_content_encoding,
"safe_headers": attempt.transport_safe_headers,
"request_id": attempt.router_request_id,
"generation_id": attempt.router_generation_id,
},
"sdk_response_snapshot": attempt.sdk_response_snapshot,
"normalized_metadata": attempt.normalized_metadata,
"software_context": attempt.software_context,
"raw_transcription": attempt.raw_transcription,
"error_category": attempt.error_category,
"error_detail": attempt.error_detail,
"failure_phase": attempt.failure_phase,
"started_at": attempt.started_at.isoformat(),
"finished_at": attempt.finished_at.isoformat(),
"duration_ms": attempt.duration_ms,
}
for attempt in attempts
]
return {
"schema_name": "transcription.evidence-export",
"schema_version": "1",
"source": {
"id": str(source.id),
"digest_sha256": source.file_hash,
"byte_size": source.file_size_bytes,
"page_number": source.page_number,
"upload_name": source.upload_name,
},
"attempts": attempt_payloads,
}
async def _read_source(
self,
*,
session: AsyncSession,
source_id: UUID,
suggestion: str = "Verify the source id and retry.",
) -> Source:
return await self._get_or_raise(
Source,
source_id,
session=session,
error=TranscriptionNotFoundError,
noun="Source",
suggestion=suggestion,
)
src/transcription/services/sources.py
+4 -182
View File
@@ -3,7 +3,6 @@
from __future__ import annotations
import asyncio
import base64
import hashlib
import logging
from collections.abc import Sequence
@@ -22,7 +21,6 @@ from pydantic import JsonValue
from pydantic import TypeAdapter
from pydantic import ValidationError
from sqlalchemy import func
from sqlalchemy import inspect as sqlalchemy_inspect
from sqlalchemy import literal
from sqlalchemy import tuple_
from sqlalchemy.ext.asyncio import async_sessionmaker
@@ -37,7 +35,6 @@ from transcription.db.models import Job
from transcription.db.models import JobSource
from transcription.db.models import JobSourceStatus
from transcription.db.models import Source
from transcription.errors import AppError
from transcription.errors import ErrorCategory
from transcription.providers import ProviderAuthError
from transcription.providers import ProviderError
@@ -50,10 +47,13 @@ from transcription.providers import TranscriptionResult
from transcription.providers import TransportEvidence
from transcription.providers import get_transcription_provider
from ..db.loading import defer
from ..db.loading import orm_attribute
from ..db.loading import selectinload
from .base import ServiceBase
from .errors import PromptLoadError
from .errors import SourceDeleteBlockedError
from .errors import TranscriptionError
from .errors import TranscriptionNotFoundError
from .source_media import lookup_source_mime_type
from .source_media import supported_source_formats
@@ -76,26 +76,6 @@ class PromptExecution(BaseModel):
top_p: float | None = Field(ge=0.0, le=1.0)
class PromptLoadError(AppError):
"""Raised when prompt artifacts cannot be loaded safely."""
class TranscriptionError(AppError):
"""Raised when transcription execution fails."""
class TranscriptionNotFoundError(TranscriptionError):
"""Raised when a transcription-related resource is not found."""
class SourceDeleteBlockedError(TranscriptionError):
"""Raised when source deletion is blocked by dependency policy."""
class CandidatePromotionError(TranscriptionError):
"""Raised when a machine attempt cannot be selected for its Source."""
@dataclass(frozen=True, slots=True)
class SourceNavigation:
"""Adjacent Source identifiers within one ordered Document."""
@@ -128,14 +108,6 @@ def build_provider_input(source: Source) -> ProviderInput:
)
@dataclass(frozen=True, slots=True)
class LatestExecutionAttempt:
"""One execution attempt plus the loader facts a caller needs to render it."""
attempt: ExecutionAttempt
transport_body_deferred: bool
class SourceService(ServiceBase):
"""Manage source records, media payloads, revisions, and page execution output."""
@@ -214,35 +186,6 @@ class SourceService(ServiceBase):
)
return source
async def read_latest_execution_attempt(
self,
*,
job_source_id: UUID,
session: AsyncSession | None = None,
) -> LatestExecutionAttempt | None:
"""Read only the latest immutable attempt for one compatibility projection.
The transport body is deferred because it can be arbitrarily large; the
returned read model reports that as a plain flag so callers never have to
inspect ORM loader state.
"""
async with self._session_scope(session) as _session:
query = (
select(ExecutionAttempt)
.options(defer(ExecutionAttempt.transport_body))
.where(ExecutionAttempt.job_source_id == job_source_id)
.order_by(
col(ExecutionAttempt.attempt_number).desc(),
col(ExecutionAttempt.id).desc(),
)
.limit(1)
)
attempt = (await _session.exec(query)).first()
if attempt is None:
return None
deferred = "transport_body" in sqlalchemy_inspect(attempt).unloaded
return LatestExecutionAttempt(attempt=attempt, transport_body_deferred=deferred)
async def read_source_navigation(
self,
source_id: UUID,
@@ -640,127 +583,6 @@ class SourceService(ServiceBase):
await self._finalize(session=_session, caller_session=session, refresh=(job, source, job_source, attempt))
return job_source
async def promote_machine_attempt(
self,
*,
source_id: UUID,
execution_attempt_id: UUID,
session: AsyncSession | None = None,
) -> Source:
"""Atomically select one successful machine attempt as the Source projection."""
async with self._session_scope(session) as _session:
source = await self._read_source(
session=_session,
source_id=source_id,
suggestion="Refresh Source Detail and retry.",
)
attempt = await _session.get(ExecutionAttempt, execution_attempt_id)
if (
attempt is None
or attempt.source_id != source_id
or attempt.status != JobSourceStatus.TRANSCRIBED
or not attempt.raw_transcription
):
raise CandidatePromotionError(
"Only a successful transcription attempt belonging to this Source can be selected",
category=ErrorCategory.VALIDATION,
suggestion="Select an available successful candidate from Source Detail.",
)
source.preferred_execution_attempt_id = attempt.id
source.raw_transcription = attempt.raw_transcription
await self._finalize(session=_session, caller_session=session, refresh=(source,))
return source
async def list_execution_attempts(
self,
*,
source_id: UUID | None = None,
job_id: UUID | None = None,
session: AsyncSession | None = None,
) -> Sequence[ExecutionAttempt]:
"""List immutable execution evidence in stable attempt order."""
async with self._session_scope(session) as _session:
query = select(ExecutionAttempt)
if source_id is not None:
query = query.where(ExecutionAttempt.source_id == source_id)
if job_id is not None:
query = query.where(ExecutionAttempt.job_id == job_id)
query = query.order_by(
col(ExecutionAttempt.job_id),
col(ExecutionAttempt.source_id),
col(ExecutionAttempt.attempt_number),
col(ExecutionAttempt.id),
)
return (await _session.exec(query)).all()
async def build_evidence_export(
self,
*,
source_id: UUID,
session: AsyncSession | None = None,
) -> dict[str, JsonValue]:
"""Build a versioned, source-reference-only evidence export."""
async with self._session_scope(session) as _session:
source = await self._read_source(session=_session, source_id=source_id)
attempts = list(await self.list_execution_attempts(source_id=source_id, session=_session))
attempt_payloads = [
{
"id": str(attempt.id),
"job_id": str(attempt.job_id),
"source_id": str(attempt.source_id),
"attempt_number": attempt.attempt_number,
"status": attempt.status.value,
"provider": attempt.provider,
"model": attempt.model,
"request_manifest": attempt.request_manifest,
"request_manifest_sha256": attempt.request_manifest_sha256,
"request_manifest_schema_version": attempt.request_manifest_schema_version,
"transport": {
"response_received": attempt.response_received,
"status_code": attempt.transport_status_code,
"body_base64": (
base64.b64encode(attempt.transport_body).decode("ascii")
if attempt.transport_body is not None
else None
),
"body_sha256": (
hashlib.sha256(attempt.transport_body).hexdigest()
if attempt.transport_body is not None
else None
),
"content_type": attempt.transport_content_type,
"content_encoding": attempt.transport_content_encoding,
"safe_headers": attempt.transport_safe_headers,
"request_id": attempt.router_request_id,
"generation_id": attempt.router_generation_id,
},
"sdk_response_snapshot": attempt.sdk_response_snapshot,
"normalized_metadata": attempt.normalized_metadata,
"software_context": attempt.software_context,
"raw_transcription": attempt.raw_transcription,
"error_category": attempt.error_category,
"error_detail": attempt.error_detail,
"failure_phase": attempt.failure_phase,
"started_at": attempt.started_at.isoformat(),
"finished_at": attempt.finished_at.isoformat(),
"duration_ms": attempt.duration_ms,
}
for attempt in attempts
]
return {
"schema_name": "transcription.evidence-export",
"schema_version": "1",
"source": {
"id": str(source.id),
"digest_sha256": source.file_hash,
"byte_size": source.file_size_bytes,
"page_number": source.page_number,
"upload_name": source.upload_name,
},
"attempts": attempt_payloads,
}
async def upsert_revision_for_source(
self,
*,
src/transcription/services/store.py
+1 -1
View File
@@ -23,10 +23,10 @@ from ..db.models import JobSourceStatus
from ..db.models import Source
from ..db.session import SessionFactory
from ..db.session import session_scope
from .errors import TranscriptionError
from .media_storage import build_stored_filename
from .media_storage import write_media_bytes
from .normalization import normalize_orientation_async
from .sources import TranscriptionError
from .sources import build_prompt_execution
from .sources import source_mime_type
from .sources import validate_source_content
src/transcription/ui/pages/sources_page.py
+13 -11
View File
@@ -14,10 +14,11 @@ from transcription.db.models import ExecutionAttempt
from transcription.db.models import JobSource
from transcription.db.models import JobSourceStatus
from transcription.db.models import Source
from transcription.services.sources import LatestExecutionAttempt
from transcription.services.sources import SourceDeleteBlockedError
from transcription.services.errors import SourceDeleteBlockedError
from transcription.services.errors import TranscriptionNotFoundError
from transcription.services.evidence import EvidenceService
from transcription.services.evidence import LatestExecutionAttempt
from transcription.services.sources import SourceService
from transcription.services.sources import TranscriptionNotFoundError
from transcription.ui.components.app_shell import render_navigation_header
from transcription.ui.components.cards import archival_card
from transcription.ui.components.confirm_delete import render_delete_actions
@@ -112,6 +113,7 @@ def register_page() -> None: # noqa: PLR0915
@ui.page("/sources/{source_id}")
async def source_detail_page(source_id: str, request: Request, session_factory: SessionFactoryDep) -> None:
sources_service = SourceService(session_factory=session_factory)
evidence_service = EvidenceService(session_factory=session_factory)
render_navigation_header(current_path="/sources")
parsed_source_id = parsed_record_id(source_id, noun="Source")
@@ -123,11 +125,11 @@ def register_page() -> None: # noqa: PLR0915
navigation = await sources_service.read_source_navigation(parsed_source_id)
latest_job_source = source.latest_job_source
latest_attempt = (
await sources_service.read_latest_execution_attempt(job_source_id=latest_job_source.id)
await evidence_service.read_latest_execution_attempt(job_source_id=latest_job_source.id)
if latest_job_source is not None
else None
)
attempts = list(await sources_service.list_execution_attempts(source_id=parsed_source_id))
attempts = list(await evidence_service.list_execution_attempts(source_id=parsed_source_id))
except TranscriptionNotFoundError:
render_record_not_found("Source")
return
@@ -158,7 +160,7 @@ def register_page() -> None: # noqa: PLR0915
"Export Evidence",
on_click=lambda: _download_evidence(
source_id=source.id,
sources_service=sources_service,
evidence_service=evidence_service,
),
icon="download",
).props("flat")
@@ -185,7 +187,7 @@ def register_page() -> None: # noqa: PLR0915
_render_machine_candidates(
source=source,
attempts=attempts,
sources_service=sources_service,
evidence_service=evidence_service,
)
_render_source_metadata_column(
source=source,
@@ -409,9 +411,9 @@ def _transport_display(latest_attempt: LatestExecutionAttempt) -> dict[str, obje
}
async def _download_evidence(*, source_id: UUID, sources_service: SourceService) -> None:
async def _download_evidence(*, source_id: UUID, evidence_service: EvidenceService) -> None:
try:
payload = await sources_service.build_evidence_export(source_id=source_id)
payload = await evidence_service.build_evidence_export(source_id=source_id)
except Exception as exc: # noqa: BLE001
show_error(exc, title="Export failed", operation="sources.evidence_export")
return
@@ -521,7 +523,7 @@ def _render_machine_candidates(
*,
source: Source,
attempts: list[ExecutionAttempt],
sources_service: SourceService,
evidence_service: EvidenceService,
) -> None:
successful = [
attempt
@@ -581,7 +583,7 @@ def _render_machine_candidates(
async def promote(candidate_id: UUID = attempt.id) -> None:
try:
await sources_service.promote_machine_attempt(
await evidence_service.promote_machine_attempt(
source_id=source.id,
execution_attempt_id=candidate_id,
)