generated from john/python-template
V4.6 Phase 4: service layer consolidation
Removes the duplicated registry CRUD, the hand-written not-found raises, and the three divergent media writers. Behavior is preserved: every existing Document Type and Person Role test passes unchanged, which is the primary proof for MED-11. [MED-11] Generic registry service - New services/registry.py owns RegistryService[ModelT]: list, list with counts, create with IntegrityError -> conflict mapping, read, update, delete with built-in and referenced guards, is_referenced, and label normalization/casefold keying. - DocumentTypeRegistry and PersonRoleRegistry declare only the model, error class, noun, short noun, retainer phrase, and reference columns. - DocumentService and PeopleService keep their public method names and delegate. Every user-facing message, error category, and suggestion string is reproduced verbatim; only the noun is templated. - Deleted _normalize_registry_label, _document_type_label_key, _normalize_role_label, _person_role_label_key, _document_type_is_referenced, and _person_role_is_referenced. [MED-12] Shared not-found lookup - ServiceBase._get_or_raise(model, id, *, session, error, noun, suggestion, options) loads by primary key or raises the caller's error type. - documents.py: local _get_document_or_raise deleted; replaced by _read_document and adopted at read_document, delete_document, and set_document_type, which previously bypassed the helper and hand-wrote the raise. - sources.py: 8 identical Source raises and 1 Job raise collapsed into _read_source / _get_or_raise. - jobs.py and people.py already funneled through local _not_found builders and were left alone. [MED-13][MED-01] Single media writer - New services/media_storage.py owns validate -> name -> mkdir -> write -> wrap OSError. The write runs in asyncio.to_thread, so uploads no longer block the event loop. - store_source_file, store_person_portrait, and store_homepage_image now share it and are async. Callers in store.py, people_page.py, and home_page.py await them. mkdir failures are now also translated to a domain error instead of escaping as a raw OSError. - homepage_store gains HomepageStorageError so its write reports like the others. [MED-14, partial] Service independence - New services/source_media.py owns SOURCE_MIME_TYPES, SOURCE_EXTENSIONS, lookup_source_mime_type, and supported_source_formats. - documents.py no longer imports services/sources.py. Its print projection uses the non-raising lookup and raises DocumentError, so DocumentService no longer emits a TranscriptionError. - api/v4_print.py imports the mapping from the policy module. - store.py and workflows.py still import sources.py; both are orchestration modules, which services.instructions.md:75-77 explicitly permits. - Splitting SourceService itself remains deferred to V4.7. [LOW-08] Query shape - list_sources_detail filters job_id with a JOIN on JobSource instead of loading every Source and filtering in Python. - read_source_navigation replaces the full ordered-id scan and .index() with two row-value comparisons bounded by LIMIT 1. - list_processing_artifacts gains the limit parameter its summary sibling already had. - build_evidence_export runs artifact integrity hashing and file reads through asyncio.to_thread. Tests - tests/test_service_boundaries.py: AST guard asserting no service module imports a sibling service module, plus a guard that the scan is non-empty. - tests/services/test_transcription_service.py: asserts the job_id filter emits a JOIN, and that navigation emits exactly two LIMIT queries. - tests/services/test_store.py: the two storage tests are now async. Verified: 276 passed, 4 skipped; ruff check clean.
This commit is contained in:
@@ -12,7 +12,7 @@ from fastapi import HTTPException
|
|||||||
from fastapi import Request
|
from fastapi import Request
|
||||||
from fastapi.responses import FileResponse
|
from fastapi.responses import FileResponse
|
||||||
|
|
||||||
from transcription.services.sources import SOURCE_MIME_TYPES
|
from transcription.services.source_media import SOURCE_MIME_TYPES
|
||||||
from transcription.services.sources import SourceService
|
from transcription.services.sources import SourceService
|
||||||
|
|
||||||
router = APIRouter(prefix="/api/v4", tags=["v4-print"])
|
router = APIRouter(prefix="/api/v4", tags=["v4-print"])
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
from abc import ABC
|
from abc import ABC
|
||||||
from collections.abc import Sequence
|
from collections.abc import Sequence
|
||||||
from contextlib import asynccontextmanager
|
from contextlib import asynccontextmanager
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
@@ -9,6 +10,8 @@ from ..config import Settings
|
|||||||
from ..config import get_settings
|
from ..config import get_settings
|
||||||
from ..db.session import resolve_session_factory
|
from ..db.session import resolve_session_factory
|
||||||
from ..db.session import session_scope
|
from ..db.session import session_scope
|
||||||
|
from ..errors import AppError
|
||||||
|
from ..errors import ErrorCategory
|
||||||
|
|
||||||
|
|
||||||
class ServiceBase(ABC):
|
class ServiceBase(ABC):
|
||||||
@@ -54,3 +57,28 @@ class ServiceBase(ABC):
|
|||||||
|
|
||||||
for obj in refresh:
|
for obj in refresh:
|
||||||
await session.refresh(obj)
|
await session.refresh(obj)
|
||||||
|
|
||||||
|
async def _get_or_raise[ModelT](
|
||||||
|
self,
|
||||||
|
model: type[ModelT],
|
||||||
|
entity_id: object,
|
||||||
|
*,
|
||||||
|
session: AsyncSession,
|
||||||
|
error: type[AppError],
|
||||||
|
noun: str,
|
||||||
|
suggestion: str,
|
||||||
|
options: Sequence[Any] = (),
|
||||||
|
) -> ModelT:
|
||||||
|
"""Load an entity by primary key or raise a not-found service error.
|
||||||
|
|
||||||
|
``noun`` and ``suggestion`` are supplied by the caller so each domain
|
||||||
|
keeps its own user-facing wording.
|
||||||
|
"""
|
||||||
|
entity = await session.get(model, entity_id, options=list(options) or None)
|
||||||
|
if entity is None:
|
||||||
|
raise error(
|
||||||
|
f"{noun} with id {entity_id} not found",
|
||||||
|
category=ErrorCategory.NOT_FOUND,
|
||||||
|
suggestion=suggestion,
|
||||||
|
)
|
||||||
|
return entity
|
||||||
|
|||||||
@@ -5,15 +5,19 @@ from dataclasses import dataclass
|
|||||||
from datetime import UTC
|
from datetime import UTC
|
||||||
from datetime import date
|
from datetime import date
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
|
|
||||||
from sqlalchemy import func
|
|
||||||
from sqlalchemy.exc import IntegrityError
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||||
from sqlalchemy.orm import selectinload
|
from sqlalchemy.orm import selectinload
|
||||||
|
from sqlmodel import SQLModel
|
||||||
from sqlmodel import col
|
from sqlmodel import col
|
||||||
from sqlmodel import select
|
from sqlmodel import select
|
||||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
|
from ..config import Settings
|
||||||
from ..db.models import Document
|
from ..db.models import Document
|
||||||
from ..db.models import DocumentPerson
|
from ..db.models import DocumentPerson
|
||||||
from ..db.models import DocumentType
|
from ..db.models import DocumentType
|
||||||
@@ -21,7 +25,9 @@ from ..db.registries import AUTHOR_ROLE_SEMANTIC_KEY
|
|||||||
from ..errors import AppError
|
from ..errors import AppError
|
||||||
from ..errors import ErrorCategory
|
from ..errors import ErrorCategory
|
||||||
from .base import ServiceBase
|
from .base import ServiceBase
|
||||||
from .sources import source_mime_type
|
from .registry import RegistryService
|
||||||
|
from .source_media import lookup_source_mime_type
|
||||||
|
from .source_media import supported_source_formats
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -46,19 +52,23 @@ class DocumentTypeError(DocumentError):
|
|||||||
"""Raised when Document Type maintenance fails."""
|
"""Raised when Document Type maintenance fails."""
|
||||||
|
|
||||||
|
|
||||||
def _normalize_registry_label(label: str) -> str:
|
class DocumentTypeRegistry(RegistryService[DocumentType]):
|
||||||
normalized = label.strip()
|
"""Document Type registry maintenance."""
|
||||||
if not normalized:
|
|
||||||
raise DocumentTypeError(
|
|
||||||
"Document Type label is required",
|
|
||||||
category=ErrorCategory.VALIDATION,
|
|
||||||
suggestion="Enter a user-facing label and retry.",
|
|
||||||
)
|
|
||||||
return normalized
|
|
||||||
|
|
||||||
|
model = DocumentType
|
||||||
|
error = DocumentTypeError
|
||||||
|
noun = "Document Type"
|
||||||
|
short_noun = "type"
|
||||||
|
referenced_retainer = "historical Documents"
|
||||||
|
|
||||||
def _document_type_label_key(label: str) -> str:
|
def reference_model(self) -> type[SQLModel]:
|
||||||
return _normalize_registry_label(label).casefold()
|
return Document
|
||||||
|
|
||||||
|
def reference_id_column(self) -> Any:
|
||||||
|
return col(Document.id)
|
||||||
|
|
||||||
|
def reference_key_column(self) -> Any:
|
||||||
|
return col(Document.document_type_id)
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
@dataclass(frozen=True, slots=True)
|
||||||
@@ -109,6 +119,14 @@ class DocumentPrintProjection:
|
|||||||
class DocumentService(ServiceBase):
|
class DocumentService(ServiceBase):
|
||||||
"""Thin service class for managing documents in the database."""
|
"""Thin service class for managing documents in the database."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
session_factory: async_sessionmaker[AsyncSession] | None = None,
|
||||||
|
settings: Settings | None = None,
|
||||||
|
) -> None:
|
||||||
|
super().__init__(session_factory, settings)
|
||||||
|
self._document_types = DocumentTypeRegistry(self.session_factory, self.settings)
|
||||||
|
|
||||||
async def _validate_document_type(self, *, session: AsyncSession, document: Document) -> None:
|
async def _validate_document_type(self, *, session: AsyncSession, document: Document) -> None:
|
||||||
"""Validate the UUID-backed Document Type reference."""
|
"""Validate the UUID-backed Document Type reference."""
|
||||||
if document.document_type_id is None:
|
if document.document_type_id is None:
|
||||||
@@ -120,16 +138,23 @@ class DocumentService(ServiceBase):
|
|||||||
suggestion="Select a valid document type and retry.",
|
suggestion="Select a valid document type and retry.",
|
||||||
)
|
)
|
||||||
|
|
||||||
async def _get_document_or_raise(self, *, session: AsyncSession, document_id: UUID) -> Document:
|
async def _read_document(
|
||||||
"""Get a document by id or raise a not-found service error."""
|
self,
|
||||||
document = await session.get(Document, document_id)
|
*,
|
||||||
if document is None:
|
session: AsyncSession,
|
||||||
raise DocumentError(
|
document_id: UUID,
|
||||||
f"Document with id {document_id} not found",
|
options: Sequence[Any] = (),
|
||||||
category=ErrorCategory.NOT_FOUND,
|
suggestion: str = "Verify the document id and retry.",
|
||||||
suggestion="Verify the document id and retry.",
|
) -> Document:
|
||||||
)
|
return await self._get_or_raise(
|
||||||
return document
|
Document,
|
||||||
|
document_id,
|
||||||
|
session=session,
|
||||||
|
error=DocumentError,
|
||||||
|
noun="Document",
|
||||||
|
suggestion=suggestion,
|
||||||
|
options=options,
|
||||||
|
)
|
||||||
|
|
||||||
#
|
#
|
||||||
# CRUD Operations
|
# CRUD Operations
|
||||||
@@ -161,21 +186,16 @@ class DocumentService(ServiceBase):
|
|||||||
The selectinload option is used to eagerly load related jobs and sources.
|
The selectinload option is used to eagerly load related jobs and sources.
|
||||||
"""
|
"""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
document = await _session.get(
|
document = await self._read_document(
|
||||||
Document,
|
session=_session,
|
||||||
document_id,
|
document_id=document_id,
|
||||||
options=(
|
options=(
|
||||||
selectinload(Document.jobs), # pyright: ignore[reportArgumentType]
|
selectinload(Document.jobs), # pyright: ignore[reportArgumentType]
|
||||||
selectinload(Document.sources), # pyright: ignore[reportArgumentType]
|
selectinload(Document.sources), # pyright: ignore[reportArgumentType]
|
||||||
),
|
),
|
||||||
)
|
|
||||||
if document is None:
|
|
||||||
raise DocumentError(
|
|
||||||
f"Document with id {document_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Re-upload the source document and retry.",
|
suggestion="Re-upload the source document and retry.",
|
||||||
)
|
)
|
||||||
elif not document.sources:
|
if not document.sources:
|
||||||
raise MissingSourceError(
|
raise MissingSourceError(
|
||||||
f"Document with id {document_id} has no associated source records",
|
f"Document with id {document_id} has no associated source records",
|
||||||
category=ErrorCategory.NOT_FOUND,
|
category=ErrorCategory.NOT_FOUND,
|
||||||
@@ -196,21 +216,15 @@ class DocumentService(ServiceBase):
|
|||||||
"""Delete a document from the database."""
|
"""Delete a document from the database."""
|
||||||
document_id = document.id
|
document_id = document.id
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
existing = await _session.get(
|
existing = await self._read_document(
|
||||||
Document,
|
session=_session,
|
||||||
document.id,
|
document_id=document.id,
|
||||||
options=(
|
options=(
|
||||||
selectinload(Document.jobs), # pyright: ignore[reportArgumentType]
|
selectinload(Document.jobs), # pyright: ignore[reportArgumentType]
|
||||||
selectinload(Document.sources), # pyright: ignore[reportArgumentType]
|
selectinload(Document.sources), # pyright: ignore[reportArgumentType]
|
||||||
selectinload(Document.document_people), # pyright: ignore[reportArgumentType]
|
selectinload(Document.document_people), # pyright: ignore[reportArgumentType]
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
if existing is None:
|
|
||||||
raise DocumentError(
|
|
||||||
f"Document with id {document.id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Verify the document id and retry.",
|
|
||||||
)
|
|
||||||
|
|
||||||
has_jobs = bool(existing.jobs)
|
has_jobs = bool(existing.jobs)
|
||||||
has_sources = bool(existing.sources)
|
has_sources = bool(existing.sources)
|
||||||
@@ -316,7 +330,7 @@ class DocumentService(ServiceBase):
|
|||||||
DocumentPrintSource(
|
DocumentPrintSource(
|
||||||
id=source.id,
|
id=source.id,
|
||||||
page_number=source.page_number,
|
page_number=source.page_number,
|
||||||
media_type=source_mime_type(source.filename),
|
media_type=_print_media_type(source.filename),
|
||||||
current_text=_current_print_text(source.revised_text, source.raw_transcription),
|
current_text=_current_print_text(source.revised_text, source.raw_transcription),
|
||||||
)
|
)
|
||||||
for source in sorted(document.sources, key=lambda item: (item.page_number, item.id))
|
for source in sorted(document.sources, key=lambda item: (item.page_number, item.id))
|
||||||
@@ -354,13 +368,7 @@ class DocumentService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> Sequence[DocumentType]:
|
) -> Sequence[DocumentType]:
|
||||||
"""List configured document types."""
|
"""List configured document types."""
|
||||||
async with self._session_scope(session) as _session:
|
return await self._document_types.list_entries(active_only=active_only, session=session)
|
||||||
query = select(DocumentType)
|
|
||||||
if active_only:
|
|
||||||
query = query.where(col(DocumentType.is_active).is_(True))
|
|
||||||
query = query.order_by(col(DocumentType.normalized_label), col(DocumentType.id))
|
|
||||||
result = await _session.exec(query)
|
|
||||||
return result.all()
|
|
||||||
|
|
||||||
async def list_document_type_summaries(
|
async def list_document_type_summaries(
|
||||||
self,
|
self,
|
||||||
@@ -368,24 +376,17 @@ class DocumentService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> Sequence[DocumentTypeSummary]:
|
) -> Sequence[DocumentTypeSummary]:
|
||||||
"""List Document Types alphabetically with current usage counts."""
|
"""List Document Types alphabetically with current usage counts."""
|
||||||
async with self._session_scope(session) as _session:
|
rows = await self._document_types.list_entries_with_counts(session=session)
|
||||||
query = (
|
return [
|
||||||
select(DocumentType, func.count(col(Document.id)))
|
DocumentTypeSummary(
|
||||||
.outerjoin(Document, col(Document.document_type_id) == col(DocumentType.id))
|
id=document_type.id,
|
||||||
.group_by(col(DocumentType.id))
|
label=document_type.label,
|
||||||
.order_by(col(DocumentType.normalized_label), col(DocumentType.id))
|
is_active=document_type.is_active,
|
||||||
|
is_built_in=document_type.semantic_key is not None,
|
||||||
|
document_count=document_count,
|
||||||
)
|
)
|
||||||
rows = (await _session.exec(query)).all()
|
for document_type, document_count in rows
|
||||||
return [
|
]
|
||||||
DocumentTypeSummary(
|
|
||||||
id=document_type.id,
|
|
||||||
label=document_type.label,
|
|
||||||
is_active=document_type.is_active,
|
|
||||||
is_built_in=document_type.semantic_key is not None,
|
|
||||||
document_count=int(document_count),
|
|
||||||
)
|
|
||||||
for document_type, document_count in rows
|
|
||||||
]
|
|
||||||
|
|
||||||
async def create_document_type(
|
async def create_document_type(
|
||||||
self,
|
self,
|
||||||
@@ -395,22 +396,7 @@ class DocumentService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> DocumentType:
|
) -> DocumentType:
|
||||||
"""Create a UUID-identified Document Type with a unique label."""
|
"""Create a UUID-identified Document Type with a unique label."""
|
||||||
document_type = DocumentType(
|
return await self._document_types.create_entry(label=label, is_active=is_active, session=session)
|
||||||
label=_normalize_registry_label(label),
|
|
||||||
normalized_label=_document_type_label_key(label),
|
|
||||||
is_active=is_active,
|
|
||||||
)
|
|
||||||
async with self._session_scope(session) as _session:
|
|
||||||
_session.add(document_type)
|
|
||||||
try:
|
|
||||||
await self._finalize(session=_session, caller_session=session, refresh=(document_type,))
|
|
||||||
except IntegrityError as exc:
|
|
||||||
raise DocumentTypeError(
|
|
||||||
f"Document Type label {document_type.label!r} already exists",
|
|
||||||
category=ErrorCategory.CONFLICT,
|
|
||||||
suggestion="Choose a different label or edit the existing type.",
|
|
||||||
) from exc
|
|
||||||
return document_type
|
|
||||||
|
|
||||||
async def read_document_type(
|
async def read_document_type(
|
||||||
self,
|
self,
|
||||||
@@ -419,15 +405,7 @@ class DocumentService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> DocumentType:
|
) -> DocumentType:
|
||||||
"""Read a Document Type by id."""
|
"""Read a Document Type by id."""
|
||||||
async with self._session_scope(session) as _session:
|
return await self._document_types.read_entry(document_type_id, session=session)
|
||||||
document_type = await _session.get(DocumentType, document_type_id)
|
|
||||||
if document_type is None:
|
|
||||||
raise DocumentTypeError(
|
|
||||||
f"Document Type with id {document_type_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Refresh Settings and select an available Document Type.",
|
|
||||||
)
|
|
||||||
return document_type
|
|
||||||
|
|
||||||
async def update_document_type(
|
async def update_document_type(
|
||||||
self,
|
self,
|
||||||
@@ -438,27 +416,12 @@ class DocumentService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> DocumentType:
|
) -> DocumentType:
|
||||||
"""Update a Document Type label and active state."""
|
"""Update a Document Type label and active state."""
|
||||||
async with self._session_scope(session) as _session:
|
return await self._document_types.update_entry(
|
||||||
document_type = await _session.get(DocumentType, document_type_id)
|
document_type_id,
|
||||||
if document_type is None:
|
label=label,
|
||||||
raise DocumentTypeError(
|
is_active=is_active,
|
||||||
f"Document Type with id {document_type_id} not found",
|
session=session,
|
||||||
category=ErrorCategory.NOT_FOUND,
|
)
|
||||||
suggestion="Refresh Settings and select an available Document Type.",
|
|
||||||
)
|
|
||||||
document_type.label = _normalize_registry_label(label)
|
|
||||||
document_type.normalized_label = _document_type_label_key(label)
|
|
||||||
document_type.is_active = is_active
|
|
||||||
document_type.updated_at = datetime.now(UTC)
|
|
||||||
try:
|
|
||||||
await self._finalize(session=_session, caller_session=session, refresh=(document_type,))
|
|
||||||
except IntegrityError as exc:
|
|
||||||
raise DocumentTypeError(
|
|
||||||
f"Document Type label {document_type.label!r} already exists",
|
|
||||||
category=ErrorCategory.CONFLICT,
|
|
||||||
suggestion="Choose a different label or edit the existing type.",
|
|
||||||
) from exc
|
|
||||||
return document_type
|
|
||||||
|
|
||||||
async def delete_document_type(
|
async def delete_document_type(
|
||||||
self,
|
self,
|
||||||
@@ -467,28 +430,7 @@ class DocumentService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Delete an unreferenced Document Type without cascade behavior."""
|
"""Delete an unreferenced Document Type without cascade behavior."""
|
||||||
async with self._session_scope(session) as _session:
|
await self._document_types.delete_entry(document_type_id, session=session)
|
||||||
document_type = await _session.get(DocumentType, document_type_id)
|
|
||||||
if document_type is None:
|
|
||||||
raise DocumentTypeError(
|
|
||||||
f"Document Type with id {document_type_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Refresh Settings and select an available Document Type.",
|
|
||||||
)
|
|
||||||
if document_type.semantic_key is not None:
|
|
||||||
raise DocumentTypeError(
|
|
||||||
f"Built-in Document Type {document_type.label!r} cannot be deleted",
|
|
||||||
category=ErrorCategory.CONFLICT,
|
|
||||||
suggestion="Deactivate the type instead; its built-in meaning must remain available.",
|
|
||||||
)
|
|
||||||
if await self._document_type_is_referenced(session=_session, document_type=document_type):
|
|
||||||
raise DocumentTypeError(
|
|
||||||
f"Document Type {document_type.label!r} is referenced and cannot be deleted",
|
|
||||||
category=ErrorCategory.CONFLICT,
|
|
||||||
suggestion="Deactivate the type instead; historical Documents will retain it.",
|
|
||||||
)
|
|
||||||
await _session.delete(document_type)
|
|
||||||
await self._finalize(session=_session, caller_session=session)
|
|
||||||
|
|
||||||
async def is_document_type_referenced(
|
async def is_document_type_referenced(
|
||||||
self,
|
self,
|
||||||
@@ -497,29 +439,7 @@ class DocumentService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> bool:
|
) -> bool:
|
||||||
"""Return whether a Document references a Document Type."""
|
"""Return whether a Document references a Document Type."""
|
||||||
async with self._session_scope(session) as _session:
|
return await self._document_types.is_referenced(document_type_id, session=session)
|
||||||
document_type = await _session.get(DocumentType, document_type_id)
|
|
||||||
if document_type is None:
|
|
||||||
raise DocumentTypeError(
|
|
||||||
f"Document Type with id {document_type_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Refresh Settings and select an available Document Type.",
|
|
||||||
)
|
|
||||||
return await self._document_type_is_referenced(
|
|
||||||
session=_session,
|
|
||||||
document_type=document_type,
|
|
||||||
)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
async def _document_type_is_referenced(
|
|
||||||
*,
|
|
||||||
session: AsyncSession,
|
|
||||||
document_type: DocumentType,
|
|
||||||
) -> bool:
|
|
||||||
reference = (
|
|
||||||
await session.exec(select(Document.id).where(Document.document_type_id == document_type.id))
|
|
||||||
).first()
|
|
||||||
return reference is not None
|
|
||||||
|
|
||||||
async def set_document_type(
|
async def set_document_type(
|
||||||
self,
|
self,
|
||||||
@@ -530,7 +450,7 @@ class DocumentService(ServiceBase):
|
|||||||
) -> Document:
|
) -> Document:
|
||||||
"""Set a Document Type by UUID."""
|
"""Set a Document Type by UUID."""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
document = await self._get_document_or_raise(session=_session, document_id=document_id)
|
document = await self._read_document(session=_session, document_id=document_id)
|
||||||
document.document_type_id = document_type_id
|
document.document_type_id = document_type_id
|
||||||
await self._validate_document_type(session=_session, document=document)
|
await self._validate_document_type(session=_session, document=document)
|
||||||
document.updated_at = datetime.now(UTC)
|
document.updated_at = datetime.now(UTC)
|
||||||
@@ -538,6 +458,18 @@ class DocumentService(ServiceBase):
|
|||||||
return document
|
return document
|
||||||
|
|
||||||
|
|
||||||
|
def _print_media_type(filename: str) -> str:
|
||||||
|
"""Resolve a stored Source filename to its MIME type for print rendering."""
|
||||||
|
mime_type = lookup_source_mime_type(filename)
|
||||||
|
if mime_type is None:
|
||||||
|
raise DocumentError(
|
||||||
|
f"Unsupported Source format: {Path(filename).suffix.lower() or '<none>'}",
|
||||||
|
category=ErrorCategory.USER_INPUT,
|
||||||
|
suggestion=f"Use one of the supported Source formats: {supported_source_formats()}.",
|
||||||
|
)
|
||||||
|
return mime_type
|
||||||
|
|
||||||
|
|
||||||
def _current_print_text(revised_text: str | None, raw_transcription: str | None) -> str | None:
|
def _current_print_text(revised_text: str | None, raw_transcription: str | None) -> str | None:
|
||||||
selected = revised_text if revised_text is not None else raw_transcription
|
selected = revised_text if revised_text is not None else raw_transcription
|
||||||
return selected if selected is not None and selected.strip() else None
|
return selected if selected is not None and selected.strip() else None
|
||||||
|
|||||||
@@ -0,0 +1,57 @@
|
|||||||
|
"""Single implementation for persisting uploaded media bytes to disk.
|
||||||
|
|
||||||
|
Source pages, Person portraits, and homepage images all follow the same
|
||||||
|
sequence: resolve a target directory, create it, write the bytes, and translate
|
||||||
|
an ``OSError`` into a domain error. The write itself runs on a worker thread so
|
||||||
|
it never blocks the event loop.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
from pathlib import Path
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from ..errors import AppError
|
||||||
|
from ..errors import ErrorCategory
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def build_stored_filename(*, filename: str, filename_stem: str | None = None) -> str:
|
||||||
|
"""Return a safe stored filename preserving the submitted extension."""
|
||||||
|
safe_name = Path(filename).name
|
||||||
|
suffix = Path(safe_name).suffix.lower()
|
||||||
|
stem = filename_stem or str(uuid4())
|
||||||
|
return f"{stem}{suffix}"
|
||||||
|
|
||||||
|
|
||||||
|
async def write_media_bytes(
|
||||||
|
*,
|
||||||
|
target_dir: Path,
|
||||||
|
stored_name: str,
|
||||||
|
file_bytes: bytes,
|
||||||
|
error: type[AppError],
|
||||||
|
failure_message: str,
|
||||||
|
failure_suggestion: str,
|
||||||
|
log_label: str,
|
||||||
|
) -> Path:
|
||||||
|
"""Create ``target_dir`` and write ``file_bytes`` into it off the event loop."""
|
||||||
|
stored_path = target_dir / stored_name
|
||||||
|
try:
|
||||||
|
await asyncio.to_thread(_write, stored_path, file_bytes)
|
||||||
|
except OSError as exc:
|
||||||
|
raise error(
|
||||||
|
failure_message,
|
||||||
|
category=ErrorCategory.INFRA_PERSISTENT,
|
||||||
|
suggestion=failure_suggestion,
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
logger.info("Stored %s: %s", log_label, stored_path)
|
||||||
|
return stored_path
|
||||||
|
|
||||||
|
|
||||||
|
def _write(stored_path: Path, file_bytes: bytes) -> None:
|
||||||
|
stored_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
stored_path.write_bytes(file_bytes)
|
||||||
@@ -9,12 +9,14 @@ from dataclasses import dataclass
|
|||||||
from datetime import UTC
|
from datetime import UTC
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
from uuid import uuid4
|
|
||||||
|
|
||||||
from sqlalchemy import func
|
|
||||||
from sqlalchemy.exc import IntegrityError
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||||
from sqlalchemy.orm import selectinload
|
from sqlalchemy.orm import selectinload
|
||||||
|
from sqlmodel import SQLModel
|
||||||
|
from sqlmodel import col
|
||||||
from sqlmodel import select
|
from sqlmodel import select
|
||||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
@@ -27,6 +29,9 @@ from ..db.models import PersonRole
|
|||||||
from ..errors import AppError
|
from ..errors import AppError
|
||||||
from ..errors import ErrorCategory
|
from ..errors import ErrorCategory
|
||||||
from .base import ServiceBase
|
from .base import ServiceBase
|
||||||
|
from .media_storage import build_stored_filename
|
||||||
|
from .media_storage import write_media_bytes
|
||||||
|
from .registry import RegistryService
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -46,15 +51,23 @@ class PersonRoleError(PeopleError):
|
|||||||
"""Raised when Person Role maintenance fails."""
|
"""Raised when Person Role maintenance fails."""
|
||||||
|
|
||||||
|
|
||||||
def _normalize_role_label(label: str) -> str:
|
class PersonRoleRegistry(RegistryService[PersonRole]):
|
||||||
normalized = label.strip()
|
"""Person Role registry maintenance."""
|
||||||
if not normalized:
|
|
||||||
raise PersonRoleError(
|
model = PersonRole
|
||||||
"Person Role label is required",
|
error = PersonRoleError
|
||||||
category=ErrorCategory.VALIDATION,
|
noun = "Person Role"
|
||||||
suggestion="Enter a user-facing label and retry.",
|
short_noun = "role"
|
||||||
)
|
referenced_retainer = "historical relationships"
|
||||||
return normalized
|
|
||||||
|
def reference_model(self) -> type[SQLModel]:
|
||||||
|
return DocumentPerson
|
||||||
|
|
||||||
|
def reference_id_column(self) -> Any:
|
||||||
|
return col(DocumentPerson.id)
|
||||||
|
|
||||||
|
def reference_key_column(self) -> Any:
|
||||||
|
return col(DocumentPerson.role_id)
|
||||||
|
|
||||||
|
|
||||||
def normalize_family_search_id(value: str | None) -> str | None:
|
def normalize_family_search_id(value: str | None) -> str | None:
|
||||||
@@ -71,10 +84,6 @@ def normalize_family_search_id(value: str | None) -> str | None:
|
|||||||
return normalized
|
return normalized
|
||||||
|
|
||||||
|
|
||||||
def _person_role_label_key(label: str) -> str:
|
|
||||||
return _normalize_role_label(label).casefold()
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
@dataclass(frozen=True, slots=True)
|
||||||
class PersonRoleSummary:
|
class PersonRoleSummary:
|
||||||
"""Settings read model for a Person Role and its usage count."""
|
"""Settings read model for a Person Role and its usage count."""
|
||||||
@@ -97,6 +106,14 @@ class DocumentPersonInput:
|
|||||||
class PeopleService(ServiceBase):
|
class PeopleService(ServiceBase):
|
||||||
"""Manage People, relationship roles, and document-person links."""
|
"""Manage People, relationship roles, and document-person links."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
session_factory: async_sessionmaker[AsyncSession] | None = None,
|
||||||
|
settings: Settings | None = None,
|
||||||
|
) -> None:
|
||||||
|
super().__init__(session_factory, settings)
|
||||||
|
self._person_roles = PersonRoleRegistry(self.session_factory, self.settings)
|
||||||
|
|
||||||
async def create_person(self, person: Person, *, session: AsyncSession | None = None) -> Person:
|
async def create_person(self, person: Person, *, session: AsyncSession | None = None) -> Person:
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
person.family_search_id = normalize_family_search_id(person.family_search_id)
|
person.family_search_id = normalize_family_search_id(person.family_search_id)
|
||||||
@@ -217,11 +234,7 @@ class PeopleService(ServiceBase):
|
|||||||
active_only: bool = True,
|
active_only: bool = True,
|
||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> Sequence[PersonRole]:
|
) -> Sequence[PersonRole]:
|
||||||
async with self._session_scope(session) as _session:
|
return await self._person_roles.list_entries(active_only=active_only, session=session)
|
||||||
query = select(PersonRole)
|
|
||||||
if active_only:
|
|
||||||
query = query.where(PersonRole.is_active.is_(True))
|
|
||||||
return (await _session.exec(query.order_by(PersonRole.normalized_label, PersonRole.id))).all()
|
|
||||||
|
|
||||||
async def list_person_role_summaries(
|
async def list_person_role_summaries(
|
||||||
self,
|
self,
|
||||||
@@ -229,24 +242,17 @@ class PeopleService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> Sequence[PersonRoleSummary]:
|
) -> Sequence[PersonRoleSummary]:
|
||||||
"""List Person Roles alphabetically with current link counts."""
|
"""List Person Roles alphabetically with current link counts."""
|
||||||
async with self._session_scope(session) as _session:
|
rows = await self._person_roles.list_entries_with_counts(session=session)
|
||||||
query = (
|
return [
|
||||||
select(PersonRole, func.count(DocumentPerson.id))
|
PersonRoleSummary(
|
||||||
.outerjoin(DocumentPerson, DocumentPerson.role_id == PersonRole.id)
|
id=role.id,
|
||||||
.group_by(PersonRole.id)
|
label=role.label,
|
||||||
.order_by(PersonRole.normalized_label, PersonRole.id)
|
is_active=role.is_active,
|
||||||
|
is_built_in=role.semantic_key is not None,
|
||||||
|
link_count=link_count,
|
||||||
)
|
)
|
||||||
rows = (await _session.exec(query)).all()
|
for role, link_count in rows
|
||||||
return [
|
]
|
||||||
PersonRoleSummary(
|
|
||||||
id=role.id,
|
|
||||||
label=role.label,
|
|
||||||
is_active=role.is_active,
|
|
||||||
is_built_in=role.semantic_key is not None,
|
|
||||||
link_count=int(link_count),
|
|
||||||
)
|
|
||||||
for role, link_count in rows
|
|
||||||
]
|
|
||||||
|
|
||||||
async def create_person_role(
|
async def create_person_role(
|
||||||
self,
|
self,
|
||||||
@@ -256,22 +262,7 @@ class PeopleService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> PersonRole:
|
) -> PersonRole:
|
||||||
"""Create a custom Person Role with a unique label."""
|
"""Create a custom Person Role with a unique label."""
|
||||||
role = PersonRole(
|
return await self._person_roles.create_entry(label=label, is_active=is_active, session=session)
|
||||||
label=_normalize_role_label(label),
|
|
||||||
normalized_label=_person_role_label_key(label),
|
|
||||||
is_active=is_active,
|
|
||||||
)
|
|
||||||
async with self._session_scope(session) as _session:
|
|
||||||
_session.add(role)
|
|
||||||
try:
|
|
||||||
await self._finalize(session=_session, caller_session=session, refresh=(role,))
|
|
||||||
except IntegrityError as exc:
|
|
||||||
raise PersonRoleError(
|
|
||||||
f"Person Role label {role.label!r} already exists",
|
|
||||||
category=ErrorCategory.CONFLICT,
|
|
||||||
suggestion="Choose a different label or edit the existing role.",
|
|
||||||
) from exc
|
|
||||||
return role
|
|
||||||
|
|
||||||
async def read_person_role(
|
async def read_person_role(
|
||||||
self,
|
self,
|
||||||
@@ -280,15 +271,7 @@ class PeopleService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> PersonRole:
|
) -> PersonRole:
|
||||||
"""Read a Person Role by id."""
|
"""Read a Person Role by id."""
|
||||||
async with self._session_scope(session) as _session:
|
return await self._person_roles.read_entry(person_role_id, session=session)
|
||||||
role = await _session.get(PersonRole, person_role_id)
|
|
||||||
if role is None:
|
|
||||||
raise PersonRoleError(
|
|
||||||
f"Person Role with id {person_role_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Refresh Settings and select an available Person Role.",
|
|
||||||
)
|
|
||||||
return role
|
|
||||||
|
|
||||||
async def update_person_role(
|
async def update_person_role(
|
||||||
self,
|
self,
|
||||||
@@ -299,27 +282,12 @@ class PeopleService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> PersonRole:
|
) -> PersonRole:
|
||||||
"""Update mutable Person Role fields without changing semantic identity."""
|
"""Update mutable Person Role fields without changing semantic identity."""
|
||||||
async with self._session_scope(session) as _session:
|
return await self._person_roles.update_entry(
|
||||||
role = await _session.get(PersonRole, person_role_id)
|
person_role_id,
|
||||||
if role is None:
|
label=label,
|
||||||
raise PersonRoleError(
|
is_active=is_active,
|
||||||
f"Person Role with id {person_role_id} not found",
|
session=session,
|
||||||
category=ErrorCategory.NOT_FOUND,
|
)
|
||||||
suggestion="Refresh Settings and select an available Person Role.",
|
|
||||||
)
|
|
||||||
role.label = _normalize_role_label(label)
|
|
||||||
role.normalized_label = _person_role_label_key(label)
|
|
||||||
role.is_active = is_active
|
|
||||||
role.updated_at = datetime.now(UTC)
|
|
||||||
try:
|
|
||||||
await self._finalize(session=_session, caller_session=session, refresh=(role,))
|
|
||||||
except IntegrityError as exc:
|
|
||||||
raise PersonRoleError(
|
|
||||||
f"Person Role label {role.label!r} already exists",
|
|
||||||
category=ErrorCategory.CONFLICT,
|
|
||||||
suggestion="Choose a different label or edit the existing role.",
|
|
||||||
) from exc
|
|
||||||
return role
|
|
||||||
|
|
||||||
async def delete_person_role(
|
async def delete_person_role(
|
||||||
self,
|
self,
|
||||||
@@ -328,28 +296,7 @@ class PeopleService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Delete an unreferenced Person Role without cascade behavior."""
|
"""Delete an unreferenced Person Role without cascade behavior."""
|
||||||
async with self._session_scope(session) as _session:
|
await self._person_roles.delete_entry(person_role_id, session=session)
|
||||||
role = await _session.get(PersonRole, person_role_id)
|
|
||||||
if role is None:
|
|
||||||
raise PersonRoleError(
|
|
||||||
f"Person Role with id {person_role_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Refresh Settings and select an available Person Role.",
|
|
||||||
)
|
|
||||||
if role.semantic_key is not None:
|
|
||||||
raise PersonRoleError(
|
|
||||||
f"Built-in Person Role {role.label!r} cannot be deleted",
|
|
||||||
category=ErrorCategory.CONFLICT,
|
|
||||||
suggestion="Deactivate the role instead; its built-in meaning must remain available.",
|
|
||||||
)
|
|
||||||
if await self._person_role_is_referenced(session=_session, role=role):
|
|
||||||
raise PersonRoleError(
|
|
||||||
f"Person Role {role.label!r} is referenced and cannot be deleted",
|
|
||||||
category=ErrorCategory.CONFLICT,
|
|
||||||
suggestion="Deactivate the role instead; historical relationships will retain it.",
|
|
||||||
)
|
|
||||||
await _session.delete(role)
|
|
||||||
await self._finalize(session=_session, caller_session=session)
|
|
||||||
|
|
||||||
async def is_person_role_referenced(
|
async def is_person_role_referenced(
|
||||||
self,
|
self,
|
||||||
@@ -358,24 +305,7 @@ class PeopleService(ServiceBase):
|
|||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> bool:
|
) -> bool:
|
||||||
"""Return whether a document-person link references a Person Role."""
|
"""Return whether a document-person link references a Person Role."""
|
||||||
async with self._session_scope(session) as _session:
|
return await self._person_roles.is_referenced(person_role_id, session=session)
|
||||||
role = await _session.get(PersonRole, person_role_id)
|
|
||||||
if role is None:
|
|
||||||
raise PersonRoleError(
|
|
||||||
f"Person Role with id {person_role_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Refresh Settings and select an available Person Role.",
|
|
||||||
)
|
|
||||||
return await self._person_role_is_referenced(session=_session, role=role)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
async def _person_role_is_referenced(
|
|
||||||
*,
|
|
||||||
session: AsyncSession,
|
|
||||||
role: PersonRole,
|
|
||||||
) -> bool:
|
|
||||||
reference = (await session.exec(select(DocumentPerson.id).where(DocumentPerson.role_id == role.id))).first()
|
|
||||||
return reference is not None
|
|
||||||
|
|
||||||
async def read_person_role_by_semantic_key(
|
async def read_person_role_by_semantic_key(
|
||||||
self,
|
self,
|
||||||
@@ -595,7 +525,7 @@ class PeopleService(ServiceBase):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def store_person_portrait(
|
async def store_person_portrait(
|
||||||
*,
|
*,
|
||||||
person_id: UUID,
|
person_id: UUID,
|
||||||
filename: str,
|
filename: str,
|
||||||
@@ -618,16 +548,12 @@ def store_person_portrait(
|
|||||||
)
|
)
|
||||||
|
|
||||||
runtime_settings = settings or get_settings()
|
runtime_settings = settings or get_settings()
|
||||||
target_dir = runtime_settings.upload_dir / "persons" / str(person_id)
|
return await write_media_bytes(
|
||||||
target_dir.mkdir(parents=True, exist_ok=True)
|
target_dir=runtime_settings.upload_dir / "persons" / str(person_id),
|
||||||
stored_path = target_dir / f"{uuid4()}{suffix}"
|
stored_name=build_stored_filename(filename=filename),
|
||||||
try:
|
file_bytes=file_bytes,
|
||||||
stored_path.write_bytes(file_bytes)
|
error=PersonMediaError,
|
||||||
except OSError as exc:
|
failure_message="Failed to persist Person portrait media",
|
||||||
raise PersonMediaError(
|
failure_suggestion="Check media directory permissions and available disk space, then retry.",
|
||||||
"Failed to persist Person portrait media",
|
log_label="Person portrait media",
|
||||||
category=ErrorCategory.INFRA_PERSISTENT,
|
)
|
||||||
suggestion="Check media directory permissions and available disk space, then retry.",
|
|
||||||
) from exc
|
|
||||||
logger.info("Stored Person portrait media: %s", stored_path)
|
|
||||||
return stored_path
|
|
||||||
|
|||||||
@@ -0,0 +1,240 @@
|
|||||||
|
"""Shared implementation for label-keyed registry tables.
|
||||||
|
|
||||||
|
Document Types and Person Roles are the same shape: a UUID-identified row with a
|
||||||
|
user-facing ``label``, a casefolded ``normalized_label`` uniqueness key, an
|
||||||
|
``is_active`` flag, and an optional ``semantic_key`` marking built-in entries
|
||||||
|
that may be deactivated but never deleted. This module owns that behavior once
|
||||||
|
so the two registries cannot drift apart.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from abc import abstractmethod
|
||||||
|
from collections.abc import Sequence
|
||||||
|
from datetime import UTC
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Any
|
||||||
|
from uuid import UUID
|
||||||
|
|
||||||
|
from sqlalchemy import func
|
||||||
|
from sqlalchemy.exc import IntegrityError
|
||||||
|
from sqlmodel import SQLModel
|
||||||
|
from sqlmodel import col
|
||||||
|
from sqlmodel import select
|
||||||
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
|
from ..errors import AppError
|
||||||
|
from ..errors import ErrorCategory
|
||||||
|
from .base import ServiceBase
|
||||||
|
|
||||||
|
|
||||||
|
class RegistryService[ModelT: SQLModel](ServiceBase):
|
||||||
|
"""Generic create/read/update/delete behavior for a registry table.
|
||||||
|
|
||||||
|
Subclasses declare the model, the error type, the user-facing noun, and the
|
||||||
|
reference query used to decide whether an entry may be deleted.
|
||||||
|
"""
|
||||||
|
|
||||||
|
#: Registry table this service maintains.
|
||||||
|
model: type[ModelT]
|
||||||
|
#: Error raised for every failure mode of this registry.
|
||||||
|
error: type[AppError]
|
||||||
|
#: User-facing singular noun, e.g. ``"Document Type"``.
|
||||||
|
noun: str
|
||||||
|
#: Lowercase noun used inside remediation suggestions, e.g. ``"type"``.
|
||||||
|
short_noun: str
|
||||||
|
#: Subject that retains a referenced entry, e.g. ``"historical Documents"``.
|
||||||
|
referenced_retainer: str
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def reference_model(self) -> type[SQLModel]:
|
||||||
|
"""Return the table whose rows reference this registry."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def reference_id_column(self) -> Any:
|
||||||
|
"""Return the primary key column of the referencing table."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def reference_key_column(self) -> Any:
|
||||||
|
"""Return the foreign key column pointing at this registry.
|
||||||
|
|
||||||
|
Declared as methods rather than class attributes because a mapped
|
||||||
|
column stored on a plain class would be re-invoked as a descriptor.
|
||||||
|
"""
|
||||||
|
|
||||||
|
#
|
||||||
|
# Message templates
|
||||||
|
#
|
||||||
|
|
||||||
|
def _not_found(self, entry_id: UUID) -> AppError:
|
||||||
|
return self.error(
|
||||||
|
f"{self.noun} with id {entry_id} not found",
|
||||||
|
category=ErrorCategory.NOT_FOUND,
|
||||||
|
suggestion=f"Refresh Settings and select an available {self.noun}.",
|
||||||
|
)
|
||||||
|
|
||||||
|
def _duplicate_label(self, label: str) -> AppError:
|
||||||
|
return self.error(
|
||||||
|
f"{self.noun} label {label!r} already exists",
|
||||||
|
category=ErrorCategory.CONFLICT,
|
||||||
|
suggestion=f"Choose a different label or edit the existing {self.short_noun}.",
|
||||||
|
)
|
||||||
|
|
||||||
|
def normalize_label(self, label: str) -> str:
|
||||||
|
"""Strip a submitted label, rejecting blank input."""
|
||||||
|
normalized = label.strip()
|
||||||
|
if not normalized:
|
||||||
|
raise self.error(
|
||||||
|
f"{self.noun} label is required",
|
||||||
|
category=ErrorCategory.VALIDATION,
|
||||||
|
suggestion="Enter a user-facing label and retry.",
|
||||||
|
)
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def label_key(self, label: str) -> str:
|
||||||
|
"""Return the casefolded uniqueness key for a submitted label."""
|
||||||
|
return self.normalize_label(label).casefold()
|
||||||
|
|
||||||
|
#
|
||||||
|
# Reads
|
||||||
|
#
|
||||||
|
|
||||||
|
async def list_entries(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
active_only: bool = True,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> Sequence[ModelT]:
|
||||||
|
"""List registry entries alphabetically by normalized label."""
|
||||||
|
async with self._session_scope(session) as _session:
|
||||||
|
query = select(self.model)
|
||||||
|
if active_only:
|
||||||
|
query = query.where(col(self.model.is_active).is_(True))
|
||||||
|
query = query.order_by(col(self.model.normalized_label), col(self.model.id))
|
||||||
|
return (await _session.exec(query)).all()
|
||||||
|
|
||||||
|
async def list_entries_with_counts(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> Sequence[tuple[ModelT, int]]:
|
||||||
|
"""List every entry alphabetically with its current reference count."""
|
||||||
|
async with self._session_scope(session) as _session:
|
||||||
|
query = (
|
||||||
|
select(self.model, func.count(self.reference_id_column()))
|
||||||
|
.outerjoin(self.reference_model(), self.reference_key_column() == col(self.model.id))
|
||||||
|
.group_by(col(self.model.id))
|
||||||
|
.order_by(col(self.model.normalized_label), col(self.model.id))
|
||||||
|
)
|
||||||
|
rows = (await _session.exec(query)).all()
|
||||||
|
return [(entry, int(count)) for entry, count in rows]
|
||||||
|
|
||||||
|
async def read_entry(
|
||||||
|
self,
|
||||||
|
entry_id: UUID,
|
||||||
|
*,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> ModelT:
|
||||||
|
"""Read a registry entry by id."""
|
||||||
|
async with self._session_scope(session) as _session:
|
||||||
|
entry = await _session.get(self.model, entry_id)
|
||||||
|
if entry is None:
|
||||||
|
raise self._not_found(entry_id)
|
||||||
|
return entry
|
||||||
|
|
||||||
|
async def is_referenced(
|
||||||
|
self,
|
||||||
|
entry_id: UUID,
|
||||||
|
*,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> bool:
|
||||||
|
"""Return whether any row references the registry entry."""
|
||||||
|
async with self._session_scope(session) as _session:
|
||||||
|
entry = await _session.get(self.model, entry_id)
|
||||||
|
if entry is None:
|
||||||
|
raise self._not_found(entry_id)
|
||||||
|
return await self._is_referenced(session=_session, entry=entry)
|
||||||
|
|
||||||
|
async def _is_referenced(self, *, session: AsyncSession, entry: ModelT) -> bool:
|
||||||
|
query = select(self.reference_id_column()).where(self.reference_key_column() == entry.id)
|
||||||
|
return (await session.exec(query)).first() is not None
|
||||||
|
|
||||||
|
#
|
||||||
|
# Writes
|
||||||
|
#
|
||||||
|
|
||||||
|
async def create_entry(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
label: str,
|
||||||
|
is_active: bool = True,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> ModelT:
|
||||||
|
"""Create a UUID-identified entry with a unique label."""
|
||||||
|
entry = self.model(
|
||||||
|
label=self.normalize_label(label),
|
||||||
|
normalized_label=self.label_key(label),
|
||||||
|
is_active=is_active,
|
||||||
|
)
|
||||||
|
async with self._session_scope(session) as _session:
|
||||||
|
_session.add(entry)
|
||||||
|
try:
|
||||||
|
await self._finalize(session=_session, caller_session=session, refresh=(entry,))
|
||||||
|
except IntegrityError as exc:
|
||||||
|
raise self._duplicate_label(entry.label) from exc
|
||||||
|
return entry
|
||||||
|
|
||||||
|
async def update_entry(
|
||||||
|
self,
|
||||||
|
entry_id: UUID,
|
||||||
|
*,
|
||||||
|
label: str,
|
||||||
|
is_active: bool,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> ModelT:
|
||||||
|
"""Update mutable fields without changing semantic identity."""
|
||||||
|
async with self._session_scope(session) as _session:
|
||||||
|
entry = await _session.get(self.model, entry_id)
|
||||||
|
if entry is None:
|
||||||
|
raise self._not_found(entry_id)
|
||||||
|
entry.label = self.normalize_label(label)
|
||||||
|
entry.normalized_label = self.label_key(label)
|
||||||
|
entry.is_active = is_active
|
||||||
|
entry.updated_at = datetime.now(UTC)
|
||||||
|
try:
|
||||||
|
await self._finalize(session=_session, caller_session=session, refresh=(entry,))
|
||||||
|
except IntegrityError as exc:
|
||||||
|
raise self._duplicate_label(entry.label) from exc
|
||||||
|
return entry
|
||||||
|
|
||||||
|
async def delete_entry(
|
||||||
|
self,
|
||||||
|
entry_id: UUID,
|
||||||
|
*,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""Delete an unreferenced, non-built-in entry without cascade behavior."""
|
||||||
|
async with self._session_scope(session) as _session:
|
||||||
|
entry = await _session.get(self.model, entry_id)
|
||||||
|
if entry is None:
|
||||||
|
raise self._not_found(entry_id)
|
||||||
|
if entry.semantic_key is not None:
|
||||||
|
raise self.error(
|
||||||
|
f"Built-in {self.noun} {entry.label!r} cannot be deleted",
|
||||||
|
category=ErrorCategory.CONFLICT,
|
||||||
|
suggestion=(
|
||||||
|
f"Deactivate the {self.short_noun} instead; "
|
||||||
|
"its built-in meaning must remain available."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
if await self._is_referenced(session=_session, entry=entry):
|
||||||
|
raise self.error(
|
||||||
|
f"{self.noun} {entry.label!r} is referenced and cannot be deleted",
|
||||||
|
category=ErrorCategory.CONFLICT,
|
||||||
|
suggestion=(
|
||||||
|
f"Deactivate the {self.short_noun} instead; "
|
||||||
|
f"{self.referenced_retainer} will retain it."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
await _session.delete(entry)
|
||||||
|
await self._finalize(session=_session, caller_session=session)
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
"""Canonical Source media format policy.
|
||||||
|
|
||||||
|
Shared by every layer that needs to know which Source formats exist and what
|
||||||
|
MIME type each maps to. Kept free of service classes and of any service-specific
|
||||||
|
error type so no service module has to import a sibling service to use it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
SOURCE_MIME_TYPES = {
|
||||||
|
".jpg": "image/jpeg",
|
||||||
|
".jpeg": "image/jpeg",
|
||||||
|
".png": "image/png",
|
||||||
|
".tif": "image/tiff",
|
||||||
|
".tiff": "image/tiff",
|
||||||
|
".pdf": "application/pdf",
|
||||||
|
}
|
||||||
|
SOURCE_EXTENSIONS = frozenset(SOURCE_MIME_TYPES)
|
||||||
|
|
||||||
|
|
||||||
|
def lookup_source_mime_type(filename: str | Path) -> str | None:
|
||||||
|
"""Return the canonical MIME type for a filename, or ``None`` if unsupported."""
|
||||||
|
return SOURCE_MIME_TYPES.get(Path(filename).suffix.lower())
|
||||||
|
|
||||||
|
|
||||||
|
def supported_source_formats() -> str:
|
||||||
|
"""Return the supported Source extensions as a sorted display string."""
|
||||||
|
return ", ".join(sorted(SOURCE_EXTENSIONS))
|
||||||
@@ -2,6 +2,7 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
import base64
|
import base64
|
||||||
import hashlib
|
import hashlib
|
||||||
import logging
|
import logging
|
||||||
@@ -12,6 +13,7 @@ from dataclasses import dataclass
|
|||||||
from datetime import UTC
|
from datetime import UTC
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
from uuid import UUID
|
from uuid import UUID
|
||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
|
|
||||||
@@ -22,9 +24,11 @@ from pydantic import JsonValue
|
|||||||
from pydantic import TypeAdapter
|
from pydantic import TypeAdapter
|
||||||
from pydantic import ValidationError
|
from pydantic import ValidationError
|
||||||
from sqlalchemy import func
|
from sqlalchemy import func
|
||||||
|
from sqlalchemy import tuple_
|
||||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||||
from sqlalchemy.orm import defer
|
from sqlalchemy.orm import defer
|
||||||
from sqlalchemy.orm import selectinload
|
from sqlalchemy.orm import selectinload
|
||||||
|
from sqlmodel import col
|
||||||
from sqlmodel import select
|
from sqlmodel import select
|
||||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
@@ -56,19 +60,12 @@ from .normalization import ORIENTATION_PRODUCER_VERSION
|
|||||||
from .normalization import ORIENTATION_SCHEMA
|
from .normalization import ORIENTATION_SCHEMA
|
||||||
from .normalization import ORIENTATION_SCHEMA_VERSION
|
from .normalization import ORIENTATION_SCHEMA_VERSION
|
||||||
from .normalization import normalize_orientation
|
from .normalization import normalize_orientation
|
||||||
|
from .source_media import lookup_source_mime_type
|
||||||
|
from .source_media import supported_source_formats
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
DEFAULT_PROMPT_FILE = "transcribe_document.md"
|
DEFAULT_PROMPT_FILE = "transcribe_document.md"
|
||||||
SOURCE_MIME_TYPES = {
|
|
||||||
".jpg": "image/jpeg",
|
|
||||||
".jpeg": "image/jpeg",
|
|
||||||
".png": "image/png",
|
|
||||||
".tif": "image/tiff",
|
|
||||||
".tiff": "image/tiff",
|
|
||||||
".pdf": "application/pdf",
|
|
||||||
}
|
|
||||||
SOURCE_EXTENSIONS = frozenset(SOURCE_MIME_TYPES)
|
|
||||||
JSON_OBJECT_ADAPTER = TypeAdapter(dict[str, JsonValue])
|
JSON_OBJECT_ADAPTER = TypeAdapter(dict[str, JsonValue])
|
||||||
|
|
||||||
|
|
||||||
@@ -153,6 +150,24 @@ class SourceService(ServiceBase):
|
|||||||
await close()
|
await close()
|
||||||
self._provider = None
|
self._provider = None
|
||||||
|
|
||||||
|
async def _read_source(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
session: AsyncSession,
|
||||||
|
source_id: UUID,
|
||||||
|
options: Sequence[Any] = (),
|
||||||
|
suggestion: str = "Verify the source id and retry.",
|
||||||
|
) -> Source:
|
||||||
|
return await self._get_or_raise(
|
||||||
|
Source,
|
||||||
|
source_id,
|
||||||
|
session=session,
|
||||||
|
error=TranscriptionNotFoundError,
|
||||||
|
noun="Source",
|
||||||
|
suggestion=suggestion,
|
||||||
|
options=options,
|
||||||
|
)
|
||||||
|
|
||||||
async def create_source(self, source: Source, *, session: AsyncSession | None = None) -> Source:
|
async def create_source(self, source: Source, *, session: AsyncSession | None = None) -> Source:
|
||||||
"""Create a new source page record in the database."""
|
"""Create a new source page record in the database."""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
@@ -163,14 +178,7 @@ class SourceService(ServiceBase):
|
|||||||
async def read_source(self, source_id: UUID, *, session: AsyncSession | None = None) -> Source:
|
async def read_source(self, source_id: UUID, *, session: AsyncSession | None = None) -> Source:
|
||||||
"""Read an existing source page record."""
|
"""Read an existing source page record."""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
source = await _session.get(Source, source_id)
|
return await self._read_source(session=_session, source_id=source_id)
|
||||||
if source is None:
|
|
||||||
raise TranscriptionNotFoundError(
|
|
||||||
f"Source with id {source_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Verify the source id and retry.",
|
|
||||||
)
|
|
||||||
return source
|
|
||||||
|
|
||||||
async def read_source_detail(self, source_id: UUID, *, session: AsyncSession | None = None) -> Source:
|
async def read_source_detail(self, source_id: UUID, *, session: AsyncSession | None = None) -> Source:
|
||||||
"""Read a source page record with job-source context for UI detail rendering."""
|
"""Read a source page record with job-source context for UI detail rendering."""
|
||||||
@@ -222,25 +230,29 @@ class SourceService(ServiceBase):
|
|||||||
) -> SourceNavigation:
|
) -> SourceNavigation:
|
||||||
"""Return adjacent Sources ordered within the current Document."""
|
"""Return adjacent Sources ordered within the current Document."""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
source = await _session.get(Source, source_id)
|
source = await self._read_source(session=_session, source_id=source_id)
|
||||||
if source is None:
|
position = (col(Source.page_number), col(Source.id))
|
||||||
raise TranscriptionNotFoundError(
|
current = (source.page_number, source_id)
|
||||||
f"Source with id {source_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Verify the source id and retry.",
|
|
||||||
)
|
|
||||||
query = (
|
|
||||||
select(Source.id)
|
|
||||||
.where(Source.document_id == source.document_id)
|
|
||||||
.order_by(Source.page_number, Source.id) # pyright: ignore[reportArgumentType]
|
|
||||||
)
|
|
||||||
source_ids = list((await _session.exec(query)).all())
|
|
||||||
|
|
||||||
current_index = source_ids.index(source_id)
|
previous_query = (
|
||||||
return SourceNavigation(
|
select(col(Source.id))
|
||||||
previous_id=source_ids[current_index - 1] if current_index > 0 else None,
|
.where(col(Source.document_id) == source.document_id)
|
||||||
next_id=source_ids[current_index + 1] if current_index + 1 < len(source_ids) else None,
|
.where(tuple_(*position) < tuple_(*current))
|
||||||
)
|
.order_by(col(Source.page_number).desc(), col(Source.id).desc())
|
||||||
|
.limit(1)
|
||||||
|
)
|
||||||
|
next_query = (
|
||||||
|
select(col(Source.id))
|
||||||
|
.where(col(Source.document_id) == source.document_id)
|
||||||
|
.where(tuple_(*position) > tuple_(*current))
|
||||||
|
.order_by(col(Source.page_number), col(Source.id))
|
||||||
|
.limit(1)
|
||||||
|
)
|
||||||
|
|
||||||
|
return SourceNavigation(
|
||||||
|
previous_id=(await _session.exec(previous_query)).first(),
|
||||||
|
next_id=(await _session.exec(next_query)).first(),
|
||||||
|
)
|
||||||
|
|
||||||
async def update_source(self, source: Source, *, session: AsyncSession | None = None) -> Source:
|
async def update_source(self, source: Source, *, session: AsyncSession | None = None) -> Source:
|
||||||
"""Update an existing source page record."""
|
"""Update an existing source page record."""
|
||||||
@@ -256,20 +268,14 @@ class SourceService(ServiceBase):
|
|||||||
async def delete_unlinked_source(self, *, source_id: UUID, session: AsyncSession | None = None) -> None:
|
async def delete_unlinked_source(self, *, source_id: UUID, session: AsyncSession | None = None) -> None:
|
||||||
"""Delete a source only when no JobSource links exist."""
|
"""Delete a source only when no JobSource links exist."""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
source = await _session.get(
|
source = await self._read_source(
|
||||||
Source,
|
session=_session,
|
||||||
source_id,
|
source_id=source_id,
|
||||||
options=(
|
options=(
|
||||||
selectinload(Source.job_sources), # pyright: ignore[reportArgumentType]
|
selectinload(Source.job_sources), # pyright: ignore[reportArgumentType]
|
||||||
selectinload(Source.processing_artifacts), # pyright: ignore[reportArgumentType]
|
selectinload(Source.processing_artifacts), # pyright: ignore[reportArgumentType]
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
if source is None:
|
|
||||||
raise TranscriptionNotFoundError(
|
|
||||||
f"Source with id {source_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Verify the source id and retry.",
|
|
||||||
)
|
|
||||||
|
|
||||||
if source.job_sources or source.processing_artifacts:
|
if source.job_sources or source.processing_artifacts:
|
||||||
raise SourceDeleteBlockedError(
|
raise SourceDeleteBlockedError(
|
||||||
@@ -330,18 +336,12 @@ class SourceService(ServiceBase):
|
|||||||
)
|
)
|
||||||
if document_id is not None:
|
if document_id is not None:
|
||||||
query = query.where(Source.document_id == document_id)
|
query = query.where(Source.document_id == document_id)
|
||||||
|
|
||||||
result = await _session.exec(query)
|
|
||||||
sources = list(result.all())
|
|
||||||
|
|
||||||
if job_id is not None:
|
if job_id is not None:
|
||||||
sources = [
|
query = query.join(JobSource, col(JobSource.source_id) == col(Source.id)).where(
|
||||||
source
|
col(JobSource.job_id) == job_id
|
||||||
for source in sources
|
)
|
||||||
if any(job_source.job_id == job_id for job_source in source.job_sources)
|
|
||||||
]
|
|
||||||
|
|
||||||
return sources
|
return list((await _session.exec(query)).all())
|
||||||
|
|
||||||
async def create_job_source(
|
async def create_job_source(
|
||||||
self,
|
self,
|
||||||
@@ -422,20 +422,14 @@ class SourceService(ServiceBase):
|
|||||||
- Blocked when additional JobSource links exist (history/shared dependencies).
|
- Blocked when additional JobSource links exist (history/shared dependencies).
|
||||||
"""
|
"""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
source = await _session.get(
|
source = await self._read_source(
|
||||||
Source,
|
session=_session,
|
||||||
source_id,
|
source_id=source_id,
|
||||||
options=(
|
options=(
|
||||||
selectinload(Source.job_sources), # pyright: ignore[reportArgumentType]
|
selectinload(Source.job_sources), # pyright: ignore[reportArgumentType]
|
||||||
selectinload(Source.processing_artifacts), # pyright: ignore[reportArgumentType]
|
selectinload(Source.processing_artifacts), # pyright: ignore[reportArgumentType]
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
if source is None:
|
|
||||||
raise TranscriptionNotFoundError(
|
|
||||||
f"Source with id {source_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Verify the source id and retry.",
|
|
||||||
)
|
|
||||||
|
|
||||||
linked_job_sources = list(source.job_sources)
|
linked_job_sources = list(source.job_sources)
|
||||||
attempt_count = (
|
attempt_count = (
|
||||||
@@ -530,21 +524,16 @@ class SourceService(ServiceBase):
|
|||||||
) -> JobSource:
|
) -> JobSource:
|
||||||
"""Persist transcription fields for one source within a specific job."""
|
"""Persist transcription fields for one source within a specific job."""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
job = await _session.get(Job, job_id)
|
job = await self._get_or_raise(
|
||||||
if job is None:
|
Job,
|
||||||
raise TranscriptionNotFoundError(
|
job_id,
|
||||||
f"Job with id {job_id} not found",
|
session=_session,
|
||||||
category=ErrorCategory.NOT_FOUND,
|
error=TranscriptionNotFoundError,
|
||||||
suggestion="Verify the job id and retry.",
|
noun="Job",
|
||||||
)
|
suggestion="Verify the job id and retry.",
|
||||||
|
)
|
||||||
|
|
||||||
source = await _session.get(Source, source_id)
|
source = await self._read_source(session=_session, source_id=source_id)
|
||||||
if source is None:
|
|
||||||
raise TranscriptionNotFoundError(
|
|
||||||
f"Source with id {source_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Verify the source id and retry.",
|
|
||||||
)
|
|
||||||
|
|
||||||
if source.document_id != job.document_id:
|
if source.document_id != job.document_id:
|
||||||
raise TranscriptionError(
|
raise TranscriptionError(
|
||||||
@@ -677,13 +666,11 @@ class SourceService(ServiceBase):
|
|||||||
) -> Source:
|
) -> Source:
|
||||||
"""Atomically select one successful machine attempt as the Source projection."""
|
"""Atomically select one successful machine attempt as the Source projection."""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
source = await _session.get(Source, source_id)
|
source = await self._read_source(
|
||||||
if source is None:
|
session=_session,
|
||||||
raise TranscriptionNotFoundError(
|
source_id=source_id,
|
||||||
f"Source with id {source_id} not found",
|
suggestion="Refresh Source Detail and retry.",
|
||||||
category=ErrorCategory.NOT_FOUND,
|
)
|
||||||
suggestion="Refresh Source Detail and retry.",
|
|
||||||
)
|
|
||||||
attempt = await _session.get(ExecutionAttempt, execution_attempt_id)
|
attempt = await _session.get(ExecutionAttempt, execution_attempt_id)
|
||||||
if (
|
if (
|
||||||
attempt is None
|
attempt is None
|
||||||
@@ -945,6 +932,11 @@ class SourceService(ServiceBase):
|
|||||||
suggestion="Restore the expected artifact bytes before retrying.",
|
suggestion="Restore the expected artifact bytes before retrying.",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def _verify_artifacts_integrity(self, artifacts: Sequence[ProcessingArtifact]) -> None:
|
||||||
|
"""Verify a batch of artifacts; hashing and file reads run off the event loop."""
|
||||||
|
for artifact in artifacts:
|
||||||
|
self._verify_artifact_integrity(artifact)
|
||||||
|
|
||||||
def _verify_artifact_integrity(self, artifact: ProcessingArtifact) -> None:
|
def _verify_artifact_integrity(self, artifact: ProcessingArtifact) -> None:
|
||||||
if artifact.inline_payload is None:
|
if artifact.inline_payload is None:
|
||||||
self._verify_external_artifact(artifact)
|
self._verify_external_artifact(artifact)
|
||||||
@@ -961,6 +953,7 @@ class SourceService(ServiceBase):
|
|||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
source_id: UUID,
|
source_id: UUID,
|
||||||
|
limit: int = 100,
|
||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> Sequence[ProcessingArtifact]:
|
) -> Sequence[ProcessingArtifact]:
|
||||||
"""List generic artifacts associated with a Source."""
|
"""List generic artifacts associated with a Source."""
|
||||||
@@ -969,6 +962,7 @@ class SourceService(ServiceBase):
|
|||||||
select(ProcessingArtifact)
|
select(ProcessingArtifact)
|
||||||
.where(ProcessingArtifact.source_id == source_id)
|
.where(ProcessingArtifact.source_id == source_id)
|
||||||
.order_by(ProcessingArtifact.created_at, ProcessingArtifact.id)
|
.order_by(ProcessingArtifact.created_at, ProcessingArtifact.id)
|
||||||
|
.limit(limit)
|
||||||
)
|
)
|
||||||
return (await _session.exec(query)).all()
|
return (await _session.exec(query)).all()
|
||||||
|
|
||||||
@@ -998,18 +992,11 @@ class SourceService(ServiceBase):
|
|||||||
) -> dict[str, JsonValue]:
|
) -> dict[str, JsonValue]:
|
||||||
"""Build a versioned, source-reference-only evidence export."""
|
"""Build a versioned, source-reference-only evidence export."""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
source = await _session.get(Source, source_id)
|
source = await self._read_source(session=_session, source_id=source_id)
|
||||||
if source is None:
|
|
||||||
raise TranscriptionNotFoundError(
|
|
||||||
f"Source with id {source_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Verify the source id and retry.",
|
|
||||||
)
|
|
||||||
attempts = list(await self.list_execution_attempts(source_id=source_id, session=_session))
|
attempts = list(await self.list_execution_attempts(source_id=source_id, session=_session))
|
||||||
artifacts = list(await self.list_processing_artifacts(source_id=source_id, session=_session))
|
artifacts = list(await self.list_processing_artifacts(source_id=source_id, session=_session))
|
||||||
|
|
||||||
for artifact in artifacts:
|
await asyncio.to_thread(self._verify_artifacts_integrity, artifacts)
|
||||||
self._verify_artifact_integrity(artifact)
|
|
||||||
|
|
||||||
artifact_payloads = [
|
artifact_payloads = [
|
||||||
{
|
{
|
||||||
@@ -1098,13 +1085,7 @@ class SourceService(ServiceBase):
|
|||||||
) -> Source:
|
) -> Source:
|
||||||
"""Persist a human revision on a source page."""
|
"""Persist a human revision on a source page."""
|
||||||
async with self._session_scope(session) as _session:
|
async with self._session_scope(session) as _session:
|
||||||
source = await _session.get(Source, source_id)
|
source = await self._read_source(session=_session, source_id=source_id)
|
||||||
if source is None:
|
|
||||||
raise TranscriptionNotFoundError(
|
|
||||||
f"Source with id {source_id} not found",
|
|
||||||
category=ErrorCategory.NOT_FOUND,
|
|
||||||
suggestion="Verify the source id and retry.",
|
|
||||||
)
|
|
||||||
|
|
||||||
source.revised_text = text
|
source.revised_text = text
|
||||||
source.date_revised = datetime.now(UTC)
|
source.date_revised = datetime.now(UTC)
|
||||||
@@ -1303,15 +1284,13 @@ def load_prompt_text(*, prompt_name: str = DEFAULT_PROMPT_FILE, settings: Settin
|
|||||||
|
|
||||||
def source_mime_type(filename: str | Path) -> str:
|
def source_mime_type(filename: str | Path) -> str:
|
||||||
"""Return the canonical MIME type for a supported Source filename."""
|
"""Return the canonical MIME type for a supported Source filename."""
|
||||||
path = Path(filename)
|
mime_type = lookup_source_mime_type(filename)
|
||||||
suffix = path.suffix.lower()
|
|
||||||
mime_type = SOURCE_MIME_TYPES.get(suffix)
|
|
||||||
if mime_type is None:
|
if mime_type is None:
|
||||||
supported = ", ".join(sorted(SOURCE_EXTENSIONS))
|
suffix = Path(filename).suffix.lower()
|
||||||
raise TranscriptionError(
|
raise TranscriptionError(
|
||||||
f"Unsupported Source format: {suffix or '<none>'}",
|
f"Unsupported Source format: {suffix or '<none>'}",
|
||||||
category=ErrorCategory.USER_INPUT,
|
category=ErrorCategory.USER_INPUT,
|
||||||
suggestion=f"Use one of the supported Source formats: {supported}.",
|
suggestion=f"Use one of the supported Source formats: {supported_source_formats()}.",
|
||||||
)
|
)
|
||||||
return mime_type
|
return mime_type
|
||||||
|
|
||||||
|
|||||||
@@ -21,6 +21,8 @@ from ..db.models import Job
|
|||||||
from ..db.models import JobSource
|
from ..db.models import JobSource
|
||||||
from ..db.models import JobSourceStatus
|
from ..db.models import JobSourceStatus
|
||||||
from ..db.models import Source
|
from ..db.models import Source
|
||||||
|
from .media_storage import build_stored_filename
|
||||||
|
from .media_storage import write_media_bytes
|
||||||
from .sources import TranscriptionError
|
from .sources import TranscriptionError
|
||||||
from .sources import build_prompt_execution
|
from .sources import build_prompt_execution
|
||||||
from .sources import validate_source_content
|
from .sources import validate_source_content
|
||||||
@@ -74,7 +76,7 @@ async def create_document_job(
|
|||||||
prompt_execution = build_prompt_execution(settings=runtime_settings)
|
prompt_execution = build_prompt_execution(settings=runtime_settings)
|
||||||
document_id = uuid4()
|
document_id = uuid4()
|
||||||
source_id = uuid4()
|
source_id = uuid4()
|
||||||
stored_path = store_source_file(
|
stored_path = await store_source_file(
|
||||||
filename=filename,
|
filename=filename,
|
||||||
file_bytes=file_bytes,
|
file_bytes=file_bytes,
|
||||||
settings=runtime_settings,
|
settings=runtime_settings,
|
||||||
@@ -134,17 +136,18 @@ async def create_job_for_document(
|
|||||||
stored_sources: list[PendingStoredSource] = []
|
stored_sources: list[PendingStoredSource] = []
|
||||||
for filename, file_bytes in sorted_source_files:
|
for filename, file_bytes in sorted_source_files:
|
||||||
source_id = uuid4()
|
source_id = uuid4()
|
||||||
|
stored_path = await store_source_file(
|
||||||
|
filename=filename,
|
||||||
|
file_bytes=file_bytes,
|
||||||
|
settings=runtime_settings,
|
||||||
|
relative_directory=Path("documents") / str(document_id),
|
||||||
|
filename_stem=str(source_id),
|
||||||
|
)
|
||||||
stored_sources.append(
|
stored_sources.append(
|
||||||
PendingStoredSource(
|
PendingStoredSource(
|
||||||
source_id=source_id,
|
source_id=source_id,
|
||||||
original_filename=filename,
|
original_filename=filename,
|
||||||
stored_path=store_source_file(
|
stored_path=stored_path,
|
||||||
filename=filename,
|
|
||||||
file_bytes=file_bytes,
|
|
||||||
settings=runtime_settings,
|
|
||||||
relative_directory=Path("documents") / str(document_id),
|
|
||||||
filename_stem=str(source_id),
|
|
||||||
),
|
|
||||||
file_hash=_compute_file_hash(file_bytes),
|
file_hash=_compute_file_hash(file_bytes),
|
||||||
file_size_bytes=len(file_bytes),
|
file_size_bytes=len(file_bytes),
|
||||||
)
|
)
|
||||||
@@ -313,7 +316,7 @@ def _compute_file_metadata(file_bytes: bytes) -> tuple[str, int]:
|
|||||||
return _compute_file_hash(file_bytes), len(file_bytes)
|
return _compute_file_hash(file_bytes), len(file_bytes)
|
||||||
|
|
||||||
|
|
||||||
def store_source_file(
|
async def store_source_file(
|
||||||
*,
|
*,
|
||||||
filename: str,
|
filename: str,
|
||||||
file_bytes: bytes,
|
file_bytes: bytes,
|
||||||
@@ -332,45 +335,14 @@ def store_source_file(
|
|||||||
suggestion=exc.suggestion,
|
suggestion=exc.suggestion,
|
||||||
retriable=exc.retriable,
|
retriable=exc.retriable,
|
||||||
) from exc
|
) from exc
|
||||||
return _store_file_bytes(
|
|
||||||
filename=filename,
|
upload_dir = runtime_settings.upload_dir
|
||||||
|
return await write_media_bytes(
|
||||||
|
target_dir=upload_dir if relative_directory is None else upload_dir / relative_directory,
|
||||||
|
stored_name=build_stored_filename(filename=filename, filename_stem=filename_stem),
|
||||||
file_bytes=file_bytes,
|
file_bytes=file_bytes,
|
||||||
settings=runtime_settings,
|
error=SourceStorageError,
|
||||||
relative_directory=relative_directory,
|
failure_message="Failed to persist Source file",
|
||||||
filename_stem=filename_stem,
|
failure_suggestion="Check upload directory permissions and available disk space, then retry.",
|
||||||
|
log_label="Source file",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _store_file_bytes(
|
|
||||||
*,
|
|
||||||
filename: str,
|
|
||||||
file_bytes: bytes,
|
|
||||||
settings: Settings,
|
|
||||||
relative_directory: Path | None = None,
|
|
||||||
filename_stem: str | None = None,
|
|
||||||
) -> Path:
|
|
||||||
upload_dir = settings.upload_dir
|
|
||||||
target_dir = upload_dir if relative_directory is None else upload_dir / relative_directory
|
|
||||||
target_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
stored_name = _build_stored_filename(filename=filename, filename_stem=filename_stem)
|
|
||||||
stored_path = target_dir / stored_name
|
|
||||||
|
|
||||||
try:
|
|
||||||
stored_path.write_bytes(file_bytes)
|
|
||||||
except OSError as exc:
|
|
||||||
raise SourceStorageError(
|
|
||||||
"Failed to persist Source file",
|
|
||||||
category=ErrorCategory.INFRA_PERSISTENT,
|
|
||||||
suggestion="Check upload directory permissions and available disk space, then retry.",
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
logger.info("Stored Source file: %s", stored_path)
|
|
||||||
return stored_path
|
|
||||||
|
|
||||||
|
|
||||||
def _build_stored_filename(*, filename: str, filename_stem: str | None = None) -> str:
|
|
||||||
safe_name = Path(filename).name
|
|
||||||
suffix = Path(safe_name).suffix.lower()
|
|
||||||
stem = filename_stem or str(uuid4())
|
|
||||||
return f"{stem}{suffix}"
|
|
||||||
|
|||||||
@@ -4,11 +4,18 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
from transcription.errors import AppError
|
||||||
|
from transcription.services.media_storage import write_media_bytes
|
||||||
|
|
||||||
HOME_PAGE_DIR = Path(__file__).resolve().parents[3] / "data" / "homepage"
|
HOME_PAGE_DIR = Path(__file__).resolve().parents[3] / "data" / "homepage"
|
||||||
HOME_PAGE_MARKDOWN_PATH = HOME_PAGE_DIR / "homepage.md"
|
HOME_PAGE_MARKDOWN_PATH = HOME_PAGE_DIR / "homepage.md"
|
||||||
SUPPORTED_IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp", ".tif", ".tiff"}
|
SUPPORTED_IMAGE_SUFFIXES = {".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp", ".tif", ".tiff"}
|
||||||
|
|
||||||
|
|
||||||
|
class HomepageStorageError(AppError):
|
||||||
|
"""Raised when homepage media cannot be persisted."""
|
||||||
|
|
||||||
|
|
||||||
def ensure_homepage_storage() -> None:
|
def ensure_homepage_storage() -> None:
|
||||||
"""Create the homepage storage directory when needed."""
|
"""Create the homepage storage directory when needed."""
|
||||||
HOME_PAGE_DIR.mkdir(parents=True, exist_ok=True)
|
HOME_PAGE_DIR.mkdir(parents=True, exist_ok=True)
|
||||||
@@ -28,18 +35,22 @@ def save_homepage_markdown(markdown_text: str) -> None:
|
|||||||
HOME_PAGE_MARKDOWN_PATH.write_text(markdown_text, encoding="utf-8")
|
HOME_PAGE_MARKDOWN_PATH.write_text(markdown_text, encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
def store_homepage_image(*, filename: str, file_bytes: bytes) -> Path:
|
async def store_homepage_image(*, filename: str, file_bytes: bytes) -> Path:
|
||||||
"""Persist an uploaded homepage image in the shared homepage folder."""
|
"""Persist an uploaded homepage image in the shared homepage folder."""
|
||||||
ensure_homepage_storage()
|
|
||||||
|
|
||||||
safe_name = Path(filename).name
|
safe_name = Path(filename).name
|
||||||
if not safe_name:
|
if not safe_name:
|
||||||
msg = "Homepage image filename is required"
|
msg = "Homepage image filename is required"
|
||||||
raise ValueError(msg)
|
raise ValueError(msg)
|
||||||
|
|
||||||
stored_path = HOME_PAGE_DIR / safe_name
|
return await write_media_bytes(
|
||||||
stored_path.write_bytes(file_bytes)
|
target_dir=HOME_PAGE_DIR,
|
||||||
return stored_path
|
stored_name=safe_name,
|
||||||
|
file_bytes=file_bytes,
|
||||||
|
error=HomepageStorageError,
|
||||||
|
failure_message="Failed to persist homepage image",
|
||||||
|
failure_suggestion="Check homepage directory permissions and available disk space, then retry.",
|
||||||
|
log_label="homepage image",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def list_homepage_images() -> list[Path]:
|
def list_homepage_images() -> list[Path]:
|
||||||
|
|||||||
@@ -84,7 +84,7 @@ def register_page() -> None:
|
|||||||
|
|
||||||
async def on_upload(event) -> None:
|
async def on_upload(event) -> None:
|
||||||
payload = await event.file.read()
|
payload = await event.file.read()
|
||||||
preview_image[0] = store_homepage_image(filename=event.file.name, file_bytes=payload)
|
preview_image[0] = await store_homepage_image(filename=event.file.name, file_bytes=payload)
|
||||||
ui.notify(f"Uploaded {event.file.name}", type="positive")
|
ui.notify(f"Uploaded {event.file.name}", type="positive")
|
||||||
render_image_panel.refresh()
|
render_image_panel.refresh()
|
||||||
|
|
||||||
|
|||||||
@@ -492,7 +492,7 @@ def _bind_portrait_file_picker(portrait_path_input: ui.input, *, settings: Setti
|
|||||||
async def on_portrait_selected(event) -> None:
|
async def on_portrait_selected(event) -> None:
|
||||||
payload = await event.file.read()
|
payload = await event.file.read()
|
||||||
try:
|
try:
|
||||||
stored_path = store_person_portrait(
|
stored_path = await store_person_portrait(
|
||||||
person_id=person_id,
|
person_id=person_id,
|
||||||
filename=event.file.name,
|
filename=event.file.name,
|
||||||
file_bytes=payload,
|
file_bytes=payload,
|
||||||
|
|||||||
@@ -114,11 +114,12 @@ async def test_create_document_job_stores_source_under_document_id_directory(asy
|
|||||||
assert created_job.prompt_name == "transcribe_document.md"
|
assert created_job.prompt_name == "transcribe_document.md"
|
||||||
|
|
||||||
|
|
||||||
def test_store_person_portrait_stores_file_under_person_id_directory(tmp_path):
|
@pytest.mark.asyncio
|
||||||
|
async def test_store_person_portrait_stores_file_under_person_id_directory(tmp_path):
|
||||||
settings = Settings(openrouter_api_key="test-key", upload_dir=tmp_path)
|
settings = Settings(openrouter_api_key="test-key", upload_dir=tmp_path)
|
||||||
person_id = uuid4()
|
person_id = uuid4()
|
||||||
|
|
||||||
stored_path = store_person_portrait(
|
stored_path = await store_person_portrait(
|
||||||
person_id=person_id,
|
person_id=person_id,
|
||||||
filename="portrait.png",
|
filename="portrait.png",
|
||||||
file_bytes=b"portrait-bytes",
|
file_bytes=b"portrait-bytes",
|
||||||
@@ -144,8 +145,9 @@ def test_source_mime_type_uses_canonical_source_policy(filename, expected_mime_t
|
|||||||
assert source_mime_type(filename) == expected_mime_type
|
assert source_mime_type(filename) == expected_mime_type
|
||||||
|
|
||||||
|
|
||||||
def test_source_storage_rejects_unsupported_format(tmp_path):
|
@pytest.mark.asyncio
|
||||||
|
async def test_source_storage_rejects_unsupported_format(tmp_path):
|
||||||
settings = Settings(openrouter_api_key="test-key", upload_dir=tmp_path)
|
settings = Settings(openrouter_api_key="test-key", upload_dir=tmp_path)
|
||||||
|
|
||||||
with pytest.raises(SourceStorageError):
|
with pytest.raises(SourceStorageError):
|
||||||
store_source_file(filename="page.txt", file_bytes=b"text", settings=settings)
|
await store_source_file(filename="page.txt", file_bytes=b"text", settings=settings)
|
||||||
|
|||||||
@@ -3,6 +3,7 @@
|
|||||||
from uuid import uuid4
|
from uuid import uuid4
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
from sqlalchemy import event
|
||||||
|
|
||||||
from transcription.config import Settings
|
from transcription.config import Settings
|
||||||
from transcription.db.models import Document
|
from transcription.db.models import Document
|
||||||
@@ -232,3 +233,112 @@ class TestSourceServiceRevisionUpsert:
|
|||||||
|
|
||||||
with pytest.raises(SourceDeleteBlockedError):
|
with pytest.raises(SourceDeleteBlockedError):
|
||||||
await transcriptions.delete_unlinked_source(source_id=source.id)
|
await transcriptions.delete_unlinked_source(source_id=source.id)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.integration
|
||||||
|
class TestSourceServiceQueryShape:
|
||||||
|
"""LOW-08: reads must filter and bound in SQL, not in Python."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_list_sources_detail_filters_job_id_with_a_join(self, default_session_factory):
|
||||||
|
documents = DocumentService(session_factory=default_session_factory)
|
||||||
|
jobs = JobService(session_factory=default_session_factory)
|
||||||
|
transcriptions = SourceService(session_factory=default_session_factory)
|
||||||
|
|
||||||
|
document = Document(id=uuid4(), name="join-filter")
|
||||||
|
await documents.create_document(document=document)
|
||||||
|
job = Job(document_id=document.id, status=JobStatus.QUEUED)
|
||||||
|
other_job = Job(document_id=document.id, status=JobStatus.QUEUED)
|
||||||
|
await jobs.create_job(job=job)
|
||||||
|
await jobs.create_job(job=other_job)
|
||||||
|
|
||||||
|
linked = Source(
|
||||||
|
document_id=document.id,
|
||||||
|
page_number=1,
|
||||||
|
upload_name="linked.jpg",
|
||||||
|
filename="linked.jpg",
|
||||||
|
file_path="uploads/linked.jpg",
|
||||||
|
file_hash="7" * 64,
|
||||||
|
file_size_bytes=1,
|
||||||
|
)
|
||||||
|
unlinked = Source(
|
||||||
|
document_id=document.id,
|
||||||
|
page_number=2,
|
||||||
|
upload_name="unlinked.jpg",
|
||||||
|
filename="unlinked.jpg",
|
||||||
|
file_path="uploads/unlinked.jpg",
|
||||||
|
file_hash="8" * 64,
|
||||||
|
file_size_bytes=1,
|
||||||
|
)
|
||||||
|
async with transcriptions._session_scope() as session:
|
||||||
|
session.add_all((linked, unlinked))
|
||||||
|
await session.flush()
|
||||||
|
session.add(JobSource(job_id=job.id, source_id=linked.id, status=JobSourceStatus.PENDING))
|
||||||
|
session.add(JobSource(job_id=other_job.id, source_id=unlinked.id, status=JobSourceStatus.PENDING))
|
||||||
|
await session.commit()
|
||||||
|
|
||||||
|
statements: list[str] = []
|
||||||
|
|
||||||
|
async with transcriptions._session_scope() as session:
|
||||||
|
bind = session.get_bind()
|
||||||
|
|
||||||
|
def capture(_conn, _cursor, statement, *_rest):
|
||||||
|
statements.append(statement)
|
||||||
|
|
||||||
|
event.listen(bind, "before_cursor_execute", capture)
|
||||||
|
try:
|
||||||
|
sources = await transcriptions.list_sources_detail(job_id=job.id, session=session)
|
||||||
|
finally:
|
||||||
|
event.remove(bind, "before_cursor_execute", capture)
|
||||||
|
|
||||||
|
assert [source.id for source in sources] == [linked.id]
|
||||||
|
|
||||||
|
primary = next(item for item in statements if item.lstrip().upper().startswith("SELECT"))
|
||||||
|
assert "JOIN" in primary.upper()
|
||||||
|
assert "JOBSOURCE" in primary.upper().replace("_", "")
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_read_source_navigation_does_not_scan_every_sibling(self, default_session_factory):
|
||||||
|
documents = DocumentService(session_factory=default_session_factory)
|
||||||
|
transcriptions = SourceService(session_factory=default_session_factory)
|
||||||
|
|
||||||
|
document = Document(id=uuid4(), name="navigation-bounds")
|
||||||
|
await documents.create_document(document=document)
|
||||||
|
|
||||||
|
pages = [
|
||||||
|
Source(
|
||||||
|
document_id=document.id,
|
||||||
|
page_number=page_number,
|
||||||
|
upload_name=f"page-{page_number}.jpg",
|
||||||
|
filename=f"page-{page_number}.jpg",
|
||||||
|
file_path=f"uploads/page-{page_number}.jpg",
|
||||||
|
file_hash=str(page_number) * 64,
|
||||||
|
file_size_bytes=1,
|
||||||
|
)
|
||||||
|
for page_number in range(1, 5)
|
||||||
|
]
|
||||||
|
async with transcriptions._session_scope() as session:
|
||||||
|
session.add_all(pages)
|
||||||
|
await session.commit()
|
||||||
|
for page in pages:
|
||||||
|
await session.refresh(page)
|
||||||
|
|
||||||
|
statements: list[str] = []
|
||||||
|
|
||||||
|
async with transcriptions._session_scope() as session:
|
||||||
|
bind = session.get_bind()
|
||||||
|
|
||||||
|
def capture(_conn, _cursor, statement, *_rest):
|
||||||
|
statements.append(statement)
|
||||||
|
|
||||||
|
event.listen(bind, "before_cursor_execute", capture)
|
||||||
|
try:
|
||||||
|
navigation = await transcriptions.read_source_navigation(pages[1].id, session=session)
|
||||||
|
finally:
|
||||||
|
event.remove(bind, "before_cursor_execute", capture)
|
||||||
|
|
||||||
|
assert navigation.previous_id == pages[0].id
|
||||||
|
assert navigation.next_id == pages[2].id
|
||||||
|
|
||||||
|
adjacency = [item for item in statements if "LIMIT" in item.upper()]
|
||||||
|
assert len(adjacency) == 2, statements
|
||||||
|
|||||||
@@ -0,0 +1,65 @@
|
|||||||
|
"""Structural rules for the services package.
|
||||||
|
|
||||||
|
`.github/instructions/services.instructions.md:13` requires that service classes
|
||||||
|
stay independent of one another. Shared behavior belongs in a neutral module
|
||||||
|
(`base.py`, `registry.py`, `source_media.py`, `media_storage.py`), and any
|
||||||
|
operation spanning two services belongs in an orchestration module.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import ast
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
SERVICES_DIR = Path(__file__).resolve().parents[1] / "src" / "transcription" / "services"
|
||||||
|
|
||||||
|
# Modules that intentionally compose several services rather than owning one table.
|
||||||
|
ORCHESTRATION_MODULES = frozenset({"store", "workflows", "__init__"})
|
||||||
|
|
||||||
|
|
||||||
|
def _module_paths() -> list[Path]:
|
||||||
|
return sorted(SERVICES_DIR.glob("*.py"))
|
||||||
|
|
||||||
|
|
||||||
|
def _defines_service_class(tree: ast.Module) -> bool:
|
||||||
|
return any(
|
||||||
|
isinstance(node, ast.ClassDef) and node.name.endswith("Service") and node.name != "RegistryService"
|
||||||
|
for node in tree.body
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _service_modules() -> dict[str, ast.Module]:
|
||||||
|
modules: dict[str, ast.Module] = {}
|
||||||
|
for path in _module_paths():
|
||||||
|
if path.stem in ORCHESTRATION_MODULES:
|
||||||
|
continue
|
||||||
|
tree = ast.parse(path.read_text(encoding="utf-8"))
|
||||||
|
if _defines_service_class(tree):
|
||||||
|
modules[path.stem] = tree
|
||||||
|
return modules
|
||||||
|
|
||||||
|
|
||||||
|
def _imported_sibling_modules(tree: ast.Module) -> set[str]:
|
||||||
|
imported: set[str] = set()
|
||||||
|
for node in ast.walk(tree):
|
||||||
|
if isinstance(node, ast.ImportFrom) and node.level == 1 and node.module:
|
||||||
|
imported.add(node.module.split(".")[0])
|
||||||
|
elif isinstance(node, ast.ImportFrom) and node.level == 0 and node.module:
|
||||||
|
parts = node.module.split(".")
|
||||||
|
if parts[:2] == ["transcription", "services"] and len(parts) > 2:
|
||||||
|
imported.add(parts[2])
|
||||||
|
return imported
|
||||||
|
|
||||||
|
|
||||||
|
def test_service_modules_are_discovered():
|
||||||
|
"""Guard the guard: the rule below is meaningless if nothing is scanned."""
|
||||||
|
assert set(_service_modules()) >= {"documents", "jobs", "people", "sources"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_service_module_imports_another_service_module():
|
||||||
|
"""MED-14: a service module must not depend on a sibling service module."""
|
||||||
|
modules = _service_modules()
|
||||||
|
violations = {
|
||||||
|
name: sorted(_imported_sibling_modules(tree) & set(modules) - {name}) for name, tree in modules.items()
|
||||||
|
}
|
||||||
|
assert {name: found for name, found in violations.items() if found} == {}
|
||||||
Reference in New Issue
Block a user