V4.11 Added tags + lots of little changes to the UI
Quality Gate / gate (push) Failing after 12s

This commit is contained in:
Jim Lancaster
2026-08-22 18:32:52 -05:00
parent 63c21d4a14
commit 0d554c0648
36 changed files with 932 additions and 116 deletions
+29 -22
View File
@@ -79,29 +79,36 @@ Small, low-risk structural-parity fixes across Person/Job/Document detail pages:
--- ---
## V4.11 — Bigger UI Features ## V4.11 — Approved Scope
- **Tags** (supersedes "collections"): many-to-many tagging for documents, - **Tags** (supersedes "collections"): many-to-many tagging for Documents with
reusing the Settings registry pattern (autocomplete against existing tags, Settings-style management (same pattern as Document Types and Person Roles),
managed like Document Types/Person Roles). Since you want dedicated UI to autocomplete-capable assignment, and a dedicated **Tags** entry point for
browse/filter by tag, include a tag-filter view (e.g. a "Tags" entry point browse/filter-by-tag workflows.
showing documents grouped/filterable by tag) as part of this version, not - **Source Detail simplification**: remove the separate **Transcription Text**
deferred — that's the UI surface that makes tags actually replace collections card; show Source image + Editable Revision + Source/SourceJob metadata in a
day-to-day, not just a data field. 3-column top layout, then keep Candidate Machine Transcriptions below the
- **UI theme selection**: kept simple, per your confirmation — 24 curated image/revision area.
themes (e.g. Archival/Sepia, Light, Dark, High-contrast), a single stored - **Integrity reconciliation checks in tests**:
preference, swapped via existing CSS variables/Tailwind tokens in - document folder count under `UPLOAD_DIR/documents` must equal `document`
`theme.py`. No open-ended theme builder. row count.
- **Edit Transcription sub-page**: source image + transcription text + - source file count under each `UPLOAD_DIR/documents/{document_id}` folder
editable revision side-by-side, keeping the existing Save Revision/Reset must equal `source` row count for that Document.
buttons. You noted the exact interaction/layout needs to be clarified when - failures should include actionable mismatch details (missing row/folder or
this is scoped — treat this as a design-first item: confirm the two/three file/source mapping).
column breakpoint behavior and what happens on narrow windows before - **UI table updates**:
building. - Archival Documents: remove **Archive Ref**, add **# Sources**.
- **Source detail page reflow** (move Candidate Machine Transcriptions to the - Archival Entities: People: remove **Display Name** and **Maiden Name**
bottom of the image column; align Source Metadata's top edge with columns, add **FamilySearch ID**.
Transcription Text): do this *after* the Edit Transcription sub-page above, - Transcription Pipeline Jobs: add **# Sources**.
since that page will reuse/rearrange the same boxes — avoids reflowing twice. - **Create Processing Job page**: Provider and Model must be selectable for new
job creation.
Deferred out of this release:
- UI theme selection.
- Settings-based `.env` editing and runtime controls.
- Person table structural redesign (removing/splitting name fields).
--- ---
+32 -4
View File
@@ -6,10 +6,11 @@ This document is the field-accurate Version 4 schema contract aligned to `src/tr
- `src/transcription/db/models.py:60-78` (status and purpose enums) - `src/transcription/db/models.py:60-78` (status and purpose enums)
- `src/transcription/db/models.py:80-120` (`DocumentType`, `PersonRole`) - `src/transcription/db/models.py:80-120` (`DocumentType`, `PersonRole`)
- `src/transcription/db/models.py:122-207` (`Document`, `Person`, `DocumentPerson`) - `src/transcription/db/models.py:122-148` (`Tag`, `Document`)
- `src/transcription/db/models.py:208-255` (`Job`) - `src/transcription/db/models.py:149-255` (`Person`, `DocumentPerson`, `DocumentTag`)
- `src/transcription/db/models.py:273-386` (`Source`, `JobSource`) - `src/transcription/db/models.py:256-303` (`Job`)
- `src/transcription/db/models.py:387-445` (`ExecutionAttempt`) - `src/transcription/db/models.py:304-417` (`Source`, `JobSource`)
- `src/transcription/db/models.py:418-476` (`ExecutionAttempt`)
## Entity Relationship Overview ## Entity Relationship Overview
@@ -19,8 +20,10 @@ erDiagram
Document ||--o{ Job : has Document ||--o{ Job : has
Document ||--o{ Source : has Document ||--o{ Source : has
Document ||--o{ DocumentPerson : links Document ||--o{ DocumentPerson : links
Document ||--o{ DocumentTag : tagged
Person ||--o{ DocumentPerson : links Person ||--o{ DocumentPerson : links
PersonRole ||--o{ DocumentPerson : labels PersonRole ||--o{ DocumentPerson : labels
Tag ||--o{ DocumentTag : labels
Job ||--o{ JobSource : includes Job ||--o{ JobSource : includes
Source ||--o{ JobSource : participates Source ||--o{ JobSource : participates
JobSource ||--o{ ExecutionAttempt : attempts JobSource ||--o{ ExecutionAttempt : attempts
@@ -74,6 +77,18 @@ erDiagram
| `created_at` | `datetime` | default now | | `created_at` | `datetime` | default now |
| `updated_at` | `datetime` | default now, onupdate | | `updated_at` | `datetime` | default now, onupdate |
### `Tag`
| Field | Type | Notes |
| :--- | :--- | :--- |
| `id` | `UUID` | PK |
| `semantic_key` | `str \| None` | nullable unique, indexed |
| `label` | `str` | required |
| `normalized_label` | `str` | unique, indexed |
| `is_active` | `bool` | default `True` |
| `created_at` | `datetime` | default now |
| `updated_at` | `datetime` | default now, onupdate |
### `Document` ### `Document`
| Field | Type | Notes | | Field | Type | Notes |
@@ -124,6 +139,19 @@ erDiagram
Constraint: Constraint:
- `UniqueConstraint(document_id, person_id)` named `uq_document_person` - `UniqueConstraint(document_id, person_id)` named `uq_document_person`
### `DocumentTag`
| Field | Type | Notes |
| :--- | :--- | :--- |
| `id` | `UUID` | PK |
| `document_id` | `UUID` | FK -> `document.id`, indexed |
| `tag_id` | `UUID` | FK -> `tag.id`, indexed |
| `created_at` | `datetime` | default now |
| `updated_at` | `datetime` | default now, onupdate |
Constraint:
- `UniqueConstraint(document_id, tag_id)` named `uq_document_tag`
### `Job` ### `Job`
| Field | Type | Notes | | Field | Type | Notes |
+5 -2
View File
@@ -22,9 +22,10 @@ Documents manages the archival record for each historical artifact independently
- The title is **Archival Documents**. - The title is **Archival Documents**.
- **Create new document** opens the create route. - **Create new document** opens the create route.
- The table defaults to Document Title order and supports search and column sorting. - The table defaults to Document Title order and supports search and column sorting.
- Columns are Document Title, Type, Author, Document Date, and Archive Ref. - Columns are Document Title, Type, Author, Document Date, and # Sources.
- Document Title is left-aligned; the remaining columns are centered. - Document Title is left-aligned; the remaining columns are centered.
- Author lists all linked people in the `author` role. - Author lists all linked people in the `author` role.
- # Sources reflects the count of linked Source rows for each Document.
- Date display prefers exact date, then approximate date, then `Unknown`. - Date display prefers exact date, then approximate date, then `Unknown`.
- Selecting a row opens Document Detail. - Selecting a row opens Document Detail.
- No records displays `No documents found in repository.` - No records displays `No documents found in repository.`
@@ -43,12 +44,14 @@ Optional:
- Document location. - Document location.
- Archive identifier. - Archive identifier.
- Notes. - Notes.
- Tags.
- Linked People, with exactly one Person Role per linked Person. - Linked People, with exactly one Person Role per linked Person.
Rules: Rules:
- Exact date must parse as `YYYY-MM-DD`; browser presentation may follow locale. - Exact date must parse as `YYYY-MM-DD`; browser presentation may follow locale.
- Existing people appear with disambiguating labels. - Existing people appear with disambiguating labels.
- Tag assignment supports selecting existing tags and adding new labels inline.
- **Create new person** opens Person creation. - **Create new person** opens Person creation.
- `person_id` may preselect that Person in the author role on Document creation. - `person_id` may preselect that Person in the author role on Document creation.
- An invalid requested Person produces a warning rather than a broken form. - An invalid requested Person produces a warning rather than a broken form.
@@ -65,7 +68,7 @@ Rules:
- The heading shows name, type, and internal ID. - The heading shows name, type, and internal ID.
- The first Source, when present, appears in the dark-room viewer. - The first Source, when present, appears in the dark-room viewer.
- Archival Metadata shows authors, Document Type, compact Document date, location, and archive identifier. Notes appear in a separate archival-notes block within the same card. - Archival Metadata shows authors, Document Type, tags, compact Document date, location, and archive identifier. Notes appear in a separate archival-notes block within the same card.
- System Logistics shows created and updated timestamps. - System Logistics shows created and updated timestamps.
- Related People are grouped by role and link to Person Detail. - Related People are grouped by role and link to Person Detail.
- **Sources & Pipeline Jobs** shows counts and actions for filtered Sources, Document Jobs, and adding a Job. - **Sources & Pipeline Jobs** shows counts and actions for filtered Sources, Document Jobs, and adding a Job.
+2 -2
View File
@@ -19,7 +19,7 @@ Jobs manages transcription processing runs. A Job belongs to one Document, links
- The title is **Transcription Pipeline Jobs**. - The title is **Transcription Pipeline Jobs**.
- **Create job** opens Job creation and **Refresh** reloads the table. - **Create job** opens Job creation and **Refresh** reloads the table.
- Columns are Job ID, Status, Document Name, Retries, and Updated. - Columns are Job ID, Status, Document Name, # Sources, Retries, and Updated.
- Updated is the primary date/sort field. - Updated is the primary date/sort field.
- Search covers Job ID, document name, and status. - Search covers Job ID, document name, and status.
- Status is displayed as a semantic status chip. - Status is displayed as a semantic status chip.
@@ -31,7 +31,7 @@ Jobs manages transcription processing runs. A Job belongs to one Document, links
- A Target Document and at least one source file are required. - A Target Document and at least one source file are required.
- `document_id` may preselect a Target Document. - `document_id` may preselect a Target Document.
- If no Documents exist, the page explains the prerequisite and links to Document creation with a return path. - If no Documents exist, the page explains the prerequisite and links to Document creation with a return path.
- Provider and Model are optional request overrides. - Provider and Model are selectable when creating a new Job.
- Upload accepts JPEG, PNG, TIFF, and PDF files and supports multiple/folder selection. - Upload accepts JPEG, PNG, TIFF, and PDF files and supports multiple/folder selection.
- The visible upload queue is sorted alphabetically by original filename. - The visible upload queue is sorted alphabetically by original filename.
- Files can be removed individually or cleared before submission. - Files can be removed individually or cleared before submission.
+3 -2
View File
@@ -19,8 +19,9 @@ People manages reusable historical-person records. A Person may appear in many D
- The title is **Archival Entities: People**. - The title is **Archival Entities: People**.
- **Create new person** opens the create route. - **Create new person** opens the create route.
- The table defaults to Full Name order and supports search and column sorting. - The table defaults to Full Name order and supports search and column sorting.
- Columns are Full Name, Display Name, Maiden Name, Birth Date, and Death Date. - Columns are Full Name, FamilySearch ID, Birth Date, Death Date, and # Documents.
- Full Name is left-aligned; Display Name, Maiden Name, and date columns are centered. - Full Name is left-aligned; FamilySearch ID, date columns, and # Documents are centered.
- # Documents reflects how many linked Documents each Person is connected to.
- Birth and death values independently prefer exact date, then approximate date, then `Unknown`. - Birth and death values independently prefer exact date, then approximate date, then `Unknown`.
- Selecting a row opens Person Detail. - Selecting a row opens Person Detail.
- No records displays `No person records found in repository.` - No records displays `No person records found in repository.`
+4 -3
View File
@@ -8,7 +8,7 @@ Settings manages installation-local registries and editable text assets from one
| Route | Purpose | | Route | Purpose |
| --- | --- | | --- | --- |
| `/settings` | Manage Document Types, Person Roles, Prompts, and Home Page Text. | | `/settings` | Manage Document Types, Person Roles, Tags, Prompts, and Home Page Text. |
## Behavior ## Behavior
@@ -16,15 +16,16 @@ Settings manages installation-local registries and editable text assets from one
- Configuration surfaces are grouped as tabs: - Configuration surfaces are grouped as tabs:
- **Document Types** - **Document Types**
- **Person Roles** - **Person Roles**
- **Tags**
- **Prompts** - **Prompts**
- **Home Page Text** - **Home Page Text**
- Document Types and Person Roles support Add/Edit/Delete with existing guardrails. - Document Types, Person Roles, and Tags support Add/Edit/Delete with existing guardrails.
- Prompts exposes only `transcribe_document.md` for editing and restore-from-backup. - Prompts exposes only `transcribe_document.md` for editing and restore-from-backup.
- Home Page Text edits the same Markdown content rendered on `/homepage`. - Home Page Text edits the same Markdown content rendered on `/homepage`.
## Acceptance Checklist ## Acceptance Checklist
- `/ui/settings` renders all four tabs. - `/ui/settings` renders all five tabs.
- Registry and prompt workflows keep existing validation and error handling. - Registry and prompt workflows keep existing validation and error handling.
- Saving Home Page Text persists content for the homepage view. - Saving Home Page Text persists content for the homepage view.
+3 -4
View File
@@ -34,13 +34,12 @@ The list accepts optional `document_id` and `job_id` query parameters. Document
- **Delete Source** opens the guarded delete route. - **Delete Source** opens the guarded delete route.
- Previous and Next navigate only among Sources belonging to the same Document in page order; unavailable boundary actions are disabled. - Previous and Next navigate only among Sources belonging to the same Document in page order; unavailable boundary actions are disabled.
- The media viewer resolves the stored Source path through the configured upload root. - The media viewer resolves the stored Source path through the configured upload root.
- Transcription Text is read-only and displays the preferred machine projection, with a legacy latest-JobSource - The top layout is three columns: Source image, Editable Revision, and Source/SourceJob metadata.
fallback only when no Source projection exists. - Editable Revision is seeded from an existing revision or the preferred machine transcription.
- Editable Revision is seeded from an existing revision or the machine transcription.
- Source Metadata shows upload name, stored filename, page number, Document Name, Document ID, and stored path. Source ID appears in the page-header subtitle. - Source Metadata shows upload name, stored filename, page number, Document Name, Document ID, and stored path. Source ID appears in the page-header subtitle.
- SourceJob Metadata shows latest status, Job ID, execution time, provider, model, prompt, and failure detail. - SourceJob Metadata shows latest status, Job ID, execution time, provider, model, prompt, and failure detail.
- Revision Logistics shows revised state, last-revised time, and upload time. - Revision Logistics shows revised state, last-revised time, and upload time.
- Candidate Machine Transcriptions remains compact until a candidate is expanded, then compares it with the preferred - Candidate Machine Transcriptions appears below the image/revision area, remains compact until expanded, then compares it with the preferred
machine result and requires confirmation before **Use this transcription**. machine result and requires confirmation before **Use this transcription**.
- Candidate promotion does not alter a human revision. Empty states distinguish no machine result from no candidates. - Candidate promotion does not alter a human revision. Empty states distinguish no machine result from no candidates.
- An orientation-normalized artifact appears in evidence only when recognized metadata required a physical rotation. - An orientation-normalized artifact appears in evidence only when recognized metadata required a physical rotation.
+31
View File
@@ -0,0 +1,31 @@
# Tags Page Contract
## Purpose
Tags provides a dedicated browse/filter entry point for document tagging workflows.
## Route
| Route | Purpose |
| --- | --- |
| `/tags` | Browse Documents grouped by Tag and filter to one Tag. |
## Behavior
- The page title is **Tags**.
- When no tags exist, the page shows `No tags are configured yet.`
- A Tag filter select allows narrowing to one tag.
- Each rendered group header includes the tag label and document count.
- Document names are clickable and open Document Detail.
## Acceptance Checklist
- `/ui/tags` renders successfully from the main navigation.
- Group counts match the number of linked Documents per Tag.
- Filtering hides non-matching tag groups.
## Implementation Anchors
- `src/transcription/ui/pages/tags_page.py`
- `src/transcription/services/documents.py`
- `tests/ui/test_tags_page.py`
+2
View File
@@ -29,9 +29,11 @@ from transcription.db.engine import get_database_url
EXPORT_TABLE_ORDER = ( EXPORT_TABLE_ORDER = (
"document_type", "document_type",
"person_role", "person_role",
"tag",
"document", "document",
"person", "person",
"document_person", "document_person",
"document_tag",
"job", "job",
"source", "source",
"job_source", "job_source",
+52
View File
@@ -119,6 +119,28 @@ class PersonRole(SQLModel, table=True):
) )
class Tag(SQLModel, table=True):
"""Registry of labels that can be attached to Documents."""
__tablename__ = "tag"
id: UUID = Field(default_factory=uuid4, primary_key=True)
semantic_key: str | None = Field(default=None, index=True, unique=True)
label: str
normalized_label: str = Field(index=True, unique=True)
is_active: bool = True
created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
updated_at: datetime = Field(
default_factory=lambda: datetime.now(UTC),
sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
)
document_tags: list["DocumentTag"] = Relationship(
back_populates="tag_ref",
sa_relationship_kwargs={"lazy": "raise"},
)
class Document(SQLModel, table=True): class Document(SQLModel, table=True):
"""An historical document.""" """An historical document."""
@@ -141,6 +163,10 @@ class Document(SQLModel, table=True):
document_people: list["DocumentPerson"] = Relationship( document_people: list["DocumentPerson"] = Relationship(
back_populates="document", sa_relationship_kwargs={"lazy": "raise"} back_populates="document", sa_relationship_kwargs={"lazy": "raise"}
) )
document_tags: list["DocumentTag"] = Relationship(
back_populates="document",
sa_relationship_kwargs={"lazy": "raise"},
)
document_type_ref: Optional["DocumentType"] = Relationship( document_type_ref: Optional["DocumentType"] = Relationship(
back_populates="documents", sa_relationship_kwargs={"lazy": "raise"} back_populates="documents", sa_relationship_kwargs={"lazy": "raise"}
) )
@@ -205,6 +231,32 @@ class DocumentPerson(SQLModel, table=True):
) )
class DocumentTag(SQLModel, table=True):
"""Associates Documents with Tags."""
__tablename__ = "document_tag"
id: UUID = Field(default_factory=uuid4, primary_key=True)
document_id: UUID = Field(foreign_key="document.id", index=True)
tag_id: UUID = Field(foreign_key="tag.id", index=True)
created_at: datetime = Field(default_factory=lambda: datetime.now(UTC))
updated_at: datetime = Field(
default_factory=lambda: datetime.now(UTC),
sa_column_kwargs={"onupdate": lambda: datetime.now(UTC)},
)
__table_args__ = (UniqueConstraint("document_id", "tag_id", name="uq_document_tag"),)
document: Optional["Document"] = Relationship(
back_populates="document_tags",
sa_relationship_kwargs={"lazy": "raise"},
)
tag_ref: Optional["Tag"] = Relationship(
back_populates="document_tags",
sa_relationship_kwargs={"lazy": "raise"},
)
class Job(SQLModel, table=True): class Job(SQLModel, table=True):
"""A transcription job tied to a single document.""" """A transcription job tied to a single document."""
+164
View File
@@ -20,7 +20,9 @@ from ..db.loading import orm_attribute
from ..db.loading import selectinload from ..db.loading import selectinload
from ..db.models import Document from ..db.models import Document
from ..db.models import DocumentPerson from ..db.models import DocumentPerson
from ..db.models import DocumentTag
from ..db.models import DocumentType from ..db.models import DocumentType
from ..db.models import Tag
from ..db.registries import AUTHOR_ROLE_SEMANTIC_KEY from ..db.registries import AUTHOR_ROLE_SEMANTIC_KEY
from ..errors import AppError from ..errors import AppError
from ..errors import ErrorCategory from ..errors import ErrorCategory
@@ -53,6 +55,10 @@ class DocumentTypeError(DocumentError):
"""Raised when Document Type maintenance fails.""" """Raised when Document Type maintenance fails."""
class TagError(DocumentError):
"""Raised when Tag maintenance fails."""
class DocumentTypeRegistry(RegistryService[DocumentType]): class DocumentTypeRegistry(RegistryService[DocumentType]):
"""Document Type registry maintenance.""" """Document Type registry maintenance."""
@@ -75,6 +81,28 @@ class DocumentTypeRegistry(RegistryService[DocumentType]):
type DocumentTypeSummary = RegistrySummary type DocumentTypeSummary = RegistrySummary
class TagRegistry(RegistryService[Tag]):
"""Tag registry maintenance."""
model = Tag
error = TagError
noun = "Tag"
short_noun = "tag"
referenced_retainer = "historical Documents"
def reference_model(self) -> type[SQLModel]:
return DocumentTag
def reference_id_column(self) -> Any:
return col(DocumentTag.id)
def reference_key_column(self) -> Any:
return col(DocumentTag.tag_id)
type TagSummary = RegistrySummary
@dataclass(frozen=True, slots=True) @dataclass(frozen=True, slots=True)
class DocumentPrintSource: class DocumentPrintSource:
id: UUID id: UUID
@@ -119,6 +147,7 @@ class DocumentService(ServiceBase):
) -> None: ) -> None:
super().__init__(session_factory, settings) super().__init__(session_factory, settings)
self._document_types = DocumentTypeRegistry(self.session_factory, self.settings) self._document_types = DocumentTypeRegistry(self.session_factory, self.settings)
self._tags = TagRegistry(self.session_factory, self.settings)
async def _validate_document_type(self, *, session: AsyncSession, document: Document) -> None: async def _validate_document_type(self, *, session: AsyncSession, document: Document) -> None:
"""Validate the UUID-backed Document Type reference.""" """Validate the UUID-backed Document Type reference."""
@@ -272,6 +301,8 @@ class DocumentService(ServiceBase):
selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.person)), selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.person)),
selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)), selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
selectinload(Document.document_type_ref), selectinload(Document.document_type_ref),
selectinload(Document.document_tags).selectinload(orm_attribute(DocumentTag.tag_ref)),
selectinload(Document.sources),
) )
result = await _session.exec(query) result = await _session.exec(query)
return result.all() return result.all()
@@ -287,6 +318,7 @@ class DocumentService(ServiceBase):
selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.person)), selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.person)),
selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)), selectinload(Document.document_people).selectinload(orm_attribute(DocumentPerson.role_ref)),
selectinload(Document.document_type_ref), selectinload(Document.document_type_ref),
selectinload(Document.document_tags).selectinload(orm_attribute(DocumentTag.tag_ref)),
) )
.where(Document.id == document_id) .where(Document.id == document_id)
.execution_options(populate_existing=True) .execution_options(populate_existing=True)
@@ -362,6 +394,15 @@ class DocumentService(ServiceBase):
"""List configured document types.""" """List configured document types."""
return await self._document_types.list_entries(active_only=active_only, session=session) return await self._document_types.list_entries(active_only=active_only, session=session)
async def list_tags(
self,
*,
active_only: bool = True,
session: AsyncSession | None = None,
) -> Sequence[Tag]:
"""List configured tags."""
return await self._tags.list_entries(active_only=active_only, session=session)
async def list_document_type_summaries( async def list_document_type_summaries(
self, self,
*, *,
@@ -380,6 +421,24 @@ class DocumentService(ServiceBase):
for document_type, document_count in rows for document_type, document_count in rows
] ]
async def list_tag_summaries(
self,
*,
session: AsyncSession | None = None,
) -> Sequence[TagSummary]:
"""List Tags alphabetically with current usage counts."""
rows = await self._tags.list_entries_with_counts(session=session)
return [
RegistrySummary(
id=tag.id,
label=tag.label,
is_active=tag.is_active,
is_built_in=tag.semantic_key is not None,
reference_count=document_count,
)
for tag, document_count in rows
]
async def create_document_type( async def create_document_type(
self, self,
*, *,
@@ -390,6 +449,16 @@ class DocumentService(ServiceBase):
"""Create a UUID-identified Document Type with a unique label.""" """Create a UUID-identified Document Type with a unique label."""
return await self._document_types.create_entry(label=label, is_active=is_active, session=session) return await self._document_types.create_entry(label=label, is_active=is_active, session=session)
async def create_tag(
self,
*,
label: str,
is_active: bool = True,
session: AsyncSession | None = None,
) -> Tag:
"""Create a UUID-identified Tag with a unique label."""
return await self._tags.create_entry(label=label, is_active=is_active, session=session)
async def read_document_type( async def read_document_type(
self, self,
document_type_id: UUID, document_type_id: UUID,
@@ -399,6 +468,15 @@ class DocumentService(ServiceBase):
"""Read a Document Type by id.""" """Read a Document Type by id."""
return await self._document_types.read_entry(document_type_id, session=session) return await self._document_types.read_entry(document_type_id, session=session)
async def read_tag(
self,
tag_id: UUID,
*,
session: AsyncSession | None = None,
) -> Tag:
"""Read a Tag by id."""
return await self._tags.read_entry(tag_id, session=session)
async def update_document_type( async def update_document_type(
self, self,
document_type_id: UUID, document_type_id: UUID,
@@ -415,6 +493,22 @@ class DocumentService(ServiceBase):
session=session, session=session,
) )
async def update_tag(
self,
tag_id: UUID,
*,
label: str,
is_active: bool,
session: AsyncSession | None = None,
) -> Tag:
"""Update a Tag label and active state."""
return await self._tags.update_entry(
tag_id,
label=label,
is_active=is_active,
session=session,
)
async def delete_document_type( async def delete_document_type(
self, self,
document_type_id: UUID, document_type_id: UUID,
@@ -424,6 +518,15 @@ class DocumentService(ServiceBase):
"""Delete an unreferenced Document Type without cascade behavior.""" """Delete an unreferenced Document Type without cascade behavior."""
await self._document_types.delete_entry(document_type_id, session=session) await self._document_types.delete_entry(document_type_id, session=session)
async def delete_tag(
self,
tag_id: UUID,
*,
session: AsyncSession | None = None,
) -> None:
"""Delete an unreferenced Tag without cascade behavior."""
await self._tags.delete_entry(tag_id, session=session)
async def is_document_type_referenced( async def is_document_type_referenced(
self, self,
document_type_id: UUID, document_type_id: UUID,
@@ -433,6 +536,15 @@ class DocumentService(ServiceBase):
"""Return whether a Document references a Document Type.""" """Return whether a Document references a Document Type."""
return await self._document_types.is_referenced(document_type_id, session=session) return await self._document_types.is_referenced(document_type_id, session=session)
async def is_tag_referenced(
self,
tag_id: UUID,
*,
session: AsyncSession | None = None,
) -> bool:
"""Return whether a Document references a Tag."""
return await self._tags.is_referenced(tag_id, session=session)
async def set_document_type( async def set_document_type(
self, self,
*, *,
@@ -448,6 +560,58 @@ class DocumentService(ServiceBase):
await self._finalize(session=_session, caller_session=session, refresh=(document,)) await self._finalize(session=_session, caller_session=session, refresh=(document,))
return document return document
async def sync_document_tags_by_labels(
self,
*,
document_id: UUID,
labels: Sequence[str],
session: AsyncSession | None = None,
) -> None:
"""Replace a Document's tag set using label-based assignment."""
normalized_labels = [self._tags.normalize_label(label) for label in labels]
deduplicated_labels = list(dict.fromkeys(normalized_labels))
label_keys = [self._tags.label_key(label) for label in deduplicated_labels]
async with self._session_scope(session) as _session:
existing_document = await _session.get(Document, document_id)
if existing_document is None:
raise DocumentError(
f"Document with id {document_id} not found",
category=ErrorCategory.NOT_FOUND,
suggestion="Refresh and select an existing document.",
)
existing_tags = (
(await _session.exec(select(Tag).where(col(Tag.normalized_label).in_(label_keys))))
.all()
if label_keys
else []
)
tags_by_key = {tag.normalized_label: tag for tag in existing_tags}
selected_tag_ids: set[UUID] = set()
for label in deduplicated_labels:
key = self._tags.label_key(label)
tag = tags_by_key.get(key)
if tag is None:
tag = await self._tags.create_entry(label=label, is_active=True, session=_session)
tags_by_key[key] = tag
selected_tag_ids.add(tag.id)
links = (
await _session.exec(select(DocumentTag).where(DocumentTag.document_id == document_id))
).all()
existing_ids = {link.tag_id for link in links}
for link in links:
if link.tag_id not in selected_tag_ids:
await _session.delete(link)
for tag_id in selected_tag_ids - existing_ids:
_session.add(DocumentTag(document_id=document_id, tag_id=tag_id))
await self._finalize(session=_session, caller_session=session)
def _print_media_type(filename: str) -> str: def _print_media_type(filename: str) -> str:
"""Resolve a stored Source filename to its MIME type for print rendering.""" """Resolve a stored Source filename to its MIME type for print rendering."""
+2 -1
View File
@@ -217,7 +217,8 @@ class PeopleService(ServiceBase):
async def list_people(self, *, session: AsyncSession | None = None) -> Sequence[Person]: async def list_people(self, *, session: AsyncSession | None = None) -> Sequence[Person]:
async with self._session_scope(session) as _session: async with self._session_scope(session) as _session:
return (await _session.exec(select(Person))).all() query = select(Person).options(selectinload(Person.document_people))
return (await _session.exec(query)).all()
async def list_person_roles( async def list_person_roles(
self, self,
+4
View File
@@ -44,6 +44,7 @@ async def create_document_with_people(
*, *,
document: Document, document: Document,
links: list[DocumentPersonInput], links: list[DocumentPersonInput],
tag_labels: list[str],
documents: DocumentService, documents: DocumentService,
people: PeopleService, people: PeopleService,
) -> Document: ) -> Document:
@@ -51,6 +52,7 @@ async def create_document_with_people(
async with transaction_scope(session_factory=documents.session_factory) as session: async with transaction_scope(session_factory=documents.session_factory) as session:
created = await documents.create_document(document, session=session) created = await documents.create_document(document, session=session)
await people.sync_document_people(document_id=created.id, links=links, session=session) await people.sync_document_people(document_id=created.id, links=links, session=session)
await documents.sync_document_tags_by_labels(document_id=created.id, labels=tag_labels, session=session)
return created return created
@@ -58,6 +60,7 @@ async def update_document_with_people(
*, *,
document: Document, document: Document,
links: list[DocumentPersonInput], links: list[DocumentPersonInput],
tag_labels: list[str],
documents: DocumentService, documents: DocumentService,
people: PeopleService, people: PeopleService,
) -> Document: ) -> Document:
@@ -65,6 +68,7 @@ async def update_document_with_people(
async with transaction_scope(session_factory=documents.session_factory) as session: async with transaction_scope(session_factory=documents.session_factory) as session:
updated = await documents.update_document(document, session=session) updated = await documents.update_document(document, session=session)
await people.sync_document_people(document_id=updated.id, links=links, session=session) await people.sync_document_people(document_id=updated.id, links=links, session=session)
await documents.sync_document_tags_by_labels(document_id=updated.id, labels=tag_labels, session=session)
return updated return updated
+2
View File
@@ -13,6 +13,7 @@ from transcription.ui.pages.people_page import register_page as register_people_
from transcription.ui.pages.print_preview_page import register_page as register_print_preview_page from transcription.ui.pages.print_preview_page import register_page as register_print_preview_page
from transcription.ui.pages.settings_page import register_page as register_settings_page from transcription.ui.pages.settings_page import register_page as register_settings_page
from transcription.ui.pages.sources_page import register_page as register_sources_page from transcription.ui.pages.sources_page import register_page as register_sources_page
from transcription.ui.pages.tags_page import register_page as register_tags_page
from transcription.ui.resources import read_css from transcription.ui.resources import read_css
from transcription.ui.theme import apply_archival_theme from transcription.ui.theme import apply_archival_theme
@@ -37,6 +38,7 @@ def register_pages(app: FastAPI) -> None:
_register_global_styles(app) _register_global_styles(app)
register_home_page() register_home_page()
register_documents_page() register_documents_page()
register_tags_page()
register_people_page() register_people_page()
register_print_preview_page() register_print_preview_page()
register_sources_page() register_sources_page()
@@ -8,6 +8,7 @@ from transcription.ui.resources import read_svg
NAV_ITEMS: tuple[tuple[str, str, str], ...] = ( NAV_ITEMS: tuple[tuple[str, str, str], ...] = (
("Documents", "/documents", "description"), ("Documents", "/documents", "description"),
("Tags", "/tags", "sell"),
("People", "/people", "group"), ("People", "/people", "group"),
("Sources", "/sources", "folder"), ("Sources", "/sources", "folder"),
("Jobs", "/jobs", "work_history"), ("Jobs", "/jobs", "work_history"),
@@ -22,6 +23,8 @@ def _is_active_path(*, current_path: str, item_path: str) -> bool:
return current_path == "/documents" or current_path.startswith("/documents/") return current_path == "/documents" or current_path.startswith("/documents/")
if item_path == "/people": if item_path == "/people":
return current_path == "/people" or current_path.startswith("/people/") return current_path == "/people" or current_path.startswith("/people/")
if item_path == "/tags":
return current_path == "/tags" or current_path.startswith("/tags/")
if item_path == "/sources": if item_path == "/sources":
return current_path == "/sources" or current_path.startswith("/sources/") return current_path == "/sources" or current_path.startswith("/sources/")
if item_path == "/settings": if item_path == "/settings":
@@ -23,7 +23,7 @@ class DocumentTableRow:
document_type: str document_type: str
authors: str authors: str
document_date: str document_date: str
archive_identifier: str source_count: int
def _serialize_rows(rows: Sequence[DocumentTableRow]) -> list[dict[str, Any]]: def _serialize_rows(rows: Sequence[DocumentTableRow]) -> list[dict[str, Any]]:
@@ -34,7 +34,7 @@ def _serialize_rows(rows: Sequence[DocumentTableRow]) -> list[dict[str, Any]]:
"document_type": row.document_type or "Unspecified", "document_type": row.document_type or "Unspecified",
"authors": row.authors or "Not set", "authors": row.authors or "Not set",
"document_date": row.document_date, "document_date": row.document_date,
"archive_identifier": row.archive_identifier or "N/A", "source_count": row.source_count,
} }
for row in rows for row in rows
] ]
@@ -87,17 +87,17 @@ def render_documents_table(rows: Sequence[DocumentTableRow]) -> None:
"style": "width: 14%;", "style": "width: 14%;",
}, },
{ {
"name": "archive_identifier", "name": "source_count",
"label": "Archive Ref", "label": "# Sources",
"field": "archive_identifier", "field": "source_count",
"sortable": True, "sortable": True,
"classes": "font-mono text-xs", "classes": "font-mono",
"align": "center", "align": "center",
"style": "width: 20%;", "style": "width: 20%;",
}, },
], ],
default_sort_by="name", default_sort_by="name",
search_placeholder="Search documents by title, type, or reference...", search_placeholder="Search documents by title, type, or author...",
on_row_click_id=lambda doc_id: ui.navigate.to(f"/documents/{doc_id}"), on_row_click_id=lambda doc_id: ui.navigate.to(f"/documents/{doc_id}"),
) )
+13 -4
View File
@@ -24,6 +24,7 @@ class JobTableRow:
id: UUID id: UUID
status: str status: str
document_name: str document_name: str
source_count: int
retry_count: int retry_count: int
date_created: str date_created: str
date_updated: str date_updated: str
@@ -45,6 +46,7 @@ def _serialize_rows(rows: Sequence[JobTableRow]) -> list[dict[str, Any]]:
"id": str(row.id), "id": str(row.id),
"status": row.status.lower(), "status": row.status.lower(),
"document_name": row.document_name, "document_name": row.document_name,
"source_count": row.source_count,
"retry_count": row.retry_count, "retry_count": row.retry_count,
"date_created": _format_timestamp(row.date_created), "date_created": _format_timestamp(row.date_created),
"date_updated": _format_timestamp(row.date_updated), "date_updated": _format_timestamp(row.date_updated),
@@ -71,7 +73,7 @@ def render_jobs_table(rows: Sequence[JobTableRow]) -> None:
"field": "id", "field": "id",
"sortable": True, "sortable": True,
"classes": "font-mono text-xs", "classes": "font-mono text-xs",
"style": "width: 30%;", "style": "width: 26%;",
}, },
{ {
"name": "status", "name": "status",
@@ -79,7 +81,7 @@ def render_jobs_table(rows: Sequence[JobTableRow]) -> None:
"field": "status", "field": "status",
"sortable": True, "sortable": True,
"classes": "font-mono", "classes": "font-mono",
"style": "width: 15%;", "style": "width: 13%;",
}, },
{ {
"name": "document_name", "name": "document_name",
@@ -88,7 +90,14 @@ def render_jobs_table(rows: Sequence[JobTableRow]) -> None:
"sortable": True, "sortable": True,
"classes": "font-serif text-left ui-table-cell-wrap", "classes": "font-serif text-left ui-table-cell-wrap",
"align": "left", "align": "left",
"style": "width: 35%;", "style": "width: 31%;",
},
{
"name": "source_count",
"label": "# Sources",
"field": "source_count",
"sortable": True,
"style": "width: 8%;",
}, },
{ {
"name": "retry_count", "name": "retry_count",
@@ -102,7 +111,7 @@ def render_jobs_table(rows: Sequence[JobTableRow]) -> None:
"label": "Updated", "label": "Updated",
"field": "updated_sort", "field": "updated_sort",
"sortable": True, "sortable": True,
"style": "width: 12%;", "style": "width: 14%;",
}, },
], ],
default_sort_by="updated_sort", default_sort_by="updated_sort",
+14 -21
View File
@@ -20,10 +20,10 @@ class PersonTableRow:
id: UUID id: UUID
full_name: str full_name: str
display_name: str family_search_id: str
maiden_name: str
birth_date: str birth_date: str
death_date: str death_date: str
document_count: int
def _serialize_rows(rows: Sequence[PersonTableRow]) -> list[dict[str, Any]]: def _serialize_rows(rows: Sequence[PersonTableRow]) -> list[dict[str, Any]]:
@@ -31,10 +31,10 @@ def _serialize_rows(rows: Sequence[PersonTableRow]) -> list[dict[str, Any]]:
{ {
"id": str(row.id), "id": str(row.id),
"full_name": row.full_name, "full_name": row.full_name,
"display_name": row.display_name or "Not set", "family_search_id": row.family_search_id or "Not set",
"maiden_name": row.maiden_name or "N/A",
"birth_date": row.birth_date or "Unknown", "birth_date": row.birth_date or "Unknown",
"death_date": row.death_date or "Unknown", "death_date": row.death_date or "Unknown",
"document_count": row.document_count,
} }
for row in rows for row in rows
] ]
@@ -57,22 +57,7 @@ def render_people_table(rows: Sequence[PersonTableRow]) -> None:
"sortable": True, "sortable": True,
"classes": "font-serif font-semibold text-left ui-table-cell-wrap", "classes": "font-serif font-semibold text-left ui-table-cell-wrap",
}, },
{ {"name": "family_search_id", "label": "FamilySearch ID", "field": "family_search_id", "sortable": True},
"name": "display_name",
"label": "Display Name",
"field": "display_name",
"sortable": True,
"classes": "ui-table-cell-wrap",
"align": "center",
},
{
"name": "maiden_name",
"label": "Maiden Name",
"field": "maiden_name",
"sortable": True,
"classes": "ui-table-cell-wrap",
"align": "center",
},
{ {
"name": "birth_date", "name": "birth_date",
"label": "Birth Date", "label": "Birth Date",
@@ -89,9 +74,17 @@ def render_people_table(rows: Sequence[PersonTableRow]) -> None:
"classes": "font-mono text-xs", "classes": "font-mono text-xs",
"align": "center", "align": "center",
}, },
{
"name": "document_count",
"label": "# Documents",
"field": "document_count",
"sortable": True,
"align": "center",
"classes": "font-mono",
},
], ],
default_sort_by="full_name", default_sort_by="full_name",
search_placeholder="Search people by name, birth date, or death date...", search_placeholder="Search people by name, FamilySearch ID, or dates...",
on_row_click_id=lambda person_id: ui.navigate.to(f"/people/{person_id}"), on_row_click_id=lambda person_id: ui.navigate.to(f"/people/{person_id}"),
) )
+67 -5
View File
@@ -10,6 +10,7 @@ from fastapi import Request
from fastapi.responses import RedirectResponse from fastapi.responses import RedirectResponse
from nicegui import ui from nicegui import ui
from transcription.config import Settings
from transcription.db.models import Document from transcription.db.models import Document
from transcription.db.registries import AUTHOR_ROLE_SEMANTIC_KEY from transcription.db.registries import AUTHOR_ROLE_SEMANTIC_KEY
from transcription.errors import ErrorCategory from transcription.errors import ErrorCategory
@@ -35,12 +36,14 @@ from transcription.ui.components.guards import parsed_record_id
from transcription.ui.components.guards import render_record_not_found from transcription.ui.components.guards import render_record_not_found
from transcription.ui.components.linked_people import LinkedPeopleEditor from transcription.ui.components.linked_people import LinkedPeopleEditor
from transcription.ui.components.linked_people import StagedLinkedPerson from transcription.ui.components.linked_people import StagedLinkedPerson
from transcription.ui.components.media_urls import resolve_media_url
from transcription.ui.components.primitives import destructive_button from transcription.ui.components.primitives import destructive_button
from transcription.ui.components.primitives import render_empty_state from transcription.ui.components.primitives import render_empty_state
from transcription.ui.components.primitives import section_header_row from transcription.ui.components.primitives import section_header_row
from transcription.ui.components.table.documents import DocumentTableRow from transcription.ui.components.table.documents import DocumentTableRow
from transcription.ui.components.table.documents import render_documents_table from transcription.ui.components.table.documents import render_documents_table
from transcription.ui.components.viewers import dark_room_viewer from transcription.ui.components.viewers import dark_room_viewer
from transcription.ui.runtime import resolve_runtime_settings
from transcription.ui.theme import page_header from transcription.ui.theme import page_header
from ...db.session import SessionFactoryDep from ...db.session import SessionFactoryDep
@@ -58,6 +61,7 @@ class DocumentFormFields:
location: ui.input location: ui.input
archive: ui.input archive: ui.input
notes: ui.textarea notes: ui.textarea
tags: ui.select
def register_page() -> None: # noqa: PLR0915 def register_page() -> None: # noqa: PLR0915
@@ -75,6 +79,7 @@ def register_page() -> None: # noqa: PLR0915
people = sorted(await people_service.list_people(), key=lambda item: item.full_name.casefold()) people = sorted(await people_service.list_people(), key=lambda item: item.full_name.casefold())
role_catalog = list(await people_service.list_person_roles(active_only=False)) role_catalog = list(await people_service.list_person_roles(active_only=False))
type_catalog = await document_service.list_document_types() type_catalog = await document_service.list_document_types()
tag_catalog = await document_service.list_tags(active_only=True)
requested_person_id = parse_uuid(request.query_params.get("person_id")) requested_person_id = parse_uuid(request.query_params.get("person_id"))
staged_links: list[StagedLinkedPerson] = [] staged_links: list[StagedLinkedPerson] = []
if requested_person_id is not None and any(person.id == requested_person_id for person in people): if requested_person_id is not None and any(person.id == requested_person_id for person in people):
@@ -101,6 +106,7 @@ def register_page() -> None: # noqa: PLR0915
) )
form = _render_document_form_fields( form = _render_document_form_fields(
type_options={str(doc_type.id): doc_type.label for doc_type in type_catalog}, type_options={str(doc_type.id): doc_type.label for doc_type in type_catalog},
tag_options=[tag.label for tag in tag_catalog],
linked_people=linked_people, linked_people=linked_people,
) )
@@ -137,6 +143,7 @@ def register_page() -> None: # noqa: PLR0915
action=lambda: create_document_with_people( action=lambda: create_document_with_people(
document=candidate, document=candidate,
links=linked_people.values(), links=linked_people.values(),
tag_labels=_resolve_selected_tag_labels(form.tags.value),
documents=document_service, documents=document_service,
people=people_service, people=people_service,
), ),
@@ -189,16 +196,17 @@ def register_page() -> None: # noqa: PLR0915
document_type=(doc.document_type_ref.label if doc.document_type_ref is not None else ""), document_type=(doc.document_type_ref.label if doc.document_type_ref is not None else ""),
authors=", ".join(_author_names(doc)), authors=", ".join(_author_names(doc)),
document_date=compact_date(doc.document_date, doc.document_date_raw), document_date=compact_date(doc.document_date, doc.document_date_raw),
archive_identifier=doc.archive_identifier or "", source_count=len(doc.sources),
) )
for doc in documents for doc in documents
] ]
render_documents_table(rows) render_documents_table(rows)
@ui.page("/documents/{document_id}") @ui.page("/documents/{document_id}")
async def document_detail_page(document_id: str, session_factory: SessionFactoryDep) -> None: async def document_detail_page(request: Request, document_id: str, session_factory: SessionFactoryDep) -> None:
document_service = DocumentService(session_factory=session_factory) document_service = DocumentService(session_factory=session_factory)
render_navigation_header(current_path="/documents") render_navigation_header(current_path="/documents")
settings = resolve_runtime_settings(request)
parsed_doc_id = parsed_record_id(document_id, noun="Document") parsed_doc_id = parsed_record_id(document_id, noun="Document")
if parsed_doc_id is None: if parsed_doc_id is None:
@@ -237,7 +245,7 @@ def register_page() -> None: # noqa: PLR0915
) )
with ui.grid().classes("w-full grid-cols-12 gap-4"): with ui.grid().classes("w-full grid-cols-12 gap-4"):
_render_bento_viewer_zone(document) _render_bento_viewer_zone(document, base_url=str(request.base_url), settings=settings)
_render_bento_metadata_zone(document) _render_bento_metadata_zone(document)
_render_bento_relations_zone(document) _render_bento_relations_zone(document)
@@ -276,6 +284,7 @@ def register_page() -> None: # noqa: PLR0915
people = sorted(await people_service.list_people(), key=lambda item: item.full_name.casefold()) people = sorted(await people_service.list_people(), key=lambda item: item.full_name.casefold())
role_catalog = list(await people_service.list_person_roles(active_only=False)) role_catalog = list(await people_service.list_person_roles(active_only=False))
type_catalog = await document_service.list_document_types(active_only=False) type_catalog = await document_service.list_document_types(active_only=False)
tag_catalog = await document_service.list_tags(active_only=False)
linked_people = LinkedPeopleEditor( linked_people = LinkedPeopleEditor(
people=people, people=people,
roles=role_catalog, roles=role_catalog,
@@ -284,6 +293,7 @@ def register_page() -> None: # noqa: PLR0915
form = _render_document_form_fields( form = _render_document_form_fields(
document=document, document=document,
type_options={str(doc_type.id): doc_type.label for doc_type in type_catalog}, type_options={str(doc_type.id): doc_type.label for doc_type in type_catalog},
tag_options=[tag.label for tag in tag_catalog],
linked_people=linked_people, linked_people=linked_people,
) )
@@ -321,6 +331,7 @@ def register_page() -> None: # noqa: PLR0915
action=lambda: update_document_with_people( action=lambda: update_document_with_people(
document=candidate, document=candidate,
links=linked_people.values(), links=linked_people.values(),
tag_labels=_resolve_selected_tag_labels(form.tags.value),
documents=document_service, documents=document_service,
people=people_service, people=people_service,
), ),
@@ -410,6 +421,7 @@ def _render_document_form_fields(
*, *,
document: Document | None = None, document: Document | None = None,
type_options: dict[str, str], type_options: dict[str, str],
tag_options: list[str],
linked_people: LinkedPeopleEditor, linked_people: LinkedPeopleEditor,
) -> DocumentFormFields: ) -> DocumentFormFields:
with archival_card(extra_classes="gap-3"): with archival_card(extra_classes="gap-3"):
@@ -471,6 +483,26 @@ def _render_document_form_fields(
.props("outlined autogrow") .props("outlined autogrow")
.classes("w-full ui-form-surface") .classes("w-full ui-form-surface")
) )
selected_tags = (
sorted(
[
link.tag_ref.label
for link in (document.document_tags if document is not None else [])
if link.tag_ref is not None
],
key=str.casefold,
)
if document is not None
else []
)
tags_input = ui.select(
sorted(tag_options, key=str.casefold),
label="Tags",
value=selected_tags,
multiple=True,
with_input=True,
new_value_mode="add-unique",
).props("outlined use-chips").classes("w-full ui-form-surface")
linked_people.render() linked_people.render()
@@ -483,13 +515,15 @@ def _render_document_form_fields(
location=location_input, location=location_input,
archive=archive_input, archive=archive_input,
notes=notes_input, notes=notes_input,
tags=tags_input,
) )
def _render_bento_viewer_zone(document: Document) -> None: def _render_bento_viewer_zone(document: Document, *, base_url: str, settings: Settings) -> None:
with ui.column().classes("col-span-12 lg:col-span-4"): with ui.column().classes("col-span-12 lg:col-span-4"):
source_path = document.sources[0].file_path if document.sources else None source_path = document.sources[0].file_path if document.sources else None
dark_room_viewer(source_path, count_label=f"{len(document.sources)} Source(s) Linked") source_url = resolve_media_url(source_path, upload_dir=settings.upload_dir, base_url=base_url)
dark_room_viewer(source_url, count_label=f"{len(document.sources)} Source(s) Linked")
def _render_bento_metadata_zone(document: Document) -> None: def _render_bento_metadata_zone(document: Document) -> None:
@@ -502,6 +536,11 @@ def _render_bento_metadata_zone(document: Document) -> None:
"Document Type:", "Document Type:",
document.document_type_ref.label if document.document_type_ref is not None else "Not set", document.document_type_ref.label if document.document_type_ref is not None else "Not set",
) )
tags = sorted(
[link.tag_ref.label for link in document.document_tags if link.tag_ref is not None],
key=str.casefold,
)
metadata_row("Tags:", ", ".join(tags) if tags else "Not set")
metadata_row("Document Date:", compact_date(document.document_date, document.document_date_raw)) metadata_row("Document Date:", compact_date(document.document_date, document.document_date_raw))
metadata_row("Location Created:", document.location_created or "Not set") metadata_row("Location Created:", document.location_created or "Not set")
metadata_row("Archive Identifier:", document.archive_identifier or "Not set") metadata_row("Archive Identifier:", document.archive_identifier or "Not set")
@@ -587,3 +626,26 @@ def _author_names(document: Document) -> list[str]:
), ),
key=str.casefold, key=str.casefold,
) )
def _resolve_selected_tag_labels(value: object) -> list[str]:
def flatten(item: object) -> list[str]:
if item is None:
return []
if isinstance(item, str):
return [item]
if isinstance(item, dict):
if "value" in item:
return flatten(item.get("value"))
if "label" in item:
return flatten(item.get("label"))
return []
if isinstance(item, (list, tuple, set)):
values: list[str] = []
for child in item:
values.extend(flatten(child))
return values
return [str(item)]
labels = [candidate.strip() for candidate in flatten(value) if candidate and candidate.strip()]
return list(dict.fromkeys(labels))
+16 -5
View File
@@ -79,6 +79,7 @@ def register_page() -> None: # noqa: PLR0915
id=job.id, id=job.id,
status=job.status.value, status=job.status.value,
document_name=job.document.name if job.document is not None else "Unknown document", document_name=job.document.name if job.document is not None else "Unknown document",
source_count=len(job.job_sources),
retry_count=job.retry_count, retry_count=job.retry_count,
date_created=job.date_created.isoformat(), date_created=job.date_created.isoformat(),
date_updated=job.date_updated.isoformat(), date_updated=job.date_updated.isoformat(),
@@ -146,20 +147,30 @@ def register_page() -> None: # noqa: PLR0915
document_select.value = requested_document_id document_select.value = requested_document_id
with ui.row().classes("w-full gap-3 grid grid-cols-1 md:grid-cols-2"): with ui.row().classes("w-full gap-3 grid grid-cols-1 md:grid-cols-2"):
provider_options = [settings.provider.value]
model_options = list(settings.provider_models) or ([settings.provider_model] if settings.provider_model else [])
if locked_source is not None: if locked_source is not None:
provider_input = ( provider_input = (
ui.input(label="Provider", value=settings.provider.value) ui.select(provider_options, label="Provider", value=settings.provider.value)
.props("outlined readonly") .props("outlined disable")
.classes("ui-form-surface") .classes("ui-form-surface")
) )
model_input = ( model_input = (
ui.select(list(settings.provider_models), label="Model", value=settings.provider_model) ui.select(model_options, label="Model", value=settings.provider_model)
.props("outlined") .props("outlined")
.classes("ui-form-surface") .classes("ui-form-surface")
) )
else: else:
provider_input = ui.input(label="Provider").props("outlined").classes("ui-form-surface") provider_input = (
model_input = ui.input(label="Model").props("outlined").classes("ui-form-surface") ui.select(provider_options, label="Provider", value=settings.provider.value)
.props("outlined")
.classes("ui-form-surface")
)
model_input = (
ui.select(model_options, label="Model", value=settings.provider_model)
.props("outlined use-input")
.classes("ui-form-surface")
)
if locked_source is None: if locked_source is None:
_render_upload_section(uploaded_files) _render_upload_section(uploaded_files)
+2 -2
View File
@@ -98,10 +98,10 @@ def register_page() -> None: # noqa: PLR0915
PersonTableRow( PersonTableRow(
id=person.id, id=person.id,
full_name=person.full_name, full_name=person.full_name,
display_name=person.display_name or "", family_search_id=person.family_search_id or "",
maiden_name=person.maiden_name or "",
birth_date=compact_date(person.birth_date, person.birth_date_raw), birth_date=compact_date(person.birth_date, person.birth_date_raw),
death_date=compact_date(person.death_date, person.death_date_raw), death_date=compact_date(person.death_date, person.death_date_raw),
document_count=len(person.document_people),
) )
for person in people for person in people
] ]
+117
View File
@@ -271,6 +271,120 @@ def register_page(*, settings: Settings) -> None: # noqa: PLR0915
ui.button("Edit", icon="edit", on_click=lambda: open_role_editor(creating=False)).props("flat") ui.button("Edit", icon="edit", on_click=lambda: open_role_editor(creating=False)).props("flat")
destructive_button("Delete", icon="delete", on_click=delete_selected_role) destructive_button("Delete", icon="delete", on_click=delete_selected_role)
@ui.refreshable
async def render_tags() -> None: # noqa: PLR0915
with archival_card("Tags"):
ui.label("Tags are listed alphabetically. Select one row to edit or delete it.").classes(
"text-xs ui-text-muted mb-3"
)
tags_outcome = await run_ui_action(
operation="settings.tags.list",
title="Tags unavailable",
action=documents.list_tag_summaries,
)
if not tags_outcome.ok:
return
tags = tags_outcome.value or ()
if not tags:
render_empty_state("No Tags are configured.", extra_classes="mt-3")
rows = [
{
"id": str(item.id),
"label": item.label,
"reference_count": item.reference_count,
"is_active": item.is_active,
"is_built_in": item.is_built_in,
}
for item in tags
]
table = render_registry_table(
rows,
count_field="reference_count",
count_label="Documents",
)
async def save_tag(
*,
item_id: UUID | None,
label: str,
is_active: bool,
) -> bool:
async def _save_tag() -> None:
if item_id is None:
await documents.create_tag(label=label, is_active=is_active)
return
await documents.update_tag(
item_id,
label=label,
is_active=is_active,
)
save_outcome = await run_ui_action(
operation="settings.tags.save",
title="Tag save failed",
action=_save_tag,
)
if not save_outcome.ok:
return False
ui.notify("Tag saved", type="positive")
render_tags.refresh()
return True
def open_tag_editor(*, creating: bool) -> None:
selected = _selected_table_row(table)
if not creating and selected is None:
ui.notify("Select one Tag to edit.", type="warning")
return
if creating:
item_id = None
current_label = ""
current_active = True
else:
assert selected is not None
item_id = UUID(str(selected["id"]))
current_label = str(selected["label"])
current_active = bool(selected["is_active"])
with ui.dialog() as dialog, ui.card().classes("w-full max-w-lg ui-card-surface"):
ui.label("Add Tag" if creating else "Edit Tag").classes("text-lg font-semibold")
label_input = ui.input("Label", value=current_label).props("outlined").classes("w-full")
active_input = ui.checkbox("Active", value=current_active)
async def submit() -> None:
saved = await save_tag(
item_id=item_id,
label=str(label_input.value or ""),
is_active=bool(active_input.value),
)
if saved:
dialog.close()
with ui.row().classes("w-full justify-end gap-2"):
ui.button("Cancel", on_click=dialog.close).props("flat")
ui.button("Save", icon="save", on_click=submit).classes("ui-btn-primary")
dialog.open()
async def delete_selected_tag() -> None:
selected = _selected_table_row(table)
if selected is None:
ui.notify("Select one Tag to delete.", type="warning")
return
delete_outcome = await run_ui_action(
operation="settings.tags.delete",
title="Tag deletion failed",
action=lambda: documents.delete_tag(UUID(str(selected["id"]))),
)
if not delete_outcome.ok:
return
ui.notify("Tag deleted", type="positive")
render_tags.refresh()
with ui.row().classes("w-full items-center gap-2 mt-3"):
ui.button("Add", icon="add", on_click=lambda: open_tag_editor(creating=True)).classes(
"ui-btn-primary"
)
ui.button("Edit", icon="edit", on_click=lambda: open_tag_editor(creating=False)).props("flat")
destructive_button("Delete", icon="delete", on_click=delete_selected_tag)
@ui.refreshable @ui.refreshable
async def render_prompts() -> None: async def render_prompts() -> None:
with archival_card("Prompts"): with archival_card("Prompts"):
@@ -381,6 +495,7 @@ def register_page(*, settings: Settings) -> None: # noqa: PLR0915
with ui.tabs().classes("w-full") as tabs: with ui.tabs().classes("w-full") as tabs:
document_types_tab = ui.tab("Document Types") document_types_tab = ui.tab("Document Types")
person_roles_tab = ui.tab("Person Roles") person_roles_tab = ui.tab("Person Roles")
tags_tab = ui.tab("Tags")
prompts_tab = ui.tab("Prompts") prompts_tab = ui.tab("Prompts")
home_page_text_tab = ui.tab("Home Page Text") home_page_text_tab = ui.tab("Home Page Text")
@@ -389,6 +504,8 @@ def register_page(*, settings: Settings) -> None: # noqa: PLR0915
await render_document_types() await render_document_types()
with ui.tab_panel(person_roles_tab): with ui.tab_panel(person_roles_tab):
await render_person_roles() await render_person_roles()
with ui.tab_panel(tags_tab):
await render_tags()
with ui.tab_panel(prompts_tab): with ui.tab_panel(prompts_tab):
await render_prompts() await render_prompts()
with ui.tab_panel(home_page_text_tab): with ui.tab_panel(home_page_text_tab):
+21 -25
View File
@@ -176,24 +176,26 @@ def register_page() -> None: # noqa: PLR0915
extra_classes="text-xs", extra_classes="text-xs",
) )
with ui.grid().classes("w-full grid-cols-12 gap-4"): with ui.grid().classes("w-full grid-cols-12 gap-4"):
with ui.column().classes("col-span-12 lg:col-span-4 gap-2"): with ui.column().classes("col-span-12 lg:col-span-8 gap-4"):
_render_source_navigation(navigation.previous_id, navigation.next_id) with ui.grid().classes("w-full grid-cols-1 lg:grid-cols-2 gap-4"):
_render_source_viewer_zone( with ui.column().classes("gap-2"):
source, _render_source_navigation(navigation.previous_id, navigation.next_id)
settings=resolve_runtime_settings(request), _render_source_viewer_zone(
request=request, source,
settings=resolve_runtime_settings(request),
request=request,
)
_render_source_transcription_column(
source=source,
original_transcription=original_transcription,
latest_job_source=latest_job_source,
sources_service=sources_service,
)
_render_machine_candidates(
source=source,
attempts=attempts,
evidence_service=evidence_service,
) )
_render_source_transcription_column(
source=source,
original_transcription=original_transcription,
latest_job_source=latest_job_source,
sources_service=sources_service,
)
_render_machine_candidates(
source=source,
attempts=attempts,
evidence_service=evidence_service,
)
_render_source_metadata_column( _render_source_metadata_column(
source=source, source=source,
latest_job_source=latest_job_source, latest_job_source=latest_job_source,
@@ -301,7 +303,7 @@ def _render_source_transcription_column(
latest_job_source: JobSource | None, latest_job_source: JobSource | None,
sources_service: SourceService, sources_service: SourceService,
) -> None: ) -> None:
with ui.column().classes("col-span-12 lg:col-span-4 gap-4"): with ui.column().classes("gap-4"):
_render_source_transcription_zone( _render_source_transcription_zone(
source=source, source=source,
original_transcription=original_transcription, original_transcription=original_transcription,
@@ -474,12 +476,6 @@ def _render_source_transcription_zone(
latest_job_source: JobSource | None, latest_job_source: JobSource | None,
sources_service: SourceService, sources_service: SourceService,
) -> None: ) -> None:
with archival_card(title="Transcription Text"):
if original_transcription:
ui.label(original_transcription).classes("p-2 ui-note-box text-xs whitespace-pre-wrap")
else:
render_empty_state("No transcription text available yet.", italic=True)
with archival_card(title="Editable Revision"): with archival_card(title="Editable Revision"):
seed_revision = source.revised_text if source.revised_text is not None else (original_transcription or "") seed_revision = source.revised_text if source.revised_text is not None else (original_transcription or "")
revision_input = ( revision_input = (
@@ -565,7 +561,7 @@ def _render_machine_candidates(
None, None,
) )
with ui.column().classes("col-span-12 lg:col-span-8 gap-2"): # noqa: PLR1702, SIM117 with ui.column().classes("w-full gap-2"): # noqa: PLR1702, SIM117
with archival_card(title="Candidate Machine Transcriptions"): with archival_card(title="Candidate Machine Transcriptions"):
if preferred_attempt is not None: if preferred_attempt is not None:
_render_attempt_warnings(preferred_attempt, label="Preferred machine transcription warnings") _render_attempt_warnings(preferred_attempt, label="Preferred machine transcription warnings")
+92
View File
@@ -0,0 +1,92 @@
"""Tags browse and filter page registration."""
from __future__ import annotations
from nicegui import ui
from transcription.services.documents import DocumentService
from transcription.ui.components.app_shell import render_navigation_header
from transcription.ui.components.cards import archival_card
from transcription.ui.components.error_presenter import run_ui_action
from transcription.ui.components.primitives import render_empty_state
from transcription.ui.components.primitives import section_header_row
from transcription.ui.theme import page_header
from ...db.session import SessionFactoryDep
def register_page() -> None:
"""Register the tags browse/filter route."""
@ui.page("/tags")
async def tags_page(session_factory: SessionFactoryDep) -> None:
document_service = DocumentService(session_factory=session_factory)
render_navigation_header(current_path="/tags")
tags_outcome = await run_ui_action(
operation="tags.list",
title="Tags unavailable",
action=document_service.list_tag_summaries,
)
if not tags_outcome.ok:
return
tag_summaries = tags_outcome.value or ()
tag_labels = [item.label for item in tag_summaries]
documents_outcome = await run_ui_action(
operation="documents.list",
title="Documents unavailable",
action=document_service.list_documents,
)
if not documents_outcome.ok:
return
documents = documents_outcome.value or ()
with ui.column().classes("w-full max-w-7xl mx-auto p-4 gap-4"):
with section_header_row():
page_header("Tags", subtitle="Browse documents by tag.")
if not tag_summaries:
with archival_card(extra_classes="p-8 text-center"):
render_empty_state("No tags are configured yet.")
return
selected_tag = (
ui.select(tag_labels, label="Filter by tag")
.props("outlined clearable use-input")
.classes("w-full md:w-96 ui-form-surface")
)
@ui.refreshable
def render_groups() -> None:
selected = str(selected_tag.value or "").strip()
with ui.column().classes("w-full gap-3"):
rendered_any = False
for summary in tag_summaries:
if selected and summary.label != selected:
continue
tagged_documents = [
document
for document in documents
if any(
link.tag_ref is not None and link.tag_ref.id == summary.id
for link in document.document_tags
)
]
if not tagged_documents:
continue
rendered_any = True
with archival_card(title=f"{summary.label} ({len(tagged_documents)})"):
for document in sorted(tagged_documents, key=lambda item: item.name.casefold()):
ui.button(
document.name,
on_click=lambda _=None, doc_id=document.id: ui.navigate.to(f"/documents/{doc_id}"),
icon="description",
).props("flat dense no-caps").classes("self-start ui-link-primary text-xs")
if not rendered_any:
with archival_card(extra_classes="p-6"):
render_empty_state("No documents match this tag filter.", italic=True)
selected_tag.on_value_change(lambda _event: render_groups.refresh())
render_groups()
+26
View File
@@ -5,13 +5,16 @@ from datetime import datetime
from uuid import uuid4 from uuid import uuid4
import pytest import pytest
from sqlmodel import select
from transcription.config import Settings from transcription.config import Settings
from transcription.db.models import Document from transcription.db.models import Document
from transcription.db.models import DocumentPerson from transcription.db.models import DocumentPerson
from transcription.db.models import DocumentTag
from transcription.db.models import Job from transcription.db.models import Job
from transcription.db.models import Person from transcription.db.models import Person
from transcription.db.models import Source from transcription.db.models import Source
from transcription.db.models import Tag
from transcription.services.documents import DocumentDeleteBlockedError from transcription.services.documents import DocumentDeleteBlockedError
from transcription.services.documents import DocumentError from transcription.services.documents import DocumentError
from transcription.services.documents import DocumentService from transcription.services.documents import DocumentService
@@ -311,3 +314,26 @@ async def test_update_document_person_changes_role_id(default_session_factory):
) )
assert updated.role_id == recipient_role.id assert updated.role_id == recipient_role.id
@pytest.mark.asyncio
async def test_sync_document_tags_by_labels_creates_and_replaces_tags(default_session_factory):
documents = DocumentService(session_factory=default_session_factory)
document = await documents.create_document(Document(id=uuid4(), name="tagged-doc"))
await documents.sync_document_tags_by_labels(document_id=document.id, labels=["Family", "Census"])
await documents.sync_document_tags_by_labels(document_id=document.id, labels=["Census", "Research"])
async with documents._session_scope() as session:
links = (await session.exec(select(DocumentTag).where(DocumentTag.document_id == document.id))).all()
tags = (await session.exec(select(Tag))).all()
assert len(links) == 2
linked_ids = {link.tag_id for link in links}
linked_labels = {tag.label for tag in tags if tag.id in linked_ids}
assert linked_labels == {"Census", "Research"}
listed = await documents.list_documents()
assert len(listed) == 1
listed_labels = {link.tag_ref.label for link in listed[0].document_tags if link.tag_ref is not None}
assert listed_labels == {"Census", "Research"}
+3
View File
@@ -31,6 +31,7 @@ async def test_create_document_with_people_rolls_back_on_invalid_person(default_
await create_document_with_people( await create_document_with_people(
document=Document(name="Must roll back"), document=Document(name="Must roll back"),
links=[DocumentPersonInput(person_id=uuid4(), role_id=role.id)], links=[DocumentPersonInput(person_id=uuid4(), role_id=role.id)],
tag_labels=[],
documents=documents, documents=documents,
people=people, people=people,
) )
@@ -48,6 +49,7 @@ async def test_update_document_with_people_rolls_back_document_and_links(default
document = await create_document_with_people( document = await create_document_with_people(
document=Document(name="Original name"), document=Document(name="Original name"),
links=[DocumentPersonInput(person_id=person.id, role_id=role.id)], links=[DocumentPersonInput(person_id=person.id, role_id=role.id)],
tag_labels=[],
documents=documents, documents=documents,
people=people, people=people,
) )
@@ -63,6 +65,7 @@ async def test_update_document_with_people_rolls_back_document_and_links(default
await update_document_with_people( await update_document_with_people(
document=candidate, document=candidate,
links=[DocumentPersonInput(person_id=person.id, role_id=inactive.id)], links=[DocumentPersonInput(person_id=person.id, role_id=inactive.id)],
tag_labels=[],
documents=documents, documents=documents,
people=people, people=people,
) )
+12 -3
View File
@@ -41,9 +41,11 @@ async def test_create_all_creates_expected_tables(tmp_path):
assert "document" in table_names assert "document" in table_names
assert "document_type" in table_names assert "document_type" in table_names
assert "tag" in table_names
assert "person" in table_names assert "person" in table_names
assert "person_role" in table_names assert "person_role" in table_names
assert "document_person" in table_names assert "document_person" in table_names
assert "document_tag" in table_names
assert "job" in table_names assert "job" in table_names
assert "source" in table_names assert "source" in table_names
assert "job_source" in table_names assert "job_source" in table_names
@@ -130,15 +132,19 @@ async def test_create_all_declares_hot_path_indexes(tmp_path):
database = inspect(sync_connection) database = inspect(sync_connection)
indexes = { indexes = {
table: [index["column_names"] for index in database.get_indexes(table)] table: [index["column_names"] for index in database.get_indexes(table)]
for table in ("job", "source", "job_source", "document", "document_person") for table in ("job", "source", "job_source", "document", "document_person", "document_tag")
} }
job_source_unique = [ job_source_unique = [
constraint["column_names"] constraint["column_names"]
for constraint in database.get_unique_constraints("job_source") for constraint in database.get_unique_constraints("job_source")
] ]
return indexes, job_source_unique document_tag_unique = [
constraint["column_names"]
for constraint in database.get_unique_constraints("document_tag")
]
return indexes, job_source_unique, document_tag_unique
indexes, job_source_unique = await connection.run_sync(collect) indexes, job_source_unique, document_tag_unique = await connection.run_sync(collect)
assert ["status", "date_created"] in indexes["job"] assert ["status", "date_created"] in indexes["job"]
assert ["document_id"] in indexes["job"] assert ["document_id"] in indexes["job"]
@@ -150,6 +156,9 @@ async def test_create_all_declares_hot_path_indexes(tmp_path):
assert ["document_type_id"] in indexes["document"] assert ["document_type_id"] in indexes["document"]
for column in ("document_id", "person_id", "role_id"): for column in ("document_id", "person_id", "role_id"):
assert [column] in indexes["document_person"] assert [column] in indexes["document_person"]
assert ["document_id"] in indexes["document_tag"]
assert ["tag_id"] in indexes["document_tag"]
assert ["document_id", "tag_id"] in document_tag_unique
finally: finally:
await dispose_database_runtime() await dispose_database_runtime()
+116
View File
@@ -0,0 +1,116 @@
"""Integrity checks for document/source filesystem-to-database reconciliation."""
from __future__ import annotations
from pathlib import Path
from uuid import UUID
from uuid import uuid4
import pytest
from sqlmodel import func
from sqlmodel import select
from transcription.config import Settings
from transcription.db import session_scope
from transcription.db.models import Document
from transcription.db.models import Source
def _document_folder_ids(root: Path) -> set[str]:
documents_root = root / "documents"
if not documents_root.exists():
return set()
return {entry.name for entry in documents_root.iterdir() if entry.is_dir()}
async def _document_ids(settings: Settings) -> set[str]:
async with session_scope(settings=settings) as session:
rows = await session.exec(select(Document.id))
return {str(item) for item in rows.all()}
async def _source_counts_by_document(settings: Settings) -> dict[str, int]:
async with session_scope(settings=settings) as session:
rows = await session.exec(
select(Source.document_id, func.count(Source.id)).group_by(Source.document_id)
)
return {str(document_id): int(count) for document_id, count in rows}
def _source_file_count_for_document(root: Path, document_id: str) -> int:
directory = root / "documents" / document_id
if not directory.exists():
return 0
return sum(1 for entry in directory.iterdir() if entry.is_file())
async def assert_storage_reconciliation(*, upload_dir: Path, settings: Settings) -> None:
folder_ids = _document_folder_ids(upload_dir)
doc_ids = await _document_ids(settings)
missing_in_table = sorted(folder_ids - doc_ids)
missing_in_folders = sorted(doc_ids - folder_ids)
if missing_in_table:
folder = missing_in_table[0]
raise AssertionError(
"Document directory count and document.doc_id count do not agree. "
f"./data/documents/{folder} does not appear in document table"
)
if missing_in_folders:
doc_id = missing_in_folders[0]
raise AssertionError(
"Document directory count and document.doc_id count do not agree. "
f"document.doc_id {doc_id} has no corresponding folder in ./data/documents"
)
source_counts = await _source_counts_by_document(settings)
for document_id in sorted(doc_ids):
db_count = source_counts.get(document_id, 0)
file_count = _source_file_count_for_document(upload_dir, document_id)
if file_count > db_count:
raise AssertionError(
"Source file count and source.source_id count do not agree. "
f"[UPLOAD_DIR]/documents/{document_id}/ contains file(s) with no source row"
)
if db_count > file_count:
raise AssertionError(
"Source file count and source.source_id count do not agree. "
f"source.source_id rows exist without files in [UPLOAD_DIR]/documents/{document_id}"
)
@pytest.mark.asyncio
async def test_storage_reconciliation_passes_for_matching_counts(tmp_path, default_settings: Settings):
upload_dir = tmp_path / "uploads"
document_id = str(uuid4())
document_uuid = UUID(document_id)
file_name = f"{uuid4()}.png"
(upload_dir / "documents" / document_id).mkdir(parents=True, exist_ok=True)
(upload_dir / "documents" / document_id / file_name).write_bytes(b"ok")
async with session_scope(settings=default_settings) as session:
session.add(Document(id=document_uuid, name="Doc"))
session.add(
Source(
document_id=document_uuid,
page_number=1,
upload_name=file_name,
filename=file_name,
file_path=f"documents/{document_id}/{file_name}",
file_hash="a" * 64,
file_size_bytes=2,
)
)
await session.commit()
await assert_storage_reconciliation(upload_dir=upload_dir, settings=default_settings)
@pytest.mark.asyncio
async def test_storage_reconciliation_reports_actionable_mismatch_message(tmp_path, default_settings: Settings):
upload_dir = tmp_path / "uploads"
orphan_dir = upload_dir / "documents" / str(uuid4())
orphan_dir.mkdir(parents=True, exist_ok=True)
with pytest.raises(AssertionError, match="document\\.doc_id count do not agree"):
await assert_storage_reconciliation(upload_dir=upload_dir, settings=default_settings)
+4
View File
@@ -23,6 +23,7 @@ from transcription.db import session as db_session_module
from transcription.db import session_scope from transcription.db import session_scope
from transcription.db.models import Document from transcription.db.models import Document
from transcription.db.models import DocumentPerson from transcription.db.models import DocumentPerson
from transcription.db.models import DocumentTag
from transcription.db.models import ExecutionAttempt from transcription.db.models import ExecutionAttempt
from transcription.db.models import Job from transcription.db.models import Job
from transcription.db.models import JobSource from transcription.db.models import JobSource
@@ -30,6 +31,7 @@ from transcription.db.models import JobSourceStatus
from transcription.db.models import JobStatus from transcription.db.models import JobStatus
from transcription.db.models import Person from transcription.db.models import Person
from transcription.db.models import Source from transcription.db.models import Source
from transcription.db.models import Tag
@pytest.fixture(scope="session") @pytest.fixture(scope="session")
@@ -73,11 +75,13 @@ async def clear_ui_database(
) )
async with session_scope(session_factory=app.state.runtime.session_factory) as session: async with session_scope(session_factory=app.state.runtime.session_factory) as session:
await session.exec(delete(JobSource)) await session.exec(delete(JobSource))
await session.exec(delete(DocumentTag))
await session.exec(delete(DocumentPerson)) await session.exec(delete(DocumentPerson))
await session.exec(delete(Source)) await session.exec(delete(Source))
await session.exec(delete(Job)) await session.exec(delete(Job))
await session.exec(delete(Document)) await session.exec(delete(Document))
await session.exec(delete(Person)) await session.exec(delete(Person))
await session.exec(delete(Tag))
await session.commit() await session.commit()
+11 -1
View File
@@ -14,6 +14,7 @@ from transcription.db.models import Job
from transcription.db.models import Person from transcription.db.models import Person
from transcription.db.models import PersonRole from transcription.db.models import PersonRole
from transcription.db.models import Source from transcription.db.models import Source
from transcription.ui.pages.documents_page import _resolve_selected_tag_labels
# --- Helper Fixtures --- # --- Helper Fixtures ---
@@ -81,9 +82,10 @@ class TestDocumentsPageRendering:
assert response.status_code == 200 assert response.status_code == 200
assert "1924 Postcard" in response.text assert "1924 Postcard" in response.text
assert "Postcard" in response.text assert "Postcard" in response.text
assert "PC-001" in response.text
assert "Document Date" in response.text assert "Document Date" in response.text
assert "Author" in response.text assert "Author" in response.text
assert "# Sources" in response.text
assert "Archive Ref" not in response.text
def test_document_create_page_renders_form(self, app_client): def test_document_create_page_renders_form(self, app_client):
_, client = app_client _, client = app_client
@@ -95,6 +97,7 @@ class TestDocumentsPageRendering:
assert "Document name" in response.text assert "Document name" in response.text
assert "Linked People" in response.text assert "Linked People" in response.text
assert "Document type" in response.text assert "Document type" in response.text
assert "Tags" in response.text
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_document_create_page_preselects_person_with_disambiguating_label(self, app_client): async def test_document_create_page_preselects_person_with_disambiguating_label(self, app_client):
@@ -209,3 +212,10 @@ class TestDocumentsPageRendering:
assert "Delete Document" in response.text assert "Delete Document" in response.text
assert "Delete document permanently" in response.text assert "Delete document permanently" in response.text
assert "Delete is blocked" not in response.text assert "Delete is blocked" not in response.text
def test_resolve_selected_tag_labels_handles_multiple_payload_shapes():
assert _resolve_selected_tag_labels("Family") == ["Family"]
assert _resolve_selected_tag_labels(["Family", "Research"]) == ["Family", "Research"]
assert _resolve_selected_tag_labels([{"label": "Family"}, {"value": "Research"}]) == ["Family", "Research"]
assert set(_resolve_selected_tag_labels({"value": {"Family", "Research"}})) == {"Family", "Research"}
+3
View File
@@ -55,6 +55,7 @@ class TestJobsPageRendering:
assert response.status_code == 200 assert response.status_code == 200
assert "Document Name" in response.text assert "Document Name" in response.text
assert "# Sources" in response.text
assert "Source Filename" not in response.text assert "Source Filename" not in response.text
assert "Updated" in response.text assert "Updated" in response.text
assert "Created" not in response.text assert "Created" not in response.text
@@ -112,6 +113,8 @@ class TestJobsPageRendering:
assert response.status_code == 200 assert response.status_code == 200
assert "Create Processing Job" in response.text assert "Create Processing Job" in response.text
assert "Preselected Journal Entry" in response.text assert "Preselected Journal Entry" in response.text
assert "Provider" in response.text
assert "Model" in response.text
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_job_detail_page_renders_logistics_and_links(self, app_client, seed_document_with_unlinked_job): async def test_job_detail_page_renders_logistics_and_links(self, app_client, seed_document_with_unlinked_job):
+1
View File
@@ -28,6 +28,7 @@ class TestNavigationAndMounts:
"/ui/homepage", "/ui/homepage",
"/ui/homepage/edit", "/ui/homepage/edit",
"/ui/documents", "/ui/documents",
"/ui/tags",
"/ui/people", "/ui/people",
"/ui/sources", "/ui/sources",
"/ui/jobs", "/ui/jobs",
+3
View File
@@ -16,6 +16,7 @@ class TestPageRegistration:
people_response = client.get("/ui/people") people_response = client.get("/ui/people")
sources_response = client.get("/ui/sources") sources_response = client.get("/ui/sources")
jobs_response = client.get("/ui/jobs") jobs_response = client.get("/ui/jobs")
tags_response = client.get("/ui/tags")
settings_response = client.get("/ui/settings") settings_response = client.get("/ui/settings")
assert homepage_response.status_code == 200 assert homepage_response.status_code == 200
@@ -23,9 +24,11 @@ class TestPageRegistration:
assert people_response.status_code == 200 assert people_response.status_code == 200
assert sources_response.status_code == 200 assert sources_response.status_code == 200
assert jobs_response.status_code == 200 assert jobs_response.status_code == 200
assert tags_response.status_code == 200
assert settings_response.status_code == 200 assert settings_response.status_code == 200
assert "Document Types" in settings_response.text assert "Document Types" in settings_response.text
assert "Person Roles" in settings_response.text assert "Person Roles" in settings_response.text
assert "Tags" in settings_response.text
assert "Prompts" in settings_response.text assert "Prompts" in settings_response.text
assert "Home Page Text" in settings_response.text assert "Home Page Text" in settings_response.text
assert "README.md" not in settings_response.text assert "README.md" not in settings_response.text
+22 -1
View File
@@ -41,7 +41,28 @@ class TestPeoplePageRendering:
assert response.status_code == 200 assert response.status_code == 200
assert "Ada Lovelace" in response.text assert "Ada Lovelace" in response.text
assert "Ada" in response.text assert "FamilySearch ID" in response.text
assert "# Documents" in response.text
assert "Display Name" not in response.text
assert "Maiden Name" not in response.text
@pytest.mark.asyncio
async def test_people_page_shows_document_counts(self, app_client):
_, client = app_client
async with session_scope() as session:
role = (await session.exec(select(PersonRole).where(PersonRole.semantic_key == "author"))).one()
person = Person(full_name="Counted Person")
document = Document(name="Linked For Count")
session.add_all([person, document])
await session.flush()
session.add(DocumentPerson(document_id=document.id, person_id=person.id, role_id=role.id))
await session.commit()
response = client.get("/ui/people")
assert response.status_code == 200
assert '"document_count":1' in response.text
def test_person_create_page_renders_fields(self, app_client): def test_person_create_page_renders_fields(self, app_client):
_, client = app_client _, client = app_client
+1 -2
View File
@@ -261,9 +261,8 @@ class TestSourcesPageRendering:
assert "SOURCE PAGE 1: DETAIL-SOURCE.PNG" in response.text.upper() assert "SOURCE PAGE 1: DETAIL-SOURCE.PNG" in response.text.upper()
assert "SOURCE METADATA" in response.text.upper() assert "SOURCE METADATA" in response.text.upper()
assert "SOURCEJOB METADATA" in response.text.upper() assert "SOURCEJOB METADATA" in response.text.upper()
assert "TRANSCRIPTION TEXT" in response.text.upper() assert "TRANSCRIPTION TEXT" not in response.text.upper()
assert "EDITABLE REVISION" in response.text.upper() assert "EDITABLE REVISION" in response.text.upper()
assert "original transcription text" in response.text
assert "human revision text" in response.text assert "human revision text" in response.text
assert "Save revision" in response.text assert "Save revision" in response.text
assert "Previous Page" in response.text assert "Previous Page" in response.text
+43
View File
@@ -0,0 +1,43 @@
"""Tests for the tags page route and grouped filtering behavior."""
import pytest
from transcription.db import session_scope
from transcription.db.models import Document
from transcription.services.documents import DocumentService
@pytest.mark.integration
class TestTagsPageRendering:
def test_tags_page_renders_empty_state_without_tags(self, app_client):
_, client = app_client
response = client.get("/ui/tags")
assert response.status_code == 200
assert "Tags" in response.text
assert "No tags are configured yet." in response.text
@pytest.mark.asyncio
async def test_tags_page_groups_documents_by_tag(self, app_client):
app, client = app_client
documents = DocumentService(session_factory=app.state.runtime.session_factory)
async with session_scope(session_factory=app.state.runtime.session_factory) as session:
first = Document(name="Tagged Letter")
second = Document(name="Tagged Journal")
session.add_all([first, second])
await session.flush()
await documents.sync_document_tags_by_labels(document_id=first.id, labels=["Family"], session=session)
await documents.sync_document_tags_by_labels(
document_id=second.id,
labels=["Family", "Research"],
session=session,
)
await session.commit()
response = client.get("/ui/tags")
assert response.status_code == 200
assert "Filter by tag" in response.text
assert "Tags" in response.text