Step 6 implemented

docs/step5-results.md

@@ -0,0 +1,70 @@
+Step 5 is implemented and passing ✅
+
+### What I implemented
+
+#### App/bootstrap
+- Added `src/transcription/app.py`:
+  - `create_app()` factory
+  - FastAPI lifespan startup/shutdown:
+    - `setup_logging()`
+    - `create_all()`
+    - ensure `upload_dir` and `prompt_dir`
+    - start/stop worker thread (`run_worker_loop`)
+  - includes health router
+  - registers NiceGUI pages
+
+#### API
+- Added `src/transcription/api/__init__.py`
+- Added `src/transcription/api/health.py`:
+  - `GET /healthz` returns `{"status": "ok"}`
+
+#### UI
+- Added `src/transcription/ui/upload_page.py`:
+  - upload page route
+  - accepted extensions helper
+  - upload submit helper to Step 4 service (`create_upload_job`)
+  - success/error state messaging
+- Added `src/transcription/ui/jobs_page.py`:
+  - jobs list page
+  - job detail page
+  - `fetch_jobs()` and `fetch_job_detail()` helpers
+- Updated `src/transcription/ui/__init__.py`:
+  - explicit page registration
+  - NiceGUI mounted via `ui.run_with(app, mount_path="/ui", ...)`
+
+#### Dependencies
+- Added runtime dependencies:
+  - `fastapi`
+  - `nicegui`
+
+---
+
+### Tests created and filled (scaffold → fill)
+
+- `tests/test_app.py`
+- `tests/api/test_health.py`
+- `tests/ui/test_pages_registration.py`
+- `tests/ui/test_upload_page.py`
+- `tests/ui/test_jobs_page.py`
+
+All scaffolded method names/docstrings were preserved and filled with executable assertions.
+
+---
+
+### Validation results (strict sequence)
+
+- `uv run pytest --collect-only -q` ✅
+- `uv run pytest -m unit -q` ✅
+- `uv run pytest tests/api/test_health.py -q` ✅
+- `uv run pytest tests/ui/test_pages_registration.py -q` ✅
+- `uv run pytest tests/test_app.py -q` ✅
+- `uv run pytest tests/ui/test_upload_page.py -q` ✅
+- `uv run pytest tests/ui/test_jobs_page.py -q` ✅
+- `uv run pytest -q` ✅
+
+Full suite is green.
+
+---
+
+### Note
+You’ll see one warning from FastAPI/Starlette test client about `httpx` deprecation; it does not affect correctness and all tests pass.

docs/step6-results.md

@@ -0,0 +1,50 @@
+Step 6 implemented ✅
+
+### Files added
+- `tests/integration/test_pipeline_flow.py`
+- `tests/test_traceability.py`
+
+### What was implemented
+
+#### 1) End-to-end pipeline integration coverage
+`tests/integration/test_pipeline_flow.py` now verifies:
+
+- **Success path**
+  - upload creates queued job
+  - worker processes job
+  - job becomes `transcribed`
+  - transcript text is persisted
+  - error detail remains `None`
+
+- **Failure path**
+  - upload creates queued job
+  - worker processing failure is handled
+  - job becomes `failed`
+  - transcript has `text=None`
+  - `error_detail` contains provider failure reason
+
+#### 2) MVP requirement traceability check
+`tests/test_traceability.py` adds a concrete map:
+
+- REQ-0, REQ-1, REQ-2, REQ-3, REQ-4, REQ-5, REQ-6, REQ-8, REQ-12
+- each REQ maps to one or more existing test files
+- test asserts each mapped path exists (guards against drift)
+
+---
+
+### Validation run results
+
+Executed and passed:
+
+1. `uv run pytest --collect-only -q` ✅  
+2. `uv run pytest -m unit -q` ✅  
+3. `uv run pytest -m integration -q` ✅  
+4. `uv run pytest -m "not external" -q` ✅  
+5. `uv run pytest tests/integration/test_pipeline_flow.py -q` ✅  
+6. `uv run pytest tests/ui/test_upload_page.py -q` ✅  
+7. `uv run pytest tests/ui/test_jobs_page.py -q` ✅  
+8. `uv run pytest -q` ✅  
+
+All green.  
+(Only existing non-blocking FastAPI TestClient deprecation warning remains.)
+

tests/artifacts/transcriptions/Book_Two_-_page_02.jpg.txt

@@ -11,11 +11,11 @@ BOOK 1 had 54 pages; 14 chapters. BOOK 2 has 70 pages; 18 chapters. BOOK 1 con-
 sisted largely of first generation family history. BOOK 2 throws more light on
 the second generation. Sidney promises a BOOK 3 and that may begin to do justice
 to the third generation. We suggest that Sidney get the help of Louis Shinn
-who has a chapter in this book (Chapter 16 - The Last 25 Years on the Doumecq
+who has a chapter in this book (Chapter 16 - The Last 25 Years on the Doumeeq
 Plains. Louis has the gift of seeing, recalling and telling. One sentence in
-his chapter gives a great tribute to the Doumecqers--so far as he knows no one
+his chapter gives a great tribute to the Doumeeqers--so far as he knows no one
-on the Doumecq Plains went on relief during the depression. That in a nutshell
+on the Doumeeq Plains went on relief during the depression. That in a nutshell
-shows the sturdy character of the residents of the Doumecq Plains.
+shows the sturdy character of the residents of the Doumeeq Plains.
 
 We promised in BOOK 1 that in BOOK 2 we would give the story of the trip of
 John E. Cochran and wife to Tennessee, Cuba and the Panama Canal. You will see
@@ -24,7 +24,7 @@ trips. Those chapters are worth reading and re-reading. Mr. Cochran has eyes
 to see and a pen to tell. We think the people in Tennessee will read with
 great pleasure the comments he makes on conditions today.
 
-Some who get this book will consider the group picture the best thing in the
+بعضome who get this book will consider the group picture the best thing in the
 book. It took a lot of preliminary photographing to reduce some pictures, enlarge
 others and bring out the tin types. We wish that instead of 44 faces we could
 have given 88. Do not blame Ethel Cochran-Shinn for the selection. She furnished
@@ -32,7 +32,7 @@ enough pictures but we had to take only part of them. We think there are great
 possibilities in reproducing old pictures. We wish we had a Pickard group. Some
 Pickard descendant may wish to make a collection.
 
-We are much impressed with the future possibilities of getting a complete geneol-
+We are much impressed with the future possibilities of getting a complete geneal-
 ogy of the Pickard family. Mr. Cochran has a fine chapter on the Pickards but
 to date we have not had the pleasure of finding all of the family dates. We had
 intended to give more family data in this book but it takes time to get the

tests/artifacts/transcriptions/Omie_Writes_Home.pdf.txt

@@ -5,7 +5,7 @@ model: google/gemini-2.5-flash
 JOHN E. COCHRAN
 FAMILY ASSOCIATION
 Family Only
-Home | Sibling's Stories | 1st Cousins | Ancestor's Stories | Reunion History | Next Reunion | JECMEF
+Home | Sibling's Stories | 1st Cousins [sic] | Ancestor's Stories | Reunion History | Next Reunion | JECMEF
 OMIE WRITES HOME
 Ed. Note: The following letter was written by Omie Cochran in Nome, Alaska and sent to her
 sister, Ethel Shinn, in Canfield, Idaho in 1923. It has been stored away these 63 years in the
@@ -23,9 +23,10 @@ and Polly sit up and listen and that little black
 rascal of yours would fairly sparkle with
 listening. Can't I see him listening now to all the
 yarns we told last summer?
+[photo of people with dogsleds and ice]
 You see, we-Miss Saville and I, took a trip north
 on the Buford and it was very interesting. We
-went north thru the Bering Strait into the Arctic and as far as the Ice Pack. There the captain
+went north thru [sic] the Bering Strait into the Arctic and as far as the Ice Pack. There the captain
 of our craft and some other mighty hunters went out first in kayaks and later in row boats and
 shot seven walrus. When they also took a movie man and camera, so you will likely see all
 this in the movies before I get to tell you. They came back on board and the ship went up
@@ -35,14 +36,14 @@ over 12 feet long, as big around as three cows, had no feet but toenails on thei
 flappers, no head but their body just suddenly ended with a hole for a mouth and big bristles
 
 all around it similar to a currycomb in coarseness; no ears but huge tusks of ivory. They are
-the most repulsive looking animals imaginable and tho I have always read about them I never
+the most repulsive looking animals imaginable and tho [sic] I have always read about them I never
 expect such disagreeable looking creatures. They had a rough brown hairy skin and some of
 them looked warty. They must have weighed two ton at least. Ere we got them back to Nome
 to the natives they were getting extremely odiferous—in fact, you could scarcely stay on the
 ship with any degree of comfort unless you had per chance lost your sense of smell.
-Then we went north to a few minutes beyond the 70th degree of latitude and thot for awhile
+Then we went north to a few minutes beyond the 70th degree of latitude and thot [sic] for awhile
-we would go to Wrangell Island where some men from Steffonsons ship were supposed to be
+we would go to Wrangell Island where some men from Steffonsons [sic] ship were supposed to be
-stranded but we didn't get there and instead stopped at a small native village at Cape Serdz in
+stranded but we didn't get there and instead stopped at a small native village at Cape Serdz [sic] in
 Siberia. These Eskimo were very primitive. One white squaw man lived there and had for 23
 years. He was a Swede--who else could. Their houses were circular and built up with dirt 2 or
 3 feet and then skins were stretched over it and weighted down with rocks. Inside, the room
@@ -56,21 +57,21 @@ We went ashore in 2 life boats and a launch pulling them. On the launch was a si
 playing and the rear of the last life boat was the movie man. 'Twas very thrilling.
 The other place we stopped was at Whalen, a trading post in Siberia. There these 'towerists'
 went wild. They rushed helter-skelter, hither and thither, here and there, trying to find
-something to buy. Prices raised right before your eyes. One would but something for $1.00
+something to buy. Prices raised right before your eyes. One would but [sic] something for $1.00
 and the next might have to pay $2.00, $4.00 or $10.00. That made no difference. They had to
-have it. One man I was sort of taking care of, tho he had his son along for the purpose,
+have it. One man I was sort of taking care of, tho [sic] he had his son along for the purpose,
 bought 2 ivory tusks, 1 pup, 2 moccasins, 3 or 4 billikens, 6 or 8 ivory and silver rings, one
 fishing line, hooks, floats, etc. and two bird slings. The slings have rocks at the end and the
 little natives throw them at the flocks of geese and ducks which fly close over the village and
 the slings entangle their wings and legs, sometimes more than one, and they can't fly. They
-come down and the natives capture them. There was more junk brot aboard than baggage, I
+come down and the natives capture them. There was more junk brot [sic] aboard than baggage, I
 do believe. And they say that at the first stop it was worse than here. The red flag was flying
 over Whalen and the Russian soldiers were there—a few, one or two or three, I forget the
 number.
 We got home yesterday morning at 5 a.m. but missed the first lighter in so had to stay out
 until 2:30. The girls had prepared a big meal for us and invited up the Hartfords and then let
 us talk. Miss Saville talked quite a bit. Any how if you folks don't like this I don't care, it is
-all I had to write about and I know Buster'ud listen anyway and I'd soak ole Peter's head if he
+all I had to write about and I know Buster'ud [sic] listen anyway and I'd soak ole Peter's head if he
 didn't and Polly would in my lap and I don't know much about the youngest one of yours so
 likely he would be squawling. But we did surely enjoy our trip and were gone just long enuf.
 
@@ -88,13 +89,12 @@ am rather afraid he doesn't know much. I quite remember your little timid Mauric
 he shifted his affection from his Aunt Om and yelled and howled and screamed steadily for a
 week while his mother went to S.S. Ask him is he recalls this little interview. Do you suppose
 he does?
-Ever hear from Zen? or Inez? They don't seem to be very writing inclined tho Zen has done
+Ever hear from Zen? or Inez? They don't seem to be very writing inclined tho [sic] Zen has done
 well this year. Even sent me a telegram a few weeks ago. Well, if anything else ever happens,
-I'll write again. Don't suppose it ever will, tho.
+I'll write again. Don't suppose it ever will, tho [sic].
 Lots of love to all,
 Ome
 Reprinted from Cochran Chronicles, Volume 9, Number 1, November 1986
 © JECFA 1986
 Up
-
 jecochranclan.org ~ Contact webmaster

tests/artifacts/transcriptions/Rod_Moser_Letter_-_p1.jpg.txt

@@ -2,29 +2,31 @@ source: Rod Moser Letter - p1.jpg
 provider: openrouter
 model: google/gemini-2.5-flash
 ---
-JOHN ISBILL R.T. MOSER
+JOHN ISBILL
+
 ISBILL & MOSER
 DEALERS IN
 GENERAL MERCHANDISE
 
-Vonore, Tenn. January 27- 1913-
+R. T. MOSER
 
-Dear Much Aunt Aduian
+Vonore, Tenn. January 27- 1913
-How are hour [sic] a
+Dear [Aunt?] Louisa
-few nights ago r [and] said a
+I was not home a
-letter from your folks, So
+few nights ago & I received a
+letter from your folks, so
 I decided to write you
-a few lines myself ok
+a few lines myself &
-I am contemplate [sic] a
+I am contemplating a
 trip out west next summer
-r [and] want one of Aldons to go
+& I want some of others to go
-where I am.
+when I am [there?]
-We are getting
+I am getting
-up in years r [and] remarrie[d]
+up in years & unmarried
-So you are the object of
+so you see the object of
 my trip, is to get a wife
-to put in any old maid
+& there is a lot of old maid
-or widows out there, I
+& widows out there. I
 want you to kiss them
-at my [illegible] or at them
+at my [hand?] for me at [their?]
-as soon as I get them
+as soon as I get there

tests/integration/test_pipeline_flow.py

@@ -0,0 +1,74 @@
+"""Integration tests for end-to-end upload and worker pipeline behavior."""
+
+from pathlib import Path
+
+import pytest
+from sqlmodel import select
+
+from transcription.config import Settings
+from transcription.models import Job, JobStatus, Transcript
+from transcription.providers.base import TranscriptionResult
+from transcription.services.upload import create_upload_job
+from transcription.worker import process_next_queued_job
+
+
+@pytest.mark.integration
+class TestPipelineSuccessFlow:
+    """Verify end-to-end success lifecycle behavior."""
+
+    def test_upload_then_worker_persists_transcribed_terminal_state(self, session, tmp_path: Path, monkeypatch):
+        """Upload followed by worker processing persists transcript and transcribed status."""
+        settings = Settings(openrouter_api_key="test-key", upload_dir=tmp_path)
+        upload_result = create_upload_job(
+            filename="pipeline.jpg",
+            file_bytes=b"pipeline-bytes",
+            session=session,
+            settings=settings,
+        )
+
+        def _fake_transcribe(_path: str) -> TranscriptionResult:
+            return TranscriptionResult(text="Pipeline transcript", provider="openrouter", model="test-model")
+
+        monkeypatch.setattr("transcription.worker.transcribe_document_image", _fake_transcribe)
+
+        processed = process_next_queued_job(session=session)
+        job = session.get(Job, upload_result.job_id)
+        transcript = session.exec(select(Transcript).where(Transcript.job_id == upload_result.job_id)).first()
+
+        assert processed is True
+        assert job is not None
+        assert job.status == JobStatus.TRANSCRIBED
+        assert transcript is not None
+        assert transcript.text == "Pipeline transcript"
+        assert transcript.error_detail is None
+
+
+@pytest.mark.integration
+class TestPipelineFailureFlow:
+    """Verify end-to-end failure lifecycle behavior."""
+
+    def test_upload_then_worker_persists_failed_terminal_state(self, session, tmp_path: Path, monkeypatch):
+        """Upload followed by worker processing persists error detail and failed status."""
+        settings = Settings(openrouter_api_key="test-key", upload_dir=tmp_path)
+        upload_result = create_upload_job(
+            filename="pipeline.jpg",
+            file_bytes=b"pipeline-bytes",
+            session=session,
+            settings=settings,
+        )
+
+        def _fake_transcribe(_path: str) -> TranscriptionResult:
+            raise RuntimeError("pipeline provider failure")
+
+        monkeypatch.setattr("transcription.worker.transcribe_document_image", _fake_transcribe)
+
+        processed = process_next_queued_job(session=session)
+        job = session.get(Job, upload_result.job_id)
+        transcript = session.exec(select(Transcript).where(Transcript.job_id == upload_result.job_id)).first()
+
+        assert processed is True
+        assert job is not None
+        assert job.status == JobStatus.FAILED
+        assert transcript is not None
+        assert transcript.text is None
+        assert "pipeline provider failure" in transcript.error_detail

tests/test_traceability.py

@@ -0,0 +1,61 @@
+"""Requirement-to-test traceability checks for MVP in-scope requirements."""
+
+from pathlib import Path
+
+import pytest
+
+MVP_REQUIREMENT_TEST_MAP: dict[str, list[str]] = {
+    "REQ-0": [
+        "tests/integration/test_pipeline_flow.py",
+    ],
+    "REQ-1": [
+        "tests/services/test_upload.py",
+        "tests/ui/test_upload_page.py",
+    ],
+    "REQ-2": [
+        "tests/services/test_worker.py",
+        "tests/integration/test_pipeline_flow.py",
+    ],
+    "REQ-3": [
+        "tests/services/test_worker.py",
+        "tests/ui/test_jobs_page.py",
+    ],
+    "REQ-4": [
+        "tests/services/test_worker.py",
+        "tests/integration/test_pipeline_flow.py",
+    ],
+    "REQ-5": [
+        "tests/ui/test_jobs_page.py",
+        "tests/ui/test_pages_registration.py",
+    ],
+    "REQ-6": [
+        "tests/test_app.py",
+        "tests/services/test_worker.py",
+    ],
+    "REQ-8": [
+        "tests/test_app.py",
+        "tests/test_config.py",
+    ],
+    "REQ-12": [
+        "tests/test_prompts.py",
+        "tests/services/test_transcription.py",
+    ],
+}
+
+
+@pytest.mark.integration
+class TestMvpRequirementTraceability:
+    """Verify MVP in-scope requirements are mapped to executable tests."""
+
+    def test_mvp_requirements_have_mapped_test_targets(self):
+        """Each MVP in-scope REQ id maps to at least one existing test path."""
+        project_root = Path(__file__).resolve().parents[1]
+
+        assert MVP_REQUIREMENT_TEST_MAP
+        for requirement_id, mapped_tests in MVP_REQUIREMENT_TEST_MAP.items():
+            assert requirement_id.startswith("REQ-")
+            assert mapped_tests, f"No mapped tests for {requirement_id}"
+
+            for relative_path in mapped_tests:
+                test_path = project_root / relative_path
+                assert test_path.exists(), f"Missing mapped test file for {requirement_id}: {relative_path}"