diff --git a/docs/step4.md b/docs/step4.md index 20810d2..45a0aa9 100644 --- a/docs/step4.md +++ b/docs/step4.md @@ -1,18 +1,18 @@ ## Step 4: `services/upload.py` + `worker.py` -## Objective +### Objective -Implement the MVP upload and background processing pipeline so the system can: +Implement the MVP upload and background-processing pipeline so the system can: 1. Save uploaded files into `UPLOAD_DIR` 2. Create `Document` + `Job(status="queued")` 3. Process queued jobs in a worker loop: - `queued -> processing` - - invoke Step 3 transcription + - call Step 3 transcription service - persist `Transcript` - finalize as `transcribed` or `failed` -This covers MVP Feature 1 + 2 and supports REQ-1, REQ-2, REQ-3, REQ-4, REQ-6. +This step advances MVP Feature 1 + Feature 2 and supports REQ-1, REQ-2, REQ-3, REQ-4, REQ-6. --- @@ -21,15 +21,15 @@ This covers MVP Feature 1 + 2 and supports REQ-1, REQ-2, REQ-3, REQ-4, REQ-6. ### In scope - `src/transcription/services/upload.py` - `src/transcription/worker.py` -- Upload persistence logic + job creation -- Worker polling and single-job processing lifecycle -- Tests for upload + worker behavior (deterministic, no live API) +- Upload persistence logic and initial job creation +- Worker polling and single-job lifecycle execution +- Deterministic test coverage for upload + worker (default suite) ### Out of scope -- UI wiring (Step 5) -- Job/transcript pages (Step 5) -- External queue infrastructure -- Async DB/session architecture changes (post-MVP) +- UI integration and pages (Step 5) +- Queue infrastructure beyond in-process loop +- Async DB/session architecture refactor +- Broad production hardening beyond MVP needs --- @@ -40,151 +40,223 @@ This covers MVP Feature 1 + 2 and supports REQ-1, REQ-2, REQ-3, REQ-4, REQ-6. - `src/transcription/worker.py` - `src/transcription/services/__init__.py` (export updates as needed) -### Tests +### Test files - `tests/services/test_upload.py` -- `tests/services/test_worker.py` *(recommended for mirror consistency)* +- `tests/services/test_worker.py` + +### Optional external lane (already present pattern) +- reuse `external` marker for live-provider checks where appropriate +- keep external out of default lane + +--- + +## Required MCP Prompt References (for test workflow) + +Apply these resources directly during Step 4 test creation: + +1. `resource://catalog/prompts/pytest-scaffold` +2. `resource://prompts/pytest-scaffold/document` +3. `resource://catalog/prompts/pytest-fill-scaffold` +4. `resource://prompts/pytest-fill-scaffold/document` + +And (as referenced by those prompts) apply relevant pytest skill references for: +- naming/hierarchy +- marker defaults +- SQLAlchemy sync testing behavior where applicable --- ## Design Decisions -1. **Upload service owns file persistence + initial DB records** - - Save file first, then create `Document` + `Job(queued)`. +1. **Upload service owns initial file + record creation** + - Writes file, creates `Document`, creates queued `Job`, returns IDs/path. -2. **Worker owns job lifecycle transitions** - - Transition only in worker: - - `queued -> processing` - - `processing -> transcribed|failed` +2. **Worker owns lifecycle transitions** + - Worker is the single owner of `queued -> processing -> terminal` job state changes. -3. **Worker calls Step 3 service, not provider SDK directly** - - Use `transcribe_document_image(...)` from `services/transcription.py`. +3. **Worker uses Step 3 service boundary** + - Worker calls `transcribe_document_image(...)`; no provider-specific SDK logic in worker. -4. **Failure details are always persisted** - - On failure, persist transcript row with `text=None`, `error_detail=...`. +4. **Failure information is always persisted** + - On failure: store `Transcript(text=None, error_detail=...)` and set `Job.status=failed`. -5. **Worker loop is stoppable** - - Simple in-process loop with stop signal/event and poll interval. +5. **Loop remains simple and stoppable** + - In-process polling loop with stop event and poll interval for MVP simplicity and testability. --- ## Task-by-Task Execution Checklist -## Phase A — Upload service (`services/upload.py`) +## Phase A — Implement upload service (`src/transcription/services/upload.py`) -- [ ] Create `src/transcription/services/upload.py` -- [ ] Add `UploadError` exception type -- [ ] Add result dataclass (e.g., `UploadJobResult`) with: +- [ ] Create `UploadError` exception +- [ ] Create `UploadJobResult` dataclass with: - [ ] `document_id` - [ ] `job_id` - [ ] `stored_path` - [ ] `original_filename` -- [ ] Implement safe filename handling: - - [ ] basename normalization - - [ ] collision-safe naming (e.g., UUID prefix) - - [ ] reject path traversal patterns -- [ ] Implement extension validation (`.jpg/.jpeg/.png/.tif/.tiff/.pdf`) -- [ ] Implement empty-bytes validation -- [ ] Ensure `UPLOAD_DIR` exists (`mkdir(parents=True, exist_ok=True)`) -- [ ] Persist file bytes to disk -- [ ] Insert `Document` and `Job(status=queued)` in one transaction -- [ ] Return structured `UploadJobResult` -- [ ] Add logging for upload success/failure (no sensitive payload logging) +- [ ] Add filename safety handling: + - [ ] normalize to basename + - [ ] avoid path traversal + - [ ] collision-safe stored name (e.g., UUID prefix/suffix) +- [ ] Validate upload payload: + - [ ] non-empty bytes required + - [ ] extension in supported set (`.jpg/.jpeg/.png/.tif/.tiff/.pdf`) +- [ ] Ensure upload directory exists (`mkdir(parents=True, exist_ok=True)`) +- [ ] Write file bytes to `UPLOAD_DIR` +- [ ] Persist DB records in one transaction: + - [ ] `Document(filename, file_path)` + - [ ] `Job(document_id=..., status=queued)` +- [ ] Return `UploadJobResult` +- [ ] Add logging for success/failure boundaries --- -## Phase B — Worker core (`worker.py`) +## Phase B — Implement worker core (`src/transcription/worker.py`) -- [ ] Create `src/transcription/worker.py` -- [ ] Implement `process_next_queued_job(...) -> bool` - - [ ] fetch oldest queued job - - [ ] return `False` if none found - - [ ] set job to `processing` + update timestamp - - [ ] call `transcribe_document_image(image_path=...)` - - [ ] success path: persist transcript text, mark `transcribed` - - [ ] failure path: persist error detail, mark `failed` - - [ ] return `True` when a job is processed -- [ ] Implement transcript upsert behavior (avoid duplicate unique `job_id` insert issues) -- [ ] Add structured logs for lifecycle transitions and outcomes +- [ ] Add `process_next_queued_job(...) -> bool` +- [ ] Fetch oldest queued job +- [ ] Return `False` when no queued jobs exist +- [ ] Transition picked job to `processing` and update timestamp +- [ ] Resolve associated `Document.file_path` +- [ ] Call `transcribe_document_image(image_path=...)` +- [ ] On success: + - [ ] insert/update transcript text + - [ ] clear error detail + - [ ] mark job `transcribed` + - [ ] update timestamp +- [ ] On failure: + - [ ] insert/update transcript with `text=None`, `error_detail=...` + - [ ] mark job `failed` + - [ ] update timestamp +- [ ] Commit terminal state and return `True` +- [ ] Add logs around job pickup, transition, and terminal outcome --- -## Phase C — Worker loop (`worker.py`) +## Phase C — Implement worker loop (`src/transcription/worker.py`) -- [ ] Implement `run_worker_loop(...)` - - [ ] accepts stop signal/event - - [ ] configurable `poll_interval_seconds` - - [ ] repeatedly calls `process_next_queued_job` - - [ ] sleeps when no work - - [ ] exits cleanly on stop request -- [ ] Keep implementation single-process/simple for MVP assumptions +- [ ] Add `run_worker_loop(...)` +- [ ] Accept configurable stop event/signal +- [ ] Accept configurable poll interval +- [ ] Repeatedly call `process_next_queued_job` +- [ ] Sleep only when queue is empty +- [ ] Exit cleanly when stop event is set --- -## Phase D — Service exports +## Phase D — Exports -- [ ] Update `src/transcription/services/__init__.py` exports - - [ ] include upload service symbols - - [ ] keep transcription exports intact +- [ ] Update `src/transcription/services/__init__.py` to expose upload APIs +- [ ] Keep existing transcription exports intact --- -## Phase E — Tests (scaffold -> fill) +## Phase E — Tests via MCP scaffold -> fill flow -### E1 Scaffold -- [ ] Create `tests/services/test_upload.py` with class/method skeletons + one-line docstrings -- [ ] Create `tests/services/test_worker.py` with class/method skeletons + one-line docstrings -- [ ] Validate scaffold collection: - - [ ] `uv run pytest --collect-only -q` +## E1 Scaffold (structure only) -### E2 Fill +Use scaffold prompt workflow first for: +- `src/transcription/services/upload.py` +- `src/transcription/worker.py` -#### Upload tests -- [ ] `test_create_upload_job_writes_file_and_creates_records` (`integration`) -- [ ] `test_create_upload_job_rejects_empty_bytes` (`unit`) -- [ ] `test_create_upload_job_rejects_unsupported_extension` (`unit`) -- [ ] `test_create_upload_job_uses_unique_stored_filename` (`integration`) -- [ ] `test_create_upload_job_sets_job_status_queued` (`integration`) +Expected scaffold targets: +- `tests/services/test_upload.py` +- `tests/services/test_worker.py` -#### Worker tests -- [ ] `test_process_next_queued_job_returns_false_when_queue_empty` (`integration`) -- [ ] `test_process_next_queued_job_transitions_processing_to_transcribed` (`integration`) -- [ ] `test_process_next_queued_job_persists_transcript_text_on_success` (`integration`) -- [ ] `test_process_next_queued_job_sets_failed_and_error_detail_on_failure` (`integration`) -- [ ] `test_process_next_queued_job_updates_existing_transcript_if_present` (`integration`) -- [ ] `test_run_worker_loop_stops_when_stop_event_is_set` (`unit`) +Scaffold rules: +- [ ] Class hierarchy + method names + one-line docstrings only +- [ ] No assertions or implementation details in scaffold phase +- [ ] Keep method names concise and behavior-focused -### Marker usage -- [ ] Mark pure logic tests as `@pytest.mark.unit` -- [ ] Mark DB/service orchestration tests as `@pytest.mark.integration` -- [ ] Reuse existing `external` only for future live endpoint checks (not required in Step 4) +Validation: +- [ ] `uv run pytest --collect-only -q` + +## E2 Fill scaffold (implementation) + +Use fill prompt workflow for: +- `tests/services/test_upload.py` +- `tests/services/test_worker.py` +- stack: `sqlalchemy-sync` (or `mixed` if combining pure + DB behaviors) +- marker lane preference: `unit` and `integration` as appropriate +- strategy: minimal deterministic implementation + +Fill rules (invariants): +- [ ] Preserve scaffold class names, method names, and one-line docstrings +- [ ] Do not rename/re-nest scaffolded tests unless explicitly approved +- [ ] One behavior target per test +- [ ] Minimal mocking; mock only network/nondeterministic boundaries + +Suggested test coverage: + +### `tests/services/test_upload.py` +- [ ] creates file + document + queued job (`integration`) +- [ ] rejects empty bytes (`unit`) +- [ ] rejects unsupported extension (`unit`) +- [ ] writes collision-safe unique filename (`integration`) +- [ ] persisted job status is `queued` (`integration`) + +### `tests/services/test_worker.py` +- [ ] returns `False` when queue empty (`integration`) +- [ ] transitions `queued -> processing -> transcribed` on success (`integration`) +- [ ] stores transcript text on success (`integration`) +- [ ] transitions to `failed` and stores `error_detail` on failure (`integration`) +- [ ] updates existing transcript instead of duplicate create (`integration`) +- [ ] worker loop exits when stop event set (`unit`) --- -## Phase F — Verification +## Marker Strategy + +- `unit`: pure logic tests (filename handling, loop stop behavior, validation logic) +- `integration`: DB + service orchestration tests (SQLite/session/contracts) +- `external`: opt-in live provider tests only (not part of default Step 4 lane) + +No new marker needed; reuse existing marker registration. + +--- + +## Validation Sequence (strict order) - [ ] `uv run pytest --collect-only -q` +- [ ] `uv run pytest -m unit -q` *(if unit tests touched)* - [ ] `uv run pytest tests/services/test_upload.py -q` - [ ] `uv run pytest tests/services/test_worker.py -q` -- [ ] `uv run pytest -m "unit or integration" -q` - [ ] `uv run pytest -q` --- -## Implementation Guardrails +## Reporting Requirements (after implementation) -- [ ] Do not couple worker directly to provider SDK internals -- [ ] Do not swallow exceptions silently -- [ ] Always persist terminal failure details -- [ ] Keep status transitions explicit and timestamped -- [ ] Keep default test suite deterministic and fast (no live network) +Implementation report must include: + +1. Files created/updated +2. Fixture and marker decisions +3. MCP references used and why +4. Validation command results +5. Remaining risks/open questions (only blockers) + +--- + +## Guardrails + +- Keep Step 4 independent from UI concerns. +- Do not call provider SDK directly from worker. +- Do not silently swallow exceptions. +- Always persist terminal job outcome. +- Keep default suite deterministic and fast. +- Preserve scaffold invariants during fill phase. --- ## Definition of Done (Step 4) -- [ ] Upload service persists file + creates `Document` and queued `Job` -- [ ] Worker processes queued jobs end-to-end via Step 3 service -- [ ] Success writes transcript text and final `transcribed` status -- [ ] Failure writes `error_detail` and final `failed` status -- [ ] Queue-empty worker call returns cleanly without error -- [ ] Full test suite passes with `uv run pytest -q` +Step 4 is complete when: + +- [ ] Upload service writes file and creates `Document` + queued `Job` +- [ ] Worker processes queued jobs end-to-end using Step 3 transcription service +- [ ] Success path persists transcript text and sets `transcribed` +- [ ] Failure path persists error detail and sets `failed` +- [ ] Queue-empty path returns cleanly +- [ ] New tests pass and full suite is green (`uv run pytest -q`) +- [ ] Output report includes MCP reference usage + validation evidence \ No newline at end of file diff --git a/tests/artifacts/transcriptions/Book_Two_-_page_02.jpg.txt b/tests/artifacts/transcriptions/Book_Two_-_page_02.jpg.txt new file mode 100644 index 0000000..817d991 --- /dev/null +++ b/tests/artifacts/transcriptions/Book_Two_-_page_02.jpg.txt @@ -0,0 +1,42 @@ +source: Book Two - page 02.jpg +provider: openrouter +model: google/gemini-2.5-flash +--- +BY WAY OF INTRODUCTION:- + +These few paragraphs of introduction may help you read BOOK 2 which covers a +wider range than did BOOK 1 (Pioneer Days). + +BOOK 1 had 54 pages; 14 chapters. BOOK 2 has 70 pages; 18 chapters. BOOK 1 con- +sisted largely of first generation family history. BOOK 2 throws more light on +the second generation. Sidney promises a BOOK 3 and that may begin to do justice +to the third generation. We suggest that Sidney get the help of Louis Shinn +who has a chapter in this book (Chapter 16 - The Last 25 Years on the Doumeeq +Plains. Louis has the gift of seeing, recalling and telling. One sentence in +his chapter gives a great tribute to the Doumeeqers--so far as he knows no one +on the Doumeeq Plains went on relief during the depression. That in a nutshell +shows the sturdy character of the residents of the Doumeeq Plains. + +We promised in BOOK 1 that in BOOK 2 we would give the story of the trip of +John E. Cochran and wife to Tennessee, Cuba and the Panama Canal. You will see +by the Table of Contents that the first four chapters have been given to those +trips. Those chapters are worth reading and re-reading. Mr. Cochran has eyes +to see and a pen to tell. We think the people in Tennessee will read with +great pleasure the comments he makes on conditions today. + +Some who get this book will consider the group picture the best thing in the +book. It took a lot of preliminary photographing to reduce some pictures, enlarge +others and bring out the tin types. We wish that instead of 44 faces we could +have given 88. Do not blame Ethel Cochran-Shinn for the selection. She furnished +enough pictures but we had to take only part of them. We think there are great +possibilities in reproducing old pictures. We wish we had a Pickard group. Some +Pickard descendant may wish to make a collection. + +We are much impressed with the future possibilities of getting a complete genealog- +ogy of the Pickard family. Mr. Cochran has a fine chapter on the Pickards but +to date we have not had the pleasure of finding all of the family dates. We had +intended to give more family data in this book but it takes time to get the +correct dates. Often times it requires trips to cemeteries to get dates on the +tombstones. Winter is no time to collect dates on tombstones. + +-2- diff --git a/tests/artifacts/transcriptions/Omie_Writes_Home.pdf.txt b/tests/artifacts/transcriptions/Omie_Writes_Home.pdf.txt new file mode 100644 index 0000000..f4290d3 --- /dev/null +++ b/tests/artifacts/transcriptions/Omie_Writes_Home.pdf.txt @@ -0,0 +1,111 @@ +source: Omie Writes Home.pdf +provider: openrouter +model: google/gemini-2.5-flash +--- +JOHN E. COCHRAN +FAMILY ASSOCIATION +Family Only +Home | Sibling's Stories | 1st Cousins | Ancestor's Stories | Reunion History | Next Reunion | JECMEF +OMIE WRITES HOME +Ed. Note: The following letter was written by Omie Cochran in Nome, Alaska and sent to her +sister, Ethel Shinn, in Canfield, Idaho in 1923. It has been stored away these 63 years in the +original envelope with its 2 cent stamp. The letter has a number of references to the Shinn +children. Peter was Louis; Polly was Edith; and the 'little black rascal' referred to Maurice. +Miss Saville was the nurse at the Nome Hospital that was mentioned in the article by Inez in +the family newsletter two years ago. +Nome Alaska August 26, 1923 +My Dear Ethel et al. +I don't know when I did write or when you did +but I am going to write now however and never +the less. But I wish I could talk (I can yet but I +mean to tell you all) instead and see ole Unc Pete +and Polly sit up and listen and that little black +rascal of yours would fairly sparkle with +listening. Can't I see him listening now to all the +yarns we told last summer? +You see, we-Miss Saville and I, took a trip north +on the Buford and it was very interesting. We +went north thru the Bering Strait into the Arctic and as far as the Ice Pack. There the captain +of our craft and some other mighty hunters went out first in kayaks and later in row boats and +shot seven walrus. When they also took a movie man and camera, so you will likely see all +this in the movies before I get to tell you. They came back on board and the ship went up +along the icebergs and the walrus were 'heisted' on board by the use of the crane which loads +tons of freight and the beasts were so huge that they made the pulleys just creak. They were +over 12 feet long, as big around as three cows, had no feet but toenails on their flippers or +flappers, no head but their body just suddenly ended with a hole for a mouth and big bristles + +all around it similar to a currycomb in coarseness; no ears but huge tusks of ivory. They are +the most repulsive looking animals imaginable and tho I have always read about them I never +expect such disagreeable looking creatures. They had a rough brown hairy skin and some of +them looked warty. They must have weighed two ton at least. Ere we got them back to Nome +to the natives they were getting extremely odiferous—in fact, you could scarcely stay on the +ship with any degree of comfort unless you had per chance lost your sense of smell. + +Then we went north to a few minutes beyond the 70th degree of latitude and thot for awhile +we would go to Wrangell Island where some men from Steffonsons ship were supposed to be +stranded but we didn't get there and instead stopped at a small native village at Cape Serdz in +Siberia. These Eskimo were very primitive. One white squaw man lived there and had for 23 +years. He was a Swede--who else could. Their houses were circular and built up with dirt 2 or +3 feet and then skins were stretched over it and weighted down with rocks. Inside, the room +was partitioned off at the sides with skins for sleeping quarters. In the main part they had the +fire on the ground and the fish drying on lines and the skins hanging around and the dogs and +babies and children. They wore skin clothes entirely. The women's were made like bloomers +and were heavily padded for warmth. They wore high mukluks and really looked very +comfortable. The babies were in fur skins with the fur inside and they looked like little Teddy +bears with faces. I guess they had never seen white women, not so many at one time anyway. +We went ashore in 2 life boats and a launch pulling them. On the launch was a six piece band +playing and the rear of the last life boat was the movie man. 'Twas very thrilling. + +The other place we stopped was at Whalen, a trading post in Siberia. There these 'towerists' +went wild. They rushed helter-skelter, hither and thither, here and there, trying to find +something to buy. Prices raised right before your eyes. One would but something for $1.00 +and the next might have to pay $2.00, $4.00 or $10.00. That made no difference. They had to +have it. One man I was sort of taking care of, tho he had his son along for the purpose, +bought 2 ivory tusks, 1 pup, 2 moccasins, 3 or 4 billikens, 6 or 8 ivory and silver rings, one +fishing line, hooks, floats, etc. and two bird slings. The slings have rocks at the end and the +little natives throw them at the flocks of geese and ducks which fly close over the village and +the slings entangle their wings and legs, sometimes more than one, and they can't fly. They +come down and the natives capture them. There was more junk brot aboard than baggage, I +do believe. And they say that at the first stop it was worse than here. The red flag was flying +over Whalen and the Russian soldiers were there—a few, one or two or three, I forget the +number. + +We got home yesterday morning at 5 a.m. but missed the first lighter in so had to stay out +until 2:30. The girls had prepared a big meal for us and invited up the Hartfords and then let +us talk. Miss Saville talked quite a bit. Any how if you folks don't like this I don't care, it is +all I had to write about and I know Buster'ud listen anyway and I'd soak ole Peter's head if he +didn't and Polly would in my lap and I don't know much about the youngest one of yours so +likely he would be squawling. But we did surely enjoy our trip and were gone just long enuf. + +I expect there were 150 passengers on board and almost or more of the crew and helpers. We +had a stateroom down next to the kitchen and 'twas pretty fierce for odor at times. + +I have had jobs nearly all summer but not very much in them. Next week, September 4, +school opens. I wish they would wait for a week but you know these school men. Wouldn't +make any special difference I suppose for I would just fritter away the time but still one likes +to postpone the inevitable. + +I just now stopped and re-read your letter and it sure was a bird. I don't especially blame Ray +for reading over your shoulder. It would seem, then that you have bright children. Maybe +they do know something about Geography. But it is ridiculous to speak of Louis finishing the +eighth grade. Why you and I were grown children when we finished and he is only a baby. I +am rather afraid he doesn't know much. I quite remember your little timid Maurice and how +he shifted his affection from his Aunt Om and yelled and howled and screamed steadily for a +week while his mother went to S.S. Ask him is he recalls this little interview. Do you suppose +he does? + +Ever hear from Zen? or Inez? They don't seem to be very writing inclined tho Zen has done +well this year. Even sent me a telegram a few weeks ago. Well, if anything else ever happens, +I'll write again. Don't suppose it ever will, tho. + +Lots of love to all, + +Ome + +Reprinted from Cochran Chronicles, Volume 9, Number 1, November 1986 + +© JECFA 1986 + +Up + +jecochranclan.org ~ Contact webmaster diff --git a/tests/artifacts/transcriptions/Rod_Moser_Letter_-_p1.jpg.txt b/tests/artifacts/transcriptions/Rod_Moser_Letter_-_p1.jpg.txt new file mode 100644 index 0000000..102f81a --- /dev/null +++ b/tests/artifacts/transcriptions/Rod_Moser_Letter_-_p1.jpg.txt @@ -0,0 +1,31 @@ +source: Rod Moser Letter - p1.jpg +provider: openrouter +model: google/gemini-2.5-flash +--- +JOHN ISBILL R. T. MOSER + +ISBILL & MOSER +DEALERS IN +GENERAL MERCHANDISE + +Vonore, Tenn. Jany 27 – 1913 +Dear Much Aunt Louie +How are you— +few nights ago received a +letter from your folks, So +I decided to write you +a few lines myself & +I am contemplating a +trip out west next summer +& would like for [illegible] to go +where I am. + +Am getting +up in years & unmarried +so you see the object of +my trip is to get a wife +& if there is any old maid +or widdow out there, I +want you to kiss them +at my [illegible] for me at They +as soon as I get there diff --git a/tests/fixtures/images/invalid/empty.bin b/tests/fixtures/images/invalid/empty.bin new file mode 100644 index 0000000..e69de29 diff --git a/tests/fixtures/images/invalid/wrong_extension.txt b/tests/fixtures/images/invalid/wrong_extension.txt new file mode 100644 index 0000000..2cebe82 --- /dev/null +++ b/tests/fixtures/images/invalid/wrong_extension.txt @@ -0,0 +1 @@ +not an image fixture \ No newline at end of file diff --git a/tests/fixtures/images/real/Book Two - page 02.jpg b/tests/fixtures/images/real/Book Two - page 02.jpg new file mode 100644 index 0000000..7f2267a Binary files /dev/null and b/tests/fixtures/images/real/Book Two - page 02.jpg differ diff --git a/tests/fixtures/images/real/Omie Writes Home.pdf b/tests/fixtures/images/real/Omie Writes Home.pdf new file mode 100644 index 0000000..6f30949 Binary files /dev/null and b/tests/fixtures/images/real/Omie Writes Home.pdf differ diff --git a/tests/fixtures/images/real/Rod Moser Letter - p1.jpg b/tests/fixtures/images/real/Rod Moser Letter - p1.jpg new file mode 100644 index 0000000..a1e9432 Binary files /dev/null and b/tests/fixtures/images/real/Rod Moser Letter - p1.jpg differ diff --git a/tests/fixtures/images/valid/single_page_pdf.pdf b/tests/fixtures/images/valid/single_page_pdf.pdf new file mode 100644 index 0000000..d61520a --- /dev/null +++ b/tests/fixtures/images/valid/single_page_pdf.pdf @@ -0,0 +1,2 @@ +%PDF-1.4 +%fixture diff --git a/tests/fixtures/images/valid/small_jpg.jpg b/tests/fixtures/images/valid/small_jpg.jpg new file mode 100644 index 0000000..5ed4353 --- /dev/null +++ b/tests/fixtures/images/valid/small_jpg.jpg @@ -0,0 +1 @@ +ÿØÿàfixture \ No newline at end of file diff --git a/tests/fixtures/images/valid/small_png.png b/tests/fixtures/images/valid/small_png.png new file mode 100644 index 0000000..aa62e8b --- /dev/null +++ b/tests/fixtures/images/valid/small_png.png @@ -0,0 +1,3 @@ +‰PNG + +fixture \ No newline at end of file diff --git a/tests/fixtures/images/valid/small_tiff.tiff b/tests/fixtures/images/valid/small_tiff.tiff new file mode 100644 index 0000000..bf1d7e9 Binary files /dev/null and b/tests/fixtures/images/valid/small_tiff.tiff differ diff --git a/tests/fixtures/prompts/invalid/empty_prompt.md b/tests/fixtures/prompts/invalid/empty_prompt.md new file mode 100644 index 0000000..e69de29 diff --git a/tests/fixtures/prompts/valid/transcribe_document_min.md b/tests/fixtures/prompts/valid/transcribe_document_min.md new file mode 100644 index 0000000..f2ea2db --- /dev/null +++ b/tests/fixtures/prompts/valid/transcribe_document_min.md @@ -0,0 +1,6 @@ +# Minimal Verbatim Prompt + +Transcribe the document verbatim. +Do not summarize or paraphrase. +Mark uncertain text using square brackets and a question mark. +Mark unreadable text as [illegible]. \ No newline at end of file diff --git a/tests/services/test_transcription_external.py b/tests/services/test_transcription_external.py new file mode 100644 index 0000000..7cb3aed --- /dev/null +++ b/tests/services/test_transcription_external.py @@ -0,0 +1,70 @@ +"""Opt-in external tests for real document image transcription.""" + +import os +from pathlib import Path + +import pytest + +from transcription.services.transcription import transcribe_document_image + + +HAS_OPENROUTER_KEY = bool(os.getenv("OPENROUTER_API_KEY")) + +REAL_IMAGES_DIR = Path(__file__).resolve().parents[1] / "fixtures" / "images" / "real" +ARTIFACTS_DIR = Path(__file__).resolve().parents[1] / "artifacts" / "transcriptions" +SUPPORTED_EXTENSIONS = {".jpg", ".jpeg", ".png", ".tif", ".tiff", ".pdf"} + +pytestmark = [ + pytest.mark.external, + pytest.mark.skipif( + not HAS_OPENROUTER_KEY, + reason="Set OPENROUTER_API_KEY to run external real-image tests.", + ), +] + + +def _real_image_paths() -> list[Path]: + if not REAL_IMAGES_DIR.exists(): + return [] + return sorted( + [ + p + for p in REAL_IMAGES_DIR.iterdir() + if p.is_file() and p.suffix.lower() in SUPPORTED_EXTENSIONS + ] + ) + + +def _artifact_filename(image_path: Path) -> str: + safe_stem = image_path.stem.replace(" ", "_") + safe_suffix = image_path.suffix.lower().replace(".", "") + return f"{safe_stem}.{safe_suffix}.txt" + + +class TestRealImageExternalTranscription: + """Validate transcription against real local fixtures via live provider.""" + + def test_real_image_fixture_set_exists(self): + """At least one supported real-image fixture exists for external tests.""" + assert REAL_IMAGES_DIR.exists() + assert _real_image_paths() + + @pytest.mark.parametrize("image_path", _real_image_paths(), ids=lambda p: p.name) + def test_transcribes_real_image_fixture(self, image_path: Path): + """Real fixture image produces a non-empty transcription result.""" + result = transcribe_document_image(image_path) + assert result.provider == "openrouter" + assert isinstance(result.model, str) and result.model.strip() + assert isinstance(result.text, str) and result.text.strip() + + ARTIFACTS_DIR.mkdir(parents=True, exist_ok=True) + artifact_path = ARTIFACTS_DIR / _artifact_filename(image_path) + artifact_text = ( + f"source: {image_path.name}\n" + f"provider: {result.provider}\n" + f"model: {result.model}\n" + "---\n" + f"{result.text}\n" + ) + artifact_path.write_text(artifact_text, encoding="utf-8") + assert artifact_path.exists() \ No newline at end of file