generated from john/python-template
Step 3 tested with real data. Added Step 4 implementation plan
This commit is contained in:
Jim Lancaster
+174
-102
@@ -1,18 +1,18 @@
|
||||
## Step 4: `services/upload.py` + `worker.py`
|
||||
|
||||
## Objective
|
||||
### Objective
|
||||
|
||||
Implement the MVP upload and background processing pipeline so the system can:
|
||||
Implement the MVP upload and background-processing pipeline so the system can:
|
||||
|
||||
1. Save uploaded files into `UPLOAD_DIR`
|
||||
2. Create `Document` + `Job(status="queued")`
|
||||
3. Process queued jobs in a worker loop:
|
||||
- `queued -> processing`
|
||||
- invoke Step 3 transcription
|
||||
- call Step 3 transcription service
|
||||
- persist `Transcript`
|
||||
- finalize as `transcribed` or `failed`
|
||||
|
||||
This covers MVP Feature 1 + 2 and supports REQ-1, REQ-2, REQ-3, REQ-4, REQ-6.
|
||||
This step advances MVP Feature 1 + Feature 2 and supports REQ-1, REQ-2, REQ-3, REQ-4, REQ-6.
|
||||
|
||||
---
|
||||
|
||||
@@ -21,15 +21,15 @@ This covers MVP Feature 1 + 2 and supports REQ-1, REQ-2, REQ-3, REQ-4, REQ-6.
|
||||
### In scope
|
||||
- `src/transcription/services/upload.py`
|
||||
- `src/transcription/worker.py`
|
||||
- Upload persistence logic + job creation
|
||||
- Worker polling and single-job processing lifecycle
|
||||
- Tests for upload + worker behavior (deterministic, no live API)
|
||||
- Upload persistence logic and initial job creation
|
||||
- Worker polling and single-job lifecycle execution
|
||||
- Deterministic test coverage for upload + worker (default suite)
|
||||
|
||||
### Out of scope
|
||||
- UI wiring (Step 5)
|
||||
- Job/transcript pages (Step 5)
|
||||
- External queue infrastructure
|
||||
- Async DB/session architecture changes (post-MVP)
|
||||
- UI integration and pages (Step 5)
|
||||
- Queue infrastructure beyond in-process loop
|
||||
- Async DB/session architecture refactor
|
||||
- Broad production hardening beyond MVP needs
|
||||
|
||||
---
|
||||
|
||||
@@ -40,151 +40,223 @@ This covers MVP Feature 1 + 2 and supports REQ-1, REQ-2, REQ-3, REQ-4, REQ-6.
|
||||
- `src/transcription/worker.py`
|
||||
- `src/transcription/services/__init__.py` (export updates as needed)
|
||||
|
||||
### Tests
|
||||
### Test files
|
||||
- `tests/services/test_upload.py`
|
||||
- `tests/services/test_worker.py` *(recommended for mirror consistency)*
|
||||
- `tests/services/test_worker.py`
|
||||
|
||||
### Optional external lane (already present pattern)
|
||||
- reuse `external` marker for live-provider checks where appropriate
|
||||
- keep external out of default lane
|
||||
|
||||
---
|
||||
|
||||
## Required MCP Prompt References (for test workflow)
|
||||
|
||||
Apply these resources directly during Step 4 test creation:
|
||||
|
||||
1. `resource://catalog/prompts/pytest-scaffold`
|
||||
2. `resource://prompts/pytest-scaffold/document`
|
||||
3. `resource://catalog/prompts/pytest-fill-scaffold`
|
||||
4. `resource://prompts/pytest-fill-scaffold/document`
|
||||
|
||||
And (as referenced by those prompts) apply relevant pytest skill references for:
|
||||
- naming/hierarchy
|
||||
- marker defaults
|
||||
- SQLAlchemy sync testing behavior where applicable
|
||||
|
||||
---
|
||||
|
||||
## Design Decisions
|
||||
|
||||
1. **Upload service owns file persistence + initial DB records**
|
||||
- Save file first, then create `Document` + `Job(queued)`.
|
||||
1. **Upload service owns initial file + record creation**
|
||||
- Writes file, creates `Document`, creates queued `Job`, returns IDs/path.
|
||||
|
||||
2. **Worker owns job lifecycle transitions**
|
||||
- Transition only in worker:
|
||||
- `queued -> processing`
|
||||
- `processing -> transcribed|failed`
|
||||
2. **Worker owns lifecycle transitions**
|
||||
- Worker is the single owner of `queued -> processing -> terminal` job state changes.
|
||||
|
||||
3. **Worker calls Step 3 service, not provider SDK directly**
|
||||
- Use `transcribe_document_image(...)` from `services/transcription.py`.
|
||||
3. **Worker uses Step 3 service boundary**
|
||||
- Worker calls `transcribe_document_image(...)`; no provider-specific SDK logic in worker.
|
||||
|
||||
4. **Failure details are always persisted**
|
||||
- On failure, persist transcript row with `text=None`, `error_detail=...`.
|
||||
4. **Failure information is always persisted**
|
||||
- On failure: store `Transcript(text=None, error_detail=...)` and set `Job.status=failed`.
|
||||
|
||||
5. **Worker loop is stoppable**
|
||||
- Simple in-process loop with stop signal/event and poll interval.
|
||||
5. **Loop remains simple and stoppable**
|
||||
- In-process polling loop with stop event and poll interval for MVP simplicity and testability.
|
||||
|
||||
---
|
||||
|
||||
## Task-by-Task Execution Checklist
|
||||
|
||||
## Phase A — Upload service (`services/upload.py`)
|
||||
## Phase A — Implement upload service (`src/transcription/services/upload.py`)
|
||||
|
||||
- [ ] Create `src/transcription/services/upload.py`
|
||||
- [ ] Add `UploadError` exception type
|
||||
- [ ] Add result dataclass (e.g., `UploadJobResult`) with:
|
||||
- [ ] Create `UploadError` exception
|
||||
- [ ] Create `UploadJobResult` dataclass with:
|
||||
- [ ] `document_id`
|
||||
- [ ] `job_id`
|
||||
- [ ] `stored_path`
|
||||
- [ ] `original_filename`
|
||||
- [ ] Implement safe filename handling:
|
||||
- [ ] basename normalization
|
||||
- [ ] collision-safe naming (e.g., UUID prefix)
|
||||
- [ ] reject path traversal patterns
|
||||
- [ ] Implement extension validation (`.jpg/.jpeg/.png/.tif/.tiff/.pdf`)
|
||||
- [ ] Implement empty-bytes validation
|
||||
- [ ] Ensure `UPLOAD_DIR` exists (`mkdir(parents=True, exist_ok=True)`)
|
||||
- [ ] Persist file bytes to disk
|
||||
- [ ] Insert `Document` and `Job(status=queued)` in one transaction
|
||||
- [ ] Return structured `UploadJobResult`
|
||||
- [ ] Add logging for upload success/failure (no sensitive payload logging)
|
||||
- [ ] Add filename safety handling:
|
||||
- [ ] normalize to basename
|
||||
- [ ] avoid path traversal
|
||||
- [ ] collision-safe stored name (e.g., UUID prefix/suffix)
|
||||
- [ ] Validate upload payload:
|
||||
- [ ] non-empty bytes required
|
||||
- [ ] extension in supported set (`.jpg/.jpeg/.png/.tif/.tiff/.pdf`)
|
||||
- [ ] Ensure upload directory exists (`mkdir(parents=True, exist_ok=True)`)
|
||||
- [ ] Write file bytes to `UPLOAD_DIR`
|
||||
- [ ] Persist DB records in one transaction:
|
||||
- [ ] `Document(filename, file_path)`
|
||||
- [ ] `Job(document_id=..., status=queued)`
|
||||
- [ ] Return `UploadJobResult`
|
||||
- [ ] Add logging for success/failure boundaries
|
||||
|
||||
---
|
||||
|
||||
## Phase B — Worker core (`worker.py`)
|
||||
## Phase B — Implement worker core (`src/transcription/worker.py`)
|
||||
|
||||
- [ ] Create `src/transcription/worker.py`
|
||||
- [ ] Implement `process_next_queued_job(...) -> bool`
|
||||
- [ ] fetch oldest queued job
|
||||
- [ ] return `False` if none found
|
||||
- [ ] set job to `processing` + update timestamp
|
||||
- [ ] call `transcribe_document_image(image_path=...)`
|
||||
- [ ] success path: persist transcript text, mark `transcribed`
|
||||
- [ ] failure path: persist error detail, mark `failed`
|
||||
- [ ] return `True` when a job is processed
|
||||
- [ ] Implement transcript upsert behavior (avoid duplicate unique `job_id` insert issues)
|
||||
- [ ] Add structured logs for lifecycle transitions and outcomes
|
||||
- [ ] Add `process_next_queued_job(...) -> bool`
|
||||
- [ ] Fetch oldest queued job
|
||||
- [ ] Return `False` when no queued jobs exist
|
||||
- [ ] Transition picked job to `processing` and update timestamp
|
||||
- [ ] Resolve associated `Document.file_path`
|
||||
- [ ] Call `transcribe_document_image(image_path=...)`
|
||||
- [ ] On success:
|
||||
- [ ] insert/update transcript text
|
||||
- [ ] clear error detail
|
||||
- [ ] mark job `transcribed`
|
||||
- [ ] update timestamp
|
||||
- [ ] On failure:
|
||||
- [ ] insert/update transcript with `text=None`, `error_detail=...`
|
||||
- [ ] mark job `failed`
|
||||
- [ ] update timestamp
|
||||
- [ ] Commit terminal state and return `True`
|
||||
- [ ] Add logs around job pickup, transition, and terminal outcome
|
||||
|
||||
---
|
||||
|
||||
## Phase C — Worker loop (`worker.py`)
|
||||
## Phase C — Implement worker loop (`src/transcription/worker.py`)
|
||||
|
||||
- [ ] Implement `run_worker_loop(...)`
|
||||
- [ ] accepts stop signal/event
|
||||
- [ ] configurable `poll_interval_seconds`
|
||||
- [ ] repeatedly calls `process_next_queued_job`
|
||||
- [ ] sleeps when no work
|
||||
- [ ] exits cleanly on stop request
|
||||
- [ ] Keep implementation single-process/simple for MVP assumptions
|
||||
- [ ] Add `run_worker_loop(...)`
|
||||
- [ ] Accept configurable stop event/signal
|
||||
- [ ] Accept configurable poll interval
|
||||
- [ ] Repeatedly call `process_next_queued_job`
|
||||
- [ ] Sleep only when queue is empty
|
||||
- [ ] Exit cleanly when stop event is set
|
||||
|
||||
---
|
||||
|
||||
## Phase D — Service exports
|
||||
## Phase D — Exports
|
||||
|
||||
- [ ] Update `src/transcription/services/__init__.py` exports
|
||||
- [ ] include upload service symbols
|
||||
- [ ] keep transcription exports intact
|
||||
- [ ] Update `src/transcription/services/__init__.py` to expose upload APIs
|
||||
- [ ] Keep existing transcription exports intact
|
||||
|
||||
---
|
||||
|
||||
## Phase E — Tests (scaffold -> fill)
|
||||
## Phase E — Tests via MCP scaffold -> fill flow
|
||||
|
||||
### E1 Scaffold
|
||||
- [ ] Create `tests/services/test_upload.py` with class/method skeletons + one-line docstrings
|
||||
- [ ] Create `tests/services/test_worker.py` with class/method skeletons + one-line docstrings
|
||||
- [ ] Validate scaffold collection:
|
||||
- [ ] `uv run pytest --collect-only -q`
|
||||
## E1 Scaffold (structure only)
|
||||
|
||||
### E2 Fill
|
||||
Use scaffold prompt workflow first for:
|
||||
- `src/transcription/services/upload.py`
|
||||
- `src/transcription/worker.py`
|
||||
|
||||
#### Upload tests
|
||||
- [ ] `test_create_upload_job_writes_file_and_creates_records` (`integration`)
|
||||
- [ ] `test_create_upload_job_rejects_empty_bytes` (`unit`)
|
||||
- [ ] `test_create_upload_job_rejects_unsupported_extension` (`unit`)
|
||||
- [ ] `test_create_upload_job_uses_unique_stored_filename` (`integration`)
|
||||
- [ ] `test_create_upload_job_sets_job_status_queued` (`integration`)
|
||||
Expected scaffold targets:
|
||||
- `tests/services/test_upload.py`
|
||||
- `tests/services/test_worker.py`
|
||||
|
||||
#### Worker tests
|
||||
- [ ] `test_process_next_queued_job_returns_false_when_queue_empty` (`integration`)
|
||||
- [ ] `test_process_next_queued_job_transitions_processing_to_transcribed` (`integration`)
|
||||
- [ ] `test_process_next_queued_job_persists_transcript_text_on_success` (`integration`)
|
||||
- [ ] `test_process_next_queued_job_sets_failed_and_error_detail_on_failure` (`integration`)
|
||||
- [ ] `test_process_next_queued_job_updates_existing_transcript_if_present` (`integration`)
|
||||
- [ ] `test_run_worker_loop_stops_when_stop_event_is_set` (`unit`)
|
||||
Scaffold rules:
|
||||
- [ ] Class hierarchy + method names + one-line docstrings only
|
||||
- [ ] No assertions or implementation details in scaffold phase
|
||||
- [ ] Keep method names concise and behavior-focused
|
||||
|
||||
### Marker usage
|
||||
- [ ] Mark pure logic tests as `@pytest.mark.unit`
|
||||
- [ ] Mark DB/service orchestration tests as `@pytest.mark.integration`
|
||||
- [ ] Reuse existing `external` only for future live endpoint checks (not required in Step 4)
|
||||
Validation:
|
||||
- [ ] `uv run pytest --collect-only -q`
|
||||
|
||||
## E2 Fill scaffold (implementation)
|
||||
|
||||
Use fill prompt workflow for:
|
||||
- `tests/services/test_upload.py`
|
||||
- `tests/services/test_worker.py`
|
||||
- stack: `sqlalchemy-sync` (or `mixed` if combining pure + DB behaviors)
|
||||
- marker lane preference: `unit` and `integration` as appropriate
|
||||
- strategy: minimal deterministic implementation
|
||||
|
||||
Fill rules (invariants):
|
||||
- [ ] Preserve scaffold class names, method names, and one-line docstrings
|
||||
- [ ] Do not rename/re-nest scaffolded tests unless explicitly approved
|
||||
- [ ] One behavior target per test
|
||||
- [ ] Minimal mocking; mock only network/nondeterministic boundaries
|
||||
|
||||
Suggested test coverage:
|
||||
|
||||
### `tests/services/test_upload.py`
|
||||
- [ ] creates file + document + queued job (`integration`)
|
||||
- [ ] rejects empty bytes (`unit`)
|
||||
- [ ] rejects unsupported extension (`unit`)
|
||||
- [ ] writes collision-safe unique filename (`integration`)
|
||||
- [ ] persisted job status is `queued` (`integration`)
|
||||
|
||||
### `tests/services/test_worker.py`
|
||||
- [ ] returns `False` when queue empty (`integration`)
|
||||
- [ ] transitions `queued -> processing -> transcribed` on success (`integration`)
|
||||
- [ ] stores transcript text on success (`integration`)
|
||||
- [ ] transitions to `failed` and stores `error_detail` on failure (`integration`)
|
||||
- [ ] updates existing transcript instead of duplicate create (`integration`)
|
||||
- [ ] worker loop exits when stop event set (`unit`)
|
||||
|
||||
---
|
||||
|
||||
## Phase F — Verification
|
||||
## Marker Strategy
|
||||
|
||||
- `unit`: pure logic tests (filename handling, loop stop behavior, validation logic)
|
||||
- `integration`: DB + service orchestration tests (SQLite/session/contracts)
|
||||
- `external`: opt-in live provider tests only (not part of default Step 4 lane)
|
||||
|
||||
No new marker needed; reuse existing marker registration.
|
||||
|
||||
---
|
||||
|
||||
## Validation Sequence (strict order)
|
||||
|
||||
- [ ] `uv run pytest --collect-only -q`
|
||||
- [ ] `uv run pytest -m unit -q` *(if unit tests touched)*
|
||||
- [ ] `uv run pytest tests/services/test_upload.py -q`
|
||||
- [ ] `uv run pytest tests/services/test_worker.py -q`
|
||||
- [ ] `uv run pytest -m "unit or integration" -q`
|
||||
- [ ] `uv run pytest -q`
|
||||
|
||||
---
|
||||
|
||||
## Implementation Guardrails
|
||||
## Reporting Requirements (after implementation)
|
||||
|
||||
- [ ] Do not couple worker directly to provider SDK internals
|
||||
- [ ] Do not swallow exceptions silently
|
||||
- [ ] Always persist terminal failure details
|
||||
- [ ] Keep status transitions explicit and timestamped
|
||||
- [ ] Keep default test suite deterministic and fast (no live network)
|
||||
Implementation report must include:
|
||||
|
||||
1. Files created/updated
|
||||
2. Fixture and marker decisions
|
||||
3. MCP references used and why
|
||||
4. Validation command results
|
||||
5. Remaining risks/open questions (only blockers)
|
||||
|
||||
---
|
||||
|
||||
## Guardrails
|
||||
|
||||
- Keep Step 4 independent from UI concerns.
|
||||
- Do not call provider SDK directly from worker.
|
||||
- Do not silently swallow exceptions.
|
||||
- Always persist terminal job outcome.
|
||||
- Keep default suite deterministic and fast.
|
||||
- Preserve scaffold invariants during fill phase.
|
||||
|
||||
---
|
||||
|
||||
## Definition of Done (Step 4)
|
||||
|
||||
- [ ] Upload service persists file + creates `Document` and queued `Job`
|
||||
- [ ] Worker processes queued jobs end-to-end via Step 3 service
|
||||
- [ ] Success writes transcript text and final `transcribed` status
|
||||
- [ ] Failure writes `error_detail` and final `failed` status
|
||||
- [ ] Queue-empty worker call returns cleanly without error
|
||||
- [ ] Full test suite passes with `uv run pytest -q`
|
||||
Step 4 is complete when:
|
||||
|
||||
- [ ] Upload service writes file and creates `Document` + queued `Job`
|
||||
- [ ] Worker processes queued jobs end-to-end using Step 3 transcription service
|
||||
- [ ] Success path persists transcript text and sets `transcribed`
|
||||
- [ ] Failure path persists error detail and sets `failed`
|
||||
- [ ] Queue-empty path returns cleanly
|
||||
- [ ] New tests pass and full suite is green (`uv run pytest -q`)
|
||||
- [ ] Output report includes MCP reference usage + validation evidence
|
||||
@@ -0,0 +1,42 @@
|
||||
source: Book Two - page 02.jpg
|
||||
provider: openrouter
|
||||
model: google/gemini-2.5-flash
|
||||
---
|
||||
BY WAY OF INTRODUCTION:-
|
||||
|
||||
These few paragraphs of introduction may help you read BOOK 2 which covers a
|
||||
wider range than did BOOK 1 (Pioneer Days).
|
||||
|
||||
BOOK 1 had 54 pages; 14 chapters. BOOK 2 has 70 pages; 18 chapters. BOOK 1 con-
|
||||
sisted largely of first generation family history. BOOK 2 throws more light on
|
||||
the second generation. Sidney promises a BOOK 3 and that may begin to do justice
|
||||
to the third generation. We suggest that Sidney get the help of Louis Shinn
|
||||
who has a chapter in this book (Chapter 16 - The Last 25 Years on the Doumeeq
|
||||
Plains. Louis has the gift of seeing, recalling and telling. One sentence in
|
||||
his chapter gives a great tribute to the Doumeeqers--so far as he knows no one
|
||||
on the Doumeeq Plains went on relief during the depression. That in a nutshell
|
||||
shows the sturdy character of the residents of the Doumeeq Plains.
|
||||
|
||||
We promised in BOOK 1 that in BOOK 2 we would give the story of the trip of
|
||||
John E. Cochran and wife to Tennessee, Cuba and the Panama Canal. You will see
|
||||
by the Table of Contents that the first four chapters have been given to those
|
||||
trips. Those chapters are worth reading and re-reading. Mr. Cochran has eyes
|
||||
to see and a pen to tell. We think the people in Tennessee will read with
|
||||
great pleasure the comments he makes on conditions today.
|
||||
|
||||
Some who get this book will consider the group picture the best thing in the
|
||||
book. It took a lot of preliminary photographing to reduce some pictures, enlarge
|
||||
others and bring out the tin types. We wish that instead of 44 faces we could
|
||||
have given 88. Do not blame Ethel Cochran-Shinn for the selection. She furnished
|
||||
enough pictures but we had to take only part of them. We think there are great
|
||||
possibilities in reproducing old pictures. We wish we had a Pickard group. Some
|
||||
Pickard descendant may wish to make a collection.
|
||||
|
||||
We are much impressed with the future possibilities of getting a complete genealog-
|
||||
ogy of the Pickard family. Mr. Cochran has a fine chapter on the Pickards but
|
||||
to date we have not had the pleasure of finding all of the family dates. We had
|
||||
intended to give more family data in this book but it takes time to get the
|
||||
correct dates. Often times it requires trips to cemeteries to get dates on the
|
||||
tombstones. Winter is no time to collect dates on tombstones.
|
||||
|
||||
-2-
|
||||
@@ -0,0 +1,111 @@
|
||||
source: Omie Writes Home.pdf
|
||||
provider: openrouter
|
||||
model: google/gemini-2.5-flash
|
||||
---
|
||||
JOHN E. COCHRAN
|
||||
FAMILY ASSOCIATION
|
||||
Family Only
|
||||
Home | Sibling's Stories | 1st Cousins | Ancestor's Stories | Reunion History | Next Reunion | JECMEF
|
||||
OMIE WRITES HOME
|
||||
Ed. Note: The following letter was written by Omie Cochran in Nome, Alaska and sent to her
|
||||
sister, Ethel Shinn, in Canfield, Idaho in 1923. It has been stored away these 63 years in the
|
||||
original envelope with its 2 cent stamp. The letter has a number of references to the Shinn
|
||||
children. Peter was Louis; Polly was Edith; and the 'little black rascal' referred to Maurice.
|
||||
Miss Saville was the nurse at the Nome Hospital that was mentioned in the article by Inez in
|
||||
the family newsletter two years ago.
|
||||
Nome Alaska August 26, 1923
|
||||
My Dear Ethel et al.
|
||||
I don't know when I did write or when you did
|
||||
but I am going to write now however and never
|
||||
the less. But I wish I could talk (I can yet but I
|
||||
mean to tell you all) instead and see ole Unc Pete
|
||||
and Polly sit up and listen and that little black
|
||||
rascal of yours would fairly sparkle with
|
||||
listening. Can't I see him listening now to all the
|
||||
yarns we told last summer?
|
||||
You see, we-Miss Saville and I, took a trip north
|
||||
on the Buford and it was very interesting. We
|
||||
went north thru the Bering Strait into the Arctic and as far as the Ice Pack. There the captain
|
||||
of our craft and some other mighty hunters went out first in kayaks and later in row boats and
|
||||
shot seven walrus. When they also took a movie man and camera, so you will likely see all
|
||||
this in the movies before I get to tell you. They came back on board and the ship went up
|
||||
along the icebergs and the walrus were 'heisted' on board by the use of the crane which loads
|
||||
tons of freight and the beasts were so huge that they made the pulleys just creak. They were
|
||||
over 12 feet long, as big around as three cows, had no feet but toenails on their flippers or
|
||||
flappers, no head but their body just suddenly ended with a hole for a mouth and big bristles
|
||||
|
||||
all around it similar to a currycomb in coarseness; no ears but huge tusks of ivory. They are
|
||||
the most repulsive looking animals imaginable and tho I have always read about them I never
|
||||
expect such disagreeable looking creatures. They had a rough brown hairy skin and some of
|
||||
them looked warty. They must have weighed two ton at least. Ere we got them back to Nome
|
||||
to the natives they were getting extremely odiferous—in fact, you could scarcely stay on the
|
||||
ship with any degree of comfort unless you had per chance lost your sense of smell.
|
||||
|
||||
Then we went north to a few minutes beyond the 70th degree of latitude and thot for awhile
|
||||
we would go to Wrangell Island where some men from Steffonsons ship were supposed to be
|
||||
stranded but we didn't get there and instead stopped at a small native village at Cape Serdz in
|
||||
Siberia. These Eskimo were very primitive. One white squaw man lived there and had for 23
|
||||
years. He was a Swede--who else could. Their houses were circular and built up with dirt 2 or
|
||||
3 feet and then skins were stretched over it and weighted down with rocks. Inside, the room
|
||||
was partitioned off at the sides with skins for sleeping quarters. In the main part they had the
|
||||
fire on the ground and the fish drying on lines and the skins hanging around and the dogs and
|
||||
babies and children. They wore skin clothes entirely. The women's were made like bloomers
|
||||
and were heavily padded for warmth. They wore high mukluks and really looked very
|
||||
comfortable. The babies were in fur skins with the fur inside and they looked like little Teddy
|
||||
bears with faces. I guess they had never seen white women, not so many at one time anyway.
|
||||
We went ashore in 2 life boats and a launch pulling them. On the launch was a six piece band
|
||||
playing and the rear of the last life boat was the movie man. 'Twas very thrilling.
|
||||
|
||||
The other place we stopped was at Whalen, a trading post in Siberia. There these 'towerists'
|
||||
went wild. They rushed helter-skelter, hither and thither, here and there, trying to find
|
||||
something to buy. Prices raised right before your eyes. One would but something for $1.00
|
||||
and the next might have to pay $2.00, $4.00 or $10.00. That made no difference. They had to
|
||||
have it. One man I was sort of taking care of, tho he had his son along for the purpose,
|
||||
bought 2 ivory tusks, 1 pup, 2 moccasins, 3 or 4 billikens, 6 or 8 ivory and silver rings, one
|
||||
fishing line, hooks, floats, etc. and two bird slings. The slings have rocks at the end and the
|
||||
little natives throw them at the flocks of geese and ducks which fly close over the village and
|
||||
the slings entangle their wings and legs, sometimes more than one, and they can't fly. They
|
||||
come down and the natives capture them. There was more junk brot aboard than baggage, I
|
||||
do believe. And they say that at the first stop it was worse than here. The red flag was flying
|
||||
over Whalen and the Russian soldiers were there—a few, one or two or three, I forget the
|
||||
number.
|
||||
|
||||
We got home yesterday morning at 5 a.m. but missed the first lighter in so had to stay out
|
||||
until 2:30. The girls had prepared a big meal for us and invited up the Hartfords and then let
|
||||
us talk. Miss Saville talked quite a bit. Any how if you folks don't like this I don't care, it is
|
||||
all I had to write about and I know Buster'ud listen anyway and I'd soak ole Peter's head if he
|
||||
didn't and Polly would in my lap and I don't know much about the youngest one of yours so
|
||||
likely he would be squawling. But we did surely enjoy our trip and were gone just long enuf.
|
||||
|
||||
I expect there were 150 passengers on board and almost or more of the crew and helpers. We
|
||||
had a stateroom down next to the kitchen and 'twas pretty fierce for odor at times.
|
||||
|
||||
I have had jobs nearly all summer but not very much in them. Next week, September 4,
|
||||
school opens. I wish they would wait for a week but you know these school men. Wouldn't
|
||||
make any special difference I suppose for I would just fritter away the time but still one likes
|
||||
to postpone the inevitable.
|
||||
|
||||
I just now stopped and re-read your letter and it sure was a bird. I don't especially blame Ray
|
||||
for reading over your shoulder. It would seem, then that you have bright children. Maybe
|
||||
they do know something about Geography. But it is ridiculous to speak of Louis finishing the
|
||||
eighth grade. Why you and I were grown children when we finished and he is only a baby. I
|
||||
am rather afraid he doesn't know much. I quite remember your little timid Maurice and how
|
||||
he shifted his affection from his Aunt Om and yelled and howled and screamed steadily for a
|
||||
week while his mother went to S.S. Ask him is he recalls this little interview. Do you suppose
|
||||
he does?
|
||||
|
||||
Ever hear from Zen? or Inez? They don't seem to be very writing inclined tho Zen has done
|
||||
well this year. Even sent me a telegram a few weeks ago. Well, if anything else ever happens,
|
||||
I'll write again. Don't suppose it ever will, tho.
|
||||
|
||||
Lots of love to all,
|
||||
|
||||
Ome
|
||||
|
||||
Reprinted from Cochran Chronicles, Volume 9, Number 1, November 1986
|
||||
|
||||
© JECFA 1986
|
||||
|
||||
Up
|
||||
|
||||
jecochranclan.org ~ Contact webmaster
|
||||
@@ -0,0 +1,31 @@
|
||||
source: Rod Moser Letter - p1.jpg
|
||||
provider: openrouter
|
||||
model: google/gemini-2.5-flash
|
||||
---
|
||||
JOHN ISBILL R. T. MOSER
|
||||
|
||||
ISBILL & MOSER
|
||||
DEALERS IN
|
||||
GENERAL MERCHANDISE
|
||||
|
||||
Vonore, Tenn. Jany 27 – 1913
|
||||
Dear Much Aunt Louie
|
||||
How are you—
|
||||
few nights ago received a
|
||||
letter from your folks, So
|
||||
I decided to write you
|
||||
a few lines myself &
|
||||
I am contemplating a
|
||||
trip out west next summer
|
||||
& would like for [illegible] to go
|
||||
where I am.
|
||||
|
||||
Am getting
|
||||
up in years & unmarried
|
||||
so you see the object of
|
||||
my trip is to get a wife
|
||||
& if there is any old maid
|
||||
or widdow out there, I
|
||||
want you to kiss them
|
||||
at my [illegible] for me at They
|
||||
as soon as I get there
|
||||
@@ -0,0 +1 @@
|
||||
not an image fixture
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 1.1 MiB |
BIN
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 1010 KiB |
@@ -0,0 +1,2 @@
|
||||
%PDF-1.4
|
||||
%fixture
|
||||
+1
@@ -0,0 +1 @@
|
||||
˙Ř˙ŕfixture
|
||||
|
After Width: | Height: | Size: 11 B |
+3
@@ -0,0 +1,3 @@
|
||||
‰PNG
|
||||
|
||||
fixture
|
||||
|
After Width: | Height: | Size: 15 B |
BIN
Binary file not shown.
@@ -0,0 +1,6 @@
|
||||
# Minimal Verbatim Prompt
|
||||
|
||||
Transcribe the document verbatim.
|
||||
Do not summarize or paraphrase.
|
||||
Mark uncertain text using square brackets and a question mark.
|
||||
Mark unreadable text as [illegible].
|
||||
@@ -0,0 +1,70 @@
|
||||
"""Opt-in external tests for real document image transcription."""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from transcription.services.transcription import transcribe_document_image
|
||||
|
||||
|
||||
HAS_OPENROUTER_KEY = bool(os.getenv("OPENROUTER_API_KEY"))
|
||||
|
||||
REAL_IMAGES_DIR = Path(__file__).resolve().parents[1] / "fixtures" / "images" / "real"
|
||||
ARTIFACTS_DIR = Path(__file__).resolve().parents[1] / "artifacts" / "transcriptions"
|
||||
SUPPORTED_EXTENSIONS = {".jpg", ".jpeg", ".png", ".tif", ".tiff", ".pdf"}
|
||||
|
||||
pytestmark = [
|
||||
pytest.mark.external,
|
||||
pytest.mark.skipif(
|
||||
not HAS_OPENROUTER_KEY,
|
||||
reason="Set OPENROUTER_API_KEY to run external real-image tests.",
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def _real_image_paths() -> list[Path]:
|
||||
if not REAL_IMAGES_DIR.exists():
|
||||
return []
|
||||
return sorted(
|
||||
[
|
||||
p
|
||||
for p in REAL_IMAGES_DIR.iterdir()
|
||||
if p.is_file() and p.suffix.lower() in SUPPORTED_EXTENSIONS
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def _artifact_filename(image_path: Path) -> str:
|
||||
safe_stem = image_path.stem.replace(" ", "_")
|
||||
safe_suffix = image_path.suffix.lower().replace(".", "")
|
||||
return f"{safe_stem}.{safe_suffix}.txt"
|
||||
|
||||
|
||||
class TestRealImageExternalTranscription:
|
||||
"""Validate transcription against real local fixtures via live provider."""
|
||||
|
||||
def test_real_image_fixture_set_exists(self):
|
||||
"""At least one supported real-image fixture exists for external tests."""
|
||||
assert REAL_IMAGES_DIR.exists()
|
||||
assert _real_image_paths()
|
||||
|
||||
@pytest.mark.parametrize("image_path", _real_image_paths(), ids=lambda p: p.name)
|
||||
def test_transcribes_real_image_fixture(self, image_path: Path):
|
||||
"""Real fixture image produces a non-empty transcription result."""
|
||||
result = transcribe_document_image(image_path)
|
||||
assert result.provider == "openrouter"
|
||||
assert isinstance(result.model, str) and result.model.strip()
|
||||
assert isinstance(result.text, str) and result.text.strip()
|
||||
|
||||
ARTIFACTS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
artifact_path = ARTIFACTS_DIR / _artifact_filename(image_path)
|
||||
artifact_text = (
|
||||
f"source: {image_path.name}\n"
|
||||
f"provider: {result.provider}\n"
|
||||
f"model: {result.model}\n"
|
||||
"---\n"
|
||||
f"{result.text}\n"
|
||||
)
|
||||
artifact_path.write_text(artifact_text, encoding="utf-8")
|
||||
assert artifact_path.exists()
|
||||
Reference in New Issue
Block a user