gpt-5.3 codex review: Phase 7 and the addition of the new test-effectiveness-auditor skill.

.github/skills/python-code-reviewer/skill.md

@@ -26,11 +26,15 @@ Perform thorough, evidence-based code reviews for Python projects. Every finding
 ## Review Workflow
 
 1. **Map the Repository First:** Inspect entry points, package layout, configurations, dependency manifests, and any project-specific rule files (`AGENTS.md`, `CLAUDE.md`, `.github/instructions/`). Project-specific conventions override generic advice.
-2. **Read Representative Modules:** Sample across all layers (routes/pages, UI components, services, workers, persistence, provider adapters, settings, tests) before drawing conclusions.
-3. **Verify Claims:** Run or reference project tooling (`ruff check`, `ty`, `pytest`) rather than guessing.
-4. **Prioritize Hot Paths:** Focus deeply on request handling, database sessions, background workers, and external API calls.
-5. **Enforce Read-Only Safety:** Do not modify code unless explicitly instructed.
-6. **Escalate Provenance Audits:** For evidence/provenance-heavy changes, apply invariant checks from `.github/skills/evidence-provenance-auditor/skill.md` and include pass/fail outcomes in the report.
+2. **Establish Canonical Authority First:** Read architecture/contracts (`docs/ver4/*`, `docs/invariant/*`, UI docs) and active instructions/skills before evaluating source behavior.
+3. **Read Representative Modules:** Sample across all layers (routes/pages, UI components, services, workers, persistence, provider adapters, settings, tests) before drawing conclusions.
+4. **Run Drift Analysis:** Compare documented intended behavior versus repository ground truth; identify both implementation drift and undocumented-but-repeatable conventions that should be formalized.
+5. **Assess Boundary and Coupling Health:** Evaluate UI/service/persistence/provider dependency flow, identify circular dependencies, leaky abstractions, and transaction ownership ambiguity.
+6. **Assess Invariant Placement:** For each hard rule, decide whether it belongs in docs (rationale), instructions (active steering), skills (periodic audit procedure), or deterministic tests (enforcement).
+7. **Verify Claims:** Run or reference project tooling (`ruff check`, `ty`, `pytest`) rather than guessing.
+8. **Prioritize Hot Paths:** Focus deeply on request handling, database sessions, background workers, and external API calls.
+9. **Enforce Read-Only Safety:** Do not modify code unless explicitly instructed.
+10. **Escalate Provenance Audits:** For evidence/provenance-heavy changes, apply invariant checks from `.github/skills/evidence-provenance-auditor/skill.md` and include pass/fail outcomes in the report.
 
 ## Repo-Specific Deterministic Checks (Transcription)
 
@@ -89,6 +93,12 @@ When reviewing this repository, always include explicit pass/fail checks for:
 ### 9. Duplication & Consolidation
 - Identify repeated code blocks, candidate helper abstractions, divergent patterns for identical operations, and duplicated domain constants.
 
+### 10. Architecture & Governance
+- **Architectural Drift:** Compare intended architecture rules against implementation behavior and cite concrete drift points.
+- **Systemic Health:** Evaluate domain cohesion, dependency direction, lifecycle consistency, and operational reliability seams.
+- **Invariant Routing:** Recommend the correct enforcement layer per rule (docs vs instructions vs skills vs tests).
+- **Meta-Tooling Alignment:** Recommend updates for instruction files and skills when repository patterns or contracts evolve.
+
 ## Output Report Structure & Template
 
 Generate Markdown reports in `./docs` following this exact template structure:
@@ -106,7 +116,13 @@ Generate Markdown reports in `./docs` following this exact template structure:
 
 ---
 
-## 2. Findings by Severity
+## 2. Executive Architecture Assessment
+- High-level verdict on domain cohesion, boundary clarity, and architecture fitness.
+- Top 3-5 systemic risks or bottlenecks.
+
+---
+
+## 3. Findings by Severity
 
 ### Critical Severity
 #### [CRIT-01] Title
@@ -129,7 +145,19 @@ Generate Markdown reports in `./docs` following this exact template structure:
 
 ---
 
-## 3. Stack-Specific Analysis
+## 4. Architectural Drift & Gap Analysis
+| Area / Component | Documented / Intended Rule | Actual Implementation State | Severity | Recommended Resolution |
+| :--- | :--- | :--- | :--- | :--- |
+
+---
+
+## 5. Invariant Inventory & Routing Recommendations
+| Invariant / Constraint | Current Location | Recommended Target Layer | Rationale |
+| :--- | :--- | :--- | :--- |
+
+---
+
+## 6. Stack-Specific Analysis
 - Python 3.12+ Best Practices
 - FastAPI
 - NiceGUI
@@ -141,7 +169,7 @@ Generate Markdown reports in `./docs` following this exact template structure:
 
 ---
 
-## 4. Duplication & Consolidation Report
+## 7. Duplication & Consolidation Report
 | Pattern / Duplication | Locations | Proposed Canonical Home | Estimated Lines Removed |
 | :--- | :--- | :--- | :--- |
 
@@ -150,12 +178,19 @@ Generate Markdown reports in `./docs` following this exact template structure:
 
 ---
 
-## 5. Prioritized Action Plan
-1. **Phase 1: Quick Wins (PR 1-2)**
-2. **Phase 2: Reliability & Concurrency (PR 3-4)**
-3. **Phase 3: Consolidation & Refactoring (PR 5-6)**
+## 8. Meta-Tooling & Instruction Update Recommendations
+- Required updates to docs/instructions/skills/tests to keep enforcement current.
 
 ---
 
-## 6. Preserved Strengths
+## 9. Prioritized Dependency-Ordered Action Plan
+1. **Phase 1: Blocking fixes**
+2. **Phase 2: Enforcement hardening**
+3. **Phase 3: Reliability & concurrency**
+4. **Phase 4: Consolidation & refactoring**
+5. **Phase 5: Non-blocking governance/documentation depth**
+
+---
+
+## 10. Preserved Strengths
 - Existing patterns worth maintaining.

.github/skills/test-effectiveness-auditor/skill.md

@@ -0,0 +1,96 @@
+---
+name: test-effectiveness-auditor
+description: Periodic reviewer for test-suite signal quality. Detects low-value or redundant tests, validates contract coverage, and recommends pruning or strengthening actions.
+---
+
+# Test Effectiveness Auditor
+
+Run a deterministic audit of test usefulness. Focus on whether tests catch real regressions, not whether they merely execute code.
+
+## When to Use
+
+- Monthly/quarterly test-health review.
+- Pre-release hardening when test count grows quickly.
+- After major AI-assisted test generation.
+- When suite runtime is increasing without clear quality gains.
+
+## Primary Objectives
+
+1. Identify tests that are weak, redundant, or non-diagnostic.
+2. Confirm critical contracts are guarded by meaningful assertions.
+3. Produce a prune/strengthen backlog with explicit risk and effort.
+
+## Normative References (Transcription Repo)
+
+1. `docs/ver4/*`
+2. `docs/invariant/*`
+3. `.github/instructions/*.instructions.md`
+4. `tests/test_meta_contract_guards.py`
+5. Contract-specific guards (`tests/test_service_boundaries.py`, `tests/test_ui_boundaries.py`, worker/evidence/media/error suites)
+
+## Deterministic Audit Checks
+
+### A. Contract Traceability
+- Each high-risk contract maps to at least one focused regression test file.
+- Missing mapping is a gap.
+
+### B. Assertion Strength
+- Flag tests that only assert status code, non-null, or “no exception” without validating state transitions or persisted outcomes.
+- Prefer assertions on domain effects: DB rows, status changes, error categories, evidence writes, or emitted payload shape.
+
+### C. Failure-Path Coverage
+- Critical paths must include negative-path tests (timeouts, provider errors, validation failures, cancellation paths, retries).
+- Happy-path-only coverage on critical modules is a gap.
+
+### D. Redundancy and Noise
+- Detect near-duplicate tests asserting the same behavior at multiple layers with no extra signal.
+- Recommend canonical location (unit/integration) and prune overlaps.
+
+### E. Mutation/Change Sensitivity
+- Prefer mutation testing for high-risk modules when practical.
+- If not run, identify tests likely to survive meaningful code mutations (low sensitivity).
+
+### F. Drift Guards
+- Verify config/doc/instruction contracts have deterministic guards and are current.
+- Ensure settings/docs synchronization checks remain active.
+
+## Evidence Standards
+
+- Every finding must include concrete file paths and line ranges.
+- No speculative claims.
+- Distinguish clearly between:
+  - **Confirmed ineffective tests**
+  - **Likely weak tests (needs mutation/probe confirmation)**
+
+## Output Format
+
+Produce a Markdown report in `docs/`:
+
+```markdown
+# Test Effectiveness Audit Report
+
+## 1. Executive Verdict
+- Effective / Effective with Conditions / Needs Remediation
+- Top risks to confidence
+
+## 2. Contract Coverage Matrix
+| Contract | Guarding Tests | Signal Quality | Gap | Action |
+| :--- | :--- | :--- | :--- | :--- |
+
+## 3. Weak/Redundant Test Findings
+| Finding ID | Location | Why Low-Signal | Risk | Recommendation |
+| :--- | :--- | :--- | :--- | :--- |
+
+## 4. Prune/Strengthen Backlog
+| Task ID | Goal | Files | Acceptance Criteria | Validation |
+| :--- | :--- | :--- | :--- | :--- |
+
+## 5. Confidence Recommendation
+- Go / Go with Conditions / No-Go for release confidence
+```
+
+## Decision Rules
+
+- Do not recommend deleting a test unless equivalent or stronger coverage is identified.
+- Prefer strengthening assertions before adding more tests.
+- Prioritize deterministic contract guards over broad snapshot-style tests.