Compare commits
20
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6ec12a100a | ||
|
|
9be7c27410 | ||
|
|
a157489634 | ||
|
|
b5d6e60d45 | ||
|
|
f240486a7e | ||
|
|
5005cd7001 | ||
|
|
88ff4c2c71 | ||
|
|
5b6d5aaec4 | ||
|
|
2a2700b78c | ||
|
|
7162645ff8 | ||
|
|
44edffb8b7 | ||
|
|
c6817a074e | ||
|
|
7ac90d29dd | ||
|
|
0dc06f72ca | ||
|
|
cd11ea8255 | ||
|
|
bc21643e8c | ||
|
|
1ed5856db0 | ||
|
|
70695ff218 | ||
|
|
a238fb4dc3 | ||
|
|
b6f109cf91 |
@@ -0,0 +1,49 @@
|
||||
# personal-mcp MCP Usage
|
||||
|
||||
This repository is resource-first.
|
||||
|
||||
- Canonical skill guidance lives in `docs/skills/<skill-id>/SKILL.md`.
|
||||
- The machine-facing skill contract is FastMCP's native `skill://` resource family.
|
||||
|
||||
When a task appears to match a documented implementation pattern in `personal-mcp`, use this sequence:
|
||||
|
||||
1. Prefer an already attached native skill resource.
|
||||
2. Otherwise browse native MCP resources and compare `skill://<name>/SKILL.md` descriptions.
|
||||
3. Read the best matching main skill file, or at most 2 candidate main files.
|
||||
4. Read `skill://<name>/_manifest` only when supporting material may be useful.
|
||||
5. Fetch only the relevant supporting paths from that manifest.
|
||||
6. Reconcile skill guidance with the actual repository code before proposing or making changes.
|
||||
|
||||
Preferred MCP resource order:
|
||||
|
||||
1. `skill://<name>/SKILL.md`
|
||||
2. `skill://<name>/_manifest` when needed
|
||||
3. `skill://<name>/<supporting-path>` for selected supporting files
|
||||
|
||||
Selection rules:
|
||||
|
||||
- Prefer the closest `name` and `description` match.
|
||||
- Keep context bounded; do not load many skill documents speculatively.
|
||||
- If confidence is low after reading at most two main files, ask one clarifying question before loading more context.
|
||||
|
||||
Repository-specific guidance:
|
||||
|
||||
- For tasks about adding or modifying a skill, use `skill://copilot-customization/SKILL.md` when relevant.
|
||||
- Keep skills provider-native; do not add custom skill catalogs, per-skill resource modules, or skill-specific discovery tools.
|
||||
|
||||
|
||||
## Python Checks
|
||||
|
||||
After changes run these commands to confirm functionality. Resolve any errors
|
||||
|
||||
```python
|
||||
uv run ruff check
|
||||
```
|
||||
|
||||
```python
|
||||
uv run ty check
|
||||
```
|
||||
|
||||
```python
|
||||
uv run pytest
|
||||
```
|
||||
@@ -14,4 +14,6 @@ Primary references:
|
||||
- [Prompt contract](../../docs/contracts/prompt.md)
|
||||
- [Frontmatter contract](../../docs/contracts/frontmatter.md)
|
||||
- [URI contract](../../docs/contracts/uris.md)
|
||||
- [Zensical documentation authoring skill](../../docs/skills/zensical-docs/SKILL.md)
|
||||
- `skill://zensical-docs/SKILL.md`
|
||||
|
||||
Inspect `skill://zensical-docs/_manifest` only when a supporting documentation reference is needed.
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
---
|
||||
name: FastMCP Python Guidance
|
||||
description: Route FastMCP Python changes to the Personal MCP source-reference skill.
|
||||
applyTo: '**/*.py'
|
||||
---
|
||||
|
||||
A core part of this repository is [FastMCP](https://gofastmcp.com/servers/server).
|
||||
|
||||
For FastMCP implementation or protocol questions, load `skill://mcp-details/SKILL.md` first.
|
||||
|
||||
Inspect `skill://mcp-details/_manifest` only when a source reference is needed, then read the relevant supporting path. For FastMCP Python APIs, prefer the supporting reference that covers SDKs and FastMCP.
|
||||
|
||||
This repository exposes skills through native MCP resources and prompts through native MCP prompt operations.
|
||||
|
||||
Reconcile the skill guidance with the installed FastMCP version and the repository's existing implementation before editing.
|
||||
@@ -4,14 +4,15 @@ description: Route tests edits to the Personal MCP pytesting resource.
|
||||
applyTo: 'tests/**'
|
||||
---
|
||||
|
||||
When editing files under `tests/`, use `resource://skills/pytesting/document` as the primary guidance source for test scaffolding and pytest authoring decisions.
|
||||
When editing files under `tests/`, use `skill://pytesting/SKILL.md` as the primary guidance source for test scaffolding and pytest authoring decisions.
|
||||
|
||||
Execution pattern:
|
||||
|
||||
1. Load `resource://skills/pytesting/document` first.
|
||||
2. Apply only the portions relevant to the file being edited.
|
||||
3. Keep tests focused, deterministic, and aligned with repository conventions.
|
||||
4. Include source-document links for any feature-level recommendation.
|
||||
1. Load `skill://pytesting/SKILL.md` first.
|
||||
2. Inspect `skill://pytesting/_manifest` only when a supporting reference is needed.
|
||||
3. Apply only the portions relevant to the file being edited.
|
||||
4. Keep tests focused, deterministic, and aligned with repository conventions.
|
||||
5. Include source-document links for any feature-level recommendation.
|
||||
|
||||
If task intent is ambiguous, ask one clarifying question before editing.
|
||||
|
||||
|
||||
@@ -4,12 +4,12 @@ description: Route .vscode edits to the Personal MCP VS Code configuration skill
|
||||
applyTo: '.vscode/**'
|
||||
---
|
||||
|
||||
When editing files under `.vscode/`, use `resource://skills/vscode-configuration/document` as the primary guidance source.
|
||||
When editing files under `.vscode/`, use `skill://vscode-configuration/SKILL.md` as the primary guidance source.
|
||||
|
||||
Execution pattern:
|
||||
|
||||
1. Load `resource://skills/vscode-configuration/document` first.
|
||||
2. Select only the matching reference page for the current file type:
|
||||
1. Load `skill://vscode-configuration/SKILL.md` first.
|
||||
2. Inspect `skill://vscode-configuration/_manifest` and select only the matching reference page for the current file type:
|
||||
- `launch.json` -> debug launch configurations.
|
||||
- `tasks.json` -> tasks.json project tasks.
|
||||
- `mcp.json` -> mcp.json MCP server configuration.
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Use the Zensical docs MCP resource when editing Markdown documentat
|
||||
applyTo: '**/*.md'
|
||||
---
|
||||
|
||||
When editing Markdown files in this repository, use the Zensical docs resource `resource://skills/zensical-docs/document` for relevant documentation authoring guidance.
|
||||
When editing Markdown files in this repository, use `skill://zensical-docs/SKILL.md` for relevant documentation authoring guidance. Inspect `skill://zensical-docs/_manifest` only when a supporting reference is needed.
|
||||
|
||||
Prefer Zensical-native documentation conventions when they cover the need cleanly, while preserving expected MkDocs compatibility unless the Zensical guidance intentionally diverges.
|
||||
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
## Plan: Docs-First FastMCP End State
|
||||
|
||||
Create a docs-first FastMCP architecture where all Markdown remains in docs/ as the only source of truth, each skill is Anthropic-compatible in its own directory, skill metadata lives in SKILL.md frontmatter, and packaged docs are served through importlib.resources so stdio deployments work from installed wheels.
|
||||
|
||||
**Steps**
|
||||
1. Phase 1: Define the end-state content contract. Confirm canonical structure as docs/skills/<skill-id>/SKILL.md plus docs/skills/<skill-id>/references/..., with strict per-skill ownership and no metadata.yaml sidecar. Also define stable skill-id rules (kebab-case, immutable after release). Deliverable: update the current docs/ directory with the finalized end-state content contract from this step.
|
||||
2. Phase 1: Define SKILL.md frontmatter schema with Pydantic-compatible fields: id, version, name, description, tags, capabilities, depends_on, and references manifest entries. The references manifest must map logical reference ids to relative paths so each skill can reorganize references internally without changing global server code. Depends on step 1. Deliverable: update the current docs/ directory with the finalized SKILL.md frontmatter schema from this step.
|
||||
3. Phase 1: Define URI contract with explicit break-and-replace policy. Recommend resource://catalog/skills_index, resource://catalog/skills/{skill_id}, resource://skills/{skill_id}/document, resource://skills/{skill_id}/references/{ref_id}, and resource://docs/{path*}. Evolving URIs and reference ids requires direct replacement, with no aliases or compatibility shims. Depends on steps 1-2. Deliverable: update the current docs/ directory with the finalized URI contract and break-and-replace policy from this step.
|
||||
4. Phase 2: Build a docs registry loader that reads packaged docs via importlib.resources.files(...) Traversable APIs, parses SKILL.md frontmatter, validates schema, and creates an in-memory registry keyed by skill_id. Fail fast for duplicate ids, missing files, broken reference mappings, or invalid depends_on. Depends on steps 2-3.
|
||||
5. Phase 2: Register FastMCP resources from the registry using RFC6570 templates (including wildcard paths where appropriate), read-only/idempotent annotations, explicit mime types, and on_duplicate_resources="error" for startup safety. Depends on step 4.
|
||||
6. Phase 2: Add discovery surfaces as resources first, then tool fallback. Keep catalog discovery in resources, then add ResourcesAsTools for tool-only clients. Add thin discovery tools only for parity and optional BM25/regex tool search when catalog/tool volume grows enough to affect token efficiency. Define canonical fallback tool names (`list_resources`, `read_resource`, `search_patterns`, `get_pattern_by_id`, `get_skill_document_by_id`), research host-specific naming behavior for GitHub Copilot, Cursor, Claude Desktop, and generic MCP clients, and require client-side name mapping or intentionally documented aliases when providers expose namespaced wrappers. Depends on step 5.
|
||||
7. Phase 3: Implement packaging so docs/ is copied into package resource space at build time (wheel + sdist) while docs/ remains canonical in source control. Use importlib.resources at runtime only; avoid direct filesystem assumptions. Depends on steps 4-6.
|
||||
8. Phase 3: Remove materialization coupling between skill source modules and docs. The website build reads docs/ directly, while MCP reads packaged docs resources from the installed package. This preserves one authored source with two distribution surfaces. Depends on step 7.
|
||||
9. Phase 4: Add validation and CI gates: frontmatter schema checks, URI uniqueness checks, reference integrity checks, docs build check, package content check, and stdio smoke checks that read representative skill/document resources from an installed wheel. Depends on steps 5-8.
|
||||
10. Phase 4: Add long-term maintainability guardrails: architecture decision record for URI and schema contracts, skill authoring checklist, and release checklist for evolving references safely within one skill. Parallel with step 9 after core architecture is stable.
|
||||
|
||||
|
||||
|
||||
**Relevant files**
|
||||
- /home/john/Documents/prompts/docs/index.md — Keep top-level docs entry and explain docs-first architecture contract.
|
||||
- /home/john/Documents/prompts/docs/skills — Canonical location for all skill content, including SKILL.md and references.
|
||||
- /home/john/Documents/prompts/pyproject.toml — Build inclusion rules for packaged markdown resources in wheel/sdist.
|
||||
- /home/john/Documents/prompts/src/personal_mcp/main.py — App/server startup wiring for resource registry initialization.
|
||||
- /home/john/Documents/prompts/src/personal_mcp/mcp.py — FastMCP instance composition and transform registration.
|
||||
- /home/john/Documents/prompts/src/personal_mcp/catalog/server.py — Catalog resource and fallback discovery behavior.
|
||||
- /home/john/Documents/prompts/src/personal_mcp/skills/document_loader.py — Replace file-path assumptions with importlib.resources docs registry loading.
|
||||
- /home/john/Documents/prompts/src/personal_mcp/web/materialize_skill_docs.py — De-scope or retire materialization once docs-first runtime is authoritative.
|
||||
|
||||
**Verification**
|
||||
1. Run uv run zensical build to verify docs/ remains valid and site output is stable.
|
||||
2. Run uv run pytest -q with tests that validate frontmatter parsing, URI generation, reference mapping, and catalog responses.
|
||||
3. Run a packaging integrity check using importlib.resources.files(...) to confirm packaged docs resources exist and are readable from an installed wheel.
|
||||
4. Run a stdio MCP smoke test that lists resources and reads at least one skill document and one reference document.
|
||||
5. Run fallback-client smoke tests verifying list_resources/read_resource tools work and return expected metadata for both static and templated resources, and that GitHub Copilot, Cursor, Claude Desktop, and protocol-level SDK tests use canonical tool names or documented mapped aliases.
|
||||
|
||||
**Decisions**
|
||||
- Anthropic compatibility: strict skill directory pattern with SKILL.md and references subtree.
|
||||
- Metadata strategy: YAML frontmatter in SKILL.md (no separate metadata file).
|
||||
- Discovery strategy: resource-first catalog with tool fallback for tool-only MCP clients.
|
||||
- Included scope: ideal end-state architecture, contracts, validation, and packaging for stdio operation.
|
||||
- Excluded scope: migration mechanics from current implementation, backward-compat shim details, and docs visual redesign.
|
||||
|
||||
**Further Considerations**
|
||||
1. Prefer recursive references support under each skill plus frontmatter manifest ids, so skill teams can reorganize internal reference folders without URI churn.
|
||||
2. Define a hard rule that skill_id and directory name must match exactly to eliminate namespace/slug drift classes of bugs.
|
||||
3. Do not provide URI aliases; client updates must track canonical URI contract changes directly.
|
||||
@@ -1,169 +0,0 @@
|
||||
**Phase 3 Results: Packaging Contract and Surface Decoupling (Wheel/sdist Resources + Docs-Only Authoring)**
|
||||
|
||||
This section finalizes Phase 3 by defining how authored docs are packaged as runtime resources, how runtime loading avoids filesystem assumptions, and how website and MCP distribution surfaces are decoupled while sharing one authored source.
|
||||
|
||||
### Greenfield Framing (Normative)
|
||||
|
||||
This Phase 3 design assumes a full refactor with intentional break-and-replace behavior:
|
||||
|
||||
1. No compatibility shims, aliases, adapter layers, or dual-read runtime paths.
|
||||
2. No runtime dependency on repository checkout layout.
|
||||
3. Runtime docs access is package-resource-only.
|
||||
4. Canonical authoring remains in `docs/` in source control.
|
||||
|
||||
### Research Baseline (Packaging + Runtime)
|
||||
|
||||
Authoritative references used for this phase:
|
||||
|
||||
1. Python `importlib.resources` docs (`files`, `Traversable`, and zip-safe behavior)
|
||||
2. Python packaging guidance for wheel/sdist data inclusion
|
||||
3. Hatchling build target configuration guidance for including non-code files
|
||||
4. Existing repository constraints from Steps 4-5 (registry-first, deterministic startup, resource-first discovery)
|
||||
|
||||
Best-practice conclusions applied to this design:
|
||||
|
||||
1. Package docs as build artifacts so runtime reads work from installed wheels.
|
||||
2. Keep docs source-of-truth in one place (`docs/`) and project into package resource space at build time.
|
||||
3. Avoid `Path(__file__)`/repo-root probing in runtime paths.
|
||||
4. Enforce parity across wheel and sdist so local/dev/prod behavior does not drift.
|
||||
|
||||
### Phase 3 Responsibilities (Normative)
|
||||
|
||||
Phase 3 MUST:
|
||||
|
||||
1. Ensure authored markdown under `docs/` is included in wheel and sdist artifacts.
|
||||
2. Ensure runtime docs registry/document reads use `importlib.resources` only.
|
||||
3. Ensure MCP runtime behavior is independent of current working directory or checkout structure.
|
||||
4. Ensure website docs build continues to consume source `docs/` directly.
|
||||
5. Remove materialization/path-probing coupling from runtime loader code.
|
||||
6. Preserve deterministic packaged docs layout for registry/resource URI generation.
|
||||
|
||||
### Packaging Contract (Wheel + sdist)
|
||||
|
||||
Canonical packaging behavior:
|
||||
|
||||
1. Source-authored docs remain at repository root: `docs/`.
|
||||
2. Build projects docs into package resource space under `personal_mcp/docs/` inside artifacts.
|
||||
3. Runtime anchor for docs loading is `importlib.resources.files("personal_mcp").joinpath("docs")`.
|
||||
4. Build artifacts MUST include:
|
||||
- top-level docs pages used by discovery/overview
|
||||
- `docs/skills/<skill-id>/SKILL.md`
|
||||
- `docs/skills/<skill-id>/references/**`
|
||||
|
||||
Parity requirements:
|
||||
|
||||
1. Wheel and sdist contain equivalent docs content for runtime use.
|
||||
2. Missing docs resources in either artifact is a hard validation failure.
|
||||
|
||||
### Build-System Plan (pyproject + build)
|
||||
|
||||
Primary target file:
|
||||
|
||||
1. `pyproject.toml`
|
||||
|
||||
Configuration goals:
|
||||
|
||||
1. Add explicit build inclusion rules so docs resources are shipped in wheel artifacts.
|
||||
2. Add explicit sdist inclusion rules so docs are present for source builds.
|
||||
3. Keep inclusion deterministic and auditable (no implicit glob side effects beyond intended docs content).
|
||||
4. Ensure packaged destination path matches runtime anchor (`personal_mcp/docs`).
|
||||
|
||||
Implementation note:
|
||||
|
||||
1. Use Hatchling-native inclusion mapping (for example force-include or equivalent target-level include mapping) to project `docs/` into package resource space.
|
||||
2. Prefer one clear packaging path over multiple fallback packaging mechanisms.
|
||||
|
||||
### Runtime Loader Contract (No Filesystem Assumptions)
|
||||
|
||||
Primary target file:
|
||||
|
||||
1. `src/personal_mcp/skills/document_loader.py`
|
||||
|
||||
Required runtime behavior:
|
||||
|
||||
1. Remove repository-root discovery helpers and path-probing candidates.
|
||||
2. Remove metadata-based document path overrides that bypass canonical skill layout.
|
||||
3. Resolve SKILL and reference documents via package-resource-relative paths only.
|
||||
4. Keep reads UTF-8 and deterministic.
|
||||
5. Raise explicit errors for missing packaged resources; no fallback probing.
|
||||
|
||||
Prohibited runtime behavior:
|
||||
|
||||
1. No `Path(__file__).resolve().parents[...]` lookup for docs.
|
||||
2. No implicit fallback to source-tree `docs/` during runtime reads.
|
||||
3. No slug-guessing or namespace substitution for path recovery.
|
||||
|
||||
### Surface Decoupling Contract (Website vs MCP)
|
||||
|
||||
Website surface:
|
||||
|
||||
1. Website build pipeline consumes source `docs/` directly (`uv run zensical build`).
|
||||
2. Static output (`site/`) remains a build artifact served by web mounting logic.
|
||||
|
||||
MCP surface:
|
||||
|
||||
1. MCP runtime serves docs from packaged resources loaded by registry/resource handlers.
|
||||
2. MCP does not read `site/` and does not depend on website build artifacts.
|
||||
|
||||
Decoupling guarantees:
|
||||
|
||||
1. One authored source (`docs/`), two distribution surfaces (website + MCP runtime).
|
||||
2. Changes to website serving do not alter MCP resource loading semantics.
|
||||
3. Changes to MCP runtime loader do not require website materialization logic.
|
||||
|
||||
### Integration Plan for Existing Modules
|
||||
|
||||
Primary integration targets:
|
||||
|
||||
1. `pyproject.toml`: add wheel/sdist docs inclusion mapping.
|
||||
2. `src/personal_mcp/skills/document_loader.py`: replace filesystem probing with package-resource loading.
|
||||
3. `src/personal_mcp/main.py`: keep startup composition deterministic once registry/resource registration is in place.
|
||||
4. `src/personal_mcp/mcp.py`: maintain registry-driven resource composition as canonical runtime surface.
|
||||
5. `src/personal_mcp/web/docs_mount.py`: continue static-site mount behavior without coupling to MCP runtime docs loading.
|
||||
|
||||
Cleanup targets:
|
||||
|
||||
1. Remove obsolete references to materialization-only modules if no longer present/used.
|
||||
2. Remove dead code paths that attempt source-tree fallback loading.
|
||||
|
||||
### Validation and Test Plan (Phase 3 Scope)
|
||||
|
||||
Build/package validation:
|
||||
|
||||
1. Build wheel and sdist in CI/local.
|
||||
2. Inspect artifacts to confirm `personal_mcp/docs/**` exists and includes representative skill/reference files.
|
||||
3. Install built wheel in isolated environment and verify resource reads via `importlib.resources.files(...)`.
|
||||
|
||||
Runtime validation:
|
||||
|
||||
1. Run MCP in an environment where repo-root docs paths are unavailable and confirm reads still succeed.
|
||||
2. Verify representative URIs resolve (skill document and reference document).
|
||||
3. Confirm startup fails clearly if required packaged docs resources are missing.
|
||||
|
||||
Decoupling validation:
|
||||
|
||||
1. Run `uv run zensical build` to verify website pipeline still consumes source `docs/`.
|
||||
2. Confirm MCP runtime does not require `site/` presence.
|
||||
3. Confirm web static serving behavior is unchanged when docs are built.
|
||||
|
||||
Expected command path in this repo:
|
||||
|
||||
1. `uv run pytest -q`
|
||||
2. `uv run zensical build`
|
||||
|
||||
### Acceptance Criteria for Phase 3 Completion
|
||||
|
||||
Phase 3 is complete when all are true:
|
||||
|
||||
1. Wheel and sdist include docs resources in deterministic package paths.
|
||||
2. Runtime docs loading works from installed artifacts using `importlib.resources` only.
|
||||
3. Runtime docs loading has no checkout-path dependency and no fallback probing.
|
||||
4. Website docs build remains source-docs-driven and independent of MCP runtime loading.
|
||||
5. No compatibility shims, aliases, or dual runtime loader paths exist.
|
||||
|
||||
### Non-goals for Phase 3
|
||||
|
||||
1. No Step 6 discovery-tool fallback implementation details.
|
||||
2. No URI aliasing or backward-compat transition mechanics.
|
||||
3. No redesign of skill frontmatter/schema contracts already finalized in earlier steps.
|
||||
4. No web UI visual redesign or docs IA overhaul.
|
||||
@@ -1,84 +0,0 @@
|
||||
**Step 1 Results: End-State Content Contract**
|
||||
|
||||
This section finalizes Step 1 by defining the canonical authored content model.
|
||||
|
||||
### Step Deliverable
|
||||
|
||||
- Update the current `docs/` directory with the finalized Step 1 content contract from this document.
|
||||
|
||||
### Canonical source of truth
|
||||
|
||||
- All authored Markdown lives under `docs/`.
|
||||
- MCP resources and static docs are two distribution surfaces of the same authored files.
|
||||
- No parallel authored markdown is allowed in `src/` or other package-only paths.
|
||||
|
||||
### Canonical skill shape (Anthropic-compatible)
|
||||
|
||||
Each skill is one directory under `docs/skills/`:
|
||||
|
||||
```text
|
||||
docs/
|
||||
skills/
|
||||
<skill-id>/
|
||||
SKILL.md
|
||||
references/
|
||||
... (one or more markdown files, optional nested folders)
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `SKILL.md` is required for every skill.
|
||||
- `references/` is the only place for skill-specific supporting docs.
|
||||
- Nested folders inside `references/` are allowed so a skill can reorganize internals without changing global architecture.
|
||||
- Skill directories are independent ownership boundaries; no cross-skill file writes.
|
||||
|
||||
### File placement and ownership boundaries
|
||||
|
||||
- Top-level project docs stay in `docs/*.md`.
|
||||
- Skill docs stay in `docs/skills/<skill-id>/...`.
|
||||
- A skill may link to other skills, but must not store content inside another skill's directory.
|
||||
- Server/runtime code may index and serve docs, but must not be the source of authored markdown.
|
||||
|
||||
### Metadata location constraint
|
||||
|
||||
- Skill metadata is embedded in YAML frontmatter in `SKILL.md`.
|
||||
- No `metadata.yaml` sidecar in the end state.
|
||||
- Reference lookup metadata (ids to relative paths) is declared from `SKILL.md` frontmatter, not inferred as a hidden global convention.
|
||||
|
||||
### Skill-id contract (change-friendly)
|
||||
|
||||
`skill-id` is the public identifier and SHOULD satisfy all rules below:
|
||||
|
||||
- Format: lowercase kebab-case only.
|
||||
- Character set: `a-z`, `0-9`, and `-`.
|
||||
- Must start with a letter.
|
||||
- No underscores, spaces, dots, or uppercase characters.
|
||||
- Directory name should equal `skill-id` in each committed revision.
|
||||
- Frontmatter `id` should equal directory name in each committed revision.
|
||||
- Treat `skill-id` as immutable after release; any rename is a breaking replacement and clients must move to the new id.
|
||||
|
||||
Example valid ids:
|
||||
|
||||
- `fastapi-uv-docker`
|
||||
- `zensical-docs`
|
||||
- `pytesting`
|
||||
|
||||
Example invalid ids:
|
||||
|
||||
- `fastapi_uv_docker` (underscore)
|
||||
- `Zensical-Docs` (uppercase)
|
||||
- `docs.zensical` (dot)
|
||||
|
||||
### Invariants this contract guarantees
|
||||
|
||||
- One authored source tree (`docs/`) for both website and MCP.
|
||||
- One skill directory maps to one skill identity per revision.
|
||||
- Namespace/slug drift is minimized by keeping directory and frontmatter ids aligned per revision.
|
||||
- Per-skill reference structure can evolve without changing cross-skill architecture.
|
||||
- Packaging for stdio is deterministic because authored content is path-stable.
|
||||
|
||||
### Non-goals for Step 1
|
||||
|
||||
- No URI versioning policy details yet (handled in Step 3).
|
||||
- No full frontmatter schema details yet (handled in Step 2).
|
||||
- No migration instructions from current architecture (out of scope for this plan).
|
||||
@@ -1,298 +0,0 @@
|
||||
**Step 2 Results: SKILL.md Frontmatter and FastMCP Metadata Contract**
|
||||
|
||||
This section finalizes Step 2 by defining the canonical SKILL.md frontmatter schema, separating Anthropic-supported fields from repository extension fields, and mapping frontmatter to FastMCP-native metadata surfaces for resources and tools.
|
||||
|
||||
### Step Deliverable
|
||||
|
||||
- Update the current `docs/` directory with the finalized Step 2 frontmatter and metadata contract content from this document.
|
||||
|
||||
### Anthropic Frontmatter Support (Research Baseline)
|
||||
|
||||
Across Anthropic API and Agent Skills specification surfaces:
|
||||
|
||||
- Required for custom skill bundles: `name`, `description`.
|
||||
- `name` constraints (Agent Skills API docs): 1-64 chars, lowercase letters/numbers/hyphens, no XML tags, and must not use reserved words `anthropic` or `claude`.
|
||||
- `description` constraints (Agent Skills API docs): 1-1024 chars, non-empty, no XML tags.
|
||||
|
||||
Portable optional fields from the Agent Skills specification:
|
||||
|
||||
- `license`
|
||||
- `compatibility`
|
||||
- `metadata`
|
||||
- `allowed-tools` (experimental)
|
||||
|
||||
Claude Code-specific optional fields (supported by Claude Code skills docs):
|
||||
|
||||
- `when_to_use`, `argument-hint`, `arguments`
|
||||
- `disable-model-invocation`, `user-invocable`
|
||||
- `allowed-tools`, `disallowed-tools`
|
||||
- `model`, `effort`, `context`, `agent`, `hooks`, `paths`, `shell`
|
||||
|
||||
Contract decision for this repository:
|
||||
|
||||
- Treat `name` and `description` as required in all SKILL.md files, even where a client could infer defaults.
|
||||
- Keep Anthropic-facing semantics in standard fields and keep MCP indexing metadata in a namespaced extension block.
|
||||
- Preserve forward compatibility by allowing additive optional metadata fields over time.
|
||||
|
||||
### Canonical Frontmatter Schema For This Repository
|
||||
|
||||
Use this exact two-layer pattern:
|
||||
|
||||
1. Anthropic layer (portable): top-level fields intended for Anthropic/Agent Skills behavior.
|
||||
2. Repository layer (runtime indexing): one namespaced block, `x-personal-mcp`, for MCP catalog and routing metadata.
|
||||
|
||||
Canonical shape:
|
||||
|
||||
```yaml
|
||||
---
|
||||
name: <skill-id>
|
||||
description: <what this skill does and when to use it>
|
||||
|
||||
# Optional Anthropic/Agent Skills fields (use only when needed)
|
||||
when_to_use: <extra trigger guidance>
|
||||
allowed-tools: <space-separated string or YAML list>
|
||||
disable-model-invocation: false
|
||||
user-invocable: true
|
||||
license: <optional>
|
||||
compatibility: <optional>
|
||||
|
||||
# Repository-specific metadata (authoritative for MCP indexing)
|
||||
x-personal-mcp:
|
||||
id: <skill-id>
|
||||
version: <semver>
|
||||
tags:
|
||||
- <tag>
|
||||
capabilities:
|
||||
- resource://skills/<skill-id>/document
|
||||
depends_on: []
|
||||
references:
|
||||
<ref-id>:
|
||||
path: references/<file>.md
|
||||
mime_type: text/markdown
|
||||
title: <short title>
|
||||
---
|
||||
```
|
||||
|
||||
### Repository Metadata Field Rules (`x-personal-mcp`)
|
||||
|
||||
- `id` required: must follow Step 1 skill-id rules and equal directory name.
|
||||
- `version` required: semantic version string.
|
||||
- `tags` optional: list of kebab-case discovery labels.
|
||||
- `capabilities` required: list of MCP URIs this skill publishes.
|
||||
- `depends_on` optional: list of other skill ids.
|
||||
- `references` optional map:
|
||||
- key is `ref-id` (kebab-case).
|
||||
- `path` is a skill-relative markdown path and must stay inside the same skill directory.
|
||||
- nested folders under `references/` are allowed.
|
||||
- `mime_type` defaults to `text/markdown` if omitted.
|
||||
- `title` is an optional display label.
|
||||
- renaming `ref-id` values is allowed when needed; optional aliases may be used during transitions.
|
||||
|
||||
### Pydantic Models For Frontmatter Validation
|
||||
|
||||
Define the Step 2 contract with Pydantic v2 models and change-friendly validation.
|
||||
|
||||
Normative model sketch:
|
||||
|
||||
```python
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import PurePosixPath
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator
|
||||
|
||||
SKILL_ID_RE = re.compile(r"^[a-z][a-z0-9-]*$")
|
||||
SEMVER_RE = re.compile(r"^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:[-+][0-9A-Za-z.-]+)?$")
|
||||
|
||||
|
||||
class ReferenceEntry(BaseModel):
|
||||
model_config = ConfigDict(extra="ignore", str_strip_whitespace=True)
|
||||
|
||||
path: str
|
||||
mime_type: str = "text/markdown"
|
||||
title: str | None = None
|
||||
|
||||
@field_validator("path")
|
||||
@classmethod
|
||||
def validate_reference_path(cls, value: str) -> str:
|
||||
p = PurePosixPath(value)
|
||||
if p.is_absolute() or ".." in p.parts:
|
||||
raise ValueError("reference path must be a relative in-skill path")
|
||||
if not str(p).startswith("references/"):
|
||||
raise ValueError("reference path must stay under references/")
|
||||
if p.suffix.lower() != ".md":
|
||||
raise ValueError("reference path must target a markdown file")
|
||||
return str(p)
|
||||
|
||||
|
||||
class PersonalMcpMetadata(BaseModel):
|
||||
model_config = ConfigDict(extra="ignore", str_strip_whitespace=True)
|
||||
|
||||
id: str
|
||||
version: str
|
||||
tags: list[str] = Field(default_factory=list)
|
||||
capabilities: list[str] = Field(min_length=1)
|
||||
depends_on: list[str] = Field(default_factory=list)
|
||||
references: dict[str, ReferenceEntry] = Field(default_factory=dict)
|
||||
|
||||
@field_validator("id")
|
||||
@classmethod
|
||||
def validate_id(cls, value: str) -> str:
|
||||
if not SKILL_ID_RE.fullmatch(value):
|
||||
raise ValueError("id must be lowercase kebab-case and start with a letter")
|
||||
return value
|
||||
|
||||
@field_validator("version")
|
||||
@classmethod
|
||||
def validate_version(cls, value: str) -> str:
|
||||
if not SEMVER_RE.fullmatch(value):
|
||||
raise ValueError("version must be semver")
|
||||
return value
|
||||
|
||||
@field_validator("depends_on")
|
||||
@classmethod
|
||||
def validate_depends_on(cls, value: list[str]) -> list[str]:
|
||||
for dep in value:
|
||||
if not SKILL_ID_RE.fullmatch(dep):
|
||||
raise ValueError(f"invalid depends_on skill id: {dep}")
|
||||
return value
|
||||
|
||||
@field_validator("references")
|
||||
@classmethod
|
||||
def validate_reference_ids(cls, value: dict[str, ReferenceEntry]) -> dict[str, ReferenceEntry]:
|
||||
for ref_id in value:
|
||||
if not SKILL_ID_RE.fullmatch(ref_id):
|
||||
raise ValueError(f"invalid reference id: {ref_id}")
|
||||
return value
|
||||
|
||||
@model_validator(mode="after")
|
||||
def ensure_primary_capability(self) -> "PersonalMcpMetadata":
|
||||
expected = f"resource://skills/{self.id}/document"
|
||||
if expected not in self.capabilities:
|
||||
raise ValueError(f"capabilities must include {expected}")
|
||||
return self
|
||||
|
||||
|
||||
class SkillFrontmatter(BaseModel):
|
||||
model_config = ConfigDict(extra="ignore", str_strip_whitespace=True)
|
||||
|
||||
# Anthropic/Agent Skills fields
|
||||
name: str = Field(min_length=1, max_length=64)
|
||||
description: str = Field(min_length=1, max_length=1024)
|
||||
when_to_use: str | None = None
|
||||
allowed_tools: str | list[str] | None = Field(default=None, alias="allowed-tools")
|
||||
disallowed_tools: str | list[str] | None = Field(default=None, alias="disallowed-tools")
|
||||
disable_model_invocation: bool | None = Field(default=None, alias="disable-model-invocation")
|
||||
user_invocable: bool | None = Field(default=None, alias="user-invocable")
|
||||
argument_hint: str | None = Field(default=None, alias="argument-hint")
|
||||
arguments: str | list[str] | None = None
|
||||
license: str | None = None
|
||||
compatibility: str | None = None
|
||||
metadata: dict[str, str] | None = None
|
||||
|
||||
# Repository extension block
|
||||
x_personal_mcp: PersonalMcpMetadata = Field(alias="x-personal-mcp")
|
||||
|
||||
@field_validator("name")
|
||||
@classmethod
|
||||
def validate_name(cls, value: str) -> str:
|
||||
if not SKILL_ID_RE.fullmatch(value):
|
||||
raise ValueError("name must be lowercase kebab-case and start with a letter")
|
||||
if "anthropic" in value or "claude" in value:
|
||||
raise ValueError("name must not contain reserved words anthropic or claude")
|
||||
return value
|
||||
|
||||
@model_validator(mode="after")
|
||||
def cross_validate(self) -> "SkillFrontmatter":
|
||||
if self.x_personal_mcp.id != self.name:
|
||||
raise ValueError("x-personal-mcp.id must exactly match name")
|
||||
return self
|
||||
|
||||
|
||||
def validate_skill_frontmatter(raw: dict[str, Any], skill_dir_name: str) -> SkillFrontmatter:
|
||||
model = SkillFrontmatter.model_validate(raw)
|
||||
if model.name != skill_dir_name:
|
||||
raise ValueError("frontmatter name must exactly match skill directory name")
|
||||
return model
|
||||
```
|
||||
|
||||
Validation behavior contract:
|
||||
|
||||
- Validate required core fields and relationships during registry load before FastMCP resource/tool registration.
|
||||
- Allow unknown additive fields so frontmatter can evolve without blocking startup.
|
||||
- Treat hard contract violations (missing required fields, invalid ids, broken required mappings) as startup errors.
|
||||
- Treat non-critical compatibility issues as warnings when possible.
|
||||
- Error messages should include skill path and failing field for CI readability.
|
||||
|
||||
Projection mode contract (for Anthropic API upload pipelines):
|
||||
|
||||
- Parse with `SkillFrontmatter` first.
|
||||
- Emit Anthropic-safe frontmatter with standard fields only.
|
||||
- Serialize repository metadata into standard `metadata` as namespaced keys.
|
||||
- Preserve the canonical authored source in `x-personal-mcp`; projection output is a build artifact.
|
||||
|
||||
### Anthropic Upload Compatibility Rule
|
||||
|
||||
- Anthropic documentation guarantees behavior for standard frontmatter fields but does not explicitly guarantee handling of arbitrary unknown top-level keys.
|
||||
- Therefore, publishing pipelines that target strict API compatibility should support a projection mode that emits only standard frontmatter fields for upload.
|
||||
- In projection mode, repository extension metadata is serialized into the standard `metadata` field (for example as namespaced keys or JSON-encoded values), while source-of-truth authoring remains in `x-personal-mcp`.
|
||||
|
||||
### FastMCP Native Metadata Surfaces (Research Baseline)
|
||||
|
||||
Resources (`@mcp.resource` and templates) support native definition metadata:
|
||||
|
||||
- `name`, `description`, `mime_type`, `tags`
|
||||
- `annotations` (`readOnlyHint`, `idempotentHint`)
|
||||
- `icons`
|
||||
- `meta` (custom metadata passed through to the MCP client resource object)
|
||||
- `version`
|
||||
- `enabled` (deprecated in v3; prefer server-level `mcp.enable()` / `mcp.disable()`)
|
||||
|
||||
Resources support runtime metadata:
|
||||
|
||||
- `ResourceContent.meta` (item-level)
|
||||
- `ResourceResult.meta` (result-level `_meta`)
|
||||
|
||||
Tools (`@mcp.tool`) support native definition metadata:
|
||||
|
||||
- `name`, `description`, `tags`
|
||||
- `annotations` (`title`, `readOnlyHint`, `destructiveHint`, `idempotentHint`, `openWorldHint`)
|
||||
- `icons`
|
||||
- `meta` (custom metadata passed through to the MCP client tool object)
|
||||
- `version`
|
||||
- `timeout`, `output_schema`, `run_in_thread`
|
||||
- `enabled` (deprecated in v3; prefer server-level `mcp.enable()` / `mcp.disable()`)
|
||||
|
||||
Tools support runtime metadata:
|
||||
|
||||
- `ToolResult.meta` (execution-level metadata for each call)
|
||||
|
||||
### Frontmatter To FastMCP Mapping Contract
|
||||
|
||||
At server startup, map `x-personal-mcp` fields into FastMCP registration as follows:
|
||||
|
||||
- `x-personal-mcp.id` -> canonical URI namespace and identity checks.
|
||||
- `description` -> default `description` for the primary skill document resource.
|
||||
- `x-personal-mcp.tags` -> `tags` on resources/tools.
|
||||
- `x-personal-mcp.version` -> `version` on resources/tools.
|
||||
- `x-personal-mcp.capabilities` -> registered URI list plus catalog exposure.
|
||||
- `x-personal-mcp.references[*]` -> resource templates or concrete resources with:
|
||||
- `mime_type` from reference entry (or default)
|
||||
- `meta` including `skill_id`, `ref_id`, and source `path`
|
||||
- read-only annotations for documentation resources
|
||||
- `x-personal-mcp.depends_on` -> catalog dependency graph metadata and validation checks.
|
||||
|
||||
### Invariants This Contract Guarantees
|
||||
|
||||
- Anthropic-required frontmatter stays valid for custom skill upload and Claude Code loading.
|
||||
- MCP-specific metadata remains embedded in SKILL.md frontmatter, with no `metadata.yaml` sidecar.
|
||||
- FastMCP registration uses only native metadata fields for resources/tools.
|
||||
- Reference ids and metadata can evolve with low-friction updates while internal file layout under `references/` stays refactor-friendly.
|
||||
|
||||
### Non-goals For Step 2
|
||||
|
||||
- No URI versioning/deprecation rollout policy details (handled in Step 3).
|
||||
- No migration script design from existing `metadata.yaml` files.
|
||||
- No runtime caching/indexing performance tuning details.
|
||||
@@ -1,114 +0,0 @@
|
||||
**Step 3 Results: URI Contract and Compatibility Policy**
|
||||
|
||||
This section finalizes Step 3 by defining the canonical resource URI contract, template parameter rules, and explicit compatibility/versioning policy for URIs and reference ids.
|
||||
|
||||
### Step Deliverable
|
||||
|
||||
- Update the current `docs/` directory with the finalized Step 3 URI contract and compatibility policy content from this document.
|
||||
|
||||
### Canonical URI Surface (Normative)
|
||||
|
||||
The public, preferred URIs are:
|
||||
|
||||
1. `resource://catalog/skills_index`
|
||||
2. `resource://catalog/skills/{skill_id}`
|
||||
3. `resource://skills/{skill_id}/document`
|
||||
4. `resource://skills/{skill_id}/references/{ref_id}`
|
||||
5. `resource://docs/{path*}`
|
||||
|
||||
Contract intent:
|
||||
|
||||
- Catalog URIs are discovery surfaces.
|
||||
- Skill URIs are primary per-skill guidance surfaces.
|
||||
- Docs wildcard URI is a direct authored-markdown access surface under `docs/`.
|
||||
|
||||
### URI Semantics
|
||||
|
||||
`resource://catalog/skills_index`
|
||||
|
||||
- Returns a compact list of skill records for discovery.
|
||||
- One entry per `skill_id`.
|
||||
- Must include enough metadata for client-side selection (at minimum id, name, description, tags, capabilities).
|
||||
|
||||
`resource://catalog/skills/{skill_id}`
|
||||
|
||||
- Returns one normalized record for `skill_id`.
|
||||
- Must include canonical document URI and declared reference ids.
|
||||
- Returns not-found when `skill_id` does not exist.
|
||||
|
||||
`resource://skills/{skill_id}/document`
|
||||
|
||||
- Returns the canonical `SKILL.md` authored content for that skill.
|
||||
- `skill_id` must match Step 1 stable id rules.
|
||||
|
||||
`resource://skills/{skill_id}/references/{ref_id}`
|
||||
|
||||
- Returns one reference document declared in the skill frontmatter references manifest.
|
||||
- `ref_id` is the stable public handle for that reference document.
|
||||
|
||||
`resource://docs/{path*}`
|
||||
|
||||
- Returns authored markdown at a normalized relative path under `docs/`.
|
||||
- Supports nested paths via RFC6570 wildcard expansion.
|
||||
- Typical examples: `index.md`, `usage.md`, `skills/<skill-id>/SKILL.md`, `skills/<skill-id>/references/<file>.md`.
|
||||
|
||||
### Template Parameter and Validation Rules
|
||||
|
||||
`skill_id`
|
||||
|
||||
- Lowercase kebab-case.
|
||||
- Must satisfy Step 1 stable id rules.
|
||||
|
||||
`ref_id`
|
||||
|
||||
- Lowercase kebab-case.
|
||||
- Must be declared in the skill’s references manifest.
|
||||
|
||||
`path*`
|
||||
|
||||
- Relative POSIX path only.
|
||||
- No leading slash.
|
||||
- No `..` traversal segments.
|
||||
- Resolves only inside `docs/`.
|
||||
- This surface is markdown-only in end state (`.md` files).
|
||||
|
||||
### URI Versioning Policy
|
||||
|
||||
Default rule:
|
||||
|
||||
- Keep URIs unversioned by default.
|
||||
- Allow URI and payload updates when they improve clarity or implementation simplicity.
|
||||
|
||||
Breaking-change rule:
|
||||
|
||||
- Breaking changes use direct replacement of the canonical URI family.
|
||||
- No compatibility aliases or dual URI families are maintained in this greenfield phase.
|
||||
|
||||
FastMCP version metadata usage:
|
||||
|
||||
- Resource `version` metadata MAY be used for implementation/version discovery.
|
||||
- URI readability and maintainability remain the primary contract.
|
||||
|
||||
### Reference ID Compatibility Policy
|
||||
|
||||
`ref_id` is the public identifier for a reference document, separate from file path.
|
||||
|
||||
Rules:
|
||||
|
||||
- Prefer keeping `ref_id` stable when practical.
|
||||
- File paths may change without URI churn as long as the mapped `ref_id` resolves.
|
||||
- If a reference is renamed, introduce a new `ref_id` and treat the old one as retired.
|
||||
- Avoid reusing retired `ref_id` values for unrelated content.
|
||||
|
||||
### Invariants This Contract Guarantees
|
||||
|
||||
- One canonical URI pattern per core capability surface.
|
||||
- Fast, low-friction URI evolution through direct replacement of canonical URIs.
|
||||
- A single canonical catalog URI family with no alias maintenance overhead.
|
||||
- Reference mappings can evolve with minimal churn.
|
||||
|
||||
### Non-goals For Step 3
|
||||
|
||||
- No implementation-specific transform wiring details (`VersionFilter`, mounts, provider composition).
|
||||
- No migration script mechanics for auto-generating aliases.
|
||||
- No authorization policy design for URI-level access control.
|
||||
@@ -1,248 +0,0 @@
|
||||
**Step 4 Results: Docs Registry Loader Design (importlib.resources + Fail-Fast Validation)**
|
||||
|
||||
This section finalizes Step 4 by defining a production-ready docs registry loader that reads packaged docs through Python resource APIs, parses SKILL.md frontmatter, validates schema and cross-links, and builds an immutable in-memory registry keyed by skill_id.
|
||||
|
||||
### Greenfield Framing (Normative)
|
||||
|
||||
This Step 4 design is for the greenfield target state:
|
||||
|
||||
1. No legacy metadata sidecars (`metadata.yaml`) are part of the runtime contract.
|
||||
2. No dual-loader compatibility path is required.
|
||||
3. Registry loading from packaged resources is the only runtime source of truth.
|
||||
4. Compatibility shims are prohibited.
|
||||
|
||||
### Research Baseline (Python + Design Guidance)
|
||||
|
||||
Authoritative references used for this step:
|
||||
|
||||
1. Python `importlib.resources` docs (`files`, `as_file`, `Traversable` APIs)
|
||||
2. Python `importlib.resources.abc` docs (`Traversable`, path traversal semantics, joinpath compatibility notes)
|
||||
3. Pydantic v2 model/validation docs (`model_validate`, `ValidationError`, strictness and extra handling)
|
||||
4. Python packaging guidance for including package data in wheels/sdists
|
||||
|
||||
Best-practice conclusions applied to this design:
|
||||
|
||||
1. Prefer `importlib.resources.files(<package>).joinpath(...)` over filesystem assumptions so stdio deployments from installed wheels work.
|
||||
2. Treat resources as potentially non-filesystem artifacts (zip-import compatible); only use `as_file(...)` when an actual OS path is required.
|
||||
3. Validate metadata with explicit Pydantic models and fail startup on contract violations.
|
||||
4. Keep registry load deterministic (sorted traversal, stable error messages, no hidden fallback mutations).
|
||||
5. Resolve references via manifest ids declared in frontmatter, not by global file conventions.
|
||||
|
||||
### Loader Responsibilities (Normative)
|
||||
|
||||
The Step 4 loader MUST:
|
||||
|
||||
1. Read canonical docs from package resources (not repo-root paths).
|
||||
2. Discover all skill directories under `docs/skills/` in packaged resources.
|
||||
3. For each skill, read and parse `SKILL.md` frontmatter.
|
||||
4. Validate frontmatter using the Step 2 schema contract.
|
||||
5. Validate directory/id invariants from Step 1 (directory name equals frontmatter id).
|
||||
6. Validate URI/reference semantics from Step 3 assumptions.
|
||||
7. Build a single in-memory registry keyed by `skill_id`.
|
||||
8. Fail fast on any integrity error before FastMCP resource registration.
|
||||
9. Precompute compact discovery projections so index resources can be served without reading full markdown bodies at request time.
|
||||
|
||||
### Package Resource Contract
|
||||
|
||||
Runtime anchor:
|
||||
|
||||
1. The loader resolves content from an importable package anchor, for example `personal_mcp`.
|
||||
2. Docs root is located as `files(anchor).joinpath("docs")` when docs are packaged at package root, or an equivalent configured subpath.
|
||||
3. Skill root is `docs/skills`.
|
||||
|
||||
Resource assumptions:
|
||||
|
||||
1. `SKILL.md` is UTF-8 text.
|
||||
2. Reference files declared in frontmatter are UTF-8 markdown by default unless otherwise declared.
|
||||
3. Path resolution always remains inside the same skill directory.
|
||||
|
||||
### Registry Data Model
|
||||
|
||||
Build immutable runtime records with explicit structure:
|
||||
|
||||
1. `SkillRecord`
|
||||
- `skill_id`
|
||||
- `name`
|
||||
- `description`
|
||||
- `version`
|
||||
- `tags`
|
||||
- `capabilities`
|
||||
- `depends_on`
|
||||
- `document_uri`
|
||||
- `document_relpath` (canonical resource-relative path)
|
||||
- `references` map keyed by `ref_id`
|
||||
2. `ReferenceRecord`
|
||||
- `ref_id`
|
||||
- `uri`
|
||||
- `relpath`
|
||||
- `mime_type`
|
||||
- `title`
|
||||
3. `DocsRegistry`
|
||||
- `skills_by_id: dict[str, SkillRecord]`
|
||||
- `skills_in_load_order: list[str]` (deterministic ordering)
|
||||
- helper indexes for catalog payload generation
|
||||
- `skills_summary_in_load_order: list[SkillSummaryRecord]` for progressive discovery responses
|
||||
- filter indexes (for example by tag/capability) derived once at startup
|
||||
|
||||
4. `SkillSummaryRecord`
|
||||
- `skill_id`
|
||||
- `name`
|
||||
- `description`
|
||||
- `tags`
|
||||
- `capabilities`
|
||||
- `document_uri`
|
||||
- optional `version`
|
||||
|
||||
Immutability rule:
|
||||
|
||||
1. Once built, registry records are treated as read-only for the process lifetime.
|
||||
2. No runtime mutation during requests; refresh only via process restart.
|
||||
|
||||
### Frontmatter Parsing Contract
|
||||
|
||||
`SKILL.md` parse steps:
|
||||
|
||||
1. Read full markdown text from resource.
|
||||
2. Parse YAML frontmatter block at file start (between the first two `---` delimiters).
|
||||
3. Parse YAML with safe loader semantics.
|
||||
4. Validate parsed object with Step 2 Pydantic model(s).
|
||||
5. Preserve markdown body as document content payload.
|
||||
|
||||
Parsing failure behavior:
|
||||
|
||||
1. Missing frontmatter block: startup error.
|
||||
2. Invalid YAML: startup error with skill path and YAML parser detail.
|
||||
3. Missing required fields (`name`, `description`, `x-personal-mcp` contract fields): startup error.
|
||||
|
||||
### Validation Pipeline (Fail-Fast)
|
||||
|
||||
Validation happens in this order:
|
||||
|
||||
1. Structural discovery validation
|
||||
- skill directory exists under `docs/skills`
|
||||
- required `SKILL.md` exists for each discovered skill
|
||||
2. Schema validation
|
||||
- Pydantic frontmatter validation for all required and constrained fields
|
||||
3. Identity validation
|
||||
- frontmatter `name` equals `x-personal-mcp.id`
|
||||
- frontmatter id equals skill directory name
|
||||
4. Reference manifest validation
|
||||
- unique `ref_id` keys per skill
|
||||
- each manifest path is relative, in-skill, and under `references/`
|
||||
- each manifest target exists and is a file
|
||||
5. Dependency graph validation
|
||||
- every `depends_on` target exists in discovered skill set
|
||||
- no self-dependency
|
||||
- cycle detection enabled (hard error on cycle)
|
||||
6. Capability sanity checks
|
||||
- required primary capability `resource://skills/{skill_id}/document` is present
|
||||
7. Global uniqueness checks
|
||||
- no duplicate `skill_id`
|
||||
- no duplicate canonical resource URIs generated from registry
|
||||
8. Discovery payload checks
|
||||
- summary fields required by catalog index are present and non-empty
|
||||
- summary generation does not require reading markdown body content during request handling
|
||||
|
||||
### Error Model and Reporting
|
||||
|
||||
Error handling contract:
|
||||
|
||||
1. Collect errors per validation phase for clarity, then raise one startup exception containing all findings.
|
||||
2. Error messages must include:
|
||||
- skill id (when known)
|
||||
- packaged relative path
|
||||
- violated rule
|
||||
- actionable fix hint
|
||||
3. If any error exists, registry is not published and FastMCP resource registration does not proceed.
|
||||
|
||||
Recommended exception shape:
|
||||
|
||||
1. `DocsRegistryValidationError(errors: list[RegistryIssue])`
|
||||
2. `RegistryIssue` fields: `code`, `message`, `skill_id`, `path`, `hint`
|
||||
|
||||
### Determinism and Runtime Safety
|
||||
|
||||
Determinism rules:
|
||||
|
||||
1. Traverse directories in sorted order.
|
||||
2. Normalize all stored relative paths to POSIX form.
|
||||
3. Normalize ids/tags exactly once at parse boundary.
|
||||
4. Produce stable catalog ordering to reduce client churn.
|
||||
5. Produce stable summary projections and filter indexes from the same normalized source records.
|
||||
|
||||
Runtime safety rules:
|
||||
|
||||
1. No dependence on `Path(__file__)` or repository root.
|
||||
2. No ad-hoc fallback probing across multiple locations.
|
||||
3. No lazy validation deferred until first request.
|
||||
|
||||
### Integration Plan for Existing Modules
|
||||
|
||||
Primary integration target:
|
||||
|
||||
1. Implement the canonical package-resource-based registry loader in `src/personal_mcp/skills/document_loader.py` as the only supported runtime loader path.
|
||||
|
||||
Catalog integration:
|
||||
|
||||
1. Update `src/personal_mcp/catalog/server.py` to consume the shared in-memory registry as the only catalog data source.
|
||||
2. Keep catalog payload normalization deterministic and sourced from registry records only.
|
||||
|
||||
Startup wiring:
|
||||
|
||||
1. Initialize registry once during app/server startup in `src/personal_mcp/main.py` or equivalent composition point.
|
||||
2. Pass registry to resource registration step (Step 5).
|
||||
|
||||
### Proposed Loader API Surface
|
||||
|
||||
Use a small, testable API:
|
||||
|
||||
1. `load_docs_registry(*, package_anchor: str, docs_root: str = "docs") -> DocsRegistry`
|
||||
2. `read_skill_document(registry: DocsRegistry, skill_id: str) -> DocumentPayload`
|
||||
3. `read_skill_reference(registry: DocsRegistry, skill_id: str, ref_id: str) -> DocumentPayload`
|
||||
|
||||
Design constraints:
|
||||
|
||||
1. Loader functions are pure relative to package resources and input args.
|
||||
2. No global mutable singleton required for unit tests.
|
||||
3. Caching is explicit and owned by startup composition.
|
||||
|
||||
### Test and Validation Plan (Step 4 Scope)
|
||||
|
||||
Unit tests:
|
||||
|
||||
1. valid multi-skill registry load from packaged test fixtures
|
||||
2. duplicate id detection
|
||||
3. missing SKILL.md detection
|
||||
4. invalid frontmatter field constraints
|
||||
5. broken reference target detection
|
||||
6. invalid depends_on target detection
|
||||
7. cycle detection in depends_on graph
|
||||
8. deterministic output ordering across runs
|
||||
|
||||
Packaging/runtime tests:
|
||||
|
||||
1. install built wheel in isolated env
|
||||
2. load registry via `importlib.resources.files(...)`
|
||||
3. assert representative skill document/reference are readable
|
||||
|
||||
Expected command path in this repo:
|
||||
|
||||
1. `uv run pytest -q`
|
||||
|
||||
### Acceptance Criteria for Step 4 Completion
|
||||
|
||||
Step 4 is complete when all are true:
|
||||
|
||||
1. Registry loads exclusively from packaged resources.
|
||||
2. All Step 2 and Step 3 dependent validations are enforced at startup.
|
||||
3. Invalid docs state blocks startup with actionable diagnostics.
|
||||
4. Registry is deterministic and immutable for runtime use.
|
||||
5. Catalog and later resource registration can consume registry without direct filesystem scanning.
|
||||
|
||||
### Non-goals for Step 4
|
||||
|
||||
1. No FastMCP resource registration wiring details (Step 5).
|
||||
2. No discovery-tool fallback behavior design (Step 6).
|
||||
3. No final packaging/build-system migration mechanics (Step 7).
|
||||
4. No backward-compat alias rollout mechanics in the greenfield baseline.
|
||||
5. No compatibility layer of any kind (URI aliases, dual reads, adapter shims, or legacy schema bridges).
|
||||
@@ -1,221 +0,0 @@
|
||||
**Step 5 Results: Registry-Driven FastMCP Resource Registration (RFC6570 + Startup Safety)**
|
||||
|
||||
This section finalizes Step 5 by defining how FastMCP resources are registered from the Step 4 docs registry using RFC6570 URI templates, explicit metadata, and strict duplicate-registration safety.
|
||||
|
||||
### Greenfield Framing (Normative)
|
||||
|
||||
This Step 5 design is for the greenfield target state:
|
||||
|
||||
1. Registry-driven resources are the primary and authoritative discovery/read surface.
|
||||
2. No legacy per-skill hardcoded resource registration is retained.
|
||||
3. Resource contracts are defined for net-new clients and replace prior contracts without transition shims.
|
||||
4. Step 6 tool fallback layers on top of this resource contract, not as a competing source of truth.
|
||||
5. Breaking changes are intentional in this full-refactor phase.
|
||||
|
||||
### Research Baseline (FastMCP + URI Templates)
|
||||
|
||||
Authoritative references used for this step:
|
||||
|
||||
1. FastMCP Resources and Templates docs (resource decorator, template behavior)
|
||||
2. FastMCP RFC6570 support docs (simple params, wildcard params, query params)
|
||||
3. FastMCP duplicate handling docs (`on_duplicate_resources`)
|
||||
4. FastMCP annotations guidance (`readOnlyHint`, `idempotentHint`)
|
||||
|
||||
Best-practice conclusions applied to this design:
|
||||
|
||||
1. Use URI templates for parameterized resources instead of generating N static resource handlers.
|
||||
2. Use wildcard template parameters (`{path*}`) for hierarchical docs paths.
|
||||
3. Set startup duplicate policy to `on_duplicate_resources="error"` to fail fast on contract collisions.
|
||||
4. Set explicit `mime_type` and resource annotations for all docs resources.
|
||||
5. Keep registration deterministic and sourced only from the validated Step 4 registry.
|
||||
|
||||
### Registration Responsibilities (Normative)
|
||||
|
||||
The Step 5 registration layer MUST:
|
||||
|
||||
1. Consume only the validated in-memory registry produced by Step 4.
|
||||
2. Register canonical resource discovery surfaces and skill document/reference surfaces.
|
||||
3. Use RFC6570 templates where URI patterns are parameterized.
|
||||
4. Use wildcard templates where path depth is variable.
|
||||
5. Attach read-only/idempotent annotations to documentation resources.
|
||||
6. Set explicit MIME types for all registered resources.
|
||||
7. Fail startup if duplicate URI/template keys are encountered.
|
||||
|
||||
### Canonical Resource Surface (from Registry)
|
||||
|
||||
The preferred resources registered in this phase are:
|
||||
|
||||
1. `resource://catalog/skills_index`
|
||||
2. `resource://catalog/skills_index{?q,tag,capability,cursor,limit}` (optional filtered/paginated discovery template)
|
||||
3. `resource://catalog/skills/{skill_id}`
|
||||
4. `resource://skills/{skill_id}/document`
|
||||
5. `resource://skills/{skill_id}/references/{ref_id}`
|
||||
6. `resource://docs/{path*}`
|
||||
|
||||
Registration decision rules:
|
||||
|
||||
1. Use static resource registration for fixed singleton endpoints (for example `skills_index`).
|
||||
2. Use template registration for parameterized endpoints (`{skill_id}`, `{ref_id}`) and optional discovery query templates.
|
||||
3. Use wildcard template registration for hierarchical docs routing (`{path*}`).
|
||||
4. Keep the singleton and query-template discovery surfaces semantically equivalent (same schema, query template adds filtering/pagination only).
|
||||
|
||||
### Progressive Discovery Contract
|
||||
|
||||
Discovery-first behavior for Step 5 resources:
|
||||
|
||||
1. `skills_index` returns summaries only (no embedded full SKILL.md bodies).
|
||||
2. Each summary includes canonical follow-up URIs so clients can progressively fetch detail (`catalog/skills/{skill_id}` then `skills/{skill_id}/document`).
|
||||
3. Filtered/paginated discovery uses RFC6570 query params (`q`, `tag`, `capability`, `cursor`, `limit`) with deterministic ordering.
|
||||
4. Handlers should enforce bounded page size and return explicit continuation metadata when pagination is active.
|
||||
5. Errors for unsupported filter params or invalid cursor/limit are explicit and actionable.
|
||||
|
||||
### RFC6570 Template Contract
|
||||
|
||||
Path parameters:
|
||||
|
||||
1. `{skill_id}` and `{ref_id}` are single-segment template params.
|
||||
2. `{path*}` is a wildcard param and may capture multi-segment paths separated by `/`.
|
||||
|
||||
Validation contract at resource-read time:
|
||||
|
||||
1. `skill_id` must exist in registry.
|
||||
2. `ref_id` must exist in that skill’s reference manifest.
|
||||
3. wildcard `path*` must normalize to an allowed docs-relative markdown path.
|
||||
4. invalid params return explicit not-found or validation errors (no silent fallback).
|
||||
|
||||
Template function signature contract:
|
||||
|
||||
1. Required URI params must exist as function parameters.
|
||||
2. Avoid hidden implicit params not represented in template.
|
||||
3. Keep template handlers side-effect free.
|
||||
|
||||
### Metadata and Annotation Contract
|
||||
|
||||
Each docs/resource registration should specify explicit metadata:
|
||||
|
||||
1. `mime_type`
|
||||
- skill docs and references: `text/markdown`
|
||||
- catalog payloads: `application/json`
|
||||
2. `annotations`
|
||||
- `readOnlyHint: true`
|
||||
- `idempotentHint: true`
|
||||
3. `tags`
|
||||
- include stable categories such as `catalog`, `skill-doc`, `reference`, `docs`
|
||||
4. `version`
|
||||
- project-defined version from registry metadata where applicable
|
||||
5. `meta`
|
||||
- include normalized identifiers (for example `skill_id`, `ref_id`, `source_relpath`) when useful
|
||||
|
||||
### Startup Safety and Duplicate Policy
|
||||
|
||||
FastMCP initialization contract for this phase:
|
||||
|
||||
1. Construct the root server with `on_duplicate_resources="error"`.
|
||||
2. Register all Step 5 resources during startup composition before serving traffic.
|
||||
3. Treat duplicate registration as a hard startup failure.
|
||||
|
||||
Duplicate conflict classes covered:
|
||||
|
||||
1. static URI vs static URI collision
|
||||
2. static URI vs template key collision
|
||||
3. template URI vs template URI collision
|
||||
4. conflicting registrations introduced by future aliases without explicit migration handling
|
||||
|
||||
### Registration Architecture
|
||||
|
||||
Use one dedicated registration module that converts registry records into FastMCP resources.
|
||||
|
||||
Recommended API:
|
||||
|
||||
1. `register_docs_resources(mcp: FastMCP, registry: DocsRegistry) -> None`
|
||||
|
||||
Responsibilities of `register_docs_resources`:
|
||||
|
||||
1. register singleton catalog resources
|
||||
2. register parameterized catalog/detail templates
|
||||
3. register skill document and reference templates
|
||||
4. register docs wildcard template
|
||||
5. apply shared annotations and MIME defaults consistently
|
||||
|
||||
Separation of concerns:
|
||||
|
||||
1. Step 4 validates and normalizes docs state.
|
||||
2. Step 5 only registers handlers and reads from validated registry state.
|
||||
3. Request handlers do not re-discover filesystem/package structure.
|
||||
|
||||
### Handler Behavior Contract
|
||||
|
||||
Catalog handlers:
|
||||
|
||||
1. `skills_index` returns compact deterministic discovery payload (summary records only) and supports progressive follow-up links.
|
||||
2. `skills/{skill_id}` returns one normalized detail record or not-found.
|
||||
|
||||
Skill document handlers:
|
||||
|
||||
1. `skills/{skill_id}/document` returns canonical SKILL markdown content.
|
||||
2. MIME type is always `text/markdown`.
|
||||
|
||||
Reference handlers:
|
||||
|
||||
1. `skills/{skill_id}/references/{ref_id}` resolves via frontmatter manifest mapping.
|
||||
2. MIME type is explicit from manifest or defaults to `text/markdown`.
|
||||
|
||||
Wildcard docs handler:
|
||||
|
||||
1. `docs/{path*}` serves markdown docs under canonical packaged docs tree.
|
||||
2. traversal outside docs root is blocked.
|
||||
|
||||
### Integration Plan for Existing Modules
|
||||
|
||||
Primary composition updates:
|
||||
|
||||
1. Implement registry-driven registration in [src/personal_mcp/mcp.py](src/personal_mcp/mcp.py) as the canonical resource composition path.
|
||||
2. Keep [src/personal_mcp/main.py](src/personal_mcp/main.py) responsible for startup wiring order (load registry first, then register resources).
|
||||
3. Use [src/personal_mcp/catalog/server.py](src/personal_mcp/catalog/server.py) as registry-backed handlers only.
|
||||
|
||||
Lifecycle order (required):
|
||||
|
||||
1. load and validate registry (Step 4)
|
||||
2. initialize FastMCP with duplicate error policy
|
||||
3. register all Step 5 resources/templates
|
||||
4. start server
|
||||
|
||||
### Testing Plan (Step 5 Scope)
|
||||
|
||||
Unit/integration tests:
|
||||
|
||||
1. resource registration succeeds with valid registry
|
||||
2. duplicate resource registration fails at startup
|
||||
3. `skills/{skill_id}` template resolves expected record
|
||||
4. `skills/{skill_id}/document` returns markdown with correct MIME
|
||||
5. `skills/{skill_id}/references/{ref_id}` resolves manifest-mapped file
|
||||
6. `docs/{path*}` resolves nested docs paths and blocks traversal attempts
|
||||
7. all registered docs resources include `readOnlyHint` and `idempotentHint`
|
||||
8. catalog payload order is deterministic
|
||||
9. filtered/paginated `skills_index{?q,tag,capability,cursor,limit}` responses are deterministic and schema-compatible with the singleton index response
|
||||
10. catalog index payload excludes full markdown bodies and includes follow-up URIs for progressive reads
|
||||
|
||||
Smoke tests:
|
||||
|
||||
1. list resources includes singleton and template entries
|
||||
2. read representative skill doc URI and reference URI successfully
|
||||
3. read representative wildcard docs URI successfully
|
||||
|
||||
### Acceptance Criteria for Step 5 Completion
|
||||
|
||||
Step 5 is complete when all are true:
|
||||
|
||||
1. Resource registration is fully registry-driven (no per-skill hardcoded decorators required for core docs surfaces).
|
||||
2. RFC6570 templates are used for parameterized URI families, including wildcard where needed.
|
||||
3. All docs resources declare explicit MIME types and read-only/idempotent annotations.
|
||||
4. `on_duplicate_resources="error"` is enabled and verified by tests.
|
||||
5. Startup fails safely on registration conflicts.
|
||||
|
||||
### Non-goals for Step 5
|
||||
|
||||
1. No tool fallback discovery behavior implementation (Step 6).
|
||||
2. No packaging build inclusion mechanics (Step 7).
|
||||
3. No CI gate expansion details (Step 9).
|
||||
4. No migration shims for legacy URI aliases in the greenfield baseline.
|
||||
5. No ranking-strategy implementation for discovery tools beyond what is needed to preserve deterministic resource-first discovery contracts.
|
||||
6. No backward-compat resource aliases, adapter handlers, or dual registration paths.
|
||||
@@ -1,243 +0,0 @@
|
||||
**Step 6 Results: Resource-First Discovery and Tool Fallback Contract**
|
||||
|
||||
This section finalizes Step 6 by defining discovery behavior for clients that can attach MCP resources and the fallback behavior for clients or chat surfaces that must rely on MCP tools.
|
||||
|
||||
### Step Deliverable
|
||||
|
||||
- Update the current `docs/` directory with the finalized Step 6 discovery and fallback contract content from this document.
|
||||
|
||||
### Primary Source Baseline (Repository Docs)
|
||||
|
||||
Step 6 is based on the current project contracts in:
|
||||
|
||||
1. `docs/architecture.md` (resource-first architecture and catalog role)
|
||||
2. `docs/usage.md` (operating flows, bounded loading, and fallback sequence)
|
||||
3. `docs/copilot.md` (client capability lanes and practical fallback behavior)
|
||||
4. `docs/mcp_layout.md` (shared content source and thin-tool fallback position)
|
||||
5. `docs/securing.md` (read-only/public-docs security invariant)
|
||||
|
||||
Normative conclusions from those sources:
|
||||
|
||||
1. Discovery stays resource-first.
|
||||
2. Tool fallback is allowed, thin, and read-only.
|
||||
3. Resources and tools must resolve to the same canonical authored markdown.
|
||||
4. Fallback behavior should keep context bounded and deterministic.
|
||||
|
||||
### FastMCP Source Baseline (Authoritative References)
|
||||
|
||||
Step 6 fallback behavior and compatibility-layer expectations align with:
|
||||
|
||||
1. [FastMCP server concepts](https://gofastmcp.com/servers/server)
|
||||
2. [FastMCP resources and resource templates](https://gofastmcp.com/servers/resources)
|
||||
3. [FastMCP resources-as-tools transform](https://gofastmcp.com/servers/transforms/resources-as-tools)
|
||||
4. [MCP specification: resources](https://modelcontextprotocol.io/specification/latest/server/resources)
|
||||
|
||||
Applied conclusions for this step:
|
||||
|
||||
1. Resource contracts remain canonical and should be surfaced directly when clients support resource attachment.
|
||||
2. Tool-first compatibility layers should wrap canonical resource reads rather than creating alternate authored-content stores.
|
||||
3. URI-template-backed resource identity remains stable across direct-resource and tool-compatibility access paths.
|
||||
|
||||
### Client Tool-Naming Research Baseline
|
||||
|
||||
Authoritative and client-specific references to verify during implementation:
|
||||
|
||||
1. [MCP specification: tools](https://modelcontextprotocol.io/specification/latest/server/tools)
|
||||
2. [MCP client concepts](https://modelcontextprotocol.io/docs/learn/client-concepts)
|
||||
3. [FastMCP tools](https://gofastmcp.com/servers/tools)
|
||||
4. [FastMCP resources-as-tools transform](https://gofastmcp.com/servers/transforms/resources-as-tools)
|
||||
5. [VS Code MCP servers](https://code.visualstudio.com/docs/agent-customization/mcp-servers)
|
||||
6. [VS Code MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration)
|
||||
7. [Cursor MCP documentation](https://docs.cursor.com/context/model-context-protocol)
|
||||
8. [Claude Desktop local MCP server setup](https://support.anthropic.com/en/articles/10949351-getting-started-with-local-mcp-servers-on-claude-desktop)
|
||||
|
||||
Baseline naming conclusions:
|
||||
|
||||
1. MCP protocol tool identity is the server-advertised `name` returned by `tools/list` and used in `tools/call`.
|
||||
2. FastMCP tool identity should be treated as the canonical server contract unless a tool is intentionally registered with an explicit alternate name.
|
||||
3. Clients and host integrations may display, namespace, or internally route tool names with provider-specific prefixes, but those wrappers are not canonical server tool names.
|
||||
4. Compatibility should be validated by observed `tools/list` and successful `tools/call` behavior in each target client rather than by assuming one global host naming convention.
|
||||
|
||||
### Discovery Priority Contract (Normative)
|
||||
|
||||
Preferred sequence for skill discovery and loading:
|
||||
|
||||
1. `resource://catalog/skills_index`
|
||||
2. `resource://catalog/skills/{skill_id}`
|
||||
3. `resource://skills/{skill_id}/document`
|
||||
4. `resource://skills/{skill_id}/references/{ref_id}` only when needed
|
||||
|
||||
Rules:
|
||||
|
||||
1. Start from catalog discovery before loading any skill document.
|
||||
2. Do not skip straight to broad document loading when catalog metadata can narrow choices first.
|
||||
3. Use `resource://docs/{path*}` only for direct authored-doc access outside skill-specific flows.
|
||||
|
||||
### Fallback Activation Rule
|
||||
|
||||
Fallback is used only when the active client path cannot reliably attach MCP resources (for example, tool-only chat surfaces).
|
||||
|
||||
Rules:
|
||||
|
||||
1. Keep the same discovery order semantics as the resource path.
|
||||
2. If resource attachment is available, prefer resources over tools.
|
||||
3. Tool fallback must never become a second authoritative content source.
|
||||
|
||||
### Tool Fallback Surface (Normative)
|
||||
|
||||
The fallback tool surface includes:
|
||||
|
||||
1. `list_resources`
|
||||
2. `read_resource`
|
||||
3. `search_patterns`
|
||||
4. `get_pattern_by_id`
|
||||
5. `get_skill_document_by_id`
|
||||
|
||||
Canonical naming rule:
|
||||
|
||||
1. The server-level tool contract uses the exact registered FastMCP tool names above.
|
||||
2. Clients that expose provider-prefixed names (for example, namespaced wrappers) must map those names to the canonical server tool name before invocation.
|
||||
3. `catalog_get_skill_document_by_id` is not a canonical server tool name for this contract unless an explicit alias is intentionally registered.
|
||||
|
||||
Compatibility alias policy:
|
||||
|
||||
1. Prefer canonical server tool names over aliases.
|
||||
2. Add server-side aliases only when a major client cannot reliably map its wrapper name back to the canonical name.
|
||||
3. Any alias must be read-only, delegate to the same payload builder as the canonical tool, and be documented as compatibility-only.
|
||||
4. If aliases are added, canonical and alias tools must return byte-for-byte equivalent payloads for the same input.
|
||||
|
||||
Fallback order:
|
||||
|
||||
1. call `list_resources` to inspect canonical static/template resource surfaces
|
||||
2. call `read_resource` for catalog URIs and selected skill URIs
|
||||
3. use thin catalog tools only when additional metadata-first narrowing is needed
|
||||
|
||||
Tool behavior requirements:
|
||||
|
||||
1. read-only and idempotent semantics
|
||||
2. deterministic ordering and bounded pagination
|
||||
3. explicit not-found responses (`found: false` style) where applicable
|
||||
4. payloads remain schema-aligned with catalog resources
|
||||
5. tool invocation examples and Copilot guidance must use canonical server tool names to avoid unknown-tool errors
|
||||
|
||||
### Major Client Compatibility Plan
|
||||
|
||||
Target clients and expected validation:
|
||||
|
||||
1. GitHub Copilot in VS Code
|
||||
- primary path: attach MCP resources when `MCP Resources...` is available
|
||||
- fallback path: call `list_resources`, `read_resource`, then canonical thin tools only when needed
|
||||
- validation: confirm Copilot-visible tool inventory includes or can invoke `list_resources`, `read_resource`, `search_patterns`, `get_pattern_by_id`, and `get_skill_document_by_id`
|
||||
- compatibility risk: host-generated wrapper names may differ from canonical FastMCP names; document any observed wrapper-to-canonical mapping
|
||||
2. Cursor
|
||||
- primary path: use the client MCP server configuration and resource/tool surfaces supported by the active Cursor version
|
||||
- fallback path: prefer resource-backed tools first, then canonical thin tools
|
||||
- validation: capture Cursor `tools/list` equivalent behavior and verify the canonical tool names or required host mappings
|
||||
- compatibility risk: Cursor may present MCP tools through its own UI labels or internal routing names
|
||||
3. Claude Desktop
|
||||
- primary path: configure the local MCP server and inspect advertised tools/resources in Claude Desktop
|
||||
- fallback path: invoke canonical server tool names exactly as returned by `tools/list`
|
||||
- validation: run a local smoke prompt that reads `resource://catalog/skills_index` and loads one skill document through `read_resource` or `get_skill_document_by_id`
|
||||
- compatibility risk: local server configuration and transport setup may fail before tool-name compatibility is tested
|
||||
4. Generic MCP clients and SDK-based tests
|
||||
- primary path: protocol-level `resources/list`, `resources/read`, `tools/list`, and `tools/call`
|
||||
- fallback path: none beyond the canonical tool contract
|
||||
- validation: automated smoke tests assert exact tool names returned by `tools/list` and successful calls for canonical names
|
||||
- compatibility risk: SDK/client libraries may expose helper names that differ from raw protocol names
|
||||
|
||||
Implementation checklist:
|
||||
|
||||
1. Capture each target client's advertised tool names before adding aliases.
|
||||
2. Prefer fixing documentation or client-side mapping when the server already advertises canonical names correctly.
|
||||
3. Add a server-side alias only for a confirmed major-client incompatibility.
|
||||
4. Add regression tests for canonical names, resource-backed tools, and any intentionally supported aliases.
|
||||
5. Keep public examples centered on `list_resources`/`read_resource` and canonical thin tool names.
|
||||
|
||||
### Resources-As-Tools Compatibility Layer
|
||||
|
||||
Step 6 includes a resources-as-tools compatibility layer for clients that can call tools but not attach resources.
|
||||
|
||||
Rules:
|
||||
|
||||
1. It wraps canonical resource reads rather than re-implementing content transforms.
|
||||
2. It preserves canonical URIs and metadata semantics.
|
||||
3. It does not replace the minimal catalog tools listed above.
|
||||
4. It is interoperability-driven and remains read-only.
|
||||
|
||||
### Resource/Tool Parity Contract
|
||||
|
||||
Resources and fallback tools must agree on identity and routing metadata.
|
||||
|
||||
Parity requirements:
|
||||
|
||||
1. `skill_id` and `ref_id` are identical across both paths.
|
||||
2. canonical URIs in payloads match Step 3 URI rules.
|
||||
3. skill metadata (`id`, `name`, `description`, `tags`, `capabilities`, `version`) remains consistent.
|
||||
4. document payload returned by `get_skill_document_by_id` resolves to the same canonical `SKILL.md` content as `resource://skills/{skill_id}/document`.
|
||||
|
||||
### Relevance and Ranking Contract
|
||||
|
||||
Baseline matching behavior is metadata-first and deterministic.
|
||||
|
||||
Rules:
|
||||
|
||||
1. Search primarily over normalized skill metadata (id, name, description, tags).
|
||||
2. Keep deterministic ordering and deterministic pagination behavior.
|
||||
3. Keep ranking logic transparent and bounded for predictable client behavior.
|
||||
|
||||
Optional extension policy:
|
||||
|
||||
1. BM25/regex augmentation is allowed only when catalog/tool volume meaningfully harms token efficiency or precision.
|
||||
2. Any augmentation must preserve canonical ids, URIs, and deterministic tie-breaking.
|
||||
3. Any augmentation remains discovery-only and does not create alternate content payloads.
|
||||
|
||||
### Context-Bounding and Clarification Policy
|
||||
|
||||
To prevent context bloat and improve answer quality:
|
||||
|
||||
1. load only the most relevant skill document by default
|
||||
2. load at most two skill documents in one pass unless the user explicitly asks for more
|
||||
3. if confidence is low after catalog discovery, ask one clarifying question before loading additional skill documents
|
||||
4. fetch references lazily and only when required
|
||||
|
||||
### Security and Safety Constraints
|
||||
|
||||
Fallback tools must preserve the project security invariant.
|
||||
|
||||
Rules:
|
||||
|
||||
1. tool surfaces stay documentation-only and read-only
|
||||
2. no mutation, shell execution, secret access, or private filesystem exposure
|
||||
3. all returned content remains safe to publish publicly
|
||||
|
||||
### Integration Boundaries
|
||||
|
||||
Step 6 integrates with prior steps as follows:
|
||||
|
||||
1. Step 4 provides the validated in-memory registry.
|
||||
2. Step 5 provides canonical resource registration.
|
||||
3. Step 6 adds fallback discovery/read behavior that reuses the same registry and canonical markdown sources.
|
||||
|
||||
Separation-of-concerns rule:
|
||||
|
||||
1. Catalog/resource contracts remain canonical.
|
||||
2. Fallback tools are interoperability adapters, not a parallel architecture.
|
||||
|
||||
### Acceptance Criteria for Step 6 Completion
|
||||
|
||||
Step 6 is complete when all are true:
|
||||
|
||||
1. Resource-first discovery remains the documented and implemented default path.
|
||||
2. `list_resources` and `read_resource` are available for tool-only clients.
|
||||
3. Thin catalog tools remain minimal, read-only parity surfaces.
|
||||
4. Fallback tool outputs map to canonical skill identities and URIs.
|
||||
5. Context loading is bounded and clarifying-question behavior is documented for low-confidence cases.
|
||||
6. No second content source is introduced; resources and tools resolve the same authored markdown.
|
||||
|
||||
### Non-goals for Step 6
|
||||
|
||||
1. No write or side-effecting tools.
|
||||
2. No alternate authored markdown stores or duplicated skill content pipelines.
|
||||
3. No guarantee that every client session exposes MCP resource attachment UI.
|
||||
4. No packaging/build contract changes (handled in Step 7).
|
||||
5. No CI gate expansion details (handled in later validation/governance steps).
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
name: Pytest Fill Scaffold
|
||||
description: Fill scaffolded pytest test methods with assertions, fixtures, and minimal test data while preserving concise test names and one-line intent docstrings.
|
||||
argument-hint: Target test file(s) under tests plus stack (pure-python, fastapi, sqlalchemy-sync, sqlalchemy-async, or mixed)
|
||||
agent: agent
|
||||
---
|
||||
|
||||
# Pytest Fill Scaffold
|
||||
|
||||
Use this prompt after test scaffolding exists and method names/docstrings are already in place.
|
||||
|
||||
## Inputs
|
||||
|
||||
- Target test file(s) under `tests/`.
|
||||
- Stack type:
|
||||
- `pure-python`
|
||||
- `fastapi`
|
||||
- `sqlalchemy-sync`
|
||||
- `sqlalchemy-async`
|
||||
- `mixed`
|
||||
- Optional constraints:
|
||||
- keep implementation minimal vs comprehensive
|
||||
- marker lane target (`unit`, `integration`, `smoke`)
|
||||
|
||||
## Required References
|
||||
|
||||
Load these in order and use only what matches the task:
|
||||
|
||||
1. Core defaults: [pytest scaffolding skill](../../docs/skills/pytesting/SKILL.md)
|
||||
2. Naming/hierarchy preservation: [naming and organization](../../docs/skills/pytesting/references/naming-and-organization.md)
|
||||
3. Baseline pytest fixtures/markers: [pytest docs notes](../../docs/skills/pytesting/references/pytest-docs.md)
|
||||
4. FastAPI-specific behavior (only when needed): [fastapi testing](../../docs/skills/pytesting/references/fastapi-testing.md)
|
||||
5. SQLAlchemy-specific behavior (only when needed): [sqlalchemy testing](../../docs/skills/pytesting/references/sqlalchemy-testing.md)
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Inspect target files and treat human-reviewed docstring-only scaffolds as invariant.
|
||||
2. Convert each scaffolded method into an executable test with a single behavior focus.
|
||||
3. Keep one-line docstrings for class and method intent.
|
||||
4. Add or refine fixtures at the nearest useful scope:
|
||||
- global in `tests/conftest.py` only when broadly reusable
|
||||
- subtree `conftest.py` for domain-specific fixtures
|
||||
5. Assign markers consistent with cost and dependencies:
|
||||
- `unit` for pure logic
|
||||
- `integration` for framework/DB contracts
|
||||
- `smoke` for thin critical-path checks
|
||||
6. Validate in this order:
|
||||
- `uv run pytest --collect-only -q`
|
||||
- `uv run pytest -m unit -q` when unit tests are touched
|
||||
- `uv run pytest -q` if dependencies are available
|
||||
|
||||
## Authoring Rules
|
||||
|
||||
- Prefer deterministic tests and explicit setup/teardown.
|
||||
- Keep assertions precise and readable.
|
||||
- Do not overfit tests to private implementation details.
|
||||
- If a scaffolded class or method has only a docstring body, treat its name and hierarchy as locked.
|
||||
- Do not rename, move, merge, split, or re-nest docstring-only scaffolded tests unless explicitly requested.
|
||||
- Preserve existing one-line docstrings on scaffolded classes and methods unless they are factually incorrect.
|
||||
- If stack details are missing and would change fixture strategy, ask one concise clarifying question before editing.
|
||||
|
||||
## Output Format
|
||||
|
||||
Return:
|
||||
1. Files updated.
|
||||
2. Fixture and marker decisions.
|
||||
3. Which references were used and why.
|
||||
4. Validation command results.
|
||||
5. Risks or open questions.
|
||||
@@ -0,0 +1,72 @@
|
||||
---
|
||||
name: Pytest Scaffold
|
||||
description: Plan and scaffold pytest test files, class hierarchy, and concise method names for selected Python modules in this repository.
|
||||
argument-hint: Target module path(s) in src plus scope (plan-only or scaffold)
|
||||
agent: agent
|
||||
---
|
||||
|
||||
# Pytest Scaffold
|
||||
|
||||
Use this prompt to do in one run what we have been doing manually in chat:
|
||||
1. Build a naming and hierarchy plan for tests.
|
||||
2. Scaffold test files and class/method skeletons.
|
||||
3. Keep test method names concise because intent is carried by one-line docstrings.
|
||||
|
||||
## Inputs
|
||||
|
||||
- Target module path(s) under `src/`.
|
||||
- Scope mode:
|
||||
- `plan-only`
|
||||
- `scaffold`
|
||||
- Optional constraints:
|
||||
- flattening preferences for path mapping under `tests/`
|
||||
- method naming style preference
|
||||
|
||||
## Repository Rules To Apply
|
||||
|
||||
- Use [pytest scaffolding skill](../../docs/skills/pytesting/SKILL.md) for strategy and defaults.
|
||||
- Use [naming and organization reference](../../docs/skills/pytesting/references/naming-and-organization.md) before finalizing hierarchy.
|
||||
- Use `uv run pytest --collect-only -q` as structural validation.
|
||||
- Default to a source-mirror style adapted to this repository:
|
||||
- map selected modules to `tests/` with concise path segments when requested
|
||||
- keep one test module per source module
|
||||
|
||||
## Execution Steps
|
||||
|
||||
1. Inspect current `tests/` layout and identify existing naming patterns.
|
||||
2. Propose a concise hierarchy plan first:
|
||||
- test file paths
|
||||
- class hierarchy
|
||||
- method naming pattern
|
||||
- fixture placement (`tests/conftest.py` vs subtree `conftest.py`)
|
||||
3. If scope mode is `scaffold`, implement the skeleton:
|
||||
- create missing test modules
|
||||
- create class hierarchy
|
||||
- add one-line docstrings to every class and test method
|
||||
- keep test method names short and behavior-focused
|
||||
- treat resulting docstring-only scaffolds as human-reviewed baseline for future fill-in work
|
||||
4. Validate collection with `uv run pytest --collect-only -q`.
|
||||
5. Report results:
|
||||
- files created or updated
|
||||
- collection outcome
|
||||
- any ambiguities or follow-up choices
|
||||
|
||||
## Class And Method Shape Defaults
|
||||
|
||||
- Class shape:
|
||||
- `Test<PrimarySubject>` as the top-level subject class
|
||||
- nested `Test<MethodOrArea>` classes when it improves context
|
||||
- top-level `Test<FunctionName>` classes for standalone module functions
|
||||
- Method shape:
|
||||
- `test_<short_outcome>` naming
|
||||
- one behavior target per method name
|
||||
- one-line docstring that states the full intent
|
||||
|
||||
## Output Format
|
||||
|
||||
Return:
|
||||
1. Discovery summary and references consulted.
|
||||
2. Proposed or applied test tree.
|
||||
3. Class and method naming map.
|
||||
4. Validation command results.
|
||||
5. Open questions only if they block confident completion.
|
||||
@@ -0,0 +1,183 @@
|
||||
## Goal
|
||||
|
||||
Build a local, self-hosted documentation knowledge base that can ingest software docs, generate embeddings, store them in SQLite, and expose high-quality retrieval through MCP tools and resources.
|
||||
|
||||
---
|
||||
|
||||
## Core Architectural Decisions
|
||||
|
||||
### Storage
|
||||
|
||||
Use:
|
||||
|
||||
* SQLite for metadata and document storage
|
||||
* FTS5 for keyword search
|
||||
* sqlite-vec for vector similarity search
|
||||
|
||||
Avoid a separate vector database unless scale requirements emerge.
|
||||
|
||||
---
|
||||
|
||||
### Embeddings
|
||||
|
||||
Use local embedding models via:
|
||||
|
||||
* sentence-transformers
|
||||
|
||||
Recommended model:
|
||||
|
||||
```text
|
||||
BAAI/bge-base-en-v1.5
|
||||
```
|
||||
|
||||
Store embeddings alongside document chunks.
|
||||
|
||||
---
|
||||
|
||||
### Ingestion
|
||||
|
||||
Primary sources:
|
||||
|
||||
1. Git repositories containing Markdown docs
|
||||
2. Documentation websites via Crawl4AI
|
||||
3. Sitemap-driven crawls when available
|
||||
|
||||
Pipeline:
|
||||
|
||||
```text
|
||||
Source
|
||||
↓
|
||||
Extract
|
||||
↓
|
||||
Normalize
|
||||
↓
|
||||
Chunk by headings
|
||||
↓
|
||||
Embed
|
||||
↓
|
||||
Store
|
||||
```
|
||||
|
||||
Track content hashes so unchanged documents are skipped during reindexing.
|
||||
|
||||
---
|
||||
|
||||
### Retrieval
|
||||
|
||||
Implement hybrid retrieval:
|
||||
|
||||
```text
|
||||
FTS5 keyword search
|
||||
+
|
||||
sqlite-vec similarity search
|
||||
↓
|
||||
Candidate set
|
||||
↓
|
||||
Reranker
|
||||
↓
|
||||
Final results
|
||||
```
|
||||
|
||||
Reranker:
|
||||
|
||||
```text
|
||||
BAAI/bge-reranker-v2
|
||||
```
|
||||
|
||||
The retriever owns all ranking logic.
|
||||
|
||||
---
|
||||
|
||||
### Public Interface
|
||||
|
||||
Do not expose vector search directly.
|
||||
|
||||
Expose a retrieval service through MCP:
|
||||
|
||||
```python
|
||||
search_docs(query)
|
||||
|
||||
get_context(query)
|
||||
|
||||
get_doc(path)
|
||||
```
|
||||
|
||||
The MCP layer becomes the stable API.
|
||||
|
||||
Clients never interact with embeddings or vectors.
|
||||
|
||||
---
|
||||
|
||||
## Repository Layout
|
||||
|
||||
```text
|
||||
src/
|
||||
├── knowledge/
|
||||
│ ├── models.py
|
||||
│ ├── chunking.py
|
||||
│ ├── embeddings.py
|
||||
│ ├── ingestion.py
|
||||
│ ├── sqlite_store.py
|
||||
│ ├── hybrid_search.py
|
||||
│ ├── reranker.py
|
||||
│ └── retrieval.py
|
||||
│
|
||||
├── sources/
|
||||
│ ├── git_docs.py
|
||||
│ ├── crawl4ai_docs.py
|
||||
│ └── sitemap_docs.py
|
||||
│
|
||||
├── mcp_server/
|
||||
│ ├── tools.py
|
||||
│ └── resources.py
|
||||
│
|
||||
└── cli/
|
||||
├── ingest.py
|
||||
└── reindex.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Retrieval Flow
|
||||
|
||||
```text
|
||||
User Query
|
||||
↓
|
||||
Embed Query
|
||||
↓
|
||||
FTS5 Search
|
||||
+
|
||||
Vector Search
|
||||
↓
|
||||
Merge Results
|
||||
↓
|
||||
Rerank
|
||||
↓
|
||||
Return Context Bundle
|
||||
```
|
||||
|
||||
Where a context bundle contains:
|
||||
|
||||
```python
|
||||
ContextBundle(
|
||||
passages=[...],
|
||||
citations=[...],
|
||||
related_docs=[...],
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Future Extensions
|
||||
|
||||
Without changing the architecture:
|
||||
|
||||
* Multiple documentation corpora
|
||||
* Version-aware retrieval
|
||||
* Code snippet indexing
|
||||
* MCP resources for specific topics
|
||||
* LangGraph integration
|
||||
* Docker deployment
|
||||
* Scheduled reindexing
|
||||
|
||||
The key design principle is: **treat the vector store as an internal implementation detail and expose a retrieval-oriented MCP interface instead.**
|
||||
+1
-1
@@ -47,7 +47,7 @@ RUN --mount=type=cache,target=/root/.cache/uv \
|
||||
--mount=type=bind,source=uv.lock,target=uv.lock \
|
||||
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
|
||||
--mount=type=bind,source=src/,target=src/ \
|
||||
uv sync --no-editable
|
||||
uv sync --no-editable --refresh-package prompts
|
||||
|
||||
USER appuser
|
||||
|
||||
|
||||
@@ -6,5 +6,3 @@ services:
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8765:8765"
|
||||
volumes:
|
||||
- ./docs:/app/src/personal_mcp/docs
|
||||
|
||||
@@ -1,266 +0,0 @@
|
||||
---
|
||||
icon: lucide/library
|
||||
---
|
||||
|
||||
# Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
The platform is implemented as a resource-first MCP system with an integrated static documentation surface. The same methodology content powers both MCP resources and the published docs site.
|
||||
|
||||
An MCP server is a runtime that exposes machine-readable resources and tools through stable interfaces so AI clients can discover and consume context consistently. Here, the server's role is intentionally narrow: publish canonical methodology documents as resources, keep discovery predictable through a catalog layer, and serve the same source material as pre-built static documentation.
|
||||
|
||||
The system is complete in three layers:
|
||||
|
||||
1. Canonical methodology is maintained in Markdown skill documents.
|
||||
2. Catalog resources provide normalized discovery.
|
||||
3. Zensical builds a static site from those same Markdown sources and the FastAPI app serves it in the FastMCP runtime process.
|
||||
|
||||
Prompt documents under `docs/prompts/` are also indexed and exposed as first-class catalog and prompt surfaces.
|
||||
|
||||
This architecture is anchored by three contracts:
|
||||
|
||||
1. Docs-first authored content contract under `docs/` with strict per-skill ownership.
|
||||
2. `SKILL.md` frontmatter contract with Anthropic fields plus `x-personal-mcp` metadata.
|
||||
3. Canonical resource URI contract with break-and-replace policy for contract changes.
|
||||
|
||||
Detailed contract pages:
|
||||
|
||||
1. [Content Contract](./contracts/index.md#content-contract)
|
||||
2. [Frontmatter Contract](./contracts/frontmatter.md)
|
||||
3. [URI Contract](./contracts/uris.md)
|
||||
|
||||
This architecture keeps authored content human-friendly while preserving machine-stable contracts.
|
||||
|
||||
## Intent
|
||||
|
||||
The architecture is designed to satisfy three long-term requirements:
|
||||
|
||||
1. Methodology must be editable as markdown by humans.
|
||||
2. Agents must consume stable, discoverable resource contracts, with a minimal read-only catalog tool fallback for constrained clients.
|
||||
3. Public documentation must be pre-built static output served from the application runtime without a separate docs service.
|
||||
|
||||
## System Model
|
||||
|
||||
### Pattern Modules
|
||||
|
||||
Each skill encapsulates one methodology domain in a docs-owned directory:
|
||||
|
||||
1. `docs/skills/<skill-id>/SKILL.md`
|
||||
2. `docs/skills/<skill-id>/references/...`
|
||||
|
||||
The skill document and references are the authored source of truth; runtime code indexes and serves these files without becoming a second authored source.
|
||||
|
||||
Each skill publishes resource families:
|
||||
|
||||
1. document
|
||||
|
||||
The document resource returns canonical Markdown, while clients can perform any downstream section extraction they need.
|
||||
|
||||
### Prompt Modules
|
||||
|
||||
Prompt guidance can be authored in `docs/prompts/` using either canonical prompt directories (`docs/prompts/<prompt-id>/PROMPT.md`) or legacy markdown files during migration.
|
||||
|
||||
Prompt modules publish two additive surfaces:
|
||||
|
||||
1. prompt resources for catalog and document retrieval
|
||||
2. MCP prompt objects for prompt-list/get-prompt style client workflows
|
||||
|
||||
This keeps authored markdown as source-of-truth while allowing clients to discover and invoke prompts directly.
|
||||
|
||||
### Catalog Module
|
||||
|
||||
The catalog is the canonical discovery layer and publishes normalized records for all modules. It may also expose a minimal set of read-only discovery tools that resolve back to the same canonical markdown content when a client chat surface does not expose MCP resource attachment.
|
||||
|
||||
Typical catalog resources:
|
||||
|
||||
1. resource://catalog/skills_index
|
||||
2. resource://catalog/skills_index{?q,tag,capability,cursor,limit}
|
||||
3. resource://catalog/skills/{skill_id}
|
||||
4. resource://catalog/prompts_index
|
||||
5. resource://catalog/prompts_index{?q,tag,cursor,limit}
|
||||
6. resource://catalog/prompts/{prompt_id}
|
||||
|
||||
Only canonical catalog resources are part of the runtime contract in this phase.
|
||||
|
||||
### Registry Loader
|
||||
|
||||
Importing the package does not read or parse documentation. The MCP server and FastAPI application factories request the registry when constructing a runnable server, using packaged resources through `importlib.resources.files(...)` and `Traversable` APIs.
|
||||
|
||||
Loader responsibilities:
|
||||
|
||||
1. Parse SKILL.md frontmatter for each skill.
|
||||
2. Validate schema and cross-field constraints before any resource registration.
|
||||
3. Build an in-memory registry keyed by `skill_id`.
|
||||
4. Fail fast for duplicate ids, missing markdown files, and broken reference mappings.
|
||||
|
||||
The immutable registry is cached for the process lifetime. Each Uvicorn worker constructs and retains its own registry because worker processes do not share Python objects. Registry load failure is a server-factory startup error, not a package-import error or partial runtime warning.
|
||||
|
||||
### Content Sources
|
||||
|
||||
Content is authored in markdown under `docs/` and managed as long-form reference material. Skill documents and companion references now live under `docs/skills/`, while project-authored pages remain alongside them in the docs tree. Resource handlers expose the same authored documents through stable resource URIs.
|
||||
|
||||
The repository root `docs/` directory is the only authored source. The `src/personal_mcp/docs` path is a relative symlink to that directory for source-checkout and editable-install workflows; it is not a second content tree and packaging does not depend on traversing it.
|
||||
|
||||
For wheel builds, [Hatchling forced inclusion](https://hatch.pypa.io/latest/config/build/#forced-inclusion) maps the root `docs/` tree to `personal_mcp/docs/`. The wheel therefore contains regular resource files at that destination rather than a symlink. Runtime registry loading uses [`importlib.resources.files`](https://docs.python.org/3/library/importlib.resources.html#importlib.resources.files) and `Traversable` operations from the `personal_mcp` package anchor, so it does not depend on the repository layout or current working directory.
|
||||
|
||||
### Static Docs Surface
|
||||
|
||||
Static docs are built directly from two markdown source streams:
|
||||
|
||||
1. Project-authored docs pages
|
||||
2. Skill and reference markdown pages
|
||||
|
||||
The merged docs tree is built by Zensical into static files and served by the FastAPI app.
|
||||
|
||||
Generated `site/` files are deployment assets for the human-facing static site. They are separate from the authored Markdown resources packaged under `personal_mcp/docs/`.
|
||||
|
||||
## Data Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Authored Markdown] --> C[Resource Handlers]
|
||||
B[Pattern Metadata] --> D[Catalog Resources]
|
||||
A --> E[Zensical Static Build]
|
||||
E --> H[FastAPI Static Mount]
|
||||
H --> I[Served Docs Site]
|
||||
D --> I
|
||||
```
|
||||
|
||||
## Contracts
|
||||
|
||||
### Metadata Contract
|
||||
|
||||
Each skill declares frontmatter in `docs/skills/<skill-id>/SKILL.md`.
|
||||
|
||||
For the full field-level contract, validation model, and FastMCP metadata mapping, see [Frontmatter Contract](./contracts/frontmatter.md).
|
||||
|
||||
Anthropic-facing required fields:
|
||||
|
||||
1. name
|
||||
2. description
|
||||
|
||||
Repository indexing metadata is declared in `x-personal-mcp`:
|
||||
|
||||
1. id
|
||||
2. version
|
||||
3. tags
|
||||
4. capabilities
|
||||
5. optional references map (for nested entries, overrides, and aliases)
|
||||
|
||||
No `metadata.yaml` sidecar is part of the end-state contract.
|
||||
|
||||
### URI Contract
|
||||
|
||||
Canonical resource URIs are:
|
||||
|
||||
For the full URI semantics, parameter validation rules, and compatibility policy, see [URI Contract](./contracts/uris.md).
|
||||
|
||||
1. resource://skills/<skill_id>/document
|
||||
2. resource://skills/<skill_id>/references/<ref_id>
|
||||
3. resource://catalog/skills_index
|
||||
4. resource://catalog/skills_index{?q,tag,capability,cursor,limit}
|
||||
5. resource://catalog/skills/{skill_id}
|
||||
6. resource://docs/{path*}
|
||||
7. resource://catalog/prompts_index
|
||||
8. resource://catalog/prompts_index{?q,tag,cursor,limit}
|
||||
9. resource://catalog/prompts/{prompt_id}
|
||||
10. resource://prompts/{prompt_id}/document
|
||||
|
||||
Validation rules:
|
||||
|
||||
1. `skill_id` is lowercase kebab-case and must satisfy the stable skill id contract.
|
||||
2. `ref_id` is lowercase kebab-case and must resolve from either:
|
||||
- top-level auto-discovery of `references/*.md` filename stems, or
|
||||
- an explicit `x-personal-mcp.references` entry.
|
||||
3. `path*` resolves only to normalized markdown paths under `docs/`.
|
||||
|
||||
### Resource Registration Contract
|
||||
|
||||
Resources are registered from the validated registry, not by ad hoc per-skill hardcoding.
|
||||
|
||||
Registration rules:
|
||||
|
||||
1. Use RFC6570 URI templates where appropriate.
|
||||
2. Mark documentation resources as read-only and idempotent.
|
||||
3. Set explicit mime types for resource responses.
|
||||
4. Configure duplicate URI handling with `on_duplicate="error"` for startup safety.
|
||||
|
||||
This keeps runtime behavior deterministic and prevents accidental URI collisions.
|
||||
|
||||
### Versioning Rule
|
||||
|
||||
URIs are unversioned and canonical in this phase.
|
||||
|
||||
1. Breaking URI changes are handled as direct replacement.
|
||||
2. No compatibility aliases or dual URI families are maintained.
|
||||
|
||||
## Static Hosting Pattern
|
||||
|
||||
The docs site is pre-built and served by the same FastAPI runtime process used by the MCP app.
|
||||
|
||||
Runtime behavior:
|
||||
|
||||
1. App starts.
|
||||
2. FastAPI mounts the static docs output directory.
|
||||
3. Requests to docs paths are served as static assets.
|
||||
|
||||
This provides a single deployment artifact with no runtime markdown rendering dependency.
|
||||
|
||||
## Advantages
|
||||
|
||||
### Single Source of Truth
|
||||
|
||||
Methodology is authored once and reused in both MCP resources and docs pages.
|
||||
|
||||
### High-Fidelity Agent Context
|
||||
|
||||
Resources expose the same canonical Markdown that humans author and review.
|
||||
|
||||
### Operational Simplicity
|
||||
|
||||
A single app process serves MCP and docs surfaces.
|
||||
|
||||
### Long-Term Maintainability
|
||||
|
||||
Markdown remains easy to review, while contracts remain stable for clients.
|
||||
|
||||
### Client Independence
|
||||
|
||||
Clients can use Ask, Edit, or Agent modes without requiring prompt-first orchestration. Prompt objects are available as an additive MCP surface, while resource retrieval remains the canonical source path. MCP affordances are still chat-surface-dependent: some clients or sessions expose resource attachment directly, while others make tool invocation the more reliable retrieval path.
|
||||
|
||||
## Authoring and Publishing Lifecycle
|
||||
|
||||
1. Update markdown reference content.
|
||||
2. Update metadata if capability surface changes.
|
||||
3. Build static docs with Zensical.
|
||||
4. Serve built output through FastAPI static mount.
|
||||
|
||||
## Scope and Non-Goals
|
||||
|
||||
In-scope:
|
||||
|
||||
1. Resource-first methodology delivery
|
||||
2. Catalog-based discovery
|
||||
3. Pre-built static docs hosting in app runtime
|
||||
|
||||
Out-of-scope:
|
||||
|
||||
1. Prompt-first orchestration as the primary interface
|
||||
2. Large tool inventories duplicating static guidance across skill modules
|
||||
3. Separate dynamic docs service at runtime
|
||||
|
||||
Allowed exception:
|
||||
|
||||
1. A small catalog-level tool layer is acceptable when it improves client interoperability without creating a second source of truth for skill content.
|
||||
|
||||
## Example Content Inputs
|
||||
|
||||
Existing markdown reference sets are valid examples of authored source material for this architecture:
|
||||
|
||||
1. docs/skills/pytesting/references/pytest-docs.md
|
||||
2. docs/skills/python-logging/references/python-logging-docs.md
|
||||
3. docs/skills/python-logging/references/json-file-logging.md
|
||||
4. docs/skills/fastapi-uv-docker/references/fastapi-best-practices.md
|
||||
|
||||
These inputs are treated as content sources, while resource URIs and catalog payloads remain the machine-facing contracts.
|
||||
@@ -1,262 +0,0 @@
|
||||
---
|
||||
icon: lucide/pencil
|
||||
---
|
||||
|
||||
# Authoring Guide
|
||||
|
||||
This page defines the practical authoring workflow for this repository so Markdown remains the single source of truth for both published docs and MCP resources.
|
||||
|
||||
Primary references:
|
||||
- [Skill contract](./contracts/skill_contract.md)
|
||||
- [Prompt contract](./contracts/prompt.md)
|
||||
- [Frontmatter contract](./contracts/frontmatter.md)
|
||||
- [URI contract](./contracts/uris.md)
|
||||
- [Zensical documentation authoring skill](./skills/zensical-docs/SKILL.md)
|
||||
|
||||
## What You Author
|
||||
|
||||
This repository has two primary authored content types:
|
||||
|
||||
1. Skills under `docs/skills/<skill-id>/`.
|
||||
2. Prompts under `docs/prompts/<prompt-id>/`.
|
||||
|
||||
Each module keeps one canonical document plus optional references:
|
||||
|
||||
```text
|
||||
docs/
|
||||
skills/<skill-id>/
|
||||
SKILL.md
|
||||
references/
|
||||
*.md
|
||||
|
||||
prompts/<prompt-id>/
|
||||
PROMPT.md
|
||||
references/
|
||||
*.md
|
||||
```
|
||||
|
||||
## Source Tree Ownership
|
||||
|
||||
Edit content only under the repository root `docs/` directory. The `src/personal_mcp/docs` path is a relative symlink provided so package-oriented tooling and editable installs see the same files; do not replace it with copied content or author files through a second tree.
|
||||
|
||||
[Hatchling forced inclusion](https://hatch.pypa.io/latest/config/build/#forced-inclusion) projects root `docs/` into `personal_mcp/docs/` when building the wheel. Installed code reads that destination through [`importlib.resources`](https://docs.python.org/3/library/importlib.resources.html), while Zensical continues to build the human-facing site directly from root `docs/`.
|
||||
|
||||
Package import does not load these resources. A runnable MCP or FastAPI server loads and validates them when its factory runs, then caches the immutable registry for that process. Restart initialized development or worker processes after changing authored Markdown.
|
||||
|
||||
## Authoring Principles
|
||||
|
||||
1. Keep Markdown as the canonical source and avoid duplicating content into alternate metadata files.
|
||||
2. Prefer resource-first discovery paths (`resource://catalog/...` then `resource://skills/...` or `resource://prompts/...`).
|
||||
3. Keep pages focused and composable: overview in the primary doc, details in `references/`.
|
||||
4. Use descriptive inline links for external sources instead of bare URLs.
|
||||
5. Use stable ids and slugs; renames are breaking changes and should be intentional.
|
||||
|
||||
## Skill Authoring Workflow
|
||||
|
||||
When creating or updating a skill:
|
||||
|
||||
1. Confirm slug format is lowercase kebab-case.
|
||||
2. Keep directory name, `name`, and `x-personal-mcp.id` aligned.
|
||||
3. Ensure capabilities include `resource://skills/<skill-id>/document`.
|
||||
4. Place supporting material under `references/`.
|
||||
5. Use explicit frontmatter reference entries only when you need overrides or nested mappings.
|
||||
|
||||
Recommended sequence:
|
||||
|
||||
1. Draft `SKILL.md` intent and routing sections.
|
||||
2. Add or refine `references/*.md`.
|
||||
3. Verify links and example commands.
|
||||
4. Run docs build and tests.
|
||||
|
||||
For exact metadata rules, see [Frontmatter contract](./contracts/frontmatter.md) and [Skill contract](./contracts/skill_contract.md).
|
||||
|
||||
## Prompt Authoring Workflow
|
||||
|
||||
When creating or updating a prompt module:
|
||||
|
||||
1. Keep one canonical `PROMPT.md`.
|
||||
2. Keep `name`, `x-personal-mcp.id`, and directory slug aligned.
|
||||
3. Include `resource://prompts/<prompt-id>/document` in capabilities.
|
||||
4. Define prompt arguments in `x-personal-mcp.arguments` when inputs are required.
|
||||
5. Keep long rationale and source notes in `references/` to preserve prompt clarity.
|
||||
|
||||
For exact structure, see [Prompt contract](./contracts/prompt.md).
|
||||
|
||||
## Prompt Argument Mechanics
|
||||
|
||||
When defining prompt inputs, keep argument metadata aligned with the prompt contract and runtime behavior.
|
||||
|
||||
1. Define arguments under `x-personal-mcp.arguments` as a map keyed by argument name.
|
||||
2. Argument names must match Python identifier format: `^[A-Za-z_][A-Za-z0-9_]*$`.
|
||||
3. Each argument entry supports only:
|
||||
- `title` (optional)
|
||||
- `description` (optional)
|
||||
- `required` (optional, defaults to `false`)
|
||||
4. Unknown argument fields are rejected by strict frontmatter validation.
|
||||
5. Prompt argument metadata appears in `resource://catalog/prompts/{prompt_id}`, and MCP prompt objects expose the same arguments for prompt-list/get-prompt workflows.
|
||||
6. Enum-like constraints are not a native argument field; encode allowed values in `description`.
|
||||
|
||||
### Frontmatter Safety Rules
|
||||
|
||||
Use these rules to avoid YAML parse failures in prompt and skill frontmatter:
|
||||
|
||||
1. Quote any scalar value that contains `:` (for example, `description: "Enum: skill | prompt | shim"`).
|
||||
2. Prefer quoted scalars for values with reserved YAML characters such as `#`, `{}`, `[]`, or leading `*`.
|
||||
3. If a description needs multiple lines, use a block scalar (`|`) instead of packing punctuation-heavy text into one line.
|
||||
4. Keep frontmatter keys simple and contract-bound; do not add undeclared argument fields.
|
||||
|
||||
### Validation Timing
|
||||
|
||||
Run validation immediately after frontmatter edits, not only at the end of a task:
|
||||
|
||||
1. First pass after metadata changes: `uv run zensical build`
|
||||
2. Prompt/skill load verification: `uv run pytest -q`
|
||||
3. Final full pass before completion: run the full checklist in [Validation Checklist](#validation-checklist)
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
x-personal-mcp:
|
||||
arguments:
|
||||
artifact_type:
|
||||
title: Artifact type
|
||||
description: Allowed values are skill, prompt, or shim.
|
||||
required: true
|
||||
scope_glob:
|
||||
title: Scope glob
|
||||
description: Optional applyTo glob for shim outputs.
|
||||
required: false
|
||||
```
|
||||
|
||||
References:
|
||||
|
||||
1. [Frontmatter contract](./contracts/frontmatter.md)
|
||||
2. [URI contract](./contracts/uris.md)
|
||||
3. [Resource-First Pattern Module Architecture](./architecture.md)
|
||||
4. [Prompt objects concept docs](https://modelcontextprotocol.io/docs/learn/server-concepts#prompts)
|
||||
|
||||
## Writing Quality Rules
|
||||
|
||||
Apply these defaults to all docs pages:
|
||||
|
||||
1. Prefer short sections with strong headings over long unbroken prose.
|
||||
2. Keep claims source-linked, especially for MCP, FastMCP, pytest, FastAPI, SQLAlchemy, and Zensical behavior.
|
||||
3. Prefer relative links for internal docs paths.
|
||||
4. Use code blocks for commands and configuration snippets.
|
||||
5. Keep examples minimal and actionable.
|
||||
|
||||
Source examples:
|
||||
- [Model Context Protocol docs](https://modelcontextprotocol.io/docs/getting-started/intro)
|
||||
- [FastMCP docs](https://gofastmcp.com/getting-started/welcome)
|
||||
- [Zensical docs](https://zensical.org/docs/)
|
||||
|
||||
## Authoring for GitHub Copilot
|
||||
|
||||
For resource selection or tool-based matching to work well, each skill should have:
|
||||
|
||||
1. precise `description`
|
||||
2. focused `tags`
|
||||
3. explicit `capabilities`
|
||||
4. stable `id` and slug naming
|
||||
|
||||
Weak metadata reduces Copilot match quality and increases wrong context injection.
|
||||
|
||||
### Copilot Instruction Authoring Pattern
|
||||
|
||||
If you want Copilot to use `personal-mcp` skill content more reliably, instruction files should describe three things clearly:
|
||||
|
||||
1. when MCP-backed skill guidance is relevant
|
||||
2. which retrieval path Copilot should prefer first
|
||||
3. how much skill context it should load before answering
|
||||
|
||||
Instructions strongly steer discovery behavior, but they do not force VS Code to auto-attach MCP resources. Keep wording explicit about preferred path and fallback path.
|
||||
|
||||
Repository policy:
|
||||
|
||||
1. start from catalog discovery
|
||||
2. prefer MCP resources when the current chat surface exposes resource attachment
|
||||
3. fall back to catalog tools when resource attachment is unavailable
|
||||
4. keep loaded skill context bounded
|
||||
|
||||
Suggested instruction text:
|
||||
|
||||
```md
|
||||
When a task may match a documented implementation pattern from `personal-mcp`:
|
||||
|
||||
1. Start with catalog-first discovery.
|
||||
2. Prefer MCP resources when the chat surface exposes resource attachment.
|
||||
3. If MCP resource attachment is unavailable, use `list_resources`/`read_resource` first, then thin catalog tools if needed.
|
||||
4. Load only the most relevant skill document, or at most 2 skill documents.
|
||||
5. Reconcile loaded skill guidance with the actual repository code before making changes.
|
||||
|
||||
Preferred resource order:
|
||||
|
||||
1. `resource://catalog/skills_index`
|
||||
2. `resource://catalog/skills/{skill_id}`
|
||||
3. `resource://skills/<skill-id>/document`
|
||||
4. `resource://skills/<skill-id>/references/<ref-id>` when needed
|
||||
|
||||
Preferred tool fallback order:
|
||||
|
||||
1. `list_resources`
|
||||
2. `read_resource`
|
||||
3. `search_patterns`
|
||||
4. `get_pattern_by_id`
|
||||
5. `get_skill_document_by_id`
|
||||
|
||||
Compatibility aliases for clients that use `catalog_*` naming are also available:
|
||||
|
||||
1. `catalog_search_patterns`
|
||||
2. `catalog_get_pattern_by_id`
|
||||
3. `catalog_get_skill_document_by_id`
|
||||
4. `catalog_search_prompts`
|
||||
5. `catalog_get_prompt_by_id`
|
||||
|
||||
Use canonical names first; aliases exist only to preserve interoperability when a client emits non-canonical names.
|
||||
|
||||
If confidence is low after discovery, ask one clarifying question before loading more context.
|
||||
```
|
||||
|
||||
This is guidance, not a guarantee. It defines a reliable policy while preserving the resource-first architecture.
|
||||
|
||||
Thin shim path binding guidance for MCP consumers is covered in [Skill Usage Mechanics](./usage.md).
|
||||
|
||||
## Zensical Details
|
||||
|
||||
When adding or restructuring pages:
|
||||
|
||||
1. Update navigation in `zensical.toml`.
|
||||
2. Ensure top-level pages include frontmatter with an icon.
|
||||
3. Keep naming and labels concise so navigation remains scannable.
|
||||
|
||||
Top-level page pattern:
|
||||
|
||||
```yaml
|
||||
---
|
||||
icon: lucide/pencil
|
||||
---
|
||||
```
|
||||
|
||||
## Validation Checklist
|
||||
|
||||
Run these checks before considering authoring changes complete:
|
||||
|
||||
```bash
|
||||
uv run zensical build
|
||||
uv run ruff check .
|
||||
uv run ty check
|
||||
uv run pytest
|
||||
```
|
||||
|
||||
Address any errors or warnings that result.
|
||||
|
||||
If a change only affects docs content, `uv run zensical build` is still required.
|
||||
|
||||
## Quick Authoring Checklist
|
||||
|
||||
1. Correct location (`skills/` or `prompts/`).
|
||||
2. Frontmatter id and slug alignment.
|
||||
3. Capability URI present.
|
||||
4. Links valid and descriptive.
|
||||
5. Navigation updated when needed.
|
||||
6. Validation commands passed.
|
||||
@@ -1,234 +0,0 @@
|
||||
---
|
||||
icon: lucide/braces
|
||||
---
|
||||
|
||||
# Frontmatter Contract
|
||||
|
||||
This page defines the `SKILL.md` frontmatter and FastMCP metadata contract.
|
||||
|
||||
Prompt modules use the same contract style in `docs/prompts/<prompt-id>/PROMPT.md` with prompt-specific capability and MCP-aligned prompt argument metadata.
|
||||
|
||||
## Validated Frontmatter Surface
|
||||
|
||||
The registry runtime validates a strict, standard-only frontmatter surface:
|
||||
|
||||
1. Top-level fields accepted for skills: `name`, `description`, `x-personal-mcp`.
|
||||
2. Top-level fields accepted for prompts: `name`, `description`, `x-personal-mcp`.
|
||||
3. Unknown top-level fields are rejected during registry load.
|
||||
|
||||
Skill and prompt identifier rules:
|
||||
|
||||
1. `name` is required, 1-64 chars, lowercase kebab-case, and must not contain `anthropic` or `claude`.
|
||||
2. `description` is required, 1-1024 chars.
|
||||
3. `x-personal-mcp.id` must exactly match `name`.
|
||||
4. Directory slug must exactly match `name`.
|
||||
|
||||
Capability invariants:
|
||||
|
||||
1. Skill capabilities must include `resource://skills/<skill-id>/document`.
|
||||
2. Prompt capabilities must include `resource://prompts/<prompt-id>/document`.
|
||||
|
||||
Repository contract decisions:
|
||||
|
||||
1. Treat `name` and `description` as required in all `SKILL.md` files.
|
||||
2. Keep only validated standard fields at top level.
|
||||
3. Keep MCP indexing metadata in a namespaced extension block.
|
||||
4. Reject unsupported optional top-level fields until explicit model support is added.
|
||||
|
||||
Reference specs:
|
||||
|
||||
1. MCP prompts data types: [Prompts](https://modelcontextprotocol.io/specification/latest/server/prompts)
|
||||
2. MCP schema reference for `Prompt` and `PromptArgument`: [Schema](https://modelcontextprotocol.io/specification/latest/schema)
|
||||
|
||||
## Canonical Frontmatter Schema
|
||||
|
||||
Use this two-layer pattern:
|
||||
|
||||
1. Anthropic layer: top-level fields intended for Anthropic and Agent Skills behavior.
|
||||
2. Repository layer: one namespaced block, `x-personal-mcp`, for MCP catalog and routing metadata.
|
||||
|
||||
Canonical shape:
|
||||
|
||||
```yaml
|
||||
---
|
||||
name: <skill-id>
|
||||
description: <what this skill does and when to use it>
|
||||
|
||||
# Repository-specific metadata
|
||||
x-personal-mcp:
|
||||
id: <skill-id>
|
||||
version: <semver>
|
||||
tags:
|
||||
- <tag>
|
||||
capabilities:
|
||||
- resource://skills/<skill-id>/document
|
||||
# Optional: overrides and nested references only.
|
||||
# Top-level references/*.md are auto-discovered.
|
||||
references:
|
||||
<ref-id>:
|
||||
path: references/<file>.md
|
||||
mime_type: text/markdown
|
||||
title: <short title>
|
||||
---
|
||||
```
|
||||
|
||||
## Repository Metadata Field Rules
|
||||
|
||||
Rules for `x-personal-mcp`:
|
||||
|
||||
1. `id` is required, must follow the skill id rules from the content contract, and must equal the directory name.
|
||||
2. `version` is required and must be a semantic version string.
|
||||
3. `tags` is optional and should be a list of kebab-case discovery labels.
|
||||
4. `capabilities` is required and lists the MCP URIs the skill publishes.
|
||||
5. `references` is an optional map keyed by `ref-id` for overrides and nested entries.
|
||||
|
||||
Prompt-specific additions:
|
||||
|
||||
1. `arguments` is an optional map keyed by argument name.
|
||||
2. Each argument supports optional `title`, optional `description`, and optional `required`.
|
||||
3. This aligns with MCP `PromptArgument` shape (`name`, optional `title`, optional `description`, optional `required`) where `name` is represented by the map key.
|
||||
4. Prompt `capabilities` must include `resource://prompts/<prompt-id>/document`.
|
||||
|
||||
Example prompt frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
name: initial-test-structure
|
||||
description: Generate a baseline pytest test layout for a target scope.
|
||||
x-personal-mcp:
|
||||
id: initial-test-structure
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- pytest
|
||||
- testing
|
||||
capabilities:
|
||||
- resource://prompts/initial-test-structure/document
|
||||
arguments:
|
||||
target_scope:
|
||||
title: Target scope
|
||||
description: Target package or module under test.
|
||||
required: true
|
||||
---
|
||||
```
|
||||
|
||||
Reference entry rules:
|
||||
|
||||
1. `ref-id` is lowercase kebab-case.
|
||||
2. `path` is a skill-relative markdown path and must stay inside the same skill directory.
|
||||
3. Top-level files under `references/*.md` are auto-discovered with `ref-id` derived from a normalized filename stem (lowercase kebab-case).
|
||||
4. Nested folders under `references/` are not auto-discovered and must be declared explicitly.
|
||||
5. `mime_type` defaults to `text/markdown` when omitted.
|
||||
6. `title` is an optional display label.
|
||||
7. Renaming `ref-id` values is allowed when needed; optional aliases may be used during transitions.
|
||||
|
||||
## Auto-Generated Reference IDs
|
||||
|
||||
Top-level markdown files directly under `references/` are auto-registered as MCP references even when `x-personal-mcp.references` is empty.
|
||||
|
||||
How `ref-id` is derived:
|
||||
|
||||
1. Start from the filename stem (without `.md`).
|
||||
2. Normalize to lowercase kebab-case.
|
||||
3. Publish at `resource://skills/<skill-id>/references/<ref-id>`.
|
||||
|
||||
Examples:
|
||||
|
||||
1. `references/ruff-docs.md` -> `ref-id: ruff-docs`
|
||||
2. `references/Ruff Integrations.md` -> `ref-id: ruff-integrations`
|
||||
3. `references/python_logging_docs.md` -> `ref-id: python-logging-docs`
|
||||
|
||||
When to use explicit `x-personal-mcp.references` entries:
|
||||
|
||||
1. The file is nested, for example `references/guides/ci.md`.
|
||||
2. You need to override defaults (`title`, `mime_type`, or custom `ref-id`).
|
||||
3. You need compatibility aliases during a rename.
|
||||
|
||||
## Validation Models
|
||||
|
||||
The normative runtime model uses strict Pydantic v2 validation:
|
||||
|
||||
1. Models are immutable (`frozen=True`) and reject unknown fields (`extra="forbid"`).
|
||||
2. `SkillFrontmatter` accepts only `name`, `description`, and `x-personal-mcp`.
|
||||
3. `PromptFrontmatter` accepts only `name`, `description`, and `x-personal-mcp`.
|
||||
4. `PromptArgumentEntry` accepts only optional `title`, optional `description`, and optional `required`.
|
||||
5. Skill and prompt metadata enforce semver, kebab-case ids, capability requirements, and id/name/directory consistency.
|
||||
6. Reference paths are validated as markdown files under `references/`.
|
||||
|
||||
Validation behavior contract:
|
||||
|
||||
1. Validate required core fields and relationships during registry load before FastMCP resource or tool registration.
|
||||
2. Reject unknown or unsupported fields at parse and model-validation time.
|
||||
3. Treat hard contract violations, including missing required fields, invalid ids, and broken required mappings, as startup errors.
|
||||
4. Keep failure messages path-aware and field-specific for CI readability.
|
||||
|
||||
Projection mode contract for Anthropic API upload pipelines:
|
||||
|
||||
1. Parse with `SkillFrontmatter` first.
|
||||
2. Emit Anthropic-safe frontmatter with standard fields only.
|
||||
3. Preserve `x-personal-mcp` in source-of-truth documents; projection output is a build artifact.
|
||||
|
||||
## Anthropic Upload Compatibility Rule
|
||||
|
||||
1. Anthropic documentation guarantees behavior for standard frontmatter fields but does not explicitly guarantee handling of arbitrary unknown top-level keys.
|
||||
2. Publishing pipelines that target strict API compatibility should support a projection mode that emits only standard frontmatter fields for upload.
|
||||
3. Source-of-truth authoring remains in `x-personal-mcp`; upload payload shape is an explicit build concern.
|
||||
|
||||
## FastMCP Native Metadata Surfaces
|
||||
|
||||
Resources support native definition metadata:
|
||||
|
||||
1. `name`
|
||||
2. `description`
|
||||
3. `mime_type`
|
||||
4. `tags`
|
||||
5. `annotations`, including `readOnlyHint` and `idempotentHint`
|
||||
6. `icons`
|
||||
7. `meta`
|
||||
8. `version`
|
||||
9. `enabled`, which is deprecated in FastMCP v3 in favor of server-level enable and disable controls
|
||||
|
||||
Resources also support runtime metadata through `ResourceContent.meta` and `ResourceResult.meta`.
|
||||
|
||||
Tools support native definition metadata:
|
||||
|
||||
1. `name`
|
||||
2. `description`
|
||||
3. `tags`
|
||||
4. `annotations`, including `title`, `readOnlyHint`, `destructiveHint`, `idempotentHint`, and `openWorldHint`
|
||||
5. `icons`
|
||||
6. `meta`
|
||||
7. `version`
|
||||
8. `timeout`
|
||||
9. `output_schema`
|
||||
10. `run_in_thread`
|
||||
11. `enabled`, which is deprecated in FastMCP v3 in favor of server-level enable and disable controls
|
||||
|
||||
Tools also support runtime metadata through `ToolResult.meta`.
|
||||
|
||||
## Frontmatter To FastMCP Mapping Contract
|
||||
|
||||
At server startup, map `x-personal-mcp` into FastMCP registration as follows:
|
||||
|
||||
1. `x-personal-mcp.id` defines the canonical URI namespace and identity checks.
|
||||
2. `description` becomes the default description for the primary skill document resource.
|
||||
3. `x-personal-mcp.tags` maps to resource and tool tags.
|
||||
4. `x-personal-mcp.version` maps to resource and tool version metadata.
|
||||
5. `x-personal-mcp.capabilities` becomes the registered URI list and catalog exposure.
|
||||
6. `x-personal-mcp.references[*]` becomes resource templates or concrete resources with `mime_type`, read-only annotations, and `meta` that includes `skill_id`, `ref_id`, and source `path`.
|
||||
|
||||
## Invariants
|
||||
|
||||
This contract guarantees:
|
||||
|
||||
1. Anthropic-required frontmatter stays valid for custom skill upload and Claude Code loading.
|
||||
2. MCP-specific metadata remains embedded in `SKILL.md` frontmatter, with no `metadata.yaml` sidecar.
|
||||
3. FastMCP registration uses native metadata fields for resources and tools.
|
||||
4. Reference ids and metadata can evolve with low-friction updates while internal file layout under `references/` stays refactor-friendly.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This contract does not define:
|
||||
|
||||
1. URI versioning and deprecation rollout policy details.
|
||||
2. Migration script design from existing `metadata.yaml` files.
|
||||
3. Runtime caching and indexing performance tuning.
|
||||
@@ -1,75 +0,0 @@
|
||||
---
|
||||
icon: lucide/messages-square
|
||||
---
|
||||
|
||||
# Prompt Contract
|
||||
|
||||
This page defines the canonical contract for prompts in the docs-first MCP architecture.
|
||||
|
||||
## Canonical Prompt Shape
|
||||
|
||||
Each prompt is one directory under `docs/prompts/`:
|
||||
|
||||
```mermaid
|
||||
---
|
||||
config:
|
||||
treeView:
|
||||
rowIndent: 20
|
||||
lineThickness: 2
|
||||
themeVariables:
|
||||
treeView:
|
||||
labelColor: '#FFFFFF'
|
||||
lineColor: '#FFFFFF'
|
||||
---
|
||||
treeView-beta
|
||||
"docs/"
|
||||
"... (other docs)"
|
||||
"prompts/"
|
||||
"<prompt-id>/"
|
||||
"PROMPT.md"
|
||||
"references/"
|
||||
"... (one or more markdown files, optional nested folders)"
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
1. `PROMPT.md` is required for every prompt.
|
||||
2. `references/` is the only place for prompt-specific supporting docs.
|
||||
3. Nested folders inside `references/` are allowed so a prompt can reorganize internals without changing global architecture.
|
||||
4. Prompt directories are independent ownership boundaries; no cross-prompt file writes.
|
||||
|
||||
## Metadata Location Constraint
|
||||
|
||||
1. Prompt metadata is embedded in YAML frontmatter in `PROMPT.md`.
|
||||
2. No `metadata.yaml` sidecar exists in the end state.
|
||||
3. Reference lookup metadata is documented and explicit: top-level `references/*.md` are auto-discovered from filenames, while `PROMPT.md` frontmatter declares overrides and nested mappings when needed.
|
||||
|
||||
## Prompt Id Contract
|
||||
|
||||
`prompt-id` is the public identifier and should satisfy all rules below:
|
||||
|
||||
1. Format: lowercase kebab-case only.
|
||||
2. Character set: `a-z`, `0-9`, and `-`.
|
||||
3. Must start with a letter.
|
||||
4. No underscores, spaces, dots, or uppercase characters.
|
||||
5. Directory name should equal `prompt-id` in each committed revision.
|
||||
6. Frontmatter `id` should equal directory name in each committed revision.
|
||||
7. Treat `prompt-id` as immutable after release; any rename is a breaking replacement and clients must move to the new id.
|
||||
|
||||
Valid examples:
|
||||
|
||||
1. `pytest-fill-scaffold`
|
||||
2. `review-pr-comments`
|
||||
3. `scaffold-fastapi-service`
|
||||
|
||||
Invalid examples:
|
||||
|
||||
1. `fill_pytest_scaffold`
|
||||
2. `Prompt-Template`
|
||||
3. `docs.prompt`
|
||||
|
||||
## Direct Documentation Inclusion
|
||||
|
||||
1. For direct API documentation, use mkdocstrings directives rather than pasting large code blocks.
|
||||
2. Keep manually-authored code examples short and task-focused; large implementation excerpts are out of scope for this contract.
|
||||
|
||||
@@ -1,186 +0,0 @@
|
||||
---
|
||||
icon: lucide/link
|
||||
---
|
||||
|
||||
# URI Contract
|
||||
|
||||
This page defines the canonical resource URI contract, template parameter rules, and compatibility policy.
|
||||
|
||||
Conventions in this document follow [MCP resource semantics](https://modelcontextprotocol.io/docs/learn/server-concepts#resources), [URI generic syntax (RFC3986)](https://www.rfc-editor.org/rfc/rfc3986), and [URI templates (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570).
|
||||
|
||||
## Canonical URI Surface
|
||||
|
||||
The public, preferred direct resource URIs are:
|
||||
|
||||
1. `resource://catalog/skills_index`
|
||||
2. `resource://catalog/skills/{skill_id}`
|
||||
3. `resource://skills/{skill_id}/document`
|
||||
4. `resource://skills/{skill_id}/references/{ref_id}`
|
||||
5. `resource://docs/{path*}`
|
||||
6. `resource://catalog/prompts_index`
|
||||
7. `resource://catalog/prompts/{prompt_id}`
|
||||
8. `resource://prompts/{prompt_id}/document`
|
||||
|
||||
The public, preferred resource template URIs are:
|
||||
|
||||
1. `resource://catalog/skills_index{?q,tag,capability,cursor,limit}`
|
||||
2. `resource://catalog/prompts_index{?q,tag,cursor,limit}`
|
||||
|
||||
Contract intent:
|
||||
|
||||
1. Catalog URIs are discovery surfaces.
|
||||
2. Skill URIs are the primary per-skill guidance surfaces.
|
||||
3. Catalog query templates are additive discovery helpers for filtering and pagination.
|
||||
4. The docs wildcard URI is a direct authored-markdown access surface under `docs/`.
|
||||
|
||||
Best-practice alignment:
|
||||
|
||||
1. Resource identifiers are stable and noun-oriented.
|
||||
2. Dynamic lookup variants are represented as RFC6570 templates.
|
||||
3. Resources remain read-oriented and are described with explicit MIME types.
|
||||
|
||||
## URI Semantics
|
||||
|
||||
### `resource://catalog/skills_index`
|
||||
|
||||
1. Returns a compact list of skill records for discovery.
|
||||
2. Contains one entry per `skill_id`.
|
||||
3. Includes enough metadata for client-side selection, at minimum `id`, `name`, `description`, `tags`, and `capabilities`.
|
||||
|
||||
### `resource://catalog/skills/{skill_id}`
|
||||
|
||||
1. Returns one normalized record for `skill_id`.
|
||||
2. Includes the canonical document URI and declared reference ids.
|
||||
3. Returns not found when `skill_id` does not exist.
|
||||
|
||||
### `resource://skills/{skill_id}/document`
|
||||
|
||||
1. Returns the canonical `SKILL.md` authored content for that skill.
|
||||
2. `skill_id` must satisfy the stable skill id rules from the content contract.
|
||||
|
||||
### `resource://skills/{skill_id}/references/{ref_id}`
|
||||
|
||||
1. Returns one reference document declared in the skill frontmatter references manifest.
|
||||
2. `ref_id` is the stable public handle for that reference document.
|
||||
|
||||
### `resource://docs/{path*}`
|
||||
|
||||
1. Returns authored markdown at a normalized relative path under `docs/`.
|
||||
2. Supports nested paths via [RFC6570 wildcard expansion](https://www.rfc-editor.org/rfc/rfc6570).
|
||||
3. Typical examples include `index.md`, `usage.md`, `skills/<skill-id>/SKILL.md`, and `skills/<skill-id>/references/<file>.md`.
|
||||
|
||||
### `resource://catalog/prompts_index`
|
||||
|
||||
1. Returns a compact list of prompt records for discovery.
|
||||
2. Contains one entry per `prompt_id`.
|
||||
3. Includes `id`, `name`, `description`, `tags`, `version`, and canonical document URI.
|
||||
|
||||
### `resource://catalog/skills_index{?q,tag,capability,cursor,limit}`
|
||||
|
||||
1. Returns the same record family as `resource://catalog/skills_index` with optional filtering and pagination.
|
||||
2. Query parameters are optional and composable.
|
||||
3. Unknown query keys are ignored or rejected deterministically by server policy.
|
||||
|
||||
### `resource://catalog/prompts_index{?q,tag,cursor,limit}`
|
||||
|
||||
1. Returns the same record family as `resource://catalog/prompts_index` with optional filtering and pagination.
|
||||
2. Query parameters are optional and composable.
|
||||
3. Unknown query keys are ignored or rejected deterministically by server policy.
|
||||
|
||||
### `resource://catalog/prompts/{prompt_id}`
|
||||
|
||||
1. Returns one normalized record for `prompt_id`.
|
||||
2. Includes prompt argument metadata when declared in frontmatter.
|
||||
3. Returns not found when `prompt_id` does not exist.
|
||||
|
||||
### `resource://prompts/{prompt_id}/document`
|
||||
|
||||
1. Returns the canonical prompt markdown document.
|
||||
2. `prompt_id` must satisfy lowercase kebab-case rules.
|
||||
|
||||
## Template Parameter And Validation Rules
|
||||
|
||||
### `skill_id`
|
||||
|
||||
1. Lowercase kebab-case.
|
||||
2. Must satisfy the stable skill id rules from the content contract.
|
||||
|
||||
### `ref_id`
|
||||
|
||||
1. Lowercase kebab-case.
|
||||
2. Must be declared in the skill's references manifest.
|
||||
|
||||
### `path*`
|
||||
|
||||
1. Relative POSIX path only, expressed as URI path segments under [RFC3986 path syntax](https://www.rfc-editor.org/rfc/rfc3986#section-3.3).
|
||||
2. No leading slash.
|
||||
3. No `..` traversal segments.
|
||||
4. Resolves only inside `docs/`.
|
||||
5. Markdown-only in the end state, meaning `.md` files.
|
||||
6. Any reserved URI characters in path segments must be [percent-encoded](https://www.rfc-editor.org/rfc/rfc3986#section-2.1).
|
||||
|
||||
### `prompt_id`
|
||||
|
||||
1. Lowercase kebab-case.
|
||||
2. Must be unique across prompt ids and must not collide with skill ids.
|
||||
|
||||
## URI Hygiene Rules
|
||||
|
||||
1. Use lowercase, human-readable path segments for stable discoverability.
|
||||
2. Keep identifiers immutable once public whenever practical.
|
||||
3. Keep template variables semantic (`skill_id`, `prompt_id`, `ref_id`, `path*`) and avoid overloading one variable for unrelated meanings.
|
||||
4. Do not include secrets, tokens, or user-identifying data in URI paths or query strings.
|
||||
5. Prefer additive query parameters for discovery over introducing parallel URI families, matching [MCP resource-template discovery patterns](https://modelcontextprotocol.io/docs/learn/server-concepts#resources).
|
||||
6. Return clear not-found semantics for unknown ids and invalid template resolution.
|
||||
|
||||
## URI Versioning Policy
|
||||
|
||||
Default rule:
|
||||
|
||||
1. Keep URIs unversioned by default.
|
||||
2. Allow URI and payload updates when they improve clarity or implementation simplicity.
|
||||
|
||||
Breaking-change rule:
|
||||
|
||||
1. Breaking changes use direct replacement of the canonical URI family.
|
||||
2. No compatibility aliases or dual URI families are maintained.
|
||||
|
||||
FastMCP version metadata usage:
|
||||
|
||||
1. Resource `version` metadata may be used for implementation and version discovery.
|
||||
2. URI readability and maintainability remain the primary contract.
|
||||
|
||||
## Reference Id Compatibility Policy
|
||||
|
||||
`ref_id` is the public identifier for a reference document, separate from file path.
|
||||
|
||||
Rules:
|
||||
|
||||
1. Prefer keeping `ref_id` stable when practical.
|
||||
2. File paths may change without URI churn as long as the mapped `ref_id` still resolves.
|
||||
3. If a reference is renamed, introduce a new `ref_id` and treat the old one as retired.
|
||||
4. Avoid reusing retired `ref_id` values for unrelated content.
|
||||
|
||||
## Invariants
|
||||
|
||||
This contract guarantees:
|
||||
|
||||
1. One canonical URI pattern per core capability surface.
|
||||
2. Fast, low-friction URI evolution through direct replacement of canonical URIs.
|
||||
3. A single canonical catalog URI family with no alias maintenance overhead.
|
||||
4. Reference mappings can evolve with minimal churn.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This contract does not define:
|
||||
|
||||
1. Implementation-specific transform wiring details, such as `VersionFilter`, mounts, or provider composition.
|
||||
2. Migration script mechanics for auto-generating aliases.
|
||||
3. Authorization policy design for URI-level access control.
|
||||
|
||||
## Sources
|
||||
|
||||
1. [MCP Server Concepts: Resources](https://modelcontextprotocol.io/docs/learn/server-concepts#resources)
|
||||
2. [MCP Architecture Overview](https://modelcontextprotocol.io/docs/learn/architecture)
|
||||
3. [MCP Specification Repository](https://github.com/modelcontextprotocol/spec)
|
||||
4. [RFC6570 URI Template](https://www.rfc-editor.org/rfc/rfc6570)
|
||||
-204
@@ -1,204 +0,0 @@
|
||||
---
|
||||
icon: lucide/bot
|
||||
---
|
||||
|
||||
# Copilot MCP Mechanics
|
||||
|
||||
## Purpose
|
||||
|
||||
This page explains how the GitHub Copilot extension in VS Code behaves as an MCP client when connected to `personal-mcp`, including why tools can appear while resource attachment appears unavailable.
|
||||
|
||||
## Core Model
|
||||
|
||||
Copilot interacts with MCP servers through separate capability lanes:
|
||||
|
||||
1. tools (invoked by the model during execution)
|
||||
2. resources (attached as read-only context)
|
||||
3. prompts (server-provided prompt templates)
|
||||
|
||||
These lanes are related but independently gated in the client.
|
||||
|
||||
Reliable paths are:
|
||||
|
||||
1. attach MCP resources explicitly through `Add Context > MCP Resources` or `MCP: Browse Resources`
|
||||
2. let Copilot invoke MCP tools when the task and tool descriptions make that relevant
|
||||
3. invoke MCP prompts explicitly with `/server.prompt` when your server exposes them
|
||||
|
||||
## What Actually Happens In VS Code
|
||||
|
||||
### MCP server side
|
||||
|
||||
Your server can advertise resources and serve them correctly. In this project that includes catalog resources and skill document resources.
|
||||
|
||||
### Copilot session side
|
||||
|
||||
The chat surface exposes tools, resources, and prompts through different UI paths. In practice, you can encounter sessions where tool use is available but MCP resource attachment is not exposed in `Add Context`.
|
||||
|
||||
That is why you can sometimes see MCP tools before you see `Add Context > MCP Resources`.
|
||||
|
||||
## Why The Picker Sometimes Shows Only Tools
|
||||
|
||||
`MCP Resources...` in Add Context requires at least:
|
||||
|
||||
1. at least one connected MCP server advertises resource capability
|
||||
2. the current chat surface exposes MCP resource attachment
|
||||
|
||||
If the second condition is not met, resources can be available on the server while still being absent from the picker.
|
||||
|
||||
## Practical Workflow
|
||||
|
||||
Use this sequence to confirm behavior:
|
||||
|
||||
1. run `MCP: Browse Resources` and verify resources exist
|
||||
2. use `MCP: List Servers` to verify the server is enabled and running
|
||||
3. open Copilot Chat
|
||||
4. check `Add Context` for `MCP Resources...`
|
||||
5. if still missing, restart the server and reload VS Code window
|
||||
|
||||
## Recommended Usage Pattern
|
||||
|
||||
1. rely on canonical catalog resources for discovery (`skills_index`, then `skills/{skill_id}`)
|
||||
2. fetch only selected skill documents for context
|
||||
3. keep slash commands for deterministic fallback flows
|
||||
|
||||
When resource attachment is unavailable in the active session, use ResourcesAsTools first, then thin catalog discovery tools as parity fallback:
|
||||
|
||||
1. `list_resources`
|
||||
2. `read_resource`
|
||||
3. `search_patterns`
|
||||
4. `get_pattern_by_id`
|
||||
5. `get_skill_document_by_id`
|
||||
|
||||
Canonical naming policy:
|
||||
|
||||
1. Prefer the five canonical tool names above in prompts and instructions.
|
||||
2. For compatibility with clients that emit `catalog_*` naming, the server also exposes:
|
||||
- `catalog_search_patterns`
|
||||
- `catalog_get_pattern_by_id`
|
||||
- `catalog_get_skill_document_by_id`
|
||||
3. Canonical and compatibility alias tools return equivalent payloads for the same input.
|
||||
|
||||
The first two are generated from the canonical resource surface and should be preferred in tool-only clients.
|
||||
|
||||
These should stay read-only, minimal, and schema-aligned with catalog resources.
|
||||
|
||||
For very large tool catalogs, server operators can optionally enable tool search mode (`regex` or `bm25`) while keeping `list_resources` and `read_resource` pinned as always-visible fallback tools.
|
||||
|
||||
## What To Type In Copilot Chat
|
||||
|
||||
Use prompts that tell Copilot which MCP feature path to take.
|
||||
|
||||
### If `MCP Resources...` is available
|
||||
|
||||
Use the resource attachment UI first, then ask Copilot to work from the attached material.
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
I attached the catalog resources and the FastAPI async SQLAlchemy modernization skill document. Use that context to propose a migration plan for this repo.
|
||||
```
|
||||
|
||||
If you want to keep the attachment sequence explicit, use:
|
||||
|
||||
```text
|
||||
I attached personal-mcp catalog resources first. Use them to identify the best matching skill, then work only from the selected skill document.
|
||||
```
|
||||
|
||||
### If only tools are available
|
||||
|
||||
Ask Copilot to explicitly use resource-backed tools first.
|
||||
|
||||
Example resource-backed prompt:
|
||||
|
||||
```text
|
||||
Use personal-mcp tool fallback by first calling list_resources, then read_resource for resource://catalog/skills_index and the selected resource://skills/<skill-id>/document URI. Use only that loaded skill context in your answer.
|
||||
```
|
||||
|
||||
If needed, use the thin catalog tools.
|
||||
|
||||
Example discovery prompt:
|
||||
|
||||
```text
|
||||
Use the personal-mcp catalog tools to search for the most relevant skill for FastAPI async SQLAlchemy modernization. Then load the selected skill document and use it as context for your answer.
|
||||
```
|
||||
|
||||
Example direct-load prompt:
|
||||
|
||||
```text
|
||||
Call get_skill_document_by_id for async-fastapi-sqlmodel and use that document as the main context for this task.
|
||||
```
|
||||
|
||||
Example bounded-selection prompt:
|
||||
|
||||
```text
|
||||
Search personal-mcp skills for NiceGUI UI customization, select at most 2 strong matches, load the best skill document, and answer using only that material plus the workspace code.
|
||||
```
|
||||
|
||||
## Repo Instructions Example
|
||||
|
||||
Repo instructions are the best place to teach Copilot when MCP content is relevant and which path to prefer.
|
||||
|
||||
If you add a repo-level `copilot-instructions.md`, keep the rule simple: prefer catalog-first discovery, keep loaded skill context small, and fall back to tools when resource attachment is unavailable.
|
||||
|
||||
Instructions can strongly steer behavior, but they do not guarantee that VS Code will auto-attach MCP resources for a request. For reliable resource use, either attach resources explicitly or prompt Copilot to use the fallback tools.
|
||||
|
||||
Example:
|
||||
|
||||
```md
|
||||
# MCP Usage
|
||||
|
||||
When a task may benefit from personal-mcp skills, use this sequence:
|
||||
|
||||
1. Start with personal-mcp catalog discovery when the task appears to match documented implementation patterns.
|
||||
2. Prefer MCP resources when the chat surface exposes resource attachment.
|
||||
3. If MCP resource attachment is unavailable, use `list_resources`/`read_resource` first, then thin catalog tools if needed.
|
||||
4. Load only the most relevant skill document or at most 2 skill documents.
|
||||
5. Treat skill documents as guidance, then reconcile them with the actual repository code before making changes.
|
||||
|
||||
Preferred discovery order:
|
||||
|
||||
1. `resource://catalog/skills_index`
|
||||
2. `resource://catalog/skills/{skill_id}`
|
||||
3. `resource://skills/<skill-id>/document`
|
||||
4. `resource://skills/<skill-id>/references/<ref-id>` when needed
|
||||
|
||||
Tool fallback order:
|
||||
|
||||
1. `list_resources`
|
||||
2. `read_resource`
|
||||
3. `search_patterns`
|
||||
4. `get_pattern_by_id`
|
||||
5. `get_skill_document_by_id`
|
||||
|
||||
If confidence is low after catalog discovery, ask one clarifying question before loading more skill documents.
|
||||
```
|
||||
|
||||
That instruction style does two useful things:
|
||||
|
||||
1. it tells Copilot to prefer the MCP server when relevant without forcing it on every prompt
|
||||
2. it keeps context size bounded so skill loading does not become noisy or expensive
|
||||
|
||||
If you want stronger behavior, add one more line that names the MCP server directly:
|
||||
|
||||
```md
|
||||
Use the `personal-mcp` server for skill discovery whenever the task involves documented implementation patterns available from the catalog.
|
||||
```
|
||||
|
||||
## Known Gotcha
|
||||
|
||||
A successful `resources/list` response from the server does not guarantee the resource picker appears in every Copilot session type. UI availability is session-capability-dependent.
|
||||
|
||||
## Further Reading
|
||||
|
||||
### VS Code docs
|
||||
|
||||
1. [Add and manage MCP servers](https://code.visualstudio.com/docs/agent-customization/mcp-servers)
|
||||
2. [MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration)
|
||||
3. [Manage context for AI](https://code.visualstudio.com/docs/chat/copilot-chat-context)
|
||||
4. [AI features cheat sheet](https://code.visualstudio.com/docs/agents/reference/ai-features-cheat-sheet)
|
||||
|
||||
### Project docs
|
||||
|
||||
1. [Resource-First Pattern Module Architecture](./architecture.md)
|
||||
2. [Static Docs Hosting Pattern](./mcp_layout.md)
|
||||
3. [Skill Usage Mechanics](./usage.md)
|
||||
@@ -1,201 +0,0 @@
|
||||
---
|
||||
icon: lucide/server
|
||||
---
|
||||
|
||||
# Static Docs Hosting Pattern
|
||||
|
||||
## Purpose
|
||||
|
||||
This document describes the completed layout and runtime pattern used to host a pre-built static documentation site from the same FastAPI app process that runs the FastMCP server.
|
||||
|
||||
This design intentionally avoids runtime docs rendering and avoids a separate docs hosting service.
|
||||
|
||||
It also treats Markdown as the single source of truth for both MCP resources and published docs.
|
||||
|
||||
## Completed-State Layout
|
||||
|
||||
```mermaid
|
||||
---
|
||||
config:
|
||||
treeView:
|
||||
rowIndent: 40
|
||||
lineThickness: 2
|
||||
themeVariables:
|
||||
treeView:
|
||||
labelColor: '#FFFFFF'
|
||||
lineColor: '#FFFFFF'
|
||||
---
|
||||
treeView-beta
|
||||
"project-root"
|
||||
"pyproject.toml"
|
||||
"uv.lock"
|
||||
"zensical.toml"
|
||||
"docs"
|
||||
"index.md"
|
||||
"<project-docs>.md"
|
||||
"contracts"
|
||||
"index.md"
|
||||
"<contract-pages>.md"
|
||||
"mcp_layout.md"
|
||||
"prompts"
|
||||
"<prompt-id>"
|
||||
"PROMPT.md"
|
||||
"references"
|
||||
"skills"
|
||||
"<skill-id>"
|
||||
"SKILL.md"
|
||||
"references"
|
||||
"<reference>.md"
|
||||
"site"
|
||||
"static build output"
|
||||
"src"
|
||||
"personal_mcp"
|
||||
"__init__.py"
|
||||
"main.py"
|
||||
"mcp.py"
|
||||
"catalog"
|
||||
"<catalog-modules>.py"
|
||||
"registry"
|
||||
"<registry-modules>.py"
|
||||
"web"
|
||||
"<web-modules>.py"
|
||||
"skills"
|
||||
"<skills-modules>.py"
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
1. docs contains both project-authored pages and the canonical skill Markdown tree.
|
||||
2. site contains static build output only.
|
||||
3. docs/skills contains canonical skill Markdown and reference Markdown.
|
||||
4. docs/prompts contains canonical prompt Markdown used for prompt catalog and document surfaces.
|
||||
5. MCP resources and docs site read from the same Markdown sources.
|
||||
|
||||
## Runtime Composition
|
||||
|
||||
The runtime process serves two surfaces:
|
||||
|
||||
1. MCP protocol surface from FastMCP
|
||||
2. Static docs surface from FastAPI static mount
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Docs Registry Loader] --> B[Validated In-Memory Registry]
|
||||
B --> C[FastMCP Resource Registration]
|
||||
C --> D[MCP Transport]
|
||||
C --> E[FastAPI Application]
|
||||
E --> F[Static Mount /docs]
|
||||
F --> G[Zensical site output directory]
|
||||
```
|
||||
|
||||
Runtime guarantees:
|
||||
|
||||
1. Docs registry load and validation happen before resource exposure.
|
||||
2. Duplicate resource and template registration fails startup (`on_duplicate="error"`).
|
||||
3. Resource registration is metadata-driven from SKILL frontmatter and reference manifests.
|
||||
4. Legacy per-skill Python servers and `metadata.yaml` sidecars are not part of the runtime.
|
||||
|
||||
## Build and Publish Flow
|
||||
|
||||
The docs flow is pre-build only.
|
||||
|
||||
1. Read authored docs pages and skill markdown sources.
|
||||
2. Build static site with Zensical into site.
|
||||
3. Start app and serve site directory as static files.
|
||||
|
||||
No runtime markdown conversion is required.
|
||||
|
||||
## Content Merge Pattern
|
||||
|
||||
The published docs site always contains both:
|
||||
|
||||
1. Project-authored docs pages
|
||||
2. Skill Markdown content from docs/skills/*/SKILL.md and references
|
||||
|
||||
This ensures the public docs reflect architectural guidance and the exact Markdown served by MCP.
|
||||
|
||||
## Markdown-to-Resource Mapping
|
||||
|
||||
MCP resources map directly to canonical Markdown documents.
|
||||
|
||||
Example mapping model:
|
||||
|
||||
1. docs/skills/<skill-id>/SKILL.md -> resource://skills/<skill_id>/document
|
||||
2. docs/skills/<skill-id>/references/<file>.md -> resource://skills/<skill_id>/references/<ref_id> (via frontmatter references manifest)
|
||||
3. docs/<path>.md -> resource://docs/{path*}
|
||||
|
||||
Catalog discovery resources are:
|
||||
|
||||
1. resource://catalog/skills_index
|
||||
2. resource://catalog/skills_index{?q,tag,capability,cursor,limit}
|
||||
3. resource://catalog/skills/{skill_id}
|
||||
4. resource://catalog/prompts_index
|
||||
5. resource://catalog/prompts_index{?q,tag,cursor,limit}
|
||||
6. resource://catalog/prompts/{prompt_id}
|
||||
|
||||
Registry-backed registration details:
|
||||
|
||||
1. `resource://skills/{skill_id}/document` resolves to each skill's SKILL.md.
|
||||
2. `resource://skills/{skill_id}/references/{ref_id}` resolves through frontmatter reference manifests.
|
||||
3. `resource://docs/{path*}` resolves normalized markdown paths under `docs/`.
|
||||
4. Resource metadata includes explicit mime type and read-only/idempotent annotations.
|
||||
|
||||
When clients cannot attach MCP resources directly, thin catalog tools may retrieve the same underlying skill documents indirectly. This does not create a second content source.
|
||||
|
||||
## URI Compatibility Policy
|
||||
|
||||
1. Canonical URIs are the only supported URIs in this runtime.
|
||||
2. No backward-compatibility aliases or dual registration paths are maintained.
|
||||
3. Contract changes should update clients to canonical URIs directly.
|
||||
|
||||
## Why This Pattern
|
||||
|
||||
### Operational Simplicity
|
||||
|
||||
One application process serves both protocol and static docs surfaces.
|
||||
|
||||
### Deterministic Docs
|
||||
|
||||
Published docs are immutable static assets for a given build.
|
||||
|
||||
### Documentation Fidelity
|
||||
|
||||
The docs site and MCP resources resolve from the same Markdown sources.
|
||||
|
||||
### Maintainer Experience
|
||||
|
||||
Authors continue to work in markdown while resource contracts remain machine-consumable.
|
||||
|
||||
## FastAPI Static Mount Expectations
|
||||
|
||||
The FastAPI app is expected to:
|
||||
|
||||
1. Mount static directory containing Zensical output.
|
||||
2. Serve index and asset files from that directory.
|
||||
3. Keep docs route stable across releases.
|
||||
|
||||
Recommended route conventions:
|
||||
|
||||
1. /docs for static site root
|
||||
2. /docs/* for static assets and page routes
|
||||
|
||||
## Update Lifecycle
|
||||
|
||||
For each documentation update:
|
||||
|
||||
1. Edit authored docs and skill markdown content.
|
||||
2. Rebuild static site.
|
||||
3. Restart runtime if needed.
|
||||
|
||||
This keeps docs publication explicit and predictable.
|
||||
|
||||
## Example Source Material
|
||||
|
||||
Existing reference docs remain valid content inputs in this pattern:
|
||||
|
||||
1. docs/skills/pytesting/references/pytest-docs.md
|
||||
2. docs/skills/python-logging/references/python-logging-docs.md
|
||||
3. docs/skills/python-logging/references/json-file-logging.md
|
||||
4. docs/skills/fastapi-uv-docker/references/fastapi-best-practices.md
|
||||
|
||||
These are source documents, not deployment artifacts.
|
||||
@@ -1,334 +0,0 @@
|
||||
# Basic CRUD Repository and Functions
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [SQLModel create-data tutorial](https://sqlmodel.tiangolo.com/tutorial/fastapi/multiple-models/)
|
||||
- [SQLModel update-data tutorial](https://sqlmodel.tiangolo.com/tutorial/fastapi/update-extra-data/)
|
||||
- [SQLModel select tutorial](https://sqlmodel.tiangolo.com/tutorial/select/)
|
||||
- [SQLAlchemy `AsyncSession` API](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.AsyncSession)
|
||||
|
||||
??? abstract "Decision metadata"
|
||||
- Status: adopted
|
||||
- Decision level: advisory
|
||||
- Applies to: api-runtime, workers, tests
|
||||
- Last reviewed: 2026-07-26
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Show a small SQLModel CRUD layer in two forms:
|
||||
|
||||
- independent functions for convenient standalone or composed operations;
|
||||
- a repository object that groups those functions behind one domain-oriented interface.
|
||||
|
||||
Every public operation accepts an optional `AsyncSession`. When omitted, reads resolve the cached session factory and own a short-lived session, while writes resolve the same factory and own a complete session-and-transaction scope. When supplied, reads borrow the session and writes borrow its already-active caller-owned transaction. The repository stores configuration and delegates to the same functions without changing those semantics.
|
||||
|
||||
Use the same vocabulary at every layer:
|
||||
|
||||
| Operation | Function | Repository method | Scope when session is omitted | Missing-row result |
|
||||
|---|---|---|---|---|
|
||||
| Create | `create_widget()` | `create()` | Owned transaction | Not applicable |
|
||||
| Read one | `get_widget()` | `get()` | Owned session | `None` |
|
||||
| Read many | `list_widgets()` | `list()` | Owned session | Empty list |
|
||||
| Update | `update_widget()` | `update()` | Owned transaction | `None` |
|
||||
| Delete | `delete_widget()` | `delete()` | Owned transaction | `None` |
|
||||
|
||||
Functions and repository methods both put domain arguments first. Database configuration and sessions are keyword-only infrastructure arguments. This keeps call sites analogous and makes ownership choices visible.
|
||||
|
||||
---
|
||||
|
||||
## Models
|
||||
|
||||
Start with one table model when the application does not need distinct persistence and API schemas.
|
||||
|
||||
```python
|
||||
from sqlmodel import Field
|
||||
from sqlmodel import SQLModel
|
||||
|
||||
|
||||
class Widget(SQLModel, table=True):
|
||||
id: int | None = Field(default=None, primary_key=True)
|
||||
name: str = Field(index=True)
|
||||
description: str | None = None
|
||||
```
|
||||
|
||||
This reference uses direct field arguments and full-update semantics to keep the CRUD mechanics visible. Introduce separate create, update, or public schemas only when an API boundary needs different validation, field visibility, or partial-update behavior. See [SQLModel integration](sqlmodel.md) for that larger modeling pattern.
|
||||
|
||||
---
|
||||
|
||||
## Independent CRUD Functions
|
||||
|
||||
Functions are the simplest default when grouping state or behavior in an object adds no value. Each function is a complete operation boundary: it can run standalone by resolving the cached factory from `database_url`, or compose into a caller-owned scope through `session`.
|
||||
|
||||
```python
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlmodel import select
|
||||
|
||||
from .session import session_scope
|
||||
from .session import transaction_scope
|
||||
|
||||
|
||||
async def create_widget(
|
||||
name: str,
|
||||
description: str | None = None,
|
||||
*,
|
||||
database_url: str,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget:
|
||||
async with transaction_scope(
|
||||
database_url=database_url,
|
||||
session=session,
|
||||
) as active_session:
|
||||
widget = Widget(name=name, description=description)
|
||||
active_session.add(widget)
|
||||
await active_session.flush()
|
||||
return widget
|
||||
|
||||
|
||||
async def get_widget(
|
||||
widget_id: int,
|
||||
*,
|
||||
database_url: str,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget | None:
|
||||
async with session_scope(
|
||||
database_url=database_url,
|
||||
session=session,
|
||||
) as active_session:
|
||||
return await active_session.get(Widget, widget_id)
|
||||
|
||||
|
||||
async def list_widgets(
|
||||
*,
|
||||
database_url: str,
|
||||
offset: int = 0,
|
||||
limit: int = 100,
|
||||
session: AsyncSession | None = None,
|
||||
) -> list[Widget]:
|
||||
if offset < 0:
|
||||
raise ValueError("offset must be non-negative")
|
||||
if not 1 <= limit <= 100:
|
||||
raise ValueError("limit must be between 1 and 100")
|
||||
|
||||
async with session_scope(
|
||||
database_url=database_url,
|
||||
session=session,
|
||||
) as active_session:
|
||||
statement = select(Widget).order_by(Widget.id).offset(offset).limit(limit)
|
||||
return list(await active_session.scalars(statement))
|
||||
|
||||
|
||||
async def update_widget(
|
||||
widget_id: int,
|
||||
name: str,
|
||||
description: str | None,
|
||||
*,
|
||||
database_url: str,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget | None:
|
||||
async with transaction_scope(
|
||||
database_url=database_url,
|
||||
session=session,
|
||||
) as active_session:
|
||||
widget = await active_session.get(Widget, widget_id)
|
||||
if widget is None:
|
||||
return None
|
||||
|
||||
widget.name = name
|
||||
widget.description = description
|
||||
await active_session.flush()
|
||||
return widget
|
||||
|
||||
|
||||
async def delete_widget(
|
||||
widget_id: int,
|
||||
*,
|
||||
database_url: str,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget | None:
|
||||
async with transaction_scope(
|
||||
database_url=database_url,
|
||||
session=session,
|
||||
) as active_session:
|
||||
widget = await active_session.get(Widget, widget_id)
|
||||
if widget is None:
|
||||
return None
|
||||
|
||||
await active_session.delete(widget)
|
||||
await active_session.flush()
|
||||
return widget
|
||||
```
|
||||
|
||||
Update and delete load the row through the same session that mutates it. This avoids accepting detached instances from an earlier standalone read and gives both operations an explicit `None` result that the application layer can map to a domain or HTTP error. Delete returns the loaded object for callers that need its values, but that object represents a row scheduled for deletion and must not be reused as persistent state. List operations validate their bounds and order by the primary key so pagination is deterministic. Add a unique tiebreaker whenever ordering by a non-unique field.
|
||||
|
||||
`flush()` sends pending writes and populates ordinary generated primary keys. It does not itself commit. For a standalone write, the surrounding owned `transaction_scope()` commits after the function body succeeds. For a supplied session, the caller's outer transaction retains commit and rollback ownership. Use `await active_session.refresh(widget)` only when the operation deliberately needs database-generated state that was not returned during the flush; an unconditional refresh adds another query.
|
||||
|
||||
---
|
||||
|
||||
## Repository Object
|
||||
|
||||
A repository can provide a stable domain-facing interface when several callers need the same grouped operations. It stores repeatable database configuration, never a mutable session. Every method delegates to the analogous function and exposes the same optional-session contract.
|
||||
|
||||
```python
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
class WidgetRepository:
|
||||
def __init__(self, database_url: str) -> None:
|
||||
self.database_url = database_url
|
||||
|
||||
async def create(
|
||||
self,
|
||||
name: str,
|
||||
description: str | None = None,
|
||||
*,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget:
|
||||
return await create_widget(
|
||||
name,
|
||||
description,
|
||||
database_url=self.database_url,
|
||||
session=session,
|
||||
)
|
||||
|
||||
async def get(
|
||||
self,
|
||||
widget_id: int,
|
||||
*,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget | None:
|
||||
return await get_widget(
|
||||
widget_id,
|
||||
database_url=self.database_url,
|
||||
session=session,
|
||||
)
|
||||
|
||||
async def list(
|
||||
self,
|
||||
*,
|
||||
offset: int = 0,
|
||||
limit: int = 100,
|
||||
session: AsyncSession | None = None,
|
||||
) -> list[Widget]:
|
||||
return await list_widgets(
|
||||
database_url=self.database_url,
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
session=session,
|
||||
)
|
||||
|
||||
async def update(
|
||||
self,
|
||||
widget_id: int,
|
||||
name: str,
|
||||
description: str | None,
|
||||
*,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget | None:
|
||||
return await update_widget(
|
||||
widget_id,
|
||||
name,
|
||||
description,
|
||||
database_url=self.database_url,
|
||||
session=session,
|
||||
)
|
||||
|
||||
async def delete(
|
||||
self,
|
||||
widget_id: int,
|
||||
*,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget | None:
|
||||
return await delete_widget(
|
||||
widget_id,
|
||||
database_url=self.database_url,
|
||||
session=session,
|
||||
)
|
||||
```
|
||||
|
||||
The object is intentionally thin. Tests can construct it with a test database URL or pass a transaction-scoped test session to individual methods. A caller-provided session always wins and remains open after the method returns. A standalone operation closes its owned session before returning, so returned objects are detached; load every required scalar, deferred column, and relationship explicitly before the scope exits, and do not mutate those objects expecting persistence.
|
||||
|
||||
If a read participates in a later write, pass the same session and place both operations inside the explicit transaction. This avoids splitting one use case across sessions and keeps SQLAlchemy's autobegin behavior from obscuring transaction ownership. Add a repository only when its naming, shared query policy, dependency substitution, or domain boundary improves the application. Independent functions remain a valid and often clearer design.
|
||||
|
||||
---
|
||||
|
||||
## Transaction Ownership
|
||||
|
||||
Compose multiple calls under one use-case transaction. At this boundary, a supplied session joins its already-active caller-owned transaction, while omitting the session creates a standalone session and transaction. Each nested CRUD write receives `active_session`, detects that transaction, and borrows it instead of committing independently.
|
||||
|
||||
The scope names describe exactly what they own: `session_scope()` manages session lifetime but never commits, while `transaction_scope()` manages a complete transaction only when it also creates the session. Both yield the name `active_session` because downstream CRUD code does not need to know whether the session was borrowed or owned.
|
||||
|
||||
```python
|
||||
from .session import transaction_scope
|
||||
|
||||
|
||||
async def replace_widget(
|
||||
repository: WidgetRepository,
|
||||
widget_id: int,
|
||||
replacement_name: str,
|
||||
replacement_description: str | None = None,
|
||||
*,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget | None:
|
||||
async with transaction_scope(
|
||||
database_url=repository.database_url,
|
||||
session=session,
|
||||
) as active_session:
|
||||
deleted_widget = await repository.delete(
|
||||
widget_id,
|
||||
session=active_session,
|
||||
)
|
||||
if deleted_widget is None:
|
||||
return None
|
||||
|
||||
return await repository.create(
|
||||
replacement_name,
|
||||
replacement_description,
|
||||
session=active_session,
|
||||
)
|
||||
```
|
||||
|
||||
If creation fails, deletion rolls back with it. For a caller-owned transaction, wrap the call in `async with session.begin():` and pass that session. For a standalone use case, omit the session; the outer `transaction_scope()` commits on successful exit, rolls back on exception, and closes its owned session. Do not add direct `commit()` calls to CRUD functions or repository methods because that prevents callers from composing several operations atomically. See [transaction boundaries](transactions.md) and [session management](session.md) for ownership details.
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- Storing one mutable `AsyncSession` on a long-lived repository object.
|
||||
- Constructing ad hoc factories or sessions instead of resolving the cached factory through the scope helpers.
|
||||
- Using `session_scope()` for an optional write, which would close an owned session without committing.
|
||||
- Accepting a supplied session for a write without requiring an active caller-owned transaction.
|
||||
- Calling `commit()` or `rollback()` directly instead of expressing ownership through `transaction_scope()`.
|
||||
- Accepting unbounded list queries.
|
||||
- Accepting detached ORM instances for update or delete when an identifier can be resolved in the active session.
|
||||
- Accessing unloaded attributes after a standalone repository read has closed its owned session.
|
||||
|
||||
---
|
||||
|
||||
## Operational Checks
|
||||
|
||||
- Every CRUD call receives a task-local `AsyncSession`.
|
||||
- Standalone reads resolve the cached factory by database URL and close their owned session.
|
||||
- Standalone writes resolve the cached factory and own commit, rollback, and session cleanup through `transaction_scope()`.
|
||||
- Supplied write sessions already have an active caller-owned transaction.
|
||||
- Each complete operation, service, or use-case boundary borrows an active transaction or owns a complete session-and-transaction scope.
|
||||
- List operations have pagination and deterministic ordering where required.
|
||||
- Update requires values for both mutable fields; passing `None` explicitly clears the nullable description.
|
||||
- Get, update, and delete use the same identifier and missing-row semantics.
|
||||
- Functions and repository methods use domain arguments first and keyword-only infrastructure arguments consistently.
|
||||
- Standalone reads load all state needed after their owned session closes.
|
||||
- Repository objects hold configuration or policy, never request-scoped session state.
|
||||
|
||||
---
|
||||
|
||||
## Testing Checks
|
||||
|
||||
- Create tests verify generated identifiers and persisted field values after commit.
|
||||
- Get and list tests cover found, missing, pagination, and ordering behavior.
|
||||
- List tests reject negative offsets and limits outside the supported range.
|
||||
- Update tests cover replacement of both mutable fields, including clearing the nullable description.
|
||||
- Update and delete tests cover missing identifiers without mutating the database.
|
||||
- Delete tests verify the returned row and its absence after commit.
|
||||
- Failure tests verify that a surrounding transaction rolls back all composed CRUD calls.
|
||||
- Optional-session read tests verify borrowed sessions remain open and owned sessions close without committing.
|
||||
- Optional-session write tests verify supplied transactions remain caller-owned and standalone transactions commit or roll back before closing.
|
||||
- Composition tests pass one active session through several CRUD calls and verify one atomic commit or rollback.
|
||||
@@ -1,171 +0,0 @@
|
||||
# Async SQLAlchemy Engine
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||
- [SQLAlchemy connections](https://docs.sqlalchemy.org/en/21/core/connections.html)
|
||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||
- [SQLAlchemy pooling and multiprocessing](https://docs.sqlalchemy.org/en/21/core/pooling.html#pooling-multiprocessing)
|
||||
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
||||
|
||||
---
|
||||
|
||||
## Engine Ownership Model
|
||||
|
||||
Create one async engine per process per database URL and keep engine construction independent from FastAPI.
|
||||
|
||||
- SQLAlchemy guidance: the engine is intended as a long-lived, concurrent registry over pooled DB connections, not a per-request object.
|
||||
- A cached function provides stable process-local engine identity without making framework state the only way to obtain it.
|
||||
- FastAPI lifespan starts and stops that independently defined resource; it does not contain the construction policy.
|
||||
|
||||
!!! tip "Practical rule"
|
||||
- Exactly one `create_async_engine(...)` call in the cached engine factory.
|
||||
- Zero `create_async_engine(...)` calls in request handlers.
|
||||
- Zero calls to the cached factory from repository code.
|
||||
|
||||
---
|
||||
|
||||
## Cached Engine Factory
|
||||
|
||||
Use [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) on a synchronous factory. Creating an `AsyncEngine` configures the dialect and pool; it does not need to await a database connection.
|
||||
|
||||
```python
|
||||
from functools import cache
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
|
||||
|
||||
|
||||
@cache
|
||||
def get_engine(database_url: str) -> AsyncEngine:
|
||||
return create_async_engine(
|
||||
database_url,
|
||||
pool_pre_ping=True,
|
||||
)
|
||||
|
||||
|
||||
async def dispose_engine(database_url: str) -> None:
|
||||
engine = get_engine(database_url)
|
||||
try:
|
||||
await engine.dispose()
|
||||
finally:
|
||||
get_engine.cache_clear()
|
||||
|
||||
|
||||
async def refresh_engine(database_url: str) -> AsyncEngine:
|
||||
await dispose_engine(database_url)
|
||||
return get_engine(database_url)
|
||||
```
|
||||
|
||||
The database URL is an explicit, hashable cache key. Calls with the same URL return the same engine; a different URL receives a different engine. If engine options vary at runtime, make them explicit hashable arguments too.
|
||||
|
||||
Resolve settings at the composition boundary and call `get_engine(settings.database_url)`. Do not hide settings lookup or engine creation inside feature code.
|
||||
|
||||
## Thin FastAPI Lifespan Wrapper
|
||||
|
||||
The lifespan context manager only connects the cached resource to FastAPI ownership:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||
database_url = app.state.settings.database_url
|
||||
engine = get_engine(database_url)
|
||||
app.state.engine = engine
|
||||
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
await dispose_engine(database_url)
|
||||
|
||||
|
||||
app = FastAPI(lifespan=lifespan)
|
||||
```
|
||||
|
||||
`dispose()` closes checked-in connections and replaces the pool, but it does not remove the Python object from `functools.cache`. `dispose_engine()` clears the cache even if driver cleanup raises, preventing a later lifespan run or test from retrieving that engine instance.
|
||||
|
||||
This simple cleanup assumes one configured database URL per process. If a process intentionally owns several cached engines, use a small registry with per-key removal instead of clearing the whole cache. For a fixed engine, `try/finally` is sufficient; use `AsyncExitStack` when lifespan composes multiple conditional or dynamically acquired resources.
|
||||
|
||||
When directly testing engine construction or lifespan behavior:
|
||||
|
||||
- Call `get_engine.cache_clear()` before the test to remove process-local state.
|
||||
- Dispose any engine the test creates.
|
||||
- Clear the cache again during teardown, even when the test fails.
|
||||
|
||||
---
|
||||
|
||||
## Driver URLs (Project Requirement: asyncpg + aiosqlite)
|
||||
|
||||
Use SQLAlchemy async driver URLs:
|
||||
|
||||
- PostgreSQL: `postgresql+asyncpg://user:pass@host:5432/dbname`
|
||||
- SQLite: `sqlite+aiosqlite:///./app.db`
|
||||
|
||||
!!! warning "Driver compatibility"
|
||||
- Do not mix sync drivers, for example `psycopg2`, with `create_async_engine()`.
|
||||
- Keep URL construction centralized in settings/config, not in feature modules.
|
||||
|
||||
---
|
||||
|
||||
## Pooling Defaults and Tuning
|
||||
|
||||
Default behavior is usually correct first:
|
||||
|
||||
- Async engines use async-compatible pooling (`AsyncAdaptedQueuePool`) by default.
|
||||
- Start with defaults, then tune from observed load (`pool_size`, `max_overflow`, `pool_timeout`, `pool_recycle`).
|
||||
- Enable `pool_pre_ping=True` for safer stale-connection handling in long-running services.
|
||||
|
||||
When to switch pool strategy:
|
||||
|
||||
- `NullPool` if you explicitly need no pooling (special environments, some tests, or strict cross-loop constraints).
|
||||
- Keep in mind this increases connect/disconnect churn.
|
||||
|
||||
---
|
||||
|
||||
## Disposal Semantics
|
||||
|
||||
`engine.dispose()` replaces/disposes the pool, but only checked-in connections are immediately closed.
|
||||
|
||||
Rules:
|
||||
- Dispose when the app is shutting down.
|
||||
- Dispose before reusing an engine across event loops.
|
||||
- In forked child-process initialization, use `engine.dispose(close=False)` (sync API guidance) so child processes do not touch parent-held connections.
|
||||
|
||||
Avoid relying on garbage collection for engine cleanup in async code.
|
||||
|
||||
---
|
||||
|
||||
## Event Loop and Process Boundaries
|
||||
|
||||
Do not share pooled connections across boundaries:
|
||||
|
||||
- Multiple event loops: do not reuse the same pooled async engine across loops unless you intentionally disable pooling (`NullPool`) or dispose before handoff.
|
||||
- Multiprocessing/fork: pooled connections must not be inherited for active use across process boundaries.
|
||||
|
||||
This prevents broken socket state and cross-process connection corruption.
|
||||
|
||||
---
|
||||
|
||||
## What Not to Do
|
||||
|
||||
- Create an engine inside every request dependency.
|
||||
- Create/dispose engines inside repository methods.
|
||||
- Call `get_engine()` from repositories instead of injecting their engine or session dependency.
|
||||
- Keep engine creation as a hidden side effect of import-time module globals.
|
||||
- Dispose a cached engine without clearing the cache during final teardown.
|
||||
- Use deprecated FastAPI startup/shutdown events together with lifespan.
|
||||
|
||||
---
|
||||
|
||||
## Engine Design Checklist
|
||||
|
||||
- One engine per process per DB URL.
|
||||
- Engine created by one cached, framework-independent factory.
|
||||
- Lifespan only retrieves, exposes, disposes, and uncaches the engine.
|
||||
- Async driver URL matches backend (`asyncpg` or `aiosqlite`).
|
||||
- Pooling strategy is explicit for non-default needs.
|
||||
- No request-path engine creation.
|
||||
- Tests dispose engines and clear cached state deterministically.
|
||||
@@ -1,383 +0,0 @@
|
||||
# Async SQLAlchemy Session Management
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
|
||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||
- [SQLAlchemy session basics](https://docs.sqlalchemy.org/en/21/orm/session_basics.html)
|
||||
- [FastAPI dependencies with yield](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Define one canonical session model for FastAPI + SQLAlchemy asyncio:
|
||||
|
||||
- configure one shared session factory,
|
||||
- create one AsyncSession per request or per unit-of-work,
|
||||
- never share one AsyncSession across concurrent tasks.
|
||||
|
||||
---
|
||||
|
||||
## Scope and Non-Goals
|
||||
|
||||
- In scope: session factory creation, FastAPI dependency wiring, request/task scoping, transaction demarcation.
|
||||
- Out of scope: ORM model design, query optimization strategy, schema migration tooling.
|
||||
|
||||
---
|
||||
|
||||
## Rules
|
||||
|
||||
- Create one cached `async_sessionmaker` per app-owned AsyncEngine.
|
||||
- Let repositories resolve the cached maker by database URL.
|
||||
- Use a fresh AsyncSession for each request or explicit unit-of-work.
|
||||
- Pass an `AsyncSession` directly to data-access functions.
|
||||
- Borrow a caller-provided session without closing or committing it.
|
||||
- Do not share AsyncSession across `asyncio.gather()` or parallel tasks.
|
||||
- Prefer direct dependency injection over global scoped-session patterns in new code.
|
||||
- Use explicit transaction boundaries (`async with session.begin():`) for writes.
|
||||
- When a use case accepts an optional session, borrow only an active caller-owned transaction or own the complete session-and-transaction scope.
|
||||
|
||||
---
|
||||
|
||||
## Sessions and Transactions
|
||||
|
||||
A session and a transaction solve related but different problems:
|
||||
|
||||
| Concept | Responsibility | Typical lifetime |
|
||||
| --- | --- | --- |
|
||||
| `AsyncSession` | Provides the ORM workspace: executes queries, tracks loaded and changed objects in its identity map, and flushes pending changes. It also coordinates access to a database connection. | One request, task, or explicit unit of work. |
|
||||
| Transaction | Defines the atomic database boundary: all work inside it commits together on success or rolls back together on failure. | One complete operation that must have a single outcome. |
|
||||
|
||||
A transaction belongs to a session; it is not an alternative to one. The session is the interface used by application and data-access code, while the transaction determines when that work becomes permanent. A session may coordinate sequential transactions during its lifetime, although short-lived application scopes commonly use one session for one transaction.
|
||||
|
||||
Use a session without a helper-owned commit boundary for independent reads or lower-level functions that must participate in whatever transaction their caller controls:
|
||||
|
||||
```python
|
||||
async with session_factory() as session:
|
||||
item = await find_item(session, item_id)
|
||||
```
|
||||
|
||||
Use an explicit transaction for writes, read-modify-write operations, or several statements that must succeed or fail as one unit:
|
||||
|
||||
```python
|
||||
async with session_factory.begin() as session:
|
||||
order = await create_order(session, order_data)
|
||||
await reserve_inventory(session, order)
|
||||
```
|
||||
|
||||
SQLAlchemy sessions use [autobegin](https://docs.sqlalchemy.org/en/21/orm/session_basics.html#auto-begin), so the first database operation normally starts a transaction even for a read. Therefore, “session-only” means that the surrounding helper owns only session lifetime and does not promise to commit; it does not mean that no database transaction exists. Closing such a session releases its resources and rolls back any unfinished transaction. An explicit `begin()` is valuable when application code must make the atomic boundary and commit ownership visible.
|
||||
|
||||
For most read-only operations, a session context is sufficient. Use an explicit transaction for reads when they need a defined consistency boundary, participate in a larger atomic operation, or use locking such as `SELECT ... FOR UPDATE`.
|
||||
|
||||
---
|
||||
|
||||
## Session Factory Mechanics
|
||||
|
||||
An `async_sessionmaker[AsyncSession]` is a reusable configuration object and callable session producer. It stores how sessions should be created, including the engine binding and options such as `expire_on_commit=False`. It is not itself a session, connection, or transaction, and calling it does not make a shared global `AsyncSession`.
|
||||
|
||||
Cache it by the application-owned engine so repeated composition calls return the same maker:
|
||||
|
||||
```python
|
||||
from functools import cache
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
||||
|
||||
from .engine import dispose_engine
|
||||
from .engine import get_engine
|
||||
|
||||
|
||||
@cache
|
||||
def get_session_factory(database_url: str) -> async_sessionmaker[AsyncSession]:
|
||||
return async_sessionmaker(
|
||||
bind=get_engine(database_url),
|
||||
class_=AsyncSession,
|
||||
expire_on_commit=False,
|
||||
)
|
||||
|
||||
|
||||
async def dispose_session_factory(database_url: str) -> None:
|
||||
get_session_factory.cache_clear()
|
||||
await dispose_engine(database_url)
|
||||
```
|
||||
|
||||
`functools.cache` caches by argument equality and requires hashable arguments. The database URL is an explicit string key shared with the cached engine factory. The cache retains the returned maker until `get_session_factory.cache_clear()` runs. Cache the synchronous maker function, never an async function and never a produced `AsyncSession`.
|
||||
|
||||
Each call to `session_factory()` creates a distinct `AsyncSession`. The caller that invokes the factory owns that session lifetime and must close it, normally with `async with`:
|
||||
|
||||
```python
|
||||
async with session_factory() as session:
|
||||
...
|
||||
```
|
||||
|
||||
The factory can be shared across requests and tasks. Sessions produced by it cannot be shared across concurrent tasks.
|
||||
|
||||
An `async_sessionmaker` has no connection pool or async `dispose()` method of its own. `dispose_session_factory()` means "invalidate the cached maker, then dispose its engine." Clearing the maker first ensures no subsequent composition call can retrieve a maker bound to the engine being shut down.
|
||||
|
||||
Use the helper when shutting down or replacing the database resources:
|
||||
|
||||
```python
|
||||
await dispose_session_factory(database_url)
|
||||
```
|
||||
|
||||
Otherwise, a later call can return a maker that still references the old engine object. This matters in lifespan tests, application restarts within one process, and test suites that replace engines.
|
||||
|
||||
---
|
||||
|
||||
## Optional Session Ownership
|
||||
|
||||
A small [`asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager) can make repository methods composable. It borrows an existing session when supplied; otherwise it creates and closes one from a supplied factory:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def session_scope(
|
||||
*,
|
||||
database_url: str,
|
||||
session: AsyncSession | None = None,
|
||||
) -> AsyncIterator[AsyncSession]:
|
||||
if session is not None:
|
||||
yield session
|
||||
return
|
||||
|
||||
async with get_session_factory(database_url)() as owned_session:
|
||||
yield owned_session
|
||||
```
|
||||
|
||||
The branch is intentionally explicit. Python's [`nullcontext`](https://docs.python.org/3/library/contextlib.html#contextlib.nullcontext) can express the same borrow-or-own idea, but the branch keeps ownership and typing obvious.
|
||||
|
||||
This helper manages session lifetime only:
|
||||
|
||||
- It does not close, commit, or roll back a supplied session; the caller owns it.
|
||||
- It closes a session that it creates. Closing releases resources and rolls back an unfinished transaction; it does not commit.
|
||||
- It does not start a transaction. Put `session.begin()` at the use-case boundary.
|
||||
- A supplied session wins; the cached factory is not resolved.
|
||||
- Otherwise, `database_url` selects the cached factory returned by `get_session_factory()`.
|
||||
|
||||
Do not turn this into an implicit unit-of-work helper that sometimes commits. Whether work joins an existing transaction or creates a new one must remain visible to the caller.
|
||||
|
||||
---
|
||||
|
||||
## Optional Transaction Ownership
|
||||
|
||||
Use a separate context manager when a service or use-case function must support both a caller-owned transaction and a standalone transaction. A supplied session must already be inside a transaction; otherwise the helper creates a session and transaction together with `async_sessionmaker.begin()`:
|
||||
|
||||
```python
|
||||
@asynccontextmanager
|
||||
async def transaction_scope(
|
||||
*,
|
||||
database_url: str,
|
||||
session: AsyncSession | None = None,
|
||||
) -> AsyncIterator[AsyncSession]:
|
||||
if session is not None:
|
||||
if not session.in_transaction():
|
||||
raise RuntimeError("A supplied session must have an active transaction")
|
||||
yield session
|
||||
return
|
||||
|
||||
session_factory = get_session_factory(database_url)
|
||||
async with session_factory.begin() as owned_session:
|
||||
yield owned_session
|
||||
```
|
||||
|
||||
Here, `begin()` is intentionally called on the [`async_sessionmaker`](https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin), not on an existing `AsyncSession`. The related APIs have different ownership semantics:
|
||||
|
||||
- `session_factory()` creates a session whose lifetime the surrounding code must manage; it does not commit automatically.
|
||||
- `session_factory.begin()` creates a new session and transaction together, commits on successful exit or rolls back on exceptional exit, and then closes the session.
|
||||
- `session.begin()` manages a transaction on an existing session but does not own or close that session.
|
||||
|
||||
The factory form is equivalent in ownership terms to creating a session and then entering that session's transaction:
|
||||
|
||||
```python
|
||||
async with session_factory() as owned_session:
|
||||
async with owned_session.begin():
|
||||
yield owned_session
|
||||
```
|
||||
|
||||
This helper makes transaction ownership follow the same explicit borrow-or-own mechanics as session ownership:
|
||||
|
||||
- A supplied session and its active transaction remain caller-owned. The helper does not commit, roll back, or close them.
|
||||
- Without a supplied session, the helper owns the session and transaction. Successful exit commits; exceptional exit rolls back; either path closes the session.
|
||||
- Use this helper only at a complete operation, service, or use-case boundary. A public CRUD function or repository method may be such a boundary when its optional-session contract explicitly states that omitting the session owns and commits one transaction. Never use it inside a lower-level session-required helper.
|
||||
- Do not silently begin a transaction on a supplied session. That would make commit ownership depend on hidden helper behavior.
|
||||
|
||||
Callers that supply a session make their ownership visible with an outer transaction:
|
||||
|
||||
```python
|
||||
async with session_factory() as session:
|
||||
async with session.begin():
|
||||
await run_use_case(..., session=session)
|
||||
```
|
||||
|
||||
Standalone callers omit the session and let the use case own the complete unit of work:
|
||||
|
||||
```python
|
||||
await run_use_case(...)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Repository and Function Boundaries
|
||||
|
||||
Pass the database URL to repository constructors. The repository stores repeatable database configuration, not mutable session state, and `session_scope()` resolves the cached factory when a standalone operation needs a session:
|
||||
|
||||
```python
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
|
||||
async def find_item(session: AsyncSession, item_id: int) -> Item | None:
|
||||
statement = select(Item).where(Item.id == item_id)
|
||||
return await session.scalar(statement)
|
||||
|
||||
|
||||
class ItemRepository:
|
||||
def __init__(self, database_url: str) -> None:
|
||||
self.database_url = database_url
|
||||
|
||||
async def find(
|
||||
self,
|
||||
item_id: int,
|
||||
*,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Item | None:
|
||||
async with session_scope(
|
||||
database_url=self.database_url,
|
||||
session=session,
|
||||
) as active_session:
|
||||
return await find_item(active_session, item_id)
|
||||
```
|
||||
|
||||
This split gives each layer one job:
|
||||
|
||||
- The repository object identifies its database configuration and creates a session only for a standalone call.
|
||||
- Standalone calls reuse the cached factory selected by database URL.
|
||||
- A caller can pass a session to join an existing unit of work; the repository borrows it.
|
||||
- The access function owns only the query and requires an existing `AsyncSession`.
|
||||
- Application wiring supplies the production factory.
|
||||
- Tests can use a test database URL or call `find_item()` with a transaction-scoped test session.
|
||||
|
||||
When several repository operations must share one transaction, pass the same session through each call. Put the transaction at the use-case boundary:
|
||||
|
||||
```python
|
||||
async with session_factory() as session:
|
||||
async with session.begin():
|
||||
item = await repository.find(item_id, session=session)
|
||||
await update_item(session, item, changes)
|
||||
```
|
||||
|
||||
This preserves atomicity without making repository objects hold mutable `AsyncSession` instances across calls.
|
||||
|
||||
---
|
||||
|
||||
## Canonical FastAPI Dependency Pattern
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
from fastapi import Depends
|
||||
from fastapi import Request
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
|
||||
|
||||
type SessionFactory = async_sessionmaker[AsyncSession]
|
||||
|
||||
|
||||
def resolve_session_factory(request: Request) -> SessionFactory:
|
||||
return get_session_factory(request.app.state.settings.database_url)
|
||||
|
||||
|
||||
async def get_db_session(
|
||||
session_factory: SessionFactory = Depends(resolve_session_factory),
|
||||
) -> AsyncIterator[AsyncSession]:
|
||||
async with session_factory() as session:
|
||||
yield session
|
||||
```
|
||||
|
||||
Route usage:
|
||||
|
||||
```python
|
||||
from fastapi import APIRouter, Depends
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from .session import get_db_session
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.post("/items")
|
||||
async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
|
||||
async with session.begin():
|
||||
# write operations here
|
||||
...
|
||||
return {"status": "ok"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration Guidance
|
||||
|
||||
- `expire_on_commit=False` is commonly preferred in asyncio applications to reduce accidental post-commit reload behavior.
|
||||
- `AsyncSession.refresh()` is preferred over broad expiration patterns when state refresh is needed.
|
||||
- [`async_sessionmaker.begin()`](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin) is a concise option when one scope must create a session, begin a transaction, commit on success, roll back on failure, and close. Do not use it when borrowing a caller's session.
|
||||
|
||||
## SQLModel Alignment
|
||||
|
||||
- Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per request/unit-of-work.
|
||||
- SQLModel does not replace SQLAlchemy async lifecycle primitives; it provides model declaration, validation, and typing ergonomics on top of them.
|
||||
- Do not mix ad hoc session construction with the canonical async dependency.
|
||||
|
||||
---
|
||||
|
||||
## Concurrency Rules
|
||||
|
||||
- One session per concurrent task.
|
||||
- If work fans out into parallel tasks, each task receives its own AsyncSession.
|
||||
- Pass sessions explicitly to service functions; avoid mutable global session state.
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- A singleton/global AsyncSession reused across requests.
|
||||
- Sharing one AsyncSession across parallel tasks.
|
||||
- Passing an application-global AsyncSession to a repository constructor.
|
||||
- Caching an `AsyncSession` instead of caching `async_sessionmaker`.
|
||||
- Leaving a cached maker pointing at a disposed or replaced engine.
|
||||
- Calling the session factory inside low-level access functions such as `find_item()`.
|
||||
- Hidden session creation in lower access functions with no caller control.
|
||||
- Closing or committing a session supplied by the caller.
|
||||
- Starting a new transaction inside a helper that may receive a session already in a transaction.
|
||||
- Silently starting or committing a transaction on a supplied session.
|
||||
- Mixing commit/rollback ownership across layers without a declared boundary.
|
||||
|
||||
---
|
||||
|
||||
## Operational Checks
|
||||
|
||||
- Exactly one cached `async_sessionmaker` exists per application engine.
|
||||
- Session factory caches are cleared before their engines are disposed or replaced.
|
||||
- Request handlers receive sessions from one canonical dependency.
|
||||
- No code path creates AsyncSession in module import side effects.
|
||||
- Background jobs and API handlers each create task-local sessions.
|
||||
|
||||
---
|
||||
|
||||
## Testing Checks
|
||||
|
||||
- Repository constructors accept a test database URL without FastAPI startup.
|
||||
- Session-taking access functions accept a transaction-scoped test session directly.
|
||||
- Optional-session tests verify that borrowed sessions remain open and created sessions close.
|
||||
- Optional-session tests verify that neither path commits implicitly.
|
||||
- Optional-transaction tests verify supplied sessions require an active transaction and remain caller-owned.
|
||||
- Optional-transaction tests verify owned transactions commit on success, roll back on failure, and close their sessions.
|
||||
- Cache tests clear `get_session_factory` before and after replacing engines.
|
||||
- Dependency override exists for the FastAPI session factory.
|
||||
- Rollback behavior is verified for failed write units.
|
||||
- Parallel-task tests verify no shared AsyncSession instances.
|
||||
- Lifespan tests confirm session factory is initialized and teardown-safe.
|
||||
|
||||
@@ -1,289 +0,0 @@
|
||||
# NiceGUI Page Layout And Styling
|
||||
|
||||
Use this reference to structure NiceGUI pages, choose component boundaries, apply responsive layout, and introduce custom CSS without fighting Quasar's internal geometry.
|
||||
|
||||
## Ownership And Dependency Boundaries
|
||||
|
||||
Keep dependencies flowing in one direction:
|
||||
|
||||
- pages import components and services
|
||||
- components contain presentation logic only
|
||||
- services contain business logic and do not import UI
|
||||
- bootstrap code mounts static assets and loads shared CSS once
|
||||
|
||||
Suggested module split:
|
||||
|
||||
```text
|
||||
src/my_app/
|
||||
ui/
|
||||
pages/
|
||||
components/
|
||||
static/
|
||||
services/
|
||||
api/
|
||||
```
|
||||
|
||||
Page modules should compose a route from reusable presentation and service calls. They should not own domain rules, persistence, or long-running synchronous work.
|
||||
|
||||
## Page Composition
|
||||
|
||||
Build the outer layout before styling individual controls:
|
||||
|
||||
1. Define the page shell and width constraints.
|
||||
2. Establish responsive rows, columns, gaps, and wrapping.
|
||||
3. Add semantic sections and repeated components.
|
||||
4. Configure Quasar component appearance with props.
|
||||
5. Add custom CSS only for behavior that props and utilities cannot express safely.
|
||||
|
||||
```python
|
||||
with ui.column().classes("w-full max-w-6xl mx-auto gap-6 px-4"):
|
||||
page_header(title="Inventory")
|
||||
|
||||
with ui.row().classes("w-full gap-4 flex-wrap lg:flex-nowrap items-start"):
|
||||
filters_panel().classes("w-full lg:w-72 shrink-0")
|
||||
item_grid().classes("w-full flex-1 min-w-0")
|
||||
```
|
||||
|
||||
Use stable width, minimum-width, and flex constraints so labels, icons, validation messages, and loaded content do not shift the surrounding layout.
|
||||
|
||||
## Component Extraction
|
||||
|
||||
Extract a presentation pattern to `ui/components/` when it appears on two or more pages or when it owns a meaningful interaction boundary. Keep one-off route layout in the page module.
|
||||
|
||||
```python
|
||||
def card_section(title: str, content: str) -> ui.card:
|
||||
with ui.card().classes("w-full max-w-md") as card:
|
||||
ui.label(title).classes("text-lg font-bold")
|
||||
ui.label(content).classes("text-gray-600")
|
||||
return card
|
||||
```
|
||||
|
||||
Reusable components should accept data and event callbacks rather than import page state or business services implicitly.
|
||||
|
||||
## Styling Decision Order
|
||||
|
||||
NiceGUI wraps Quasar components. Choose the styling mechanism according to what it owns:
|
||||
|
||||
1. Use Quasar props for component appearance, density, labels, and popup behavior.
|
||||
2. Use NiceGUI `.classes()` and Tailwind utilities for width, spacing, alignment, and responsive layout.
|
||||
3. Use reusable component functions for repeated visual patterns.
|
||||
4. Use `.style()` for genuinely dynamic inline values.
|
||||
5. Use minimal shared CSS only when props and utilities are insufficient.
|
||||
|
||||
Common Quasar props include:
|
||||
|
||||
- `outlined`
|
||||
- `dense`
|
||||
- `stack-label`
|
||||
- `popup-content-class`
|
||||
- `input-class`
|
||||
- `input-style`
|
||||
|
||||
Avoid overriding internal selectors such as:
|
||||
|
||||
- `.q-field__label`
|
||||
- `.q-field__native`
|
||||
- `.q-field__control`
|
||||
- `.q-field__input`
|
||||
|
||||
Quasar coordinates field height, padding, labels, values, icons, and floating-label transforms. Changing only one internal part tends to cause clipping or overlap.
|
||||
|
||||
## Responsive Layout
|
||||
|
||||
Support these layouts only:
|
||||
|
||||
- mobile: a single-column layout with wrapping toolbars and full-width controls
|
||||
- landscape desktop: $1920 \times 1080$ with side-by-side panels where they improve scanning
|
||||
- portrait desktop: $1080 \times 1920 with stacked panels or a narrow fixed sidebar
|
||||
|
||||
Build the mobile layout first, then add one desktop breakpoint when a row or grid needs more space. Prefer flex wrapping and fluid grids before adding another breakpoint. Use Tailwind classes for page layout and Quasar props for component behavior.
|
||||
|
||||
```python
|
||||
with ui.row().classes('w-full flex-wrap gap-4 lg:flex-nowrap items-start'):
|
||||
filters_panel().classes('w-full lg:w-72 shrink-0')
|
||||
item_grid().classes('w-full flex-1 min-w-0')
|
||||
```
|
||||
|
||||
Use `min-w-0` for flexible children, `flex-wrap` for toolbars, and `max-w-* mx-auto` to keep portrait layouts readable. Do not add device-specific component trees, container queries, or custom breakpoints unless a supported layout demonstrates a concrete failure.
|
||||
|
||||
## Static Assets And Shared CSS
|
||||
|
||||
- Mount static assets from the composition layer.
|
||||
- Load shared CSS once rather than injecting it from individual pages.
|
||||
- Keep custom CSS tokenized with variables and scoped to application classes.
|
||||
- Avoid broad rules against Quasar internals.
|
||||
- Verify mount paths, reverse-proxy rewrites, and cache behavior.
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
|
||||
STATIC_DIR = Path(__file__).parent / "ui" / "static"
|
||||
|
||||
app.mount("/static", StaticFiles(directory=STATIC_DIR), name="static")
|
||||
ui.add_css((STATIC_DIR / "css" / "base.css").read_text(encoding="utf-8"))
|
||||
```
|
||||
|
||||
## Responsive Dialog Pattern
|
||||
|
||||
Use whole-card scaling when a form dialog must become uniformly larger on mobile while preserving Quasar's internal proportions. Keep detached select menus unscaled and make the card itself scrollable.
|
||||
|
||||
### Use Normal Field Density
|
||||
|
||||
Normal Quasar fields are approximately `56px` high, while dense fields are approximately `40px` high. Remove `dense` when larger controls are needed.
|
||||
|
||||
```python
|
||||
ui.input("Name").props("outlined")
|
||||
ui.number("Quantity").props("outlined")
|
||||
ui.select(...).props(
|
||||
"outlined popup-content-class=app-item-detail-menu"
|
||||
)
|
||||
ui.textarea("Description").props("outlined autogrow")
|
||||
```
|
||||
|
||||
Add a scoped class to the dialog card:
|
||||
|
||||
```python
|
||||
ui.card().classes("app-detail-card app-item-detail-card")
|
||||
```
|
||||
|
||||
### Scale The Complete Card
|
||||
|
||||
```css
|
||||
:root {
|
||||
--item-dialog-scale: 1;
|
||||
--item-dialog-max-height: calc(100dvh - 3rem);
|
||||
}
|
||||
|
||||
.app-item-detail-card {
|
||||
width: min(50rem, 50vw);
|
||||
max-height: var(--item-dialog-max-height);
|
||||
overflow-y: auto;
|
||||
overscroll-behavior: contain;
|
||||
zoom: var(--item-dialog-scale);
|
||||
}
|
||||
|
||||
/* Restore Quasar's baseline if a global rule overrides it. */
|
||||
.app-item-detail-card .q-field,
|
||||
.app-item-detail-menu {
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
@media (max-width: 599px) {
|
||||
:root {
|
||||
--item-dialog-scale: 1.2;
|
||||
/* 75dvh becomes 90dvh after 1.2x zoom. */
|
||||
--item-dialog-max-height: 75dvh;
|
||||
}
|
||||
|
||||
.app-item-detail-card {
|
||||
width: 80vw;
|
||||
}
|
||||
|
||||
.app-item-detail-menu {
|
||||
font-size: 16.8px;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The main mobile tuning knob is:
|
||||
|
||||
```css
|
||||
--item-dialog-scale: 1.2;
|
||||
```
|
||||
|
||||
### Keep Detached Popups Unscaled
|
||||
|
||||
Do not apply `zoom` or `transform: scale()` to a `QSelect` popup menu. Quasar renders menus outside the dialog and positions them from the unscaled anchor geometry. Scaling the menu container afterward separates it from its field.
|
||||
|
||||
Avoid:
|
||||
|
||||
```css
|
||||
.app-item-detail-card,
|
||||
.app-item-detail-menu {
|
||||
zoom: 1.2;
|
||||
}
|
||||
```
|
||||
|
||||
Use:
|
||||
|
||||
```css
|
||||
.app-item-detail-card {
|
||||
zoom: 1.2;
|
||||
}
|
||||
|
||||
.app-item-detail-menu {
|
||||
font-size: 16.8px;
|
||||
}
|
||||
```
|
||||
|
||||
Use `popup-content-class=app-item-detail-menu` to target the detached menu and enlarge its text without changing its coordinate system.
|
||||
|
||||
### Account For Zoom When Scrolling
|
||||
|
||||
The card's pre-zoom maximum height must account for the scale:
|
||||
|
||||
\[
|
||||
\begin{aligned}
|
||||
h_{\mathrm{pre}} &= \frac{h_{\mathrm{visible}}}{s} \\
|
||||
\text{where } s &= \text{the zoom scale}
|
||||
\end{aligned}
|
||||
\]
|
||||
|
||||
For a desired visual height of `90dvh` at \(1.2\times\):
|
||||
|
||||
\[
|
||||
\frac{90\,\mathrm{dvh}}{1.2} = 75\,\mathrm{dvh}
|
||||
\]
|
||||
|
||||
Therefore:
|
||||
|
||||
```css
|
||||
--item-dialog-max-height: 75dvh;
|
||||
```
|
||||
|
||||
Apply scrolling to the card itself:
|
||||
|
||||
```css
|
||||
.app-item-detail-card {
|
||||
max-height: var(--item-dialog-max-height);
|
||||
overflow-y: auto;
|
||||
overscroll-behavior: contain;
|
||||
}
|
||||
```
|
||||
|
||||
This keeps the dimmed page stationary while the form scrolls.
|
||||
|
||||
### Match The Quasar Breakpoint
|
||||
|
||||
Quasar's extra-small breakpoint ends at `599.98px`. A mobile-only rule can use:
|
||||
|
||||
```css
|
||||
@media (max-width: 599px) {
|
||||
/* Mobile rules. */
|
||||
}
|
||||
```
|
||||
|
||||
Confirm custom breakpoint values against the target application's Quasar configuration.
|
||||
|
||||
## Validation Checklist
|
||||
|
||||
Check each completed page at these three viewports:
|
||||
|
||||
1. A representative mobile viewport, such as $390 \times 844$.
|
||||
2. Landscape desktop at $1920 \times 1080$.
|
||||
3. Portrait desktop at $1080 \times 1920$.
|
||||
|
||||
Confirm that page sections do not overlap, toolbars wrap on mobile, desktop panels use the available space without becoming excessively wide, and dialogs remain visible and scroll to their final field.
|
||||
|
||||
## Sources
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [NiceGUI element styling and props](https://nicegui.io/documentation/element)
|
||||
- [NiceGUI binding properties](https://nicegui.io/documentation/section_binding_properties)
|
||||
- [Quasar components](https://quasar.dev/vue-components)
|
||||
- [Quasar field](https://quasar.dev/vue-components/field/)
|
||||
- [Quasar select](https://quasar.dev/vue-components/select/)
|
||||
- [Tailwind responsive design](https://tailwindcss.com/docs/responsive-design)
|
||||
- [MDN `zoom`](https://developer.mozilla.org/en-US/docs/Web/CSS/zoom)
|
||||
-366
@@ -1,366 +0,0 @@
|
||||
---
|
||||
icon: lucide/workflow
|
||||
---
|
||||
|
||||
# Skill Usage Mechanics
|
||||
|
||||
## Purpose
|
||||
|
||||
This page explains practical usage mechanics for the GitHub Copilot extension in VS Code when `personal-mcp` is configured as an MCP server:
|
||||
|
||||
1. explicit `/` command flows when you want deterministic control
|
||||
2. guided skill loading when relevance can be inferred
|
||||
|
||||
The goal is to show how Copilot behaves as a client and how to shape that behavior.
|
||||
|
||||
## Mental Model
|
||||
|
||||
In Copilot Chat, there are two distinct mechanisms:
|
||||
|
||||
1. `/` commands are user-invoked orchestration shortcuts.
|
||||
2. MCP resources are server-published knowledge units that can be attached as read-only context, while MCP tools provide an execution path for discovery and retrieval.
|
||||
|
||||
In this repository, skill guidance is exposed as MCP resources, not as server-owned prompt execution. Copilot remains the orchestrator.
|
||||
|
||||
Prompt guidance is now exposed through both prompt resources and MCP prompt objects. Prompt objects are additive; authored markdown remains the canonical source.
|
||||
|
||||
## Background Mechanics
|
||||
|
||||
### What the server publishes
|
||||
|
||||
`personal-mcp` registers resources from the validated docs registry and exposes catalog discovery resources:
|
||||
|
||||
1. `resource://catalog/skills_index`
|
||||
2. `resource://catalog/skills_index{?q,tag,capability,cursor,limit}`
|
||||
3. `resource://catalog/skills/{skill_id}`
|
||||
4. `resource://catalog/prompts_index`
|
||||
5. `resource://catalog/prompts_index{?q,tag,cursor,limit}`
|
||||
6. `resource://catalog/prompts/{prompt_id}`
|
||||
|
||||
Each skill publishes a canonical Markdown document resource:
|
||||
|
||||
1. `resource://skills/<skill-id>/document`
|
||||
2. `resource://skills/<skill-id>/references/<ref-id>`
|
||||
|
||||
Prompts publish a canonical prompt document resource:
|
||||
|
||||
1. `resource://prompts/<prompt-id>/document`
|
||||
|
||||
The document payload is loaded from `docs/skills/<skill-id>/SKILL.md` and returned with metadata.
|
||||
|
||||
### What Copilot does as the client
|
||||
|
||||
When connected to MCP, Copilot can do the following at runtime:
|
||||
|
||||
1. interpret the current chat request
|
||||
2. use attached MCP resources that you provide through the chat UI
|
||||
3. invoke MCP tools when the task and tool descriptions make that relevant
|
||||
4. summarize relevant sections into working context
|
||||
5. apply guidance while generating edits or recommendations
|
||||
|
||||
This behavior is shaped by the active chat surface, prompt or instruction guidance, and available MCP tools.
|
||||
|
||||
For reliable progressive discovery, use one of these sequences:
|
||||
|
||||
1. explicit resource path: attach a catalog resource first, then attach only selected skill documents
|
||||
2. tool path: call catalog tools first, then load only selected skill documents
|
||||
|
||||
### What `/` commands do
|
||||
|
||||
`/` commands in VS Code are client-side prompt entry points (for example in prompt files). They do not replace MCP resources. In Copilot, they typically:
|
||||
|
||||
1. enforce a known sequence
|
||||
2. collect missing inputs
|
||||
3. call discovery/read steps in a predictable order
|
||||
|
||||
Think of `/` commands as orchestration shortcuts on top of MCP resources.
|
||||
|
||||
### What automatic loading means here
|
||||
|
||||
In this project, "automatic loading" should be read as a preference you express through instructions and prompts, not as a guaranteed VS Code feature that auto-attaches MCP resources.
|
||||
|
||||
In practice, there are two reliable ways to make skill content available in chat:
|
||||
|
||||
1. explicit resource attachment through `Add Context > MCP Resources` or `MCP: Browse Resources`
|
||||
2. MCP tool invocation using `list_resources`/`read_resource` (ResourcesAsTools), with thin catalog tools as parity fallback
|
||||
|
||||
For prompt content, there is a third option when the client supports MCP prompt APIs:
|
||||
|
||||
1. prompt-object discovery and invocation through MCP prompt lists and `get_prompt`
|
||||
|
||||
Instruction quality and metadata quality still matter, because they influence whether Copilot recognizes that the MCP server is relevant and chooses the tool path well.
|
||||
|
||||
## Invocation Mechanics Deep Dive
|
||||
|
||||
This section expands on how invocation works at runtime across chat entry points.
|
||||
|
||||
### Invocation Surfaces
|
||||
|
||||
A user request can arrive through one of these surfaces:
|
||||
|
||||
1. plain chat request in Ask/Edit/Agent mode
|
||||
2. slash command invocation of a prompt or skill
|
||||
3. chat request with manually attached MCP resources
|
||||
|
||||
Each surface changes how much discovery Copilot must do before applying guidance.
|
||||
|
||||
### Resolution Order
|
||||
|
||||
When multiple retrieval paths are possible, use this priority order:
|
||||
|
||||
1. attached MCP resources already in context
|
||||
2. explicit slash-command workflow steps
|
||||
3. catalog-first discovery via MCP resources
|
||||
4. tool fallback (`list_resources` then `read_resource`, then thin catalog parity tools)
|
||||
|
||||
This ordering keeps behavior predictable while minimizing unnecessary context expansion.
|
||||
|
||||
### Prompt Invocation Pipeline
|
||||
|
||||
For prompt-oriented flows, treat invocation as this sequence:
|
||||
|
||||
1. parse prompt frontmatter and argument hints
|
||||
2. validate required inputs and ask one clarifying question if blocked
|
||||
3. run bounded discovery against prompt or skill catalogs
|
||||
4. fetch only selected document resources
|
||||
5. apply instructions to produce edits, recommendations, or commands
|
||||
6. report what was loaded and why
|
||||
|
||||
Prompt objects and prompt document resources are additive mechanisms. The authored Markdown prompt document remains the canonical contract.
|
||||
|
||||
### Argument Syntax Nuance
|
||||
|
||||
Invocation strings such as target_modules=src/personal_mcp/registry/ingest/skill.py, mode=plan-only are a structured authoring convention, not a guaranteed client-level grammar.
|
||||
|
||||
In practice:
|
||||
|
||||
1. Prompt metadata defines expected argument names and intent.
|
||||
2. Prompt body instructions define how those inputs should be interpreted.
|
||||
3. Copilot may receive equivalent intent in freeform phrasing and still resolve it correctly.
|
||||
|
||||
Implication for authors:
|
||||
|
||||
1. Treat key=value examples as clarity aids for users.
|
||||
2. Do not assume strict parser enforcement unless your prompt explicitly validates and rejects malformed input.
|
||||
3. Include accepted invocation examples and one fallback freeform example so behavior is predictable for both humans and the model.
|
||||
|
||||
This distinction is important because argument hints improve discoverability, while robust prompt instructions determine actual runtime reliability.
|
||||
|
||||
### Skill Invocation Pipeline
|
||||
|
||||
For guided skill loading, use this sequence:
|
||||
|
||||
1. start from `resource://catalog/skills_index` or scoped index query
|
||||
2. inspect one or two top candidates for intent and capability fit
|
||||
3. fetch `resource://skills/<skill-id>/document`
|
||||
4. load references only when the task needs deeper detail
|
||||
5. apply only relevant sections and keep context bounded
|
||||
|
||||
This avoids the common failure mode where many skill documents are loaded up front.
|
||||
|
||||
### Determinism vs Flexibility
|
||||
|
||||
Use this decision rule:
|
||||
|
||||
1. choose slash-command invocation when repeatability and step order are critical
|
||||
2. choose guided loading when requests vary and speed matters more than strict orchestration
|
||||
3. escalate from guided loading to slash-command flow when confidence is low or conflicting skills appear
|
||||
|
||||
### Invocation Trace (What to Log in Results)
|
||||
|
||||
For transparent operation, include a concise invocation trace in task outputs:
|
||||
|
||||
1. entry surface used (plain chat, slash command, or attached resource)
|
||||
2. discovery source used (catalog resource or tool path)
|
||||
3. resources fetched (ids only)
|
||||
4. clarifying questions asked (if any)
|
||||
5. reason for fallback or escalation (if used)
|
||||
|
||||
This makes behavior auditable and easier to tune over time.
|
||||
|
||||
## Operating Pattern
|
||||
|
||||
Use both modes intentionally in Copilot Chat.
|
||||
|
||||
### Mode A: Explicit `/` command
|
||||
|
||||
Use when you need predictable, repeatable behavior across teammates.
|
||||
|
||||
Good fits:
|
||||
|
||||
1. onboarding workflows
|
||||
2. compliance-sensitive tasks
|
||||
3. repetitive scaffolding
|
||||
|
||||
### Mode B: Guided skill loading
|
||||
|
||||
Use when requests are varied and you want lower friction during normal chat.
|
||||
|
||||
Good fits:
|
||||
|
||||
1. ad hoc implementation questions
|
||||
2. mixed-topic debugging
|
||||
3. architecture tradeoff discussions
|
||||
|
||||
### Mode C: Fallback flow
|
||||
|
||||
Start with guided loading in chat; escalate to a `/` command when:
|
||||
|
||||
1. confidence is low
|
||||
2. multiple skills conflict
|
||||
3. the user wants strict repeatability
|
||||
|
||||
## Suggested Resolution Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[User request in Copilot Chat] --> B{Deterministic workflow needed?}
|
||||
B -- Yes --> C[/Run slash command/]
|
||||
C --> D[Copilot fetches known catalog and skill resources]
|
||||
B -- No --> E[Copilot uses attached resources or catalog tools]
|
||||
E --> F{Confident skill match?}
|
||||
F -- Yes --> G[Copilot fetches skill documents]
|
||||
F -- No --> H[Ask clarifying question or suggest slash command]
|
||||
D --> I[Apply guidance to task]
|
||||
G --> I
|
||||
H --> I
|
||||
```
|
||||
|
||||
## Authoring Requirements
|
||||
|
||||
Authoring rules for metadata quality and instruction patterns are maintained in [Authoring Guide](./authoring.md).
|
||||
|
||||
## Practical Guidelines
|
||||
|
||||
1. Keep `/` commands minimal and high-value.
|
||||
2. Do not duplicate full methodology text inside command files.
|
||||
3. Keep canonical guidance in `docs/skills/*/SKILL.md`.
|
||||
4. In Copilot instructions, prefer catalog-first discovery before skill fetch.
|
||||
5. Prefer small, relevant context slices over loading every skill.
|
||||
6. Keep slash commands focused on deterministic orchestration, not content duplication.
|
||||
|
||||
If you skip the catalog/index step, behavior is less predictable and may either miss relevant skills or pull too much context.
|
||||
|
||||
## Optional Tool Search Mode
|
||||
|
||||
When tool catalogs grow, FastMCP search transforms can reduce tool-list noise for tool-only clients.
|
||||
|
||||
Runtime switches:
|
||||
|
||||
1. `PERSONAL_MCP_TOOL_SEARCH=none|regex|bm25` (default `none`)
|
||||
2. `PERSONAL_MCP_TOOL_SEARCH_MAX_RESULTS=<positive int>` (default `5`)
|
||||
|
||||
Behavior:
|
||||
|
||||
1. `regex` uses deterministic regex matching for targeted queries.
|
||||
2. `bm25` uses ranked natural-language matching.
|
||||
3. `list_resources` and `read_resource` stay visible so resource-backed fallback remains primary.
|
||||
|
||||
## Failure Modes and Recovery
|
||||
|
||||
Common failure modes:
|
||||
|
||||
1. No relevant skill selected.
|
||||
2. Too many skills selected (context bloat).
|
||||
3. Stale assumptions from old metadata.
|
||||
4. Slash command bypasses normal discovery and forces the wrong skill.
|
||||
|
||||
Recovery sequence:
|
||||
|
||||
1. re-run catalog lookup
|
||||
2. narrow by tags and intent
|
||||
3. fetch only top candidates
|
||||
4. if still ambiguous, ask one clarifying question
|
||||
5. use explicit `/` workflow for deterministic fallback
|
||||
|
||||
## Checklist
|
||||
|
||||
Use this checklist when configuring GitHub Copilot in VS Code against `personal-mcp`:
|
||||
|
||||
1. confirm server connectivity
|
||||
2. verify catalog resources are readable
|
||||
3. verify at least one `resource://skills/<id>/document` can be fetched
|
||||
4. add one deterministic `/` command for fallback
|
||||
5. confirm your workspace instruction policy exists (see [Authoring Guide](./authoring.md))
|
||||
6. verify context size remains bounded
|
||||
7. validate behavior in Ask/Edit/Agent-style workflows with at least one task each
|
||||
|
||||
## Runtime Discovery Workflow
|
||||
|
||||
Use this runtime sequence in chat sessions:
|
||||
|
||||
1. Start with catalog-first discovery.
|
||||
2. Prefer MCP resources when the chat surface exposes resource attachment.
|
||||
3. Otherwise use tool fallback to load one or two likely skill documents.
|
||||
4. Prefer `list_resources`/`read_resource` first when operating in tool-only clients.
|
||||
5. If confidence is low, ask one clarifying question before loading more.
|
||||
|
||||
## Thin Shim Path Binding Pattern
|
||||
|
||||
For repositories that consume this MCP server, thin shims are a usage pattern for binding path scopes to the right MCP resources. The "thin shims" are just lightweight, repo-specific instructions files that tell Copilot to use certain MCP resources when editing files that match a pattern. That helps with ensuring Copilot uses the intended resources without too much specific goading in the prompt.
|
||||
|
||||
Use thin shims in Copilot instruction files to bind file-path scopes to:
|
||||
|
||||
1. the most relevant docs page for human-readable conventions
|
||||
2. the matching MCP resource URI for machine retrieval
|
||||
|
||||
Keep each shim short: trigger, primary resource, minimal execution pattern, and one fallback rule.
|
||||
|
||||
Recommended binding pattern:
|
||||
|
||||
1. Put shims in `.github/instructions/*.instructions.md`.
|
||||
2. Scope each shim with `applyTo` so it activates only where needed.
|
||||
3. Point to one primary `resource://skills/<skill-id>/document` URI.
|
||||
4. Link one repository docs page as the human-facing companion.
|
||||
5. Expand to references only when the task needs deeper detail.
|
||||
|
||||
Current repository examples:
|
||||
|
||||
| applyTo scope | Primary docs page | Primary MCP resource |
|
||||
| --- | --- | --- |
|
||||
| `**/*.md` | [docs/authoring.md](./authoring.md) | `resource://skills/zensical-docs/document` |
|
||||
| `tests/**` | [docs/testing.md](./testing.md) | `resource://skills/pytesting/document` |
|
||||
| `.vscode/**` | [docs/skills/vscode-configuration/SKILL.md](./skills/vscode-configuration/SKILL.md) | `resource://skills/vscode-configuration/document` |
|
||||
|
||||
Minimal shim shape:
|
||||
|
||||
```md
|
||||
---
|
||||
name: <short scope name>
|
||||
description: Route <path scope> edits to the Personal MCP <skill-id> resource.
|
||||
applyTo: '<glob>'
|
||||
---
|
||||
|
||||
When editing files matching <glob>, use `resource://skills/<skill-id>/document` as the primary guidance source.
|
||||
|
||||
Execution pattern:
|
||||
|
||||
1. Load the primary skill document first.
|
||||
2. Apply only sections relevant to the file being edited.
|
||||
3. Keep edits minimal and aligned with repository conventions.
|
||||
4. If confidence is low, ask one clarifying question before editing.
|
||||
|
||||
Companion docs page: [docs/<page>.md](./<page>.md)
|
||||
```
|
||||
|
||||
When to use thin shims:
|
||||
|
||||
1. Repositories that want thin local policy while keeping canonical guidance in MCP resources.
|
||||
2. Stable, repeated workflows with clear path ownership.
|
||||
3. Cases where teams need predictable retrieval behavior.
|
||||
|
||||
When not to use thin shims:
|
||||
|
||||
1. Broad, ambiguous tasks with unclear ownership boundaries.
|
||||
2. Cases where one shim would need many exceptions.
|
||||
3. Situations better handled by catalog-first discovery at runtime.
|
||||
|
||||
## Summary
|
||||
|
||||
The intended model is:
|
||||
|
||||
1. skills are canonical MCP resources
|
||||
2. `/` commands are explicit Copilot control shortcuts
|
||||
3. guided skill loading should be catalog-driven, bounded, and explicit about whether it is using resources or tools
|
||||
|
||||
Using all three together gives predictable control when needed and low-friction assistance by default in VS Code.
|
||||
+11
-6
@@ -1,19 +1,22 @@
|
||||
[project]
|
||||
name = "prompts"
|
||||
version = "0.1.0"
|
||||
version = "2.0.0"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"fastapi>=0.115.0",
|
||||
"fastmcp>=2.10.0",
|
||||
"pydantic-settings>=2.0.0",
|
||||
"python-json-logger>=4.1.0",
|
||||
"fastapi>=0.133.0",
|
||||
"fastmcp==4.0.0b1",
|
||||
"pydantic-settings>=2",
|
||||
"pyyaml>=6.0.2",
|
||||
"python-json-logger>=4",
|
||||
"uvicorn[standard]>=0.34.0",
|
||||
"zensical>=0.0.45",
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
constraint-dependencies = ["fastmcp-slim==4.0.0b1"]
|
||||
|
||||
[project.scripts]
|
||||
personal-mcp = "personal_mcp.main:main"
|
||||
personal-mcp = "personal_mcp.__main__:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
@@ -30,9 +33,11 @@ dev = [
|
||||
"ty>=0.0.51",
|
||||
]
|
||||
test = [
|
||||
"httpx2>=2.9.1",
|
||||
"pytest>=9.1.1",
|
||||
"pytest-asyncio>=1.4.0",
|
||||
"pytest-cov>=7.1.0",
|
||||
"pyyaml>=6.0.2",
|
||||
]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
import uvicorn
|
||||
|
||||
from .config import get_settings
|
||||
|
||||
|
||||
def main(cli: bool = True) -> None:
|
||||
"""Run the root MCP server."""
|
||||
settings = get_settings(cli=cli)
|
||||
uvicorn.run(
|
||||
"personal_mcp.web.app:create_app",
|
||||
factory=True,
|
||||
host=settings.host,
|
||||
port=settings.port,
|
||||
reload=settings.reload,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,19 +0,0 @@
|
||||
from personal_mcp.catalog.server import build_prompt_detail_payload
|
||||
from personal_mcp.catalog.server import build_prompts_index_payload
|
||||
from personal_mcp.catalog.server import build_skill_detail_payload
|
||||
from personal_mcp.catalog.server import build_skills_index_payload
|
||||
from personal_mcp.catalog.server import get_pattern_by_id_payload
|
||||
from personal_mcp.catalog.server import get_prompt_by_id_payload
|
||||
from personal_mcp.catalog.server import search_patterns_payload
|
||||
from personal_mcp.catalog.server import search_prompts_payload
|
||||
|
||||
__all__ = [
|
||||
"build_prompt_detail_payload",
|
||||
"build_prompts_index_payload",
|
||||
"build_skill_detail_payload",
|
||||
"build_skills_index_payload",
|
||||
"get_pattern_by_id_payload",
|
||||
"get_prompt_by_id_payload",
|
||||
"search_patterns_payload",
|
||||
"search_prompts_payload",
|
||||
]
|
||||
@@ -1,250 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from personal_mcp.registry.models.registry import DocsRegistry
|
||||
from personal_mcp.registry.models.registry import PromptRecord
|
||||
from personal_mcp.registry.models.registry import PromptSummaryPayload
|
||||
from personal_mcp.registry.models.registry import SkillPatternPayload
|
||||
from personal_mcp.registry.models.registry import SkillRecord
|
||||
from personal_mcp.registry.models.registry import SkillSummaryPayload
|
||||
|
||||
DEFAULT_LIMIT = 20
|
||||
MAX_LIMIT = 100
|
||||
|
||||
|
||||
def _skill_matches(
|
||||
skill: SkillRecord,
|
||||
*,
|
||||
query: str | None,
|
||||
tag: str | None,
|
||||
capability: str | None,
|
||||
) -> bool:
|
||||
if query:
|
||||
lowered = query.strip().lower()
|
||||
if lowered:
|
||||
haystack = " ".join(
|
||||
[
|
||||
skill.skill_id,
|
||||
skill.name,
|
||||
skill.description,
|
||||
" ".join(skill.tags),
|
||||
]
|
||||
).lower()
|
||||
terms = [term for term in lowered.replace("-", " ").split() if term]
|
||||
if any(term not in haystack for term in terms):
|
||||
return False
|
||||
|
||||
if tag and tag not in skill.tags:
|
||||
return False
|
||||
|
||||
return not (capability and capability not in skill.capabilities)
|
||||
|
||||
|
||||
def _prompt_matches(
|
||||
prompt: PromptRecord,
|
||||
*,
|
||||
query: str | None,
|
||||
tag: str | None,
|
||||
) -> bool:
|
||||
if query:
|
||||
lowered = query.strip().lower()
|
||||
if lowered:
|
||||
haystack = " ".join(
|
||||
[
|
||||
prompt.prompt_id,
|
||||
prompt.name,
|
||||
prompt.description,
|
||||
" ".join(prompt.tags),
|
||||
" ".join(sorted(prompt.arguments)),
|
||||
]
|
||||
).lower()
|
||||
terms = [term for term in lowered.replace("-", " ").split() if term]
|
||||
if any(term not in haystack for term in terms):
|
||||
return False
|
||||
|
||||
return not (tag and tag not in prompt.tags)
|
||||
|
||||
|
||||
def build_skills_index_payload(
|
||||
registry: DocsRegistry,
|
||||
*,
|
||||
query: str | None = None,
|
||||
tag: str | None = None,
|
||||
capability: str | None = None,
|
||||
cursor: str | None = None,
|
||||
limit: int | None = None,
|
||||
) -> dict[str, Any]:
|
||||
normalized_limit = DEFAULT_LIMIT if limit is None else max(1, min(limit, MAX_LIMIT))
|
||||
try:
|
||||
start = 0 if cursor is None else max(0, int(cursor))
|
||||
except ValueError as exc:
|
||||
raise ValueError("cursor must be an integer string") from exc
|
||||
|
||||
ordered = [registry.skills_by_id[skill_id] for skill_id in registry.skills_in_load_order]
|
||||
matches = [skill for skill in ordered if _skill_matches(skill, query=query, tag=tag, capability=capability)]
|
||||
|
||||
page = matches[start : start + normalized_limit]
|
||||
next_cursor = start + normalized_limit
|
||||
|
||||
return {
|
||||
"skills": [SkillSummaryPayload.from_record(skill).model_dump() for skill in page],
|
||||
"total": len(matches),
|
||||
"cursor": str(start),
|
||||
"limit": normalized_limit,
|
||||
"next_cursor": str(next_cursor) if next_cursor < len(matches) else None,
|
||||
}
|
||||
|
||||
|
||||
def build_skill_detail_payload(registry: DocsRegistry, skill_id: str) -> dict[str, Any]:
|
||||
if skill_id not in registry.skills_by_id:
|
||||
raise KeyError(skill_id)
|
||||
|
||||
skill = registry.skills_by_id[skill_id]
|
||||
return {
|
||||
"id": skill.skill_id,
|
||||
"name": skill.name,
|
||||
"description": skill.description,
|
||||
"version": skill.version,
|
||||
"tags": list(skill.tags),
|
||||
"capabilities": list(skill.capabilities),
|
||||
"resources": {
|
||||
"document": skill.document_uri,
|
||||
"references": {
|
||||
ref_id: {
|
||||
"uri": ref.uri,
|
||||
"mime_type": ref.mime_type,
|
||||
"title": ref.title,
|
||||
"path": ref.relpath.as_posix(),
|
||||
}
|
||||
for ref_id, ref in sorted(skill.references.items())
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def build_prompts_index_payload(
|
||||
registry: DocsRegistry,
|
||||
*,
|
||||
query: str | None = None,
|
||||
tag: str | None = None,
|
||||
cursor: str | None = None,
|
||||
limit: int | None = None,
|
||||
) -> dict[str, Any]:
|
||||
normalized_limit = DEFAULT_LIMIT if limit is None else max(1, min(limit, MAX_LIMIT))
|
||||
try:
|
||||
start = 0 if cursor is None else max(0, int(cursor))
|
||||
except ValueError as exc:
|
||||
raise ValueError("cursor must be an integer string") from exc
|
||||
|
||||
ordered = [registry.prompts_by_id[prompt_id] for prompt_id in registry.prompts_in_load_order]
|
||||
matches = [prompt for prompt in ordered if _prompt_matches(prompt, query=query, tag=tag)]
|
||||
|
||||
page = matches[start : start + normalized_limit]
|
||||
next_cursor = start + normalized_limit
|
||||
|
||||
return {
|
||||
"prompts": [PromptSummaryPayload.from_record(prompt).model_dump() for prompt in page],
|
||||
"total": len(matches),
|
||||
"cursor": str(start),
|
||||
"limit": normalized_limit,
|
||||
"next_cursor": str(next_cursor) if next_cursor < len(matches) else None,
|
||||
}
|
||||
|
||||
|
||||
def build_prompt_detail_payload(registry: DocsRegistry, prompt_id: str) -> dict[str, Any]:
|
||||
if prompt_id not in registry.prompts_by_id:
|
||||
raise KeyError(prompt_id)
|
||||
|
||||
prompt = registry.prompts_by_id[prompt_id]
|
||||
return {
|
||||
"id": prompt.prompt_id,
|
||||
"name": prompt.name,
|
||||
"description": prompt.description,
|
||||
"version": prompt.version,
|
||||
"tags": list(prompt.tags),
|
||||
"capabilities": list(prompt.capabilities),
|
||||
"resources": {
|
||||
"document": prompt.document_uri,
|
||||
},
|
||||
"arguments": {
|
||||
arg_name: arg.model_dump(exclude_none=True) for arg_name, arg in sorted(prompt.arguments.items())
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def search_patterns_payload(
|
||||
registry: DocsRegistry,
|
||||
*,
|
||||
query: str = "",
|
||||
tags: list[str] | None = None,
|
||||
skip: int = 0,
|
||||
limit: int = DEFAULT_LIMIT,
|
||||
) -> dict[str, Any]:
|
||||
normalized_skip = max(skip, 0)
|
||||
normalized_limit = max(1, min(limit, MAX_LIMIT))
|
||||
|
||||
requested_tags = [tag.strip() for tag in (tags or []) if tag and tag.strip()]
|
||||
|
||||
matches: list[SkillRecord] = []
|
||||
for skill_id in registry.skills_in_load_order:
|
||||
skill = registry.skills_by_id[skill_id]
|
||||
if not _skill_matches(skill, query=query, tag=None, capability=None):
|
||||
continue
|
||||
if requested_tags and any(tag not in skill.tags for tag in requested_tags):
|
||||
continue
|
||||
matches.append(skill)
|
||||
|
||||
page = matches[normalized_skip : normalized_skip + normalized_limit]
|
||||
return {
|
||||
"patterns": [SkillPatternPayload.from_record(skill).model_dump() for skill in page],
|
||||
"total": len(matches),
|
||||
"skip": normalized_skip,
|
||||
"limit": normalized_limit,
|
||||
}
|
||||
|
||||
|
||||
def get_pattern_by_id_payload(registry: DocsRegistry, skill_id: str) -> dict[str, Any]:
|
||||
if skill_id not in registry.skills_by_id:
|
||||
return {"found": False, "id": skill_id}
|
||||
return {"found": True, "pattern": SkillPatternPayload.from_record(registry.skills_by_id[skill_id]).model_dump()}
|
||||
|
||||
|
||||
def search_prompts_payload(
|
||||
registry: DocsRegistry,
|
||||
*,
|
||||
query: str = "",
|
||||
tags: list[str] | None = None,
|
||||
skip: int = 0,
|
||||
limit: int = DEFAULT_LIMIT,
|
||||
) -> dict[str, Any]:
|
||||
normalized_skip = max(skip, 0)
|
||||
normalized_limit = max(1, min(limit, MAX_LIMIT))
|
||||
|
||||
requested_tags = [tag.strip() for tag in (tags or []) if tag and tag.strip()]
|
||||
|
||||
matches: list[PromptRecord] = []
|
||||
for prompt_id in registry.prompts_in_load_order:
|
||||
prompt = registry.prompts_by_id[prompt_id]
|
||||
if not _prompt_matches(prompt, query=query, tag=None):
|
||||
continue
|
||||
if requested_tags and any(tag not in prompt.tags for tag in requested_tags):
|
||||
continue
|
||||
matches.append(prompt)
|
||||
|
||||
page = matches[normalized_skip : normalized_skip + normalized_limit]
|
||||
return {
|
||||
"prompts": [PromptSummaryPayload.from_record(prompt).model_dump() for prompt in page],
|
||||
"total": len(matches),
|
||||
"skip": normalized_skip,
|
||||
"limit": normalized_limit,
|
||||
}
|
||||
|
||||
|
||||
def get_prompt_by_id_payload(registry: DocsRegistry, prompt_id: str) -> dict[str, Any]:
|
||||
if prompt_id not in registry.prompts_by_id:
|
||||
return {"found": False, "id": prompt_id}
|
||||
return {
|
||||
"found": True,
|
||||
"prompt": build_prompt_detail_payload(registry, prompt_id),
|
||||
}
|
||||
@@ -1,6 +1,5 @@
|
||||
from functools import cache
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel
|
||||
from pydantic import DirectoryPath
|
||||
@@ -9,7 +8,7 @@ from pydantic_settings import BaseSettings
|
||||
from pydantic_settings import SettingsConfigDict
|
||||
|
||||
DEFAULT_ENV_FILE = Path(".env").resolve()
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
DEFAULT_SITE_DIR = Path("site").resolve()
|
||||
|
||||
|
||||
class Mounts(BaseModel):
|
||||
@@ -24,18 +23,21 @@ class Settings(BaseSettings):
|
||||
env_file=DEFAULT_ENV_FILE,
|
||||
env_prefix="PERSONAL_MCP_",
|
||||
extra="ignore",
|
||||
cli_implicit_flags=True,
|
||||
)
|
||||
|
||||
debug: bool = False
|
||||
log_level: str = "info"
|
||||
mounts: Mounts = Field(default_factory=Mounts)
|
||||
mcp_transport: Literal["http", "sse"] = "http"
|
||||
site_dir: DirectoryPath = Field(default=_REPO_ROOT / "site")
|
||||
site_dir: DirectoryPath = Field(default=DEFAULT_SITE_DIR)
|
||||
host: str = "localhost"
|
||||
port: int = 8080
|
||||
reload: bool = True
|
||||
|
||||
|
||||
@cache
|
||||
def get_settings(**overrides) -> Settings:
|
||||
return Settings(**overrides)
|
||||
def get_settings(*, cli: bool = False, **overrides) -> Settings:
|
||||
return Settings(**overrides, _cli_parse_args=cli) # pyright: ignore[reportCallIssue]
|
||||
|
||||
|
||||
def refresh_settings(**overrides):
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
../../docs
|
||||
@@ -0,0 +1,105 @@
|
||||
---
|
||||
icon: lucide/library
|
||||
---
|
||||
|
||||
# Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
The application combines a FastMCP server with a pre-built Zensical documentation site. Markdown under `docs/` is the single authored content tree, while native FastMCP providers own skill and prompt discovery.
|
||||
|
||||
The runtime has four content paths:
|
||||
|
||||
1. `SkillsDirectoryProvider` publishes native `skill://` resources from packaged skill directories.
|
||||
2. A custom prompt provider loads declarative prompt definitions from packaged Markdown.
|
||||
3. The general docs registry publishes non-skill Markdown through `resource://docs/{path*}`.
|
||||
4. FastAPI serves the pre-built `site/` directory.
|
||||
|
||||
There is no custom skill catalog, prompt catalog, or per-prompt Python module.
|
||||
|
||||
## Source Ownership
|
||||
|
||||
### Skills
|
||||
|
||||
Each skill owns one directory:
|
||||
|
||||
1. `docs/skills/<skill-id>/SKILL.md`
|
||||
2. `docs/skills/<skill-id>/<supporting-path>`
|
||||
|
||||
`SkillsDirectoryProvider` publishes:
|
||||
|
||||
1. `skill://<name>/SKILL.md`
|
||||
2. `skill://<name>/_manifest`
|
||||
3. `skill://<name>/{path*}`
|
||||
|
||||
The provider parses standard skill frontmatter and generates the manifest. The general docs registry excludes `skills/**`, so only the native provider owns this namespace.
|
||||
|
||||
### Prompts
|
||||
|
||||
Each prompt has one source: `docs/prompts/<prompt-id>/PROMPT.md`. Its nested `prompt` frontmatter owns runtime metadata and argument declarations, while its body owns canonical prose.
|
||||
|
||||
The custom provider reads packaged Markdown with `importlib.resources`, validates metadata and exact placeholder-to-argument equality, and creates native FastMCP prompt objects. It rescans on each list and get request, so an editable deployment observes file additions, edits, and deletions without a restart.
|
||||
|
||||
FastMCP exposes prompts through native `prompts/list` and `prompts/get` operations.
|
||||
|
||||
### General Docs
|
||||
|
||||
The docs registry indexes packaged Markdown for `resource://docs/{path*}`. It rejects `skills/**` because skills are provider-owned. Prompt Markdown can remain visible as general documentation, but prompt invocation is owned by the native prompt provider.
|
||||
|
||||
## Runtime Composition
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Packaged Skill Directories] --> B[SkillsDirectoryProvider]
|
||||
C[Packaged Prompt Markdown] --> D[Markdown Prompt Provider]
|
||||
F[General Markdown] --> G[Docs Registry]
|
||||
B --> H[FastMCP Server]
|
||||
D --> H
|
||||
G --> H
|
||||
H --> K[MCP Transport]
|
||||
L[Zensical Site Output] --> M[FastAPI Static Mount]
|
||||
K --> M
|
||||
```
|
||||
|
||||
Server construction is lazy with respect to package import. Each application process creates its providers and docs snapshot when the server factory runs. Skills use startup discovery, while prompts are reloaded when a client lists or gets prompts.
|
||||
|
||||
## Packaging
|
||||
|
||||
The repository root `docs/` directory is the only authored Markdown source. `src/personal_mcp/docs` is a relative symlink used by source checkouts and editable installs. Hatchling follows it and stores regular files beneath `personal_mcp/docs/` in the wheel.
|
||||
|
||||
Runtime reads are package-relative:
|
||||
|
||||
1. Prompt content and general docs use `importlib.resources` and `Traversable` APIs.
|
||||
2. `SkillsDirectoryProvider` receives the packaged `personal_mcp/docs/skills` filesystem path.
|
||||
3. No runtime content lookup depends on the current working directory.
|
||||
|
||||
## Public Contracts
|
||||
|
||||
The machine-facing surfaces are:
|
||||
|
||||
1. Native skill resources under `skill://<name>/...`.
|
||||
2. Native MCP prompt list and get operations.
|
||||
3. `resource://docs/{path*}` for general Markdown.
|
||||
|
||||
Canonical contracts are documented in:
|
||||
|
||||
1. [Prompt Contract](./contracts/prompt.md)
|
||||
2. [Skill Contract](./contracts/skill_contract.md)
|
||||
3. [Frontmatter Contract](./contracts/frontmatter.md)
|
||||
4. [URI Contract](./contracts/uris.md)
|
||||
|
||||
Only these canonical provider and protocol surfaces are registered.
|
||||
|
||||
## Static Documentation
|
||||
|
||||
Zensical builds `docs/` into `site/` before deployment. FastAPI mounts that immutable output in the same process that hosts FastMCP. Generated `site/` files are deployment assets and are never an authored source.
|
||||
|
||||
## Validation
|
||||
|
||||
Changes are accepted only after:
|
||||
|
||||
1. focused provider and protocol tests
|
||||
2. Ruff and ty checks
|
||||
3. a Zensical build
|
||||
4. the full pytest suite
|
||||
5. an installed-wheel smoke test when packaging or provider paths change
|
||||
@@ -0,0 +1,131 @@
|
||||
---
|
||||
icon: lucide/pencil
|
||||
---
|
||||
|
||||
# Authoring Guide
|
||||
|
||||
This page defines the practical workflow for maintaining skills, prompts, and project documentation while keeping root `docs/` as the only authored source.
|
||||
|
||||
Primary references:
|
||||
|
||||
1. [Skill Contract](./contracts/skill_contract.md)
|
||||
2. [Prompt Contract](./contracts/prompt.md)
|
||||
3. [Frontmatter Contract](./contracts/frontmatter.md)
|
||||
4. [URI Contract](./contracts/uris.md)
|
||||
5. [Zensical documentation skill](./skills/zensical-docs/SKILL.md)
|
||||
|
||||
## Source Tree Ownership
|
||||
|
||||
Edit content only under root `docs/`. The `src/personal_mcp/docs` path is a relative symlink for editable installs; do not author through a copied package tree.
|
||||
|
||||
Hatchling's normal package traversal follows `src/personal_mcp/docs` during wheel builds and archives the linked targets as regular files under `personal_mcp/docs/`. Do not add a `force-include` entry for root `docs/`; it duplicates those wheel paths. The installed package therefore gives `SkillsDirectoryProvider` a regular filesystem directory while Zensical builds the human site directly from root `docs/`.
|
||||
|
||||
Generated `site/` content is a build artifact and must not be edited by hand.
|
||||
|
||||
## Content Layout
|
||||
|
||||
```text
|
||||
docs/
|
||||
*.md
|
||||
contracts/
|
||||
prompts/<prompt-id>/
|
||||
PROMPT.md
|
||||
references/
|
||||
skills/<skill-name>/
|
||||
SKILL.md
|
||||
references/
|
||||
```
|
||||
|
||||
Keep skill and prompt files inside their owning directories. Relative links may cross sections, but content ownership should remain clear.
|
||||
|
||||
## Skill Authoring
|
||||
|
||||
A skill is discovered when a direct child of `docs/skills/` contains `SKILL.md`.
|
||||
|
||||
Required frontmatter:
|
||||
|
||||
```yaml
|
||||
---
|
||||
name: <skill-name>
|
||||
description: <what the skill does and when to use it>
|
||||
---
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
1. Use lowercase kebab-case for the directory and `name`.
|
||||
2. Keep `name` exactly equal to the directory name.
|
||||
3. Write a specific description because clients use it for discovery.
|
||||
4. Do not add `x-personal-mcp`, versions, tags, capabilities, or reference mappings.
|
||||
5. Put supporting material anywhere beneath the skill directory, normally under `references/`.
|
||||
6. Link supporting files from `SKILL.md` so humans and agents understand when to load them.
|
||||
|
||||
FastMCP recursively scans every skill file and generates `skill://<name>/_manifest`. Supporting-resource identity is the real relative path, not a synthetic reference id.
|
||||
|
||||
Recommended sequence:
|
||||
|
||||
1. Draft or revise `SKILL.md` routing guidance.
|
||||
2. Add focused supporting files.
|
||||
3. Verify relative links.
|
||||
4. Run the provider tests and docs build.
|
||||
5. Restart running servers because production uses `reload=False`.
|
||||
|
||||
## Prompt Authoring
|
||||
|
||||
A prompt is one self-describing `docs/prompts/<prompt-id>/PROMPT.md` file:
|
||||
|
||||
1. Create a lowercase kebab-case directory beneath `docs/prompts/`.
|
||||
2. Add a nested `prompt` frontmatter mapping with version, description, tags, and ordered arguments.
|
||||
3. Give every argument a description and explicit required flag.
|
||||
4. Add `choices` only when a string argument accepts a fixed set of values.
|
||||
5. Use each argument exactly once or more as a `{{argument_name}}` placeholder in the body.
|
||||
6. Do not add a Python component, name field, metadata sidecar, or central catalog entry.
|
||||
|
||||
The custom provider rescans prompt documents during every native list and get request. Changes in an editable checkout are therefore visible on the next request without a process restart. Invalid metadata or placeholder drift fails that request with a configuration error.
|
||||
|
||||
## Frontmatter Safety
|
||||
|
||||
1. Quote scalar values containing `:`.
|
||||
2. Quote values with reserved YAML characters such as `#`, `{}`, `[]`, or leading `*`.
|
||||
3. Use block scalars for punctuation-heavy multiline text.
|
||||
4. Keep fields within the applicable skill or documentation contract.
|
||||
|
||||
## Writing Quality
|
||||
|
||||
1. Prefer focused sections and descriptive headings.
|
||||
2. Link feature-level claims to authoritative sources.
|
||||
3. Use relative links for internal pages.
|
||||
4. Keep code examples minimal and actionable.
|
||||
5. Avoid bare URLs in prose.
|
||||
6. Load only supporting material relevant to the immediate task.
|
||||
|
||||
## Copilot Routing
|
||||
|
||||
Active instructions should point directly to native main resources:
|
||||
|
||||
1. `skill://zensical-docs/SKILL.md`
|
||||
2. `skill://pytesting/SKILL.md`
|
||||
3. `skill://vscode-configuration/SKILL.md`
|
||||
|
||||
When deeper guidance is needed, read the selected skill's `_manifest` and fetch supporting files by their listed path.
|
||||
|
||||
## Validation Checklist
|
||||
|
||||
```bash
|
||||
uv run pytest tests/skills/test_provider.py tests/web/test_mcp_skills.py -q
|
||||
uv run zensical build
|
||||
uv run ruff check .
|
||||
uv run ty check
|
||||
uv run pytest
|
||||
```
|
||||
|
||||
For packaging changes, also build and inspect an installed wheel so provider path resolution is verified outside the editable checkout.
|
||||
|
||||
## Navigation
|
||||
|
||||
When adding or moving pages:
|
||||
|
||||
1. update `zensical.toml`
|
||||
2. keep top-level page icons in frontmatter
|
||||
3. rebuild the site
|
||||
4. verify internal links and navigation labels
|
||||
@@ -0,0 +1,72 @@
|
||||
---
|
||||
icon: lucide/braces
|
||||
---
|
||||
|
||||
# Frontmatter Contract
|
||||
|
||||
This page defines frontmatter ownership for native skills and prompt documentation.
|
||||
|
||||
## Skill Frontmatter
|
||||
|
||||
Skills use the standard Agent Skills fields consumed by the [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills):
|
||||
|
||||
```yaml
|
||||
---
|
||||
name: <skill-id>
|
||||
description: <what the skill does and when to use it>
|
||||
---
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
1. `name` and `description` are required.
|
||||
2. `name` must equal the skill directory name.
|
||||
3. The repository uses lowercase kebab-case directory names.
|
||||
4. Skill frontmatter contains no `x-personal-mcp` catalog metadata.
|
||||
5. Supporting files require no frontmatter manifest. The provider discovers files recursively and generates `_manifest` with relative paths, byte sizes, and SHA256 hashes.
|
||||
|
||||
The provider uses the directory name as the URI identity and the frontmatter `description` as the main resource description. Repository tests enforce directory/name parity and reject extra skill frontmatter fields.
|
||||
|
||||
## Prompt Documentation Frontmatter
|
||||
|
||||
Each prompt stores runtime metadata in a nested `prompt` mapping beside fields consumed by the static documentation site. The runtime mapping uses this shape:
|
||||
|
||||
```yaml
|
||||
---
|
||||
icon: lucide/messages-square
|
||||
prompt:
|
||||
version: "1.0.0"
|
||||
description: Describe when to use the prompt.
|
||||
tags: [example, prompts]
|
||||
arguments: {topic: {description: "Topic to process.", required: true, choices: [first, second]}, notes: {description: "Optional constraints.", required: false}}
|
||||
---
|
||||
```
|
||||
|
||||
Prompt rules:
|
||||
|
||||
1. `version`, `description`, `tags`, and `arguments` are required; unknown fields inside `prompt` or an argument are rejected.
|
||||
2. The directory name supplies the prompt id. Do not add a duplicate `name` field.
|
||||
3. Argument names must be valid identifiers and preserve their authored mapping order.
|
||||
4. Every argument requires a non-empty `description` and explicit `required` boolean.
|
||||
5. Optional `choices` must be a non-empty list of unique, non-empty strings.
|
||||
6. Markdown placeholders must exactly match the declared argument names.
|
||||
7. Top-level fields such as `icon` remain owned by the documentation site and are not runtime prompt metadata.
|
||||
|
||||
See the MCP [prompts concept documentation](https://modelcontextprotocol.io/docs/learn/server-concepts#prompts) and [schema reference](https://modelcontextprotocol.io/specification/latest/schema) for the protocol-level prompt shape.
|
||||
|
||||
## Validation Timing
|
||||
|
||||
Skill validation is file- and provider-oriented:
|
||||
|
||||
1. `SkillsDirectoryProvider` discovers each directory containing `SKILL.md`.
|
||||
2. FastMCP parses the description and scans all files when the provider is created.
|
||||
3. Repository tests enforce the stricter standard-only frontmatter and directory/name rules.
|
||||
|
||||
Prompt validation is provider- and renderer-oriented. Every list or get request reloads and validates the authored files. A malformed definition fails the request instead of publishing a partial prompt set.
|
||||
|
||||
## Invariants
|
||||
|
||||
1. Skills remain directly portable to tools that understand standard Agent Skills directories.
|
||||
2. Native skill discovery has no parallel catalog metadata source.
|
||||
3. Prompts use FastMCP's native component metadata and protocol surface without a parallel catalog or Python component file.
|
||||
4. All authored content remains under `docs/`.
|
||||
@@ -0,0 +1,84 @@
|
||||
---
|
||||
icon: lucide/messages-square
|
||||
---
|
||||
|
||||
# Prompt Contract
|
||||
|
||||
This page defines the canonical contract for declarative prompts published through a custom [FastMCP provider](https://gofastmcp.com/servers/providers/custom).
|
||||
|
||||
## Canonical Prompt Shape
|
||||
|
||||
Each prompt is one self-describing Markdown document:
|
||||
|
||||
```mermaid
|
||||
---
|
||||
config:
|
||||
treeView:
|
||||
rowIndent: 20
|
||||
lineThickness: 2
|
||||
themeVariables:
|
||||
treeView:
|
||||
labelColor: '#FFFFFF'
|
||||
lineColor: '#FFFFFF'
|
||||
---
|
||||
treeView-beta
|
||||
"docs/prompts/"
|
||||
"<prompt-id>/"
|
||||
"PROMPT.md"
|
||||
"src/personal_mcp/prompts/"
|
||||
"content.py"
|
||||
"models.py"
|
||||
"provider.py"
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
1. The parent directory name defines the public prompt id.
|
||||
2. The nested `prompt` frontmatter block defines version, description, tags, and arguments.
|
||||
3. Argument declarations define names, descriptions, requiredness, and optional string choices.
|
||||
4. The Markdown body owns the rendered prompt prose and uses `{{argument_name}}` placeholders.
|
||||
5. Declared arguments and body placeholders must match exactly.
|
||||
6. No Python file is added when authoring a prompt.
|
||||
|
||||
## Ownership Boundary
|
||||
|
||||
1. Each `PROMPT.md` owns both its runtime metadata and prose.
|
||||
2. Python owns only generic parsing, validation, rendering, and provider behavior.
|
||||
3. There is no central prompt catalog, generated signature, or metadata sidecar.
|
||||
4. The provider scans direct children of packaged `docs/prompts/` on each list or get request.
|
||||
5. Additions, edits, and deletions become visible on the next request without restarting the server.
|
||||
6. Reload is pull-based; the provider does not watch files or emit proactive change notifications.
|
||||
|
||||
## Prompt Id Contract
|
||||
|
||||
`prompt-id` is the public identifier and should satisfy all rules below:
|
||||
|
||||
1. Format: lowercase kebab-case only.
|
||||
2. Character set: `a-z`, `0-9`, and `-`.
|
||||
3. Must start with a letter.
|
||||
4. No underscores, spaces, dots, or uppercase characters.
|
||||
5. Directory name should equal `prompt-id` in each committed revision.
|
||||
6. The provider derives the prompt name from the directory; frontmatter must not duplicate it.
|
||||
7. Treat `prompt-id` as immutable after release; a rename is a breaking replacement.
|
||||
|
||||
Valid examples:
|
||||
|
||||
1. `pytest-fill-scaffold`
|
||||
2. `review-pr-comments`
|
||||
3. `scaffold-fastapi-service`
|
||||
|
||||
Invalid examples:
|
||||
|
||||
1. `fill_pytest_scaffold`
|
||||
2. `Prompt-Template`
|
||||
3. `docs.prompt`
|
||||
|
||||
## Rendering Contract
|
||||
|
||||
1. The loader requires one leading YAML frontmatter block and validates its nested `prompt` mapping strictly.
|
||||
2. All MCP arguments are strings; `choices` optionally restricts accepted values.
|
||||
3. Missing required arguments, unknown arguments, and invalid choices fail before rendering.
|
||||
4. An omitted optional value renders as `Not provided`.
|
||||
5. Unknown prompt ids, malformed metadata, and mismatched placeholders fail immediately.
|
||||
6. Prompt content is read through [importlib resources](https://docs.python.org/3/library/importlib.resources.html) and does not depend on the working directory.
|
||||
|
||||
+15
-5
@@ -40,9 +40,9 @@ Rules:
|
||||
|
||||
## Metadata Location Constraint
|
||||
|
||||
1. Skill metadata is embedded in YAML frontmatter in `SKILL.md`.
|
||||
2. No `metadata.yaml` sidecar exists in the end state.
|
||||
3. Reference lookup metadata is documented and explicit: top-level `references/*.md` are auto-discovered from filenames, while `SKILL.md` frontmatter declares overrides and nested mappings when needed.
|
||||
1. `SKILL.md` frontmatter contains only standard `name` and `description` fields.
|
||||
2. No `metadata.yaml` sidecar or repository-specific skill metadata block exists.
|
||||
3. The provider discovers supporting files recursively; their real relative paths are published in the generated `_manifest`.
|
||||
|
||||
## Skill Id Contract
|
||||
|
||||
@@ -52,8 +52,8 @@ Rules:
|
||||
2. Character set: `a-z`, `0-9`, and `-`.
|
||||
3. Must start with a letter.
|
||||
4. No underscores, spaces, dots, or uppercase characters.
|
||||
5. Directory name should equal `skill-id` in each committed revision.
|
||||
6. Frontmatter `id` should equal directory name in each committed revision.
|
||||
5. Directory name equals `skill-id` in each committed revision.
|
||||
6. Frontmatter `name` equals the directory name.
|
||||
7. Treat `skill-id` as immutable after release; any rename is a breaking replacement and clients must move to the new id.
|
||||
|
||||
Valid examples:
|
||||
@@ -68,6 +68,16 @@ Invalid examples:
|
||||
2. `Zensical-Docs`
|
||||
3. `docs.zensical`
|
||||
|
||||
## Provider Publication
|
||||
|
||||
[`SkillsDirectoryProvider`](https://gofastmcp.com/servers/providers/skills) scans `docs/skills/` with `supporting_files="template"` and publishes:
|
||||
|
||||
1. `skill://<skill-id>/SKILL.md`
|
||||
2. `skill://<skill-id>/_manifest`
|
||||
3. `skill://<skill-id>/{path*}` for supporting files
|
||||
|
||||
Only the main file and manifest appear in `resources/list`. Clients inspect the manifest before reading supporting paths.
|
||||
|
||||
## Direct Documentation Inclusion
|
||||
|
||||
1. For direct API documentation, use mkdocstrings directives rather than pasting large code blocks.
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
icon: lucide/link
|
||||
---
|
||||
|
||||
# URI Contract
|
||||
|
||||
This page defines the public resource URI contract for native skills and general authored documentation.
|
||||
|
||||
## Native Skill URIs
|
||||
|
||||
The [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills) publishes each skill through the `skill://` scheme:
|
||||
|
||||
1. `skill://<skill-name>/SKILL.md`
|
||||
2. `skill://<skill-name>/_manifest`
|
||||
3. `skill://<skill-name>/<supporting-path>`
|
||||
|
||||
The first two are concrete resources returned by `resources/list`. Supporting files use a per-skill wildcard resource template when the provider is configured with `supporting_files="template"`:
|
||||
|
||||
```text
|
||||
skill://<skill-name>/{path*}
|
||||
```
|
||||
|
||||
### Main File
|
||||
|
||||
`skill://<skill-name>/SKILL.md` returns the canonical authored skill document. The skill directory name supplies `<skill-name>`, and the resource description comes from `SKILL.md` frontmatter.
|
||||
|
||||
### Manifest
|
||||
|
||||
`skill://<skill-name>/_manifest` returns JSON containing the skill name and every file beneath its directory. Each file entry includes:
|
||||
|
||||
1. relative POSIX path
|
||||
2. byte size
|
||||
3. SHA256 hash
|
||||
|
||||
Clients read the manifest before requesting supporting files. FastMCP client utilities such as `list_skills()` and `get_skill_manifest()` understand this contract directly.
|
||||
|
||||
### Supporting Files
|
||||
|
||||
Supporting files retain their real skill-relative paths. For example:
|
||||
|
||||
```text
|
||||
skill://pytesting/references/pytest-docs.md
|
||||
```
|
||||
|
||||
FastMCP confines reads to the selected skill directory. Absolute paths, traversal outside the directory, missing files, directories, and symlinks that resolve outside the skill root are rejected.
|
||||
|
||||
## General Docs URI
|
||||
|
||||
General authored documentation is exposed through `resource://docs/{path*}`. The wildcard accepts normalized relative POSIX Markdown paths beneath `docs/`, excludes the provider-owned `skills/` subtree, and rejects absolute paths, traversal segments, backslashes, and non-Markdown targets.
|
||||
|
||||
Prompts are MCP prompt components rather than resources. Clients discover them with the protocol `prompts/list` operation and render them with `prompts/get`.
|
||||
|
||||
## Discovery Order
|
||||
|
||||
For skills:
|
||||
|
||||
1. list resources or call FastMCP `list_skills()`
|
||||
2. select a skill by name and description
|
||||
3. read `skill://<skill-name>/SKILL.md`
|
||||
4. read `_manifest` when supporting material may be needed
|
||||
5. fetch only the supporting paths relevant to the task
|
||||
|
||||
Tool-only agents may call `search_skills` instead of retrieving the complete resource list. Search results contain only provider-derived names, descriptions, and canonical main-resource URIs; skill content remains available exclusively through the native resource contract.
|
||||
|
||||
For prompts, use the native MCP prompt APIs or their generic tool projection.
|
||||
|
||||
## Stability Policy
|
||||
|
||||
The provider and protocol surfaces documented here are the complete public contract. Contract changes replace the affected surface directly.
|
||||
|
||||
Skill renames are breaking because the directory name is part of every native skill URI. Supporting-file renames change the corresponding manifest path and URI.
|
||||
|
||||
## Sources
|
||||
|
||||
1. [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills)
|
||||
2. [MCP resources](https://modelcontextprotocol.io/specification/latest/server/resources)
|
||||
3. [RFC 3986 URI syntax](https://www.rfc-editor.org/rfc/rfc3986)
|
||||
4. [RFC 6570 URI templates](https://www.rfc-editor.org/rfc/rfc6570)
|
||||
5. [FastMCP prompts](https://gofastmcp.com/servers/prompts)
|
||||
@@ -0,0 +1,131 @@
|
||||
---
|
||||
icon: lucide/bot
|
||||
---
|
||||
|
||||
# Copilot MCP Mechanics
|
||||
|
||||
## Purpose
|
||||
|
||||
This page explains how GitHub Copilot in VS Code consumes native skill resources and prompts from `personal-mcp`.
|
||||
|
||||
## Capability Lanes
|
||||
|
||||
Copilot interacts with MCP servers through independently exposed lanes:
|
||||
|
||||
1. tools invoked during execution
|
||||
2. resources attached as read-only context
|
||||
3. server-provided prompts
|
||||
|
||||
This server publishes skills as native `skill://` resources and prompts as native MCP prompt objects. It also exposes `search_skills`, `list_resources`, and `read_resource` tools for agents whose tool catalog does not include direct MCP resource operations.
|
||||
|
||||
## VS Code Feature Coverage
|
||||
|
||||
The server uses every FastMCP feature that applies to its read-only guidance workload:
|
||||
|
||||
| Feature | Usage |
|
||||
| --- | --- |
|
||||
| Server identity | The initialize response includes a stable name, usage instructions, and a self-contained icon for VS Code's MCP server UI. |
|
||||
| Tools | Compatibility tools have display titles, structured output schemas, and read-only, idempotent, closed-world annotations. FastMCP's default schema dereferencing remains enabled for clients such as VS Code that require flat schemas. |
|
||||
| Resources | Documentation and skills use native resources and wildcard resource templates with explicit Markdown MIME types. |
|
||||
| Prompts | Declarative workflows use native prompt objects with descriptions, display titles, typed arguments, and slash-command access. |
|
||||
| Argument completion | Prompt arguments with authored `choices` are returned through `completion/complete` as the user types. |
|
||||
|
||||
[FastMCP server identity](https://gofastmcp.com/servers/server), [component icons](https://gofastmcp.com/servers/icons), [tool metadata](https://gofastmcp.com/servers/tools), and [argument completion](https://gofastmcp.com/servers/completions) define the implementation details. [VS Code's MCP documentation](https://code.visualstudio.com/docs/agent-customization/mcp-servers) describes how tools, resources, prompts, and MCP Apps appear in the client.
|
||||
|
||||
The following capabilities are conditional rather than useful by default:
|
||||
|
||||
1. MCP Apps require an interactive tool result such as a form or visualization; this server returns guidance and structured resource data only.
|
||||
2. Sampling is appropriate only when server-side work must ask VS Code to run an LLM. The current server retrieves authored content and does not generate it.
|
||||
3. Elicitation is appropriate only when a running operation needs additional user input. Prompt arguments already collect all required input before execution.
|
||||
4. Progress, client logging, and background tasks require long-running operations. Current reads and prompt rendering are bounded local operations.
|
||||
5. Client roots matter only when server behavior depends on client filesystem roots. This server reads packaged content and never traverses a client workspace.
|
||||
6. `website_url` requires a canonical public deployment URL. None is configured, so the server does not advertise a guessed address.
|
||||
|
||||
Add one of these capabilities when a concrete workflow needs it, then cover its negotiated capability and protocol response in the HTTP MCP smoke tests. See the [FastMCP Apps overview](https://gofastmcp.com/apps/overview), [sampling](https://gofastmcp.com/servers/sampling), [elicitation](https://gofastmcp.com/servers/elicitation), [progress reporting](https://gofastmcp.com/servers/progress), and [MCP context](https://gofastmcp.com/servers/context) for the activation criteria.
|
||||
|
||||
## Native Skill Resources
|
||||
|
||||
For every skill, Copilot can discover:
|
||||
|
||||
1. `skill://<name>/SKILL.md`
|
||||
2. `skill://<name>/_manifest`
|
||||
3. `skill://<name>/{path*}` supporting-file template
|
||||
|
||||
The main resource description comes from `SKILL.md`. The manifest discloses supporting paths, sizes, and SHA256 hashes. Native resources remain the only skill content and discovery contract; the tools search or delegate to that same resource surface rather than maintaining a parallel catalog.
|
||||
|
||||
## Resource Picker Availability
|
||||
|
||||
`MCP Resources...` in Add Context requires both:
|
||||
|
||||
1. a connected server advertising resource capability
|
||||
2. a chat surface that exposes MCP resource attachment
|
||||
|
||||
A successful `resources/list` response does not guarantee the picker appears in every session type. Use `MCP: Browse Resources` to distinguish server availability from chat UI availability.
|
||||
|
||||
## Recommended Workflow
|
||||
|
||||
For autonomous agents:
|
||||
|
||||
1. call `search_skills` with the task, capability, or technology
|
||||
2. compare the bounded main-skill matches
|
||||
3. call `read_resource` for one relevant `skill://<name>/SKILL.md`
|
||||
4. read `_manifest` only if supporting detail may be needed
|
||||
5. read only selected supporting files
|
||||
|
||||
For manual context attachment, browse the server's resources and attach the same bounded set of files.
|
||||
|
||||
## Prompt Examples
|
||||
|
||||
Resource attachment:
|
||||
|
||||
```text
|
||||
Use the attached personal-mcp skill as guidance, then reconcile it with the repository before proposing changes.
|
||||
```
|
||||
|
||||
Direct loading:
|
||||
|
||||
```text
|
||||
Read skill://async-fastapi-sqlmodel/SKILL.md and apply only the sections relevant to this repository.
|
||||
```
|
||||
|
||||
Supporting material:
|
||||
|
||||
```text
|
||||
Read skill://pytesting/_manifest, select the one reference relevant to async test lifecycle, and use that file with the main skill instructions.
|
||||
```
|
||||
|
||||
## Repository Instruction Pattern
|
||||
|
||||
A repo-level instruction should name the native retrieval order and context budget:
|
||||
|
||||
```md
|
||||
When a task matches a personal-mcp skill:
|
||||
|
||||
1. Prefer an already attached native skill resource.
|
||||
2. Otherwise call `search_skills` and select one `skill://<name>/SKILL.md` result by description.
|
||||
3. Call `read_resource` for the selected skill and read `_manifest` only when supporting material is needed.
|
||||
4. Load at most two candidate main files and only the relevant supporting paths.
|
||||
5. Reconcile guidance with the current repository before editing.
|
||||
```
|
||||
|
||||
Instructions steer behavior but do not force VS Code to attach resources automatically. The generic tools provide an agent-callable fallback when direct resource operations are absent from the deferred-tool catalog.
|
||||
|
||||
## Prompt Objects
|
||||
|
||||
Prompts remain separate from skills. When the client supports MCP prompt APIs, use prompt listing and `get_prompt` for parameterized workflows. Each authored `PROMPT.md` is the complete source of truth for its metadata, arguments, and prose; changes are loaded on the next prompt request.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
1. Use `MCP: List Servers` to confirm the server is enabled.
|
||||
2. Use `MCP: Browse Resources` to confirm native skill resources exist.
|
||||
3. Confirm `search_skills` and `read_resource` appear in the chat tool picker when autonomous retrieval is required.
|
||||
4. Restart the MCP server after changing skill files because production uses `reload=False`.
|
||||
5. Reload the VS Code window if the server is healthy but the resource or tool picker remains stale.
|
||||
|
||||
## Further Reading
|
||||
|
||||
1. [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills)
|
||||
2. [Add and manage MCP servers](https://code.visualstudio.com/docs/agent-customization/mcp-servers)
|
||||
3. [MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration)
|
||||
4. [Manage context for AI](https://code.visualstudio.com/docs/chat/copilot-chat-context)
|
||||
5. [Skill Usage Mechanics](./usage.md)
|
||||
@@ -25,7 +25,7 @@ uv sync
|
||||
Run the app locally with the static docs rebuilt first, using [Uvicorn factory mode](https://www.uvicorn.org/settings/#application):
|
||||
|
||||
```bash
|
||||
uv run zensical build && uv run uvicorn personal_mcp.main:create_app --factory --host 127.0.0.1 --port 8765
|
||||
uv run zensical build && uv run uvicorn personal_mcp.web.app:create_app --factory --host 127.0.0.1 --port 8765
|
||||
```
|
||||
|
||||
Build and run the Docker image with the same exposed port:
|
||||
@@ -0,0 +1,95 @@
|
||||
---
|
||||
icon: lucide/server
|
||||
---
|
||||
|
||||
# Runtime And Static Docs Layout
|
||||
|
||||
## Purpose
|
||||
|
||||
The project serves native MCP content and a pre-built documentation site from one FastAPI process. Markdown is authored once under `docs/`; runtime providers and Zensical consume that same packaged tree for different purposes.
|
||||
|
||||
## Repository Layout
|
||||
|
||||
```mermaid
|
||||
---
|
||||
config:
|
||||
treeView:
|
||||
rowIndent: 32
|
||||
lineThickness: 2
|
||||
---
|
||||
treeView-beta
|
||||
"project-root"
|
||||
"docs"
|
||||
"prompts/<prompt-id>/PROMPT.md"
|
||||
"skills/<skill-id>/SKILL.md"
|
||||
"skills/<skill-id>/<supporting-files>"
|
||||
"<general-pages>.md"
|
||||
"site"
|
||||
"static build output"
|
||||
"src/personal_mcp"
|
||||
"mcp.py"
|
||||
"prompts/content.py"
|
||||
"prompts/models.py"
|
||||
"prompts/provider.py"
|
||||
"registry/"
|
||||
"skills/provider.py"
|
||||
"web/"
|
||||
```
|
||||
|
||||
Ownership rules:
|
||||
|
||||
1. `docs/skills/` is owned exclusively by `SkillsDirectoryProvider` at runtime.
|
||||
2. Each file under `docs/prompts/` owns its prompt metadata, argument schema, and prose.
|
||||
3. The docs registry owns only general Markdown resources and explicitly excludes skills.
|
||||
4. `site/` is generated output.
|
||||
5. The deleted custom `catalog/` package is not part of the runtime.
|
||||
|
||||
## Runtime Composition
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[Packaged Skills] --> B[SkillsDirectoryProvider]
|
||||
C[Packaged Prompt Markdown] --> D[Markdown Prompt Provider]
|
||||
E[Packaged Markdown] --> F[Docs Registry]
|
||||
B --> G[FastMCP]
|
||||
D --> G
|
||||
F --> G
|
||||
G --> H[MCP Transport]
|
||||
H --> K[FastAPI Application]
|
||||
L[Pre-built site] --> M[Static /docs Mount]
|
||||
K --> M
|
||||
```
|
||||
|
||||
Runtime guarantees:
|
||||
|
||||
1. Providers are installed before serving requests.
|
||||
2. Prompt discovery rescans authored files on each list and get request.
|
||||
3. Duplicate components fail according to FastMCP's configured duplicate policy.
|
||||
4. Skills and prompts use native FastMCP component surfaces.
|
||||
5. General docs path parsing rejects traversal, backslashes, non-Markdown paths, and the skill namespace.
|
||||
|
||||
## Build And Publish Flow
|
||||
|
||||
1. Author prompt definitions and prose under `docs/prompts/`.
|
||||
2. Run `uv run zensical build` to produce `site/`.
|
||||
3. Build the wheel, which packages the authored docs under `personal_mcp/docs/`.
|
||||
4. Start the app and serve MCP plus the static site.
|
||||
|
||||
No runtime Markdown-to-HTML conversion occurs.
|
||||
|
||||
## Machine-Facing Mapping
|
||||
|
||||
1. `docs/skills/<skill-id>/SKILL.md` maps to `skill://<skill-id>/SKILL.md`.
|
||||
2. Skill supporting files map to `skill://<skill-id>/<path>`.
|
||||
3. Declarative prompt documents map to native MCP prompt names.
|
||||
4. General `docs/<path>.md` maps to `resource://docs/{path*}`.
|
||||
|
||||
The server publishes no tool projections of resources or prompts.
|
||||
|
||||
## Public Surface Policy
|
||||
|
||||
Canonical provider and protocol surfaces are the only public interfaces.
|
||||
|
||||
## Static Mount Expectations
|
||||
|
||||
The FastAPI app mounts the Zensical output, serves index and asset files, and returns a clear unavailable response when the static output is absent. The site directory is immutable for a given build and remains separate from packaged authored Markdown.
|
||||
+19
-29
@@ -1,33 +1,21 @@
|
||||
---
|
||||
name: authoring
|
||||
description: Provide a practical checklist and baseline template for authoring docs-first MCP modules and repository-specific Copilot instruction shims.
|
||||
x-personal-mcp:
|
||||
id: authoring
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- authoring
|
||||
- mcp
|
||||
- fastmcp
|
||||
- copilot
|
||||
- prompts
|
||||
- scaffolding
|
||||
capabilities:
|
||||
- resource://prompts/authoring/document
|
||||
icon: lucide/messages-square
|
||||
prompt:
|
||||
version: "1.0.0"
|
||||
description: Provide a practical checklist and baseline template for authoring docs-first MCP modules and repository-specific Copilot instruction shims.
|
||||
tags: [authoring, mcp, fastmcp, copilot, prompts, scaffolding]
|
||||
arguments:
|
||||
artifact_type:
|
||||
title: Artifact type
|
||||
description: "Enum (case-sensitive): skill | prompt | shim."
|
||||
description: Artifact type to create.
|
||||
required: true
|
||||
choices: [skill, prompt, shim]
|
||||
artifact_id:
|
||||
title: Artifact id
|
||||
description: Lowercase kebab-case id for the module or shim.
|
||||
required: true
|
||||
goal:
|
||||
title: Goal
|
||||
description: One-sentence capability statement describing what to create and when to use it.
|
||||
description: One-sentence capability statement.
|
||||
required: true
|
||||
scope_glob:
|
||||
title: Scope glob
|
||||
description: Optional applyTo glob for shim outputs.
|
||||
required: false
|
||||
---
|
||||
@@ -36,6 +24,13 @@ x-personal-mcp:
|
||||
|
||||
Use this prompt to author or update docs-first MCP modules in this repository, including repository-specific Copilot thin shims.
|
||||
|
||||
## Supplied Inputs
|
||||
|
||||
- `artifact_type`: {{artifact_type}}
|
||||
- `artifact_id`: {{artifact_id}}
|
||||
- `goal`: {{goal}}
|
||||
- `scope_glob`: {{scope_glob}}
|
||||
|
||||
## Inputs
|
||||
|
||||
1. artifact_type: one of skill, prompt, shim
|
||||
@@ -51,7 +46,7 @@ Load only what matches the requested artifact:
|
||||
2. Prompt metadata and structure: [Prompt Contract](../../contracts/prompt.md)
|
||||
3. Skill metadata and structure (only for skill outputs): [Skill Contract](../../contracts/skill_contract.md)
|
||||
4. Thin shim mechanics and path binding: [Skill Usage Mechanics](../../usage.md)
|
||||
5. Copilot resource attachment and fallback behavior: [Copilot MCP Mechanics](../../copilot.md)
|
||||
5. Copilot resource attachment behavior: [Copilot MCP Mechanics](../../copilot.md)
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -69,14 +64,9 @@ Load only what matches the requested artifact:
|
||||
7. Produce only the requested artifact type.
|
||||
8. Keep guidance deterministic and minimal, with explicit references to source docs.
|
||||
9. If artifact_type is shim:
|
||||
- bind one applyTo scope to one primary skill resource URI
|
||||
- prefer MCP resource attachment first
|
||||
- if resource attachment is unavailable, use fallback tool order:
|
||||
1. list_resources
|
||||
2. read_resource
|
||||
3. search_patterns
|
||||
4. get_pattern_by_id
|
||||
5. get_skill_document_by_id
|
||||
- bind one applyTo scope to one `skill://<name>/SKILL.md` resource URI
|
||||
- use MCP resource attachment
|
||||
- inspect the selected skill's `_manifest` only when supporting material is needed
|
||||
10. Return created or updated file paths and any validation commands that should be run.
|
||||
|
||||
## Output Contract
|
||||
+26
-31
@@ -1,41 +1,36 @@
|
||||
---
|
||||
name: greenfield-architecture
|
||||
description: Research established patterns and design a high-level architecture for a new app or library with explicit tradeoffs and test strategy.
|
||||
x-personal-mcp:
|
||||
id: greenfield-architecture
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- architecture
|
||||
- planning
|
||||
- greenfield
|
||||
- design
|
||||
- testing
|
||||
- prompts
|
||||
capabilities:
|
||||
- resource://prompts/greenfield-architecture/document
|
||||
arguments:
|
||||
scope_type:
|
||||
title: Scope type
|
||||
description: "Scope type: app or library."
|
||||
required: true
|
||||
intent_document:
|
||||
title: Intent document
|
||||
description: Optional full document describing goals, context, and desired outcomes.
|
||||
required: false
|
||||
problem_domain:
|
||||
title: Problem domain
|
||||
description: Domain and business goal for the new app or library when no full intent document is provided.
|
||||
required: false
|
||||
constraints:
|
||||
title: Constraints
|
||||
description: Runtime, deployment, and non-functional constraints.
|
||||
required: false
|
||||
icon: lucide/messages-square
|
||||
prompt:
|
||||
version: "1.0.0"
|
||||
description: Research established patterns and design a high-level architecture for a new app or library with explicit tradeoffs and test strategy.
|
||||
tags: [architecture, planning, greenfield, design, testing, prompts]
|
||||
arguments:
|
||||
scope_type:
|
||||
description: Scope type to design.
|
||||
required: true
|
||||
choices: [app, library]
|
||||
intent_document:
|
||||
description: Optional full document describing goals and context.
|
||||
required: false
|
||||
problem_domain:
|
||||
description: Problem domain and business goal.
|
||||
required: false
|
||||
constraints:
|
||||
description: Runtime, deployment, and non-functional constraints.
|
||||
required: false
|
||||
---
|
||||
|
||||
# Greenfield Architecture Planner
|
||||
|
||||
Use this prompt to design a new software app or library architecture in generic terms.
|
||||
|
||||
## Supplied Inputs
|
||||
|
||||
- `scope_type`: {{scope_type}}
|
||||
- `intent_document`: {{intent_document}}
|
||||
- `problem_domain`: {{problem_domain}}
|
||||
- `constraints`: {{constraints}}
|
||||
|
||||
## Inputs
|
||||
|
||||
1. intent_document: optional full document that explains goals, context, constraints, and desired outcomes
|
||||
@@ -0,0 +1,68 @@
|
||||
---
|
||||
icon: lucide/messages-square
|
||||
prompt:
|
||||
version: "1.1.0"
|
||||
description: Create a responsive sample page layout for a user-supplied domain and return paste-ready HTML and CSS for JSFiddle.
|
||||
tags: [frontend, html, css, jsfiddle, layout, prototyping, prompts]
|
||||
arguments:
|
||||
domain:
|
||||
description: Product, service, organization, or subject represented by the page.
|
||||
required: true
|
||||
layout_brief:
|
||||
description: Optional page type, sections, priorities, or visual constraints.
|
||||
required: false
|
||||
---
|
||||
|
||||
# JSFiddle Page Layout
|
||||
|
||||
Create a polished sample page layout for the supplied domain. The result must run by pasting the markup and styles into the [JSFiddle](https://jsfiddle.net/) HTML and CSS panes.
|
||||
|
||||
## Supplied Inputs
|
||||
|
||||
- `domain`: {{domain}}
|
||||
- `layout_brief`: {{layout_brief}}
|
||||
|
||||
## Inputs
|
||||
|
||||
1. `domain`: the product, service, organization, or subject represented by the page, including its intended audience when known
|
||||
2. `layout_brief`: optional page type, required sections, content priorities, visual direction, or constraints
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Infer the page's primary purpose, audience, content hierarchy, and most important user action from the inputs.
|
||||
2. If the domain does not provide enough information to choose a useful page type or primary action, ask one concise clarification question before generating code.
|
||||
3. Choose a visual direction and information density appropriate to the domain. Build the usable page itself, not a marketing explanation of the page.
|
||||
4. Write semantic HTML with realistic domain-specific sample content. Do not use placeholder text such as lorem ipsum.
|
||||
5. Build the layout with modern CSS, using [CSS Grid](https://css-tricks.com/complete-guide-css-grid-layout/) for two-dimensional page structure and [Flexbox](https://css-tricks.com/snippets/css/a-guide-to-flexbox/) for one-dimensional alignment where each fits naturally.
|
||||
6. Make the page responsive at narrow mobile and desktop widths without horizontal overflow, overlapping content, or clipped text.
|
||||
7. Keep the example self-contained. Use no JavaScript, build tools, external stylesheets, images, or icon libraries unless the layout brief explicitly requires them.
|
||||
8. Include accessible landmarks, heading order, labels, focus styles, color contrast, and reduced-motion handling when animation is present.
|
||||
9. Use CSS custom properties for the color, typography, spacing, border, and shadow system. Avoid generic framework styling and tailor the visual language to the domain.
|
||||
|
||||
## Design References
|
||||
|
||||
Use these references as comparative guidance, not as templates to copy. Select principles that fit the domain and layout brief, and do not reproduce a vendor's visual language unless the user requests it.
|
||||
|
||||
1. [Material Design 3 foundations](https://m3.material.io/foundations) for current approaches to layout, interaction states, design tokens, and adaptable UI systems.
|
||||
2. [Apple Human Interface Guidelines](https://developer.apple.com/design/human-interface-guidelines/) for contemporary principles covering hierarchy, typography, controls, and platform-aware interaction.
|
||||
3. [web.dev responsive web design basics](https://web.dev/articles/responsive-web-design-basics) for content-led breakpoints, flexible layouts, and input-aware responsiveness.
|
||||
4. [Web Content Accessibility Guidelines (WCAG) 2.2](https://www.w3.org/TR/WCAG22/) as the accessibility baseline for structure, contrast, focus, reflow, and target sizing.
|
||||
|
||||
## Output Contract
|
||||
|
||||
Return exactly two fenced code blocks in this order:
|
||||
|
||||
1. An `html` block containing only the content for JSFiddle's HTML pane.
|
||||
2. A `css` block containing only the content for JSFiddle's CSS pane.
|
||||
|
||||
Do not include setup instructions, design commentary, JavaScript, or prose outside the two code blocks.
|
||||
|
||||
## Quality Rules
|
||||
|
||||
1. Prefer semantic elements such as `header`, `nav`, `main`, `section`, `article`, `aside`, and `footer` when they match the content.
|
||||
2. Reserve large display type for a true hero or primary page title; keep operational interfaces compact and easy to scan.
|
||||
3. Use cards only for repeated items or genuinely framed tools. Do not place cards inside cards.
|
||||
4. Use stable responsive constraints for grids, controls, media, and navigation so dynamic content does not shift the layout unexpectedly.
|
||||
5. Avoid decorative gradients, floating color blobs, excessive rounding, and one-note palettes unless they are explicitly appropriate to the domain.
|
||||
6. Ensure controls look and behave like their purpose, with visible hover and keyboard-focus states.
|
||||
7. Keep all visible copy relevant to the fictional domain rather than describing the mockup or its implementation.
|
||||
+27
-37
@@ -1,29 +1,21 @@
|
||||
---
|
||||
name: mcp-consumer-repo-shim
|
||||
description: Create one repository-specific thin shim instruction file that binds a file scope to a user-selected Personal MCP skill resource and enforces resource-first Copilot retrieval behavior.
|
||||
x-personal-mcp:
|
||||
id: mcp-consumer-repo-shim
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- copilot
|
||||
- mcp
|
||||
- instructions
|
||||
- shims
|
||||
- prompts
|
||||
capabilities:
|
||||
- resource://prompts/mcp-consumer-repo-shim/document
|
||||
icon: lucide/messages-square
|
||||
prompt:
|
||||
version: "1.0.0"
|
||||
description: Create one repository-specific thin shim instruction file that binds a file scope to a user-selected Personal MCP skill resource.
|
||||
tags: [copilot, mcp, instructions, shims, prompts]
|
||||
arguments:
|
||||
apply_to_glob:
|
||||
description: File glob scope for the shim applyTo field, such as tests/** or **/*.md.
|
||||
description: File glob scope for the shim applyTo field.
|
||||
required: true
|
||||
primary_skill_resource:
|
||||
description: Primary skill resource URI, usually resource://skills/<skill-id>/document.
|
||||
description: Primary native skill:// resource URI.
|
||||
required: true
|
||||
shim_title:
|
||||
description: Human-readable name for the instruction shim frontmatter.
|
||||
description: Optional human-readable instruction shim name.
|
||||
required: false
|
||||
companion_docs_page:
|
||||
description: Optional relative docs link for human-facing companion guidance.
|
||||
description: Optional relative companion documentation link.
|
||||
required: false
|
||||
---
|
||||
|
||||
@@ -31,6 +23,13 @@ x-personal-mcp:
|
||||
|
||||
Use this prompt to generate exactly one repository-scoped Copilot instruction shim for an MCP consumer repository.
|
||||
|
||||
## Supplied Inputs
|
||||
|
||||
- `apply_to_glob`: {{apply_to_glob}}
|
||||
- `primary_skill_resource`: {{primary_skill_resource}}
|
||||
- `shim_title`: {{shim_title}}
|
||||
- `companion_docs_page`: {{companion_docs_page}}
|
||||
|
||||
## Inputs
|
||||
|
||||
- Required:
|
||||
@@ -45,30 +44,26 @@ Use this prompt to generate exactly one repository-scoped Copilot instruction sh
|
||||
Load only sections relevant to the requested shim:
|
||||
|
||||
1. Thin shim pattern and scope guidance: [Skill Usage Mechanics](../../usage.md)
|
||||
2. VS Code Copilot MCP behavior and fallback mechanics: [Copilot MCP Mechanics](../../copilot.md)
|
||||
2. VS Code Copilot MCP resource behavior: [Copilot MCP Mechanics](../../copilot.md)
|
||||
3. Authoring workflow and validation checklist: [Authoring Guide](../../authoring.md)
|
||||
4. Instruction metadata expectations and examples: [Copilot customization skill](../../skills/copilot-customization/SKILL.md)
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Validate that apply_to_glob and primary_skill_resource are present.
|
||||
2. If either value is missing or ambiguous, ask exactly one clarifying question before generating output.
|
||||
3. Generate one .instructions.md file content block only.
|
||||
4. Keep the shim concise and deterministic:
|
||||
2. Validate that primary_skill_resource uses the `skill://<skill-name>/SKILL.md` form.
|
||||
3. If either value is missing or ambiguous, ask exactly one clarifying question before generating output.
|
||||
4. Generate one .instructions.md file content block only.
|
||||
5. Keep the shim concise and deterministic:
|
||||
- include YAML frontmatter with name, description, and applyTo
|
||||
- include a primary rule that uses the selected primary_skill_resource first
|
||||
- include a bounded execution pattern (load primary doc, apply only relevant sections, keep edits minimal)
|
||||
5. Include VS Code/Copilot integration mechanics in the shim body:
|
||||
- prefer MCP resource attachment when available
|
||||
- if attachment is unavailable, use tool fallback order:
|
||||
1. list_resources
|
||||
2. read_resource
|
||||
3. search_patterns
|
||||
4. get_pattern_by_id
|
||||
5. get_skill_document_by_id
|
||||
6. Include VS Code/Copilot integration mechanics in the shim body:
|
||||
- use MCP resource attachment
|
||||
- inspect `_manifest` only when the task needs supporting material
|
||||
- ask one clarifying question when confidence is low
|
||||
6. If companion_docs_page is provided, include it as a companion docs link line.
|
||||
7. Do not generate additional files, code changes, or batch shim packs.
|
||||
7. If companion_docs_page is provided, include it as a companion docs link line.
|
||||
8. Do not generate additional files, code changes, or batch shim packs.
|
||||
|
||||
## Output Format
|
||||
|
||||
@@ -98,12 +93,7 @@ Execution pattern:
|
||||
2. Apply only sections relevant to the file being edited.
|
||||
3. Keep edits minimal and aligned with repository conventions.
|
||||
4. Prefer MCP resource attachment when available in the current chat surface.
|
||||
5. If MCP resource attachment is unavailable, use tool fallback in this order:
|
||||
1. list_resources
|
||||
2. read_resource
|
||||
3. search_patterns
|
||||
4. get_pattern_by_id
|
||||
5. get_skill_document_by_id
|
||||
5. Read the selected skill's `_manifest` only when supporting material is needed.
|
||||
6. If confidence is low, ask one clarifying question before editing.
|
||||
|
||||
Companion docs page: <optional-relative-doc-link>
|
||||
@@ -0,0 +1,90 @@
|
||||
---
|
||||
icon: lucide/messages-square
|
||||
prompt:
|
||||
version: "1.0.0"
|
||||
description: Extract a user-selected component from a JSFiddle page layout and implement it as a reusable NiceGUI render function.
|
||||
tags: [nicegui, components, frontend, refactoring, jsfiddle, prompts]
|
||||
arguments:
|
||||
component:
|
||||
description: Visible label, semantic role, or selector identifying the component.
|
||||
required: true
|
||||
source_layout:
|
||||
description: Optional source HTML and CSS.
|
||||
required: false
|
||||
target_location:
|
||||
description: Optional target NiceGUI page, module, or package.
|
||||
required: false
|
||||
behavior_requirements:
|
||||
description: Optional interactions, state, callbacks, or variations.
|
||||
required: false
|
||||
---
|
||||
|
||||
# NiceGUI Component Extraction
|
||||
|
||||
Extract one user-selected component from the output of the [JSFiddle Page Layout](../jsfiddle-page-layout/PROMPT.md) prompt and implement it as a reusable NiceGUI component in the target repository.
|
||||
|
||||
## Supplied Inputs
|
||||
|
||||
- `component`: {{component}}
|
||||
- `source_layout`: {{source_layout}}
|
||||
- `target_location`: {{target_location}}
|
||||
- `behavior_requirements`: {{behavior_requirements}}
|
||||
|
||||
## Inputs
|
||||
|
||||
1. `component`: required visible label, semantic role, or selector identifying the component to extract
|
||||
2. `source_layout`: optional HTML and CSS; when omitted, use the latest applicable JSFiddle page layout output in the conversation
|
||||
3. `target_location`: optional target page, module, or package; infer it from the repository when omitted
|
||||
4. `behavior_requirements`: optional interactions, state, callbacks, or content variations
|
||||
|
||||
If the selected component or source layout cannot be identified unambiguously, ask one concise clarification question before editing.
|
||||
|
||||
## Required References
|
||||
|
||||
Apply these references before implementation:
|
||||
|
||||
1. Package boundaries, dependency direction, and page or component ownership: [NiceGUI Application Architecture](../../skills/nicegui/references/architecture.md)
|
||||
2. Responsive layout, Quasar props, Tailwind utilities, and shared CSS: [NiceGUI Styling and Customization](../../skills/nicegui/references/styling-and-customization.md)
|
||||
3. Typed UI state, propagation, mutable defaults, binding strictness, and version checks: [Binding Dataclasses Deep Dive](../../skills/nicegui/references/binding-dataclasses.md)
|
||||
|
||||
## Workflow
|
||||
|
||||
1. Locate the selected region in the source HTML and CSS, including its responsive rules, states, and dependencies on surrounding layout.
|
||||
2. Inspect the target repository's NiceGUI version, package structure, component conventions, shared CSS loading, and nearest page call site.
|
||||
3. Define the smallest reusable API for the component:
|
||||
- name the public function `render_<component_name>` using snake_case
|
||||
- accept content, typed state, and event callbacks as explicit parameters
|
||||
- keep business rules, persistence, and service access outside the component
|
||||
- preserve an established return-value convention; otherwise return the component's root NiceGUI element
|
||||
4. Translate semantic HTML into native NiceGUI and Quasar elements. Do not embed the original page wholesale with `ui.html` when standard components express the structure.
|
||||
5. Recreate only the CSS needed by the extracted component:
|
||||
- use Quasar props for component appearance and behavior
|
||||
- use NiceGUI classes and Tailwind utilities for spacing, sizing, alignment, and responsive layout
|
||||
- use scoped shared CSS only where props and utilities are insufficient
|
||||
- do not override Quasar field internals or duplicate globally loaded styles
|
||||
6. Model editable or shared component state with a typed `@binding.bindable_dataclass` only when binding improves the interaction:
|
||||
- use `field(default_factory=...)` for mutable defaults
|
||||
- scope state to the appropriate page, client, or user
|
||||
- keep binding transforms pure and inexpensive
|
||||
- assign updated collections back to bound fields instead of relying on in-place mutation
|
||||
7. Integrate the render function at the nearest target page or call site without moving unrelated page composition or domain logic into the component.
|
||||
8. Preserve accessibility, focus behavior, text wrapping, stable dimensions, and the source layout's visual hierarchy.
|
||||
9. Run the narrowest available tests, lint, and type checks for the changed files. For visual components, verify representative mobile, landscape desktop, and portrait desktop viewports when browser tooling is available.
|
||||
|
||||
## Output Contract
|
||||
|
||||
Complete the implementation in the target repository, then report:
|
||||
|
||||
1. Files created or updated.
|
||||
2. The `render_*` function signature and its state or callback contract.
|
||||
3. Any deliberate visual or interaction differences from the JSFiddle source.
|
||||
4. Validation commands and outcomes, including viewport checks when performed.
|
||||
|
||||
## Quality Rules
|
||||
|
||||
1. Extract exactly the requested component and its necessary local dependencies.
|
||||
2. Prefer the target repository's established patterns over introducing a new abstraction style.
|
||||
3. Keep the component presentation-focused and reusable across pages with compatible data.
|
||||
4. Do not add a bindable dataclass for static content or event-local state that is clearer as ordinary parameters.
|
||||
5. Do not create a second component tree for mobile; use responsive classes and stable layout constraints.
|
||||
6. Keep custom CSS tokenized, scoped to the component, and loaded once by the application's composition layer.
|
||||
+26
-25
@@ -1,35 +1,36 @@
|
||||
---
|
||||
name: pytest-fill-scaffold
|
||||
description: Fill scaffolded pytest test methods with assertions, fixtures, and minimal test data while preserving concise test names and one-line intent docstrings.
|
||||
x-personal-mcp:
|
||||
id: pytest-fill-scaffold
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- pytest
|
||||
- testing
|
||||
- scaffolding
|
||||
- prompts
|
||||
capabilities:
|
||||
- resource://prompts/pytest-fill-scaffold/document
|
||||
arguments:
|
||||
target_files:
|
||||
description: Target test file paths under tests/.
|
||||
required: true
|
||||
stack:
|
||||
description: Runtime stack type for fixture and marker choices.
|
||||
required: true
|
||||
strategy:
|
||||
description: Balance between minimal and comprehensive implementation.
|
||||
required: false
|
||||
marker_lane:
|
||||
description: Preferred marker lane when applicable.
|
||||
required: false
|
||||
icon: lucide/messages-square
|
||||
prompt:
|
||||
version: "1.0.0"
|
||||
description: Fill scaffolded pytest methods with assertions, fixtures, and minimal test data while preserving reviewed structure.
|
||||
tags: [pytest, testing, scaffolding, prompts]
|
||||
arguments:
|
||||
target_files:
|
||||
description: Target test file paths under tests/.
|
||||
required: true
|
||||
stack:
|
||||
description: Runtime stack type for fixture and marker choices.
|
||||
required: true
|
||||
choices: [pure-python, fastapi, sqlalchemy-sync, sqlalchemy-async, mixed]
|
||||
strategy:
|
||||
description: Optional minimal or comprehensive implementation preference.
|
||||
required: false
|
||||
marker_lane:
|
||||
description: Optional pytest marker lane.
|
||||
required: false
|
||||
---
|
||||
|
||||
# Pytest Fill Scaffold
|
||||
|
||||
Use this prompt after test scaffolding exists and method names/docstrings are already in place.
|
||||
|
||||
## Supplied Inputs
|
||||
|
||||
- `target_files`: {{target_files}}
|
||||
- `stack`: {{stack}}
|
||||
- `strategy`: {{strategy}}
|
||||
- `marker_lane`: {{marker_lane}}
|
||||
|
||||
## Inputs
|
||||
|
||||
- Target test file(s) under tests/.
|
||||
+17
-16
@@ -1,28 +1,22 @@
|
||||
---
|
||||
name: pytest-scaffold
|
||||
description: Plan and optionally scaffold pytest file and class structure for selected Python modules while preserving concise behavior-focused test names and one-line intent docstrings.
|
||||
x-personal-mcp:
|
||||
id: pytest-scaffold
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- pytest
|
||||
- testing
|
||||
- scaffolding
|
||||
- prompts
|
||||
capabilities:
|
||||
- resource://prompts/pytest-scaffold/document
|
||||
icon: lucide/messages-square
|
||||
prompt:
|
||||
version: "1.0.0"
|
||||
description: Plan and optionally scaffold pytest file and class structure for selected Python modules.
|
||||
tags: [pytest, testing, scaffolding, prompts]
|
||||
arguments:
|
||||
target_modules:
|
||||
description: Target module path(s) under src/.
|
||||
description: Target module paths under src/.
|
||||
required: true
|
||||
mode:
|
||||
description: Execution mode, either plan-only or scaffold.
|
||||
description: Whether to plan only or create scaffold files.
|
||||
required: true
|
||||
choices: [plan-only, scaffold]
|
||||
path_strategy:
|
||||
description: Optional mapping preference for src to tests paths.
|
||||
description: Optional src-to-tests path mapping preference.
|
||||
required: false
|
||||
naming_style:
|
||||
description: Optional preference for concise method naming style.
|
||||
description: Optional concise test naming preference.
|
||||
required: false
|
||||
---
|
||||
|
||||
@@ -30,6 +24,13 @@ x-personal-mcp:
|
||||
|
||||
Use this prompt to consistently plan and scaffold pytest test modules for selected Python source modules.
|
||||
|
||||
## Supplied Inputs
|
||||
|
||||
- `target_modules`: {{target_modules}}
|
||||
- `mode`: {{mode}}
|
||||
- `path_strategy`: {{path_strategy}}
|
||||
- `naming_style`: {{naming_style}}
|
||||
|
||||
## Inputs
|
||||
|
||||
- Required:
|
||||
+24
-50
@@ -1,21 +1,6 @@
|
||||
---
|
||||
name: async-fastapi-sqlmodel
|
||||
description: 'Explain and apply async database principles for FastAPI, SQLAlchemy 2.x, and SQLModel. Use when: learning or reviewing AsyncEngine and AsyncSession lifecycles, FastAPI lifespan and yield dependencies, transaction boundaries, concurrency safety, implicit ORM I/O, AsyncExitStack, pooling, testing, or SQLModel integration.'
|
||||
x-personal-mcp:
|
||||
id: async-fastapi-sqlmodel
|
||||
version: 1.1.0
|
||||
tags:
|
||||
- fastapi
|
||||
- sqlalchemy
|
||||
- sqlmodel
|
||||
- async
|
||||
- asyncio
|
||||
- database
|
||||
- transactions
|
||||
- resource-lifecycle
|
||||
- architecture
|
||||
capabilities:
|
||||
- resource://skills/async-fastapi-sqlmodel/document
|
||||
description: 'Explain and apply async database principles for FastAPI, SQLAlchemy 2.x, and SQLModel. Use when: learning or reviewing cached AsyncEngine and session-factory lifecycles, AsyncSession scopes and injection, FastAPI lifespan and yield dependencies, transaction boundaries, concurrency safety, implicit ORM I/O, pooling, testing, or SQLModel integration.'
|
||||
---
|
||||
|
||||
# Async FastAPI, SQLAlchemy, and SQLModel
|
||||
@@ -24,6 +9,8 @@ Use this skill to explain how an async database layer works, why the recommended
|
||||
|
||||
Primary targets: PostgreSQL with asyncpg and SQLite with aiosqlite.
|
||||
|
||||
Engine and session mechanics mirror the [`nicegui-db` template repository](https://forgejo.john-stream.com/john/nicegui-db). Treat that template as the implementation baseline, then explain the rationale, lifecycle constraints, and tradeoffs behind its cached engines, session factories, context managers, dependency wiring, and `with_session` decorator. Source-specific claims in the references link to the reviewed template commit so behavior remains auditable as the template evolves.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Explain an async engine, session factory, session, connection, or transaction.
|
||||
@@ -51,7 +38,7 @@ Keep three ownership scopes distinct:
|
||||
|
||||
| Scope | Object | Purpose | Typical owner |
|
||||
|---|---|---|---|
|
||||
| Application process | `AsyncEngine` and `async_sessionmaker` | Dialect, connection pool, and repeatable session configuration | FastAPI lifespan |
|
||||
| Application process | Cached `AsyncEngine` and lifespan-owned `async_sessionmaker` | Dialect, connection pool, schema initialization, and repeatable session configuration | FastAPI lifespan |
|
||||
| Request or concurrent task | `AsyncSession` | Mutable ORM identity map and transactional state | A `yield` dependency or explicit unit of work |
|
||||
| Atomic operation | `SessionTransaction` | Commit all changes together or roll them back together | Service or use-case boundary |
|
||||
|
||||
@@ -61,16 +48,16 @@ The engine is a long-lived factory and pool, not a single database connection. T
|
||||
|
||||
### Match lifetime to ownership
|
||||
|
||||
- Create one `AsyncEngine` per process and database configuration in the normal case.
|
||||
- Dispose it explicitly in an awaitable shutdown path; garbage collection cannot reliably await async driver cleanup.
|
||||
- Configure `async_sessionmaker` once and call it to create short-lived sessions.
|
||||
- Resolve one cached `AsyncEngine` per database URL during the active application lifecycle.
|
||||
- Enter one owning engine scope per URL; initialize registered SQLModel metadata by default, then dispose the engine and clear cached resolution on exit.
|
||||
- Configure the application `async_sessionmaker` inside the engine lifecycle; use the template's cached factory resolver only for standalone helpers that cannot receive the application factory.
|
||||
- Close each session deterministically with `async with` or a FastAPI dependency that yields once.
|
||||
|
||||
See [engine lifecycle](references/engine.md) and [session management](references/session.md).
|
||||
|
||||
### Isolate mutable session state
|
||||
|
||||
An `AsyncSession` represents one stateful transaction in progress. Never use one session in multiple concurrent tasks, including branches of `asyncio.gather()`. Give each task its own session and pass sessions explicitly rather than relying on mutable scoped globals.
|
||||
An `AsyncSession` represents one stateful transaction in progress. Never use one session in multiple concurrent tasks, including branches of `asyncio.gather()`. Give each task its own session. Template-style `@with_session` functions inject one only when the `session` argument is omitted; a supplied session remains caller-owned, and explicit `None` is forwarded unchanged.
|
||||
|
||||
See [session management](references/session.md).
|
||||
|
||||
@@ -92,9 +79,9 @@ See [transaction boundaries](references/transactions.md).
|
||||
|
||||
FastAPI lifespan owns resources shared by many requests. A dependency with one `yield` owns request-scoped resources and runs cleanup after use. These are related context-manager mechanisms but solve different lifetime problems.
|
||||
|
||||
Use `AsyncExitStack` when lifespan acquires a variable, conditional, or mixed collection of context-managed resources. It records cleanup as resources are acquired and unwinds callbacks in reverse order. A single engine with one cleanup callback can use a plain `try/finally`; `AsyncExitStack` is a composition tool, not a requirement.
|
||||
Use `AsyncExitStack` when lifespan acquires a variable, conditional, or mixed collection of context-managed resources. It records cleanup as resources are acquired and unwinds callbacks in reverse order. A single engine should use the direct engine context manager; `AsyncExitStack` is a composition tool, not a requirement.
|
||||
|
||||
See [engine lifecycle](references/engine.md).
|
||||
See [FastAPI database integration](references/fastapi.md).
|
||||
|
||||
### Use SQLModel as the primary modeling layer
|
||||
|
||||
@@ -110,6 +97,14 @@ Pool sizing, overflow, recycle, pre-ping, isolation, statement timeouts, and hea
|
||||
|
||||
See [observability and resilience](references/observability.md).
|
||||
|
||||
### Test through the production seam
|
||||
|
||||
Keep the production engine and session-factory construction path intact in tests. Select a dedicated PostgreSQL, local SQLite, or in-memory SQLite URL at that seam, then override the request-session dependency only for the test lifetime. Use a test-scoped outer transaction with SAVEPOINT-backed session commits when application code calls `commit()`; it exercises normal transaction behavior while cleanup remains deterministic.
|
||||
|
||||
In-memory SQLite is suitable for serial tests. For multiple simultaneous sessions, use a named shared-cache SQLite URL or a temporary file, and retain PostgreSQL integration coverage for PostgreSQL-specific behavior.
|
||||
|
||||
See [database testing and fixture data](references/testing.md).
|
||||
|
||||
## Reference Map
|
||||
|
||||
| Concept | Reference |
|
||||
@@ -117,41 +112,19 @@ See [observability and resilience](references/observability.md).
|
||||
| Engine lifecycle and ownership | [Engine lifecycle reference](references/engine.md) |
|
||||
| Session factory and scope | [Session management reference](references/session.md) |
|
||||
| Transaction boundaries | [Transaction boundaries reference](references/transactions.md) |
|
||||
| Lifespan composition | [Engine lifecycle reference](references/engine.md) |
|
||||
| Dependency injection | [Session management reference](references/session.md) |
|
||||
| FastAPI lifespan composition | [FastAPI integration reference](references/fastapi.md) |
|
||||
| FastAPI dependency injection | [FastAPI integration reference](references/fastapi.md) |
|
||||
| Implicit I/O control in ORM | [Implicit I/O reference](references/implicit_io.md) |
|
||||
| Observability and resilience | [Observability reference](references/observability.md) |
|
||||
| SQLModel-first modeling | [SQLModel integration reference](references/sqlmodel.md) |
|
||||
| CRUD repository and standalone functions | [Basic CRUD reference](references/crud.md) |
|
||||
| Test database selection and fixture data | [Database testing reference](references/testing.md) |
|
||||
|
||||
## Canonical Composition Pattern
|
||||
|
||||
This example shows the ownership boundaries. Adapt state storage and dependency wiring to the application's conventions.
|
||||
The framework-independent primitives live in [engine lifecycle](references/engine.md), [session management](references/session.md), and [transaction boundaries](references/transactions.md). Their canonical FastAPI adaptation, including lifespan state and `Annotated` dependencies, lives in [FastAPI database integration](references/fastapi.md).
|
||||
|
||||
```python
|
||||
from contextlib import AsyncExitStack, asynccontextmanager
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
from fastapi import FastAPI
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||
async with AsyncExitStack() as stack:
|
||||
engine = create_async_engine(settings.database_url)
|
||||
stack.push_async_callback(engine.dispose)
|
||||
|
||||
session_factory = async_sessionmaker(engine, expire_on_commit=False)
|
||||
app.state.session_factory = session_factory
|
||||
yield
|
||||
|
||||
|
||||
async def get_session() -> AsyncIterator[AsyncSession]:
|
||||
async with app.state.session_factory() as session:
|
||||
yield session
|
||||
```
|
||||
|
||||
For direct construction without `AsyncExitStack`, put `await engine.dispose()` in a `finally` block. For background work that outlives a request, create a new session inside that task instead of retaining the request's session.
|
||||
For background work that outlives a request, inject the shared factory and create a new session inside that task instead of retaining the request's session.
|
||||
|
||||
## Explanation Procedure
|
||||
|
||||
@@ -173,6 +146,7 @@ When reviewing code, verify:
|
||||
- Relationship and deferred-column access cannot surprise the event loop with implicit I/O.
|
||||
- Pool and timeout settings are justified by deployment behavior.
|
||||
- Tests exercise rollback, cleanup, concurrency, and lifespan behavior where relevant.
|
||||
- Tests use a dedicated database target and preserve production session mechanics.
|
||||
|
||||
## Anti-Patterns to Flag
|
||||
|
||||
@@ -0,0 +1,294 @@
|
||||
# Basic CRUD Repository and Functions
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [SQLModel create-data tutorial](https://sqlmodel.tiangolo.com/tutorial/fastapi/multiple-models/)
|
||||
- [SQLModel update-data tutorial](https://sqlmodel.tiangolo.com/tutorial/fastapi/update-extra-data/)
|
||||
- [SQLModel select tutorial](https://sqlmodel.tiangolo.com/tutorial/select/)
|
||||
- [SQLAlchemy `AsyncSession` API](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.AsyncSession)
|
||||
- [`nicegui-db` service functions](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/services/my_table.py)
|
||||
|
||||
??? abstract "Decision metadata"
|
||||
- Status: adopted
|
||||
- Decision level: advisory
|
||||
- Applies to: api-runtime, workers, tests
|
||||
- Last reviewed: 2026-08-06
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Show a small SQLModel CRUD layer in two forms:
|
||||
|
||||
- independent functions for convenient standalone or composed operations;
|
||||
- a repository object that groups those functions behind one domain-oriented interface.
|
||||
|
||||
Template-style public functions use `@with_session` and accept an optional `AsyncSession`. When the argument is omitted, the decorator resolves the cached session factory and owns a short-lived session. When supplied, the function borrows the session without controlling its lifetime or transaction. The decorator does not commit, so standalone writes need a visible transaction strategy; repository methods remain explicit-session operations for predictable composition.
|
||||
|
||||
Use the same vocabulary at every layer:
|
||||
|
||||
| Operation | Function | Repository method | Scope when session is omitted | Missing-row result |
|
||||
|---|---|---|---|---|
|
||||
| Create | `create_widget()` | `create()` | Owned session; no implicit commit | Not applicable |
|
||||
| Read one | `get_widget()` | `get()` | Owned session | `None` |
|
||||
| Read many | `list_widgets()` | `list()` | Owned session | Empty list |
|
||||
| Update | `update_widget()` | `update()` | Owned session; no implicit commit | `None` |
|
||||
| Delete | `delete_widget()` | `delete()` | Owned session; no implicit commit | `None` |
|
||||
|
||||
Functions and repository methods both put domain arguments first. Database configuration and sessions are keyword-only infrastructure arguments. This keeps call sites analogous and makes ownership choices visible.
|
||||
|
||||
---
|
||||
|
||||
## Models
|
||||
|
||||
Start with one table model when the application does not need distinct persistence and API schemas.
|
||||
|
||||
```python
|
||||
from sqlmodel import Field
|
||||
from sqlmodel import SQLModel
|
||||
|
||||
|
||||
class Widget(SQLModel, table=True):
|
||||
id: int | None = Field(default=None, primary_key=True)
|
||||
name: str = Field(index=True)
|
||||
description: str | None = None
|
||||
```
|
||||
|
||||
This reference uses direct field arguments and full-update semantics to keep the CRUD mechanics visible. Introduce separate create, update, or public schemas only when an API boundary needs different validation, field visibility, or partial-update behavior. See [SQLModel integration](sqlmodel.md) for that larger modeling pattern.
|
||||
|
||||
---
|
||||
|
||||
## Independent CRUD Functions
|
||||
|
||||
Functions are the simplest default when grouping state or behavior in an object adds no value. Decorate public service functions when both standalone reads and explicit composition are useful. The assertion narrows the optional type after decorator injection and catches accidental explicit `None` calls.
|
||||
|
||||
```python
|
||||
from sqlmodel import select
|
||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||
|
||||
from .session import with_session
|
||||
|
||||
|
||||
@with_session
|
||||
async def create_widget(
|
||||
name: str,
|
||||
description: str | None = None,
|
||||
*,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget:
|
||||
assert session is not None, "Session must be provided by with_session decorator"
|
||||
widget = Widget(name=name, description=description)
|
||||
session.add(widget)
|
||||
await session.flush()
|
||||
return widget
|
||||
|
||||
|
||||
@with_session
|
||||
async def get_widget(
|
||||
widget_id: int,
|
||||
*,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget | None:
|
||||
assert session is not None, "Session must be provided by with_session decorator"
|
||||
return await session.get(Widget, widget_id)
|
||||
|
||||
|
||||
@with_session
|
||||
async def list_widgets(
|
||||
*,
|
||||
offset: int = 0,
|
||||
limit: int = 100,
|
||||
session: AsyncSession | None = None,
|
||||
) -> list[Widget]:
|
||||
assert session is not None, "Session must be provided by with_session decorator"
|
||||
if offset < 0:
|
||||
raise ValueError("offset must be non-negative")
|
||||
if not 1 <= limit <= 100:
|
||||
raise ValueError("limit must be between 1 and 100")
|
||||
|
||||
statement = select(Widget).order_by(Widget.id).offset(offset).limit(limit)
|
||||
return list(await session.scalars(statement))
|
||||
|
||||
|
||||
@with_session
|
||||
async def update_widget(
|
||||
widget_id: int,
|
||||
name: str,
|
||||
description: str | None,
|
||||
*,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget | None:
|
||||
assert session is not None, "Session must be provided by with_session decorator"
|
||||
widget = await session.get(Widget, widget_id)
|
||||
if widget is None:
|
||||
return None
|
||||
|
||||
widget.name = name
|
||||
widget.description = description
|
||||
await session.flush()
|
||||
return widget
|
||||
|
||||
|
||||
@with_session
|
||||
async def delete_widget(
|
||||
widget_id: int,
|
||||
*,
|
||||
session: AsyncSession | None = None,
|
||||
) -> Widget | None:
|
||||
assert session is not None, "Session must be provided by with_session decorator"
|
||||
widget = await session.get(Widget, widget_id)
|
||||
if widget is None:
|
||||
return None
|
||||
|
||||
await session.delete(widget)
|
||||
await session.flush()
|
||||
return widget
|
||||
```
|
||||
|
||||
Update and delete load the row through the same session that mutates it. This avoids accepting detached instances from an earlier standalone read and gives both operations an explicit `None` result that the application layer can map to a domain or HTTP error. Delete returns the loaded object for callers that need its values, but that object represents a row scheduled for deletion and must not be reused as persistent state. List operations validate their bounds and order by the primary key so pagination is deterministic. Add a unique tiebreaker whenever ordering by a non-unique field.
|
||||
|
||||
`flush()` sends pending writes and populates ordinary generated primary keys. It does not itself commit. A decorated write called without a session will therefore roll back when its owned session closes unless the function explicitly commits. Prefer passing a transaction-scoped session so several writes compose atomically. Use `await session.refresh(widget)` only when the operation deliberately needs database-generated state that was not returned during the flush; an unconditional refresh adds another query.
|
||||
|
||||
---
|
||||
|
||||
## Repository Object
|
||||
|
||||
A repository can provide a stable domain-facing interface when several callers need the same grouped operations. It remains stateless here: every method requires a session and delegates to the analogous function.
|
||||
|
||||
```python
|
||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||
|
||||
class WidgetRepository:
|
||||
async def create(
|
||||
self,
|
||||
session: AsyncSession,
|
||||
name: str,
|
||||
description: str | None = None,
|
||||
) -> Widget:
|
||||
return await create_widget(
|
||||
name,
|
||||
description,
|
||||
session=session,
|
||||
)
|
||||
|
||||
async def get(
|
||||
self,
|
||||
session: AsyncSession,
|
||||
widget_id: int,
|
||||
) -> Widget | None:
|
||||
return await get_widget(widget_id, session=session)
|
||||
|
||||
async def list(
|
||||
self,
|
||||
session: AsyncSession,
|
||||
*,
|
||||
offset: int = 0,
|
||||
limit: int = 100,
|
||||
) -> list[Widget]:
|
||||
return await list_widgets(
|
||||
offset=offset,
|
||||
limit=limit,
|
||||
session=session,
|
||||
)
|
||||
|
||||
async def update(
|
||||
self,
|
||||
session: AsyncSession,
|
||||
widget_id: int,
|
||||
name: str,
|
||||
description: str | None,
|
||||
) -> Widget | None:
|
||||
return await update_widget(
|
||||
widget_id,
|
||||
name,
|
||||
description,
|
||||
session=session,
|
||||
)
|
||||
|
||||
async def delete(
|
||||
self,
|
||||
session: AsyncSession,
|
||||
widget_id: int,
|
||||
) -> Widget | None:
|
||||
return await delete_widget(widget_id, session=session)
|
||||
```
|
||||
|
||||
The object is intentionally thin. Tests pass a transaction-scoped test session directly. The caller always owns that session and its transaction, and the repository never closes or commits it.
|
||||
|
||||
If a read participates in a later write, pass the same session and place both operations inside the explicit transaction. This avoids splitting one use case across sessions and keeps SQLAlchemy's autobegin behavior from obscuring transaction ownership. Add a repository only when its naming, shared query policy, dependency substitution, or domain boundary improves the application. Independent functions remain a valid and often clearer design.
|
||||
|
||||
---
|
||||
|
||||
## Transaction Ownership
|
||||
|
||||
Compose multiple calls under one use-case transaction. `db_transaction_scope()` owns the standalone engine, factory, session, and transaction lifetimes. Decorated CRUD functions detect the supplied session and borrow it; repository methods receive it directly.
|
||||
|
||||
```python
|
||||
from .session import db_transaction_scope
|
||||
|
||||
|
||||
async def replace_widget(
|
||||
repository: WidgetRepository,
|
||||
widget_id: int,
|
||||
replacement_name: str,
|
||||
replacement_description: str | None = None,
|
||||
) -> Widget | None:
|
||||
async with db_transaction_scope() as active_session:
|
||||
deleted_widget = await repository.delete(
|
||||
active_session,
|
||||
widget_id,
|
||||
)
|
||||
if deleted_widget is None:
|
||||
return None
|
||||
|
||||
return await repository.create(
|
||||
active_session,
|
||||
replacement_name,
|
||||
replacement_description,
|
||||
)
|
||||
```
|
||||
|
||||
If creation fails, deletion rolls back with it. Inside an already-running application, prefer `async with session_factory.begin()` or `async with session.begin()` over `db_transaction_scope()` so the application-owned engine and factory remain in use. Do not add direct `commit()` calls to CRUD functions or repository methods because that prevents callers from composing several operations atomically. See [transaction boundaries](transactions.md) and [session management](session.md) for ownership details.
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- Storing one mutable `AsyncSession` on a long-lived repository object.
|
||||
- Creating sessions manually inside functions already using `@with_session`.
|
||||
- Passing database configuration through every CRUD call instead of injecting a session at the data-access boundary.
|
||||
- Assuming decorator-owned write sessions commit on close.
|
||||
- Forwarding explicit `session=None` when decorator injection was intended.
|
||||
- Accepting unbounded list queries.
|
||||
- Accepting detached ORM instances for update or delete when an identifier can be resolved in the active session.
|
||||
- Accessing unloaded attributes after a standalone repository read has closed its owned session.
|
||||
|
||||
---
|
||||
|
||||
## Operational Checks
|
||||
|
||||
- Every CRUD call receives a task-local `AsyncSession`.
|
||||
- Standalone reads create and close a session at the service or application boundary.
|
||||
- Standalone reads may omit `session`; decorated writes receive a transaction-scoped session or explicitly own their commit policy.
|
||||
- Supplied write sessions remain caller-owned.
|
||||
- Each complete write operation declares a visible transaction boundary.
|
||||
- List operations have pagination and deterministic ordering where required.
|
||||
- Update requires values for both mutable fields; passing `None` explicitly clears the nullable description.
|
||||
- Get, update, and delete use the same identifier and missing-row semantics.
|
||||
- Decorated functions accept an optional keyword-only session; repository methods require one explicitly.
|
||||
- Standalone service reads load all state needed after their owned session closes.
|
||||
- Repository objects hold query policy when useful, never database configuration or request-scoped session state.
|
||||
|
||||
---
|
||||
|
||||
## Testing Checks
|
||||
|
||||
- Create tests verify generated identifiers and persisted field values after commit.
|
||||
- Get and list tests cover found, missing, pagination, and ordering behavior.
|
||||
- List tests reject negative offsets and limits outside the supported range.
|
||||
- Update tests cover replacement of both mutable fields, including clearing the nullable description.
|
||||
- Update and delete tests cover missing identifiers without mutating the database.
|
||||
- Delete tests verify the returned row and its absence after commit.
|
||||
- Failure tests verify that a surrounding transaction rolls back all composed CRUD calls.
|
||||
- Optional-session read tests verify borrowed sessions remain open and owned sessions close without committing.
|
||||
- Decorated write tests verify supplied transactions remain caller-owned and omitted sessions do not imply a commit.
|
||||
- Composition tests pass one active session through several CRUD calls and verify one atomic commit or rollback.
|
||||
@@ -0,0 +1,281 @@
|
||||
# Async SQLAlchemy Engine
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
|
||||
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||
- [SQLAlchemy connections](https://docs.sqlalchemy.org/en/21/core/connections.html)
|
||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||
- [SQLAlchemy pooling and multiprocessing](https://docs.sqlalchemy.org/en/21/core/pooling.html#pooling-multiprocessing)
|
||||
- [SQLAlchemy SQLite transaction control](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#enabling-non-legacy-sqlite-transactional-modes-with-the-sqlite3-or-aiosqlite-driver)
|
||||
- [SQLAlchemy SQLite foreign-key support](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#foreign-key-support)
|
||||
- [SQLite PRAGMA reference](https://www.sqlite.org/pragma.html)
|
||||
- [`nicegui-db` engine implementation](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/db/engine.py)
|
||||
|
||||
---
|
||||
|
||||
## Engine Ownership Model
|
||||
|
||||
Resolve one async engine for each database URL within an application, worker, command, or test lifecycle.
|
||||
|
||||
- SQLAlchemy guidance: the engine is intended as a long-lived, concurrent registry over pooled DB connections, not a per-operation object.
|
||||
- `get_engine(database_url)` owns URL-keyed engine construction and caching.
|
||||
- The composition root enters `engine_scope(database_url)` once and therefore owns initialization and disposal.
|
||||
- Services and repositories receive a session or session factory; they do not resolve an engine.
|
||||
|
||||
!!! tip "Practical rule"
|
||||
- Exactly one cached engine for each database URL during an active application-owned lifecycle.
|
||||
- Exactly one active owning `engine_scope()` for a given URL.
|
||||
- Zero `create_async_engine(...)` calls in feature code.
|
||||
- Zero engine lookup or disposal calls in repository code.
|
||||
|
||||
---
|
||||
|
||||
## Cached Engine Resolution
|
||||
|
||||
[`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) makes the database URL the engine identity:
|
||||
|
||||
```python
|
||||
from functools import cache
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
|
||||
|
||||
@cache
|
||||
def get_engine(database_url: str) -> AsyncEngine:
|
||||
engine = create_async_engine(database_url, pool_pre_ping=True)
|
||||
if engine.dialect.name == "sqlite":
|
||||
configure_aiosqlite_engine(engine)
|
||||
return engine
|
||||
```
|
||||
|
||||
Repeated calls with the same exact URL return the same `AsyncEngine`; different URLs produce independent cache entries. Construction configures the dialect and pool but normally does not open a database connection until the first operation. SQLite event listeners are installed only when a new cached engine is constructed, before its first connection.
|
||||
|
||||
Resolve settings into the final URL before calling `get_engine()`. Services and repositories should not call it directly: the cache controls construction identity, not ownership.
|
||||
|
||||
## Owning Engine Scope
|
||||
|
||||
Use one [`asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager) to pair cached resolution and optional schema initialization with disposal:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncGenerator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||
from sqlmodel import SQLModel
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def engine_scope(
|
||||
database_url: str,
|
||||
*,
|
||||
initialize: bool = True,
|
||||
) -> AsyncGenerator[AsyncEngine]:
|
||||
engine = get_engine(database_url)
|
||||
if initialize:
|
||||
await initialize_db(database_url)
|
||||
|
||||
try:
|
||||
yield engine
|
||||
finally:
|
||||
await dispose_engine(database_url)
|
||||
|
||||
|
||||
async def initialize_db(database_url: str) -> None:
|
||||
from . import models # noqa: F401
|
||||
|
||||
engine = get_engine(database_url)
|
||||
async with engine.begin() as connection:
|
||||
await connection.run_sync(SQLModel.metadata.create_all)
|
||||
|
||||
|
||||
async def dispose_engine(database_url: str) -> None:
|
||||
engine = get_engine(database_url)
|
||||
try:
|
||||
await engine.dispose()
|
||||
finally:
|
||||
get_engine.cache_clear()
|
||||
```
|
||||
|
||||
The code that enters `engine_scope()` owns the engine. It keeps that scope open for the complete application, worker, command, or test lifecycle and passes the yielded engine into session-factory construction. Successful and exceptional exits both dispose the pool and invalidate cached engine resolution.
|
||||
|
||||
Initialization imports the model package so every table is registered, then runs `SQLModel.metadata.create_all()` in `engine.begin()`. This is suitable for the template and focused tests. Use migrations instead when schema evolution is part of the deployment contract. Pass `initialize=False` only when another owner provisions the schema or a test is directly exercising construction without schema setup.
|
||||
|
||||
`dispose_engine()` clears the complete function cache, not only the requested URL. This matches the template and is safe under its intended single-database lifecycle. Applications that own several simultaneously active database URLs need per-key lifecycle management rather than this global invalidation behavior.
|
||||
|
||||
Workers, scripts, and other composition roots enter `database_scope()` directly:
|
||||
|
||||
```python
|
||||
async with database_scope(settings.database_url) as session_factory:
|
||||
await run_worker(session_factory)
|
||||
```
|
||||
|
||||
`database_scope()` is defined in [session management](session.md). It enters `engine_scope()` and creates the factory bound to the yielded engine.
|
||||
|
||||
Do not overlap two owning scopes for the same URL. Both resolve the same cached engine, and the first scope to exit disposes it and clears the cache while the other still refers to it. For several fixed databases, use one non-overlapping owner per URL and account for global cache invalidation; use [`AsyncExitStack`](https://docs.python.org/3/library/contextlib.html#contextlib.AsyncExitStack) only after adopting lifecycle semantics that support several simultaneous owners.
|
||||
|
||||
When directly testing engine construction or lifecycle behavior, enter `engine_scope()` in the test or fixture. Exiting the context disposes the engine even when the test fails and clears the cache for the next lifecycle.
|
||||
|
||||
See [FastAPI database integration](fastapi.md) for adapting `database_scope()` to application lifespan and dependency injection.
|
||||
|
||||
---
|
||||
|
||||
## Driver URLs (Project Requirement: asyncpg + aiosqlite)
|
||||
|
||||
Use SQLAlchemy async driver URLs:
|
||||
|
||||
- PostgreSQL: `postgresql+asyncpg://user:pass@host:5432/dbname`
|
||||
- SQLite: `sqlite+aiosqlite:///./app.db`
|
||||
|
||||
!!! warning "Driver compatibility"
|
||||
- Do not mix sync drivers, for example `psycopg2`, with `create_async_engine()`.
|
||||
- Keep URL construction centralized in settings/config, not in feature modules.
|
||||
|
||||
---
|
||||
|
||||
## SQLite Connection and Transaction Policy
|
||||
|
||||
SQLite settings do not form one indivisible bundle:
|
||||
|
||||
- `PRAGMA foreign_keys=ON` is a correctness requirement when the schema declares foreign keys. SQLite requires it on every connection, including the connection used by `metadata.create_all()`.
|
||||
- Disabling the driver's implicit `BEGIN` and emitting `BEGIN` from SQLAlchemy provides non-legacy transaction behavior for `aiosqlite`. This makes SELECT, DDL, and SAVEPOINT behavior participate in SQLAlchemy's transaction boundary consistently.
|
||||
- `PRAGMA busy_timeout` is a per-connection lock-wait policy. Choose the duration from the application's latency and contention requirements.
|
||||
- `PRAGMA journal_mode=WAL` is an optional file-database concurrency policy. WAL persists in the database file, cannot be enabled for an in-memory database, and is not a substitute for transaction control.
|
||||
|
||||
Install instance-level listeners exactly once, immediately after constructing an `aiosqlite` engine and before its first connection:
|
||||
|
||||
```python
|
||||
from sqlalchemy import event
|
||||
from sqlalchemy.engine import Connection
|
||||
from sqlalchemy.engine.interfaces import DBAPIConnection
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||
|
||||
|
||||
def configure_aiosqlite_engine(
|
||||
engine: AsyncEngine,
|
||||
*,
|
||||
busy_timeout_ms: int | None = 30_000,
|
||||
enable_wal: bool = False,
|
||||
) -> None:
|
||||
if engine.dialect.name != "sqlite" or engine.dialect.driver != "aiosqlite":
|
||||
raise ValueError("Expected a sqlite+aiosqlite engine")
|
||||
if busy_timeout_ms is not None and busy_timeout_ms < 0:
|
||||
raise ValueError("busy_timeout_ms must be non-negative")
|
||||
|
||||
@event.listens_for(engine.sync_engine, "connect")
|
||||
def configure_connection(dbapi_connection: DBAPIConnection, _: object) -> None:
|
||||
dbapi_connection.isolation_level = None
|
||||
cursor = dbapi_connection.cursor()
|
||||
try:
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
if busy_timeout_ms is not None:
|
||||
cursor.execute(f"PRAGMA busy_timeout={busy_timeout_ms}")
|
||||
if enable_wal:
|
||||
cursor.execute("PRAGMA journal_mode=WAL")
|
||||
journal_mode = cursor.fetchone()
|
||||
if journal_mode is None or journal_mode[0].lower() != "wal":
|
||||
raise RuntimeError("SQLite could not enable WAL mode")
|
||||
finally:
|
||||
cursor.close()
|
||||
|
||||
@event.listens_for(engine.sync_engine, "begin")
|
||||
def begin_transaction(connection: Connection) -> None:
|
||||
connection.exec_driver_sql("BEGIN")
|
||||
```
|
||||
|
||||
The `connect` listener receives the adapted synchronous DBAPI connection exposed by `engine.sync_engine`; event callbacks themselves are synchronous even though application queries use the async engine. Setting `isolation_level=None` and adding the `begin` listener are one transaction-control strategy and must remain paired. Do not combine this pair with SQLAlchemy's driver-level `AUTOCOMMIT` isolation mode.
|
||||
|
||||
The default above enables foreign keys and modern transaction boundaries for file and in-memory databases. Enable WAL only for a file-backed database after deciding that its read/write concurrency model is appropriate. Treat `30_000` as an example policy, not a universal default; `connect_args={"timeout": 30.0}` at engine construction is another way to configure the underlying SQLite lock timeout.
|
||||
|
||||
---
|
||||
|
||||
## Pooling Defaults and Tuning
|
||||
|
||||
Default behavior is usually correct first:
|
||||
|
||||
- Async engines use async-compatible pooling (`AsyncAdaptedQueuePool`) by default.
|
||||
- Start with defaults, then tune from observed load (`pool_size`, `max_overflow`, `pool_timeout`, `pool_recycle`).
|
||||
- Enable `pool_pre_ping=True` for safer stale-connection handling in long-running services.
|
||||
|
||||
When to switch pool strategy:
|
||||
|
||||
- `NullPool` if you explicitly need no pooling (special environments, some tests, or strict cross-loop constraints).
|
||||
- Keep in mind this increases connect/disconnect churn.
|
||||
|
||||
### When `StaticPool` Is Appropriate
|
||||
|
||||
Use [`StaticPool`](https://docs.sqlalchemy.org/en/21/core/pooling.html#sqlalchemy.pool.StaticPool) only when every checkout must reuse one DBAPI connection and all database access is serialized. Typical cases are:
|
||||
|
||||
- A serial test suite using a private in-memory SQLite database. The `sqlite+aiosqlite://` URL already selects `StaticPool` automatically, so specifying `poolclass=StaticPool` is normally redundant.
|
||||
- A narrowly scoped SQLite engine that must preserve connection-local state, such as temporary tables, across SQLAlchemy connection or session checkouts.
|
||||
|
||||
When explicit configuration is required:
|
||||
|
||||
```python
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
from sqlalchemy.pool import StaticPool
|
||||
|
||||
engine = create_async_engine(
|
||||
"sqlite+aiosqlite:///./test.db",
|
||||
poolclass=StaticPool,
|
||||
)
|
||||
```
|
||||
|
||||
`StaticPool` is not a general performance optimization or a way to make SQLite concurrent. All sessions share one underlying connection and its single transaction state, so one session's `COMMIT` or `ROLLBACK` can interfere with another session. Do not use it when several sessions or tasks may access the engine concurrently. For concurrent in-memory work, use a named shared-cache SQLite URL so pooled connections have independent transaction state, or use a temporary file database. See [SQLite test targets](testing.md#sqlite-targets) for those patterns.
|
||||
|
||||
---
|
||||
|
||||
## Disposal Semantics
|
||||
|
||||
`dispose_engine(database_url)` resolves the cached engine, awaits `engine.dispose()`, and clears the engine cache in a `finally` block. `engine.dispose()` replaces/disposes the pool, but only checked-in connections are immediately closed.
|
||||
|
||||
Rules:
|
||||
- Dispose when the app is shutting down.
|
||||
- Clear cached resolution even when disposal raises, so a later lifecycle cannot receive the failed engine object.
|
||||
- Dispose before reusing an engine across event loops.
|
||||
- In forked child-process initialization, use `engine.dispose(close=False)` (sync API guidance) so child processes do not touch parent-held connections.
|
||||
|
||||
Avoid relying on garbage collection for engine cleanup in async code.
|
||||
|
||||
---
|
||||
|
||||
## Event Loop and Process Boundaries
|
||||
|
||||
Do not share pooled connections across boundaries:
|
||||
|
||||
- Multiple event loops: do not reuse the same pooled async engine across loops unless you intentionally disable pooling (`NullPool`) or dispose before handoff.
|
||||
- Multiprocessing/fork: pooled connections must not be inherited for active use across process boundaries.
|
||||
|
||||
This prevents broken socket state and cross-process connection corruption.
|
||||
|
||||
---
|
||||
|
||||
## What Not to Do
|
||||
|
||||
- Create an engine inside each operation or unit of work.
|
||||
- Create/dispose engines inside repository methods.
|
||||
- Resolve an engine from repositories instead of injecting a session dependency.
|
||||
- Keep engine creation as a hidden side effect of import-time module globals.
|
||||
- Keep a session factory alive after its bound engine scope exits.
|
||||
- Enter overlapping engine scopes for the same cached URL.
|
||||
- Treat `cache_clear()` as per-URL invalidation when it clears every cached engine.
|
||||
- Use `metadata.create_all()` as a substitute for required production migrations.
|
||||
- Install the same SQLite event listeners more than once on one engine.
|
||||
- Enable WAL blindly for in-memory SQLite or treat a busy timeout as a concurrency guarantee.
|
||||
|
||||
---
|
||||
|
||||
## Engine Design Checklist
|
||||
|
||||
- One cached engine per exact database URL during an active lifecycle.
|
||||
- One owning engine scope per URL, with no overlapping owners.
|
||||
- Cached resolution, optional initialization, disposal, and cache invalidation follow one framework-independent lifecycle.
|
||||
- The composition root enters the database scope once and keeps it open until shutdown.
|
||||
- Session factory created inside, and never outlives, its engine scope.
|
||||
- Model registration occurs before `metadata.create_all()` when initialization is enabled.
|
||||
- Async driver URL matches backend (`asyncpg` or `aiosqlite`).
|
||||
- `aiosqlite` foreign-key and transaction listeners installed once before first use.
|
||||
- WAL enabled only as an explicit policy for a file-backed SQLite database.
|
||||
- Pooling strategy is explicit for non-default needs.
|
||||
- No feature-path engine creation.
|
||||
- Tests enter the same scope and receive deterministic disposal plus cache cleanup.
|
||||
@@ -0,0 +1,192 @@
|
||||
# FastAPI Database Integration
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
||||
- [FastAPI dependencies with `yield`](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
|
||||
- [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||
- [`nicegui-db` application lifespan](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/ui/app.py)
|
||||
- [`nicegui-db` database dependencies](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/ui/dependency.py)
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Connect the framework-independent database tools to FastAPI:
|
||||
|
||||
- lifespan enters one application-owned `database_scope()`,
|
||||
- application state holds settings and the resulting session factory,
|
||||
- dependencies create one session per request,
|
||||
- `Annotated` aliases make route ownership concise and explicit.
|
||||
|
||||
The underlying resource and transaction rules remain in [engine lifecycle](engine.md), [session management](session.md), and [transaction boundaries](transactions.md).
|
||||
|
||||
---
|
||||
|
||||
## Lifespan Ownership
|
||||
|
||||
Enter `database_scope()` once for the complete application lifecycle. Store the session factory, not the engine, because request code needs sessions rather than direct pool access:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncGenerator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
|
||||
from .config import Settings
|
||||
from .config import get_database_url
|
||||
from .db import database_scope
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(settings: Settings, app: FastAPI) -> AsyncGenerator[None]:
|
||||
app.state.settings = settings
|
||||
db_url = get_database_url(settings)
|
||||
|
||||
try:
|
||||
async with database_scope(db_url) as session_factory:
|
||||
app.state.session_factory = session_factory
|
||||
yield
|
||||
finally:
|
||||
del app.state.settings
|
||||
del app.state.session_factory
|
||||
```
|
||||
|
||||
The application factory binds `settings` to lifespan, for example with `partial(lifespan, settings)`. Lifespan does not construct resources per request. It enters the same framework-independent scope used by scripts, workers, and tests, keeps that scope open while requests are served, and lets it dispose the engine and clear cached engine resolution during shutdown.
|
||||
|
||||
The template's unconditional `del app.state.session_factory` mirrors an expected successful startup. If `database_scope()` raises before assignment, cleanup can raise `AttributeError` and obscure the startup error. A production hardening option is to assign a sentinel before the `try` or delete conditionally; that changes failure behavior and is not part of the exact template mechanics.
|
||||
|
||||
Only store the engine too when application-level code genuinely needs direct Core operations, pool instrumentation, or engine-specific diagnostics. Routes and repositories should normally receive an `AsyncSession`.
|
||||
|
||||
---
|
||||
|
||||
## Session Factory Dependency
|
||||
|
||||
A synchronous dependency retrieves the already-created factory from application state:
|
||||
|
||||
```python
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import Depends
|
||||
from fastapi import Request
|
||||
|
||||
from .session import SessionFactory
|
||||
|
||||
|
||||
def _get_session_factory(request: Request) -> SessionFactory:
|
||||
return request.app.state.session_factory
|
||||
|
||||
|
||||
type SessionFactoryDep = Annotated[SessionFactory, Depends(_get_session_factory)]
|
||||
```
|
||||
|
||||
`Depends()` does not create or cache a factory here. It only exposes the lifespan-owned object. This function is also the narrow seam that tests can override when they need a different factory.
|
||||
|
||||
---
|
||||
|
||||
## Request Session Dependencies
|
||||
|
||||
Use a session-only dependency for reads and other request conversations that must not commit implicitly:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncGenerator
|
||||
|
||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||
|
||||
|
||||
async def _get_session(session_factory: SessionFactoryDep) -> AsyncGenerator[AsyncSession]:
|
||||
async with session_factory() as owned_session:
|
||||
yield owned_session
|
||||
|
||||
|
||||
type SessionDep = Annotated[AsyncSession, Depends(_get_session)]
|
||||
```
|
||||
|
||||
The dependency creates and closes one session per request. Closing rolls back any unfinished autobegun transaction; it does not commit.
|
||||
|
||||
---
|
||||
|
||||
## Route Usage
|
||||
|
||||
Read route:
|
||||
|
||||
```python
|
||||
@router.get("/items/{item_id}")
|
||||
async def get_item(item_id: int, session: SessionDep) -> Item | None:
|
||||
return await find_item(session, item_id)
|
||||
```
|
||||
|
||||
Write route:
|
||||
|
||||
```python
|
||||
@router.post("/items")
|
||||
async def create_item(payload: ItemCreate, session: SessionDep) -> Item:
|
||||
async with session.begin():
|
||||
return await insert_item(session, payload)
|
||||
```
|
||||
|
||||
The template exposes only `SessionDep`; it does not hide commit behavior in dependency teardown. Choose one visible write convention per application:
|
||||
|
||||
- place `async with session.begin():` around a complete write unit, which commits on success and rolls back on exception; or
|
||||
- call `await session.commit()` explicitly after all writes when the route is the complete unit, as the template's simple UI action does.
|
||||
|
||||
The context-manager form scales better to several statements and makes exception rollback visible. Direct `commit()` is concise but requires the route to preserve the single-commit invariant and handle any recovery needs. Do not combine both conventions in one route. Lower-level data-access functions receive the existing session and remain unaware of FastAPI.
|
||||
|
||||
---
|
||||
|
||||
## Background Work
|
||||
|
||||
A request session belongs to that request and must not be retained by a background task. Inject or otherwise provide the application session factory, then create a new session inside the task:
|
||||
|
||||
```python
|
||||
async def run_background_job(session_factory: SessionFactory) -> None:
|
||||
async with session_factory.begin() as session:
|
||||
await process_pending_items(session)
|
||||
```
|
||||
|
||||
If work must survive application shutdown, it needs an independently owned worker lifecycle rather than the FastAPI lifespan-owned factory.
|
||||
|
||||
---
|
||||
|
||||
## Testing and Overrides
|
||||
|
||||
Override the narrow dependency that matches the test objective:
|
||||
|
||||
- Override `_get_session_factory` to preserve production request-session behavior with a test factory.
|
||||
- Override `_get_session` when a test must inject one transaction-scoped session directly.
|
||||
- Verify each lifespan receives a fresh engine and session factory and removes application state during teardown.
|
||||
- Remove overrides during teardown so mutable application state does not leak between tests.
|
||||
|
||||
```python
|
||||
app.dependency_overrides[_get_session] = get_test_session
|
||||
try:
|
||||
yield app
|
||||
finally:
|
||||
app.dependency_overrides.pop(_get_session, None)
|
||||
```
|
||||
|
||||
See [database testing](testing.md) for outer transactions, SAVEPOINT-backed fixtures, and database target selection.
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- Creating an engine or session factory in a request dependency.
|
||||
- Reading settings and constructing database resources from repositories.
|
||||
- Storing one mutable `AsyncSession` on `app.state`.
|
||||
- Sharing a request session with concurrent or background tasks.
|
||||
- Assuming `SessionDep` commits when dependency cleanup runs.
|
||||
- Keeping `app.state.session_factory` after its `database_scope()` exits.
|
||||
- Using deprecated startup and shutdown event handlers alongside lifespan.
|
||||
|
||||
---
|
||||
|
||||
## Integration Checklist
|
||||
|
||||
- Lifespan enters exactly one `database_scope()` for each application lifecycle.
|
||||
- Application state stores the yielded session factory.
|
||||
- Session dependencies create and close one session per request.
|
||||
- The session dependency owns request session closure but not commit behavior.
|
||||
- Routes use `Annotated` aliases and receive sessions, not engines.
|
||||
- Background tasks create their own sessions from a still-live factory.
|
||||
- Tests override and restore dependencies deterministically.
|
||||
+6
-4
@@ -8,13 +8,15 @@ Purpose: concept registry for the principles, mechanics, and implementation guid
|
||||
|
||||
| Concept | File | Status | Decision Level | Owner | Last Reviewed |
|
||||
|---|---|---|---|---|---|
|
||||
| Engine lifecycle and ownership | [engine.md](engine.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
||||
| Session factory and scope | [session.md](session.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
||||
| Engine lifecycle and ownership | [engine.md](engine.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||
| Session factory and scope | [session.md](session.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||
| FastAPI lifespan and dependency injection | [fastapi.md](fastapi.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||
| Transaction boundaries | [transactions.md](transactions.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
||||
| Implicit ORM I/O under asyncio | [implicit_io.md](implicit_io.md) | adopted | advisory | platform/backend | 2026-06-17 |
|
||||
| Observability and resilience | [observability.md](observability.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
||||
| SQLModel modeling and async boundaries | [sqlmodel.md](sqlmodel.md) | adopted | mandatory | platform/backend | 2026-07-26 |
|
||||
| Basic CRUD repository and functions | [crud.md](crud.md) | adopted | advisory | platform/backend | 2026-07-26 |
|
||||
| SQLModel modeling and async boundaries | [sqlmodel.md](sqlmodel.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||
| Basic CRUD repository and functions | [crud.md](crud.md) | adopted | advisory | platform/backend | 2026-08-06 |
|
||||
| Test database targets and fixture data | [testing.md](testing.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||
|
||||
---
|
||||
|
||||
@@ -0,0 +1,404 @@
|
||||
# Async SQLAlchemy Session Management
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
|
||||
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||
- [Python `inspect.signature`](https://docs.python.org/3/library/inspect.html#inspect.signature)
|
||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||
- [SQLAlchemy session basics](https://docs.sqlalchemy.org/en/21/orm/session_basics.html)
|
||||
- [`nicegui-db` session implementation](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/db/session.py)
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Define one canonical session model for SQLAlchemy asyncio:
|
||||
|
||||
- configure a lifespan-owned factory or resolve a URL-keyed cached factory,
|
||||
- create one AsyncSession per task or unit of work,
|
||||
- let callers supply a session when they already own the scope,
|
||||
- never share one AsyncSession across concurrent tasks.
|
||||
|
||||
---
|
||||
|
||||
## Scope and Non-Goals
|
||||
|
||||
- In scope: session factory creation, task scoping, and transaction demarcation.
|
||||
- Out of scope: framework dependency wiring, ORM model design, query optimization strategy, and schema migration tooling.
|
||||
|
||||
---
|
||||
|
||||
## Rules
|
||||
|
||||
- Create the application `async_sessionmaker` inside `database_scope()` and store it in application state for request dependencies.
|
||||
- Use `get_session_factory(db_url)` and `resolve_session_factory()` for standalone decorated operations that do not receive the application factory.
|
||||
- Use a fresh AsyncSession for each task or explicit unit of work.
|
||||
- Let reusable service functions accept `AsyncSession | None` and apply `@with_session` when standalone invocation is useful.
|
||||
- Pass an `AsyncSession` directly when composing several calls under one caller-owned scope.
|
||||
- Borrow a caller-provided session without beginning, closing, committing, or rolling it back.
|
||||
- Do not share AsyncSession across `asyncio.gather()` or parallel tasks.
|
||||
- Prefer direct dependency injection over global scoped-session patterns in new code.
|
||||
- Use explicit transaction boundaries (`async with session.begin():`) for writes.
|
||||
- Use `db_transaction_scope()` when a standalone operation must own engine, factory, session, and transaction lifetimes together.
|
||||
- Use `begin_nested()` directly and only when partial rollback through a database SAVEPOINT is required.
|
||||
|
||||
---
|
||||
|
||||
## Sessions and Transactions
|
||||
|
||||
A session and a transaction solve related but different problems:
|
||||
|
||||
| Concept | Responsibility | Typical lifetime |
|
||||
| --- | --- | --- |
|
||||
| `AsyncSession` | Provides the ORM workspace: executes queries, tracks loaded and changed objects in its identity map, and flushes pending changes. It also coordinates access to a database connection. | One task or explicit unit of work. |
|
||||
| Transaction | Defines the atomic database boundary: all work inside it commits together on success or rolls back together on failure. | One complete operation that must have a single outcome. |
|
||||
|
||||
A transaction belongs to a session; it is not an alternative to one. The session is the interface used by application and data-access code, while the transaction determines when that work becomes permanent. A session may coordinate sequential transactions during its lifetime, although short-lived application scopes commonly use one session for one transaction.
|
||||
|
||||
Use a session without a helper-owned commit boundary for independent reads or lower-level functions that must participate in whatever transaction their caller controls:
|
||||
|
||||
```python
|
||||
async with session_factory() as session:
|
||||
item = await find_item(session, item_id)
|
||||
```
|
||||
|
||||
Use an explicit transaction for writes, read-modify-write operations, or several statements that must succeed or fail as one unit:
|
||||
|
||||
```python
|
||||
async with session_factory.begin() as session:
|
||||
order = await create_order(session, order_data)
|
||||
await reserve_inventory(session, order)
|
||||
```
|
||||
|
||||
SQLAlchemy sessions use [autobegin](https://docs.sqlalchemy.org/en/21/orm/session_basics.html#auto-begin), so the first database operation normally starts a transaction even for a read. Therefore, “session-only” means that the surrounding helper owns only session lifetime and does not promise to commit; it does not mean that no database transaction exists. Closing such a session releases its resources and rolls back any unfinished transaction. An explicit `begin()` is valuable when application code must make the atomic boundary and commit ownership visible.
|
||||
|
||||
For most read-only operations, a session context is sufficient. Use an explicit transaction for reads when they need a defined consistency boundary, participate in a larger atomic operation, or use locking such as `SELECT ... FOR UPDATE`.
|
||||
|
||||
---
|
||||
|
||||
## Session Factory Mechanics
|
||||
|
||||
An `async_sessionmaker[AsyncSession]` is a reusable configuration object and callable session producer. It stores how sessions should be created, including the engine binding and options such as `expire_on_commit=False`. It is not itself a session, connection, or transaction, and calling it does not make a shared global `AsyncSession`.
|
||||
|
||||
The template exposes two construction paths with the same session options.
|
||||
|
||||
The application-owned path creates a factory inside the engine lifecycle:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncGenerator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||
|
||||
from .engine import engine_scope
|
||||
|
||||
type SessionFactory = async_sessionmaker[AsyncSession]
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def database_scope(
|
||||
db_url: str,
|
||||
*,
|
||||
auto_flush: bool = True,
|
||||
) -> AsyncGenerator[SessionFactory]:
|
||||
async with engine_scope(db_url) as engine:
|
||||
yield async_sessionmaker(
|
||||
bind=engine,
|
||||
class_=AsyncSession,
|
||||
expire_on_commit=False,
|
||||
autoflush=auto_flush,
|
||||
)
|
||||
```
|
||||
|
||||
FastAPI lifespan enters this path once and stores the yielded factory on application state. The factory must not outlive the scope because its bound engine is disposed on exit.
|
||||
|
||||
The standalone path caches a factory by URL and `auto_flush` policy:
|
||||
|
||||
```python
|
||||
from functools import cache
|
||||
|
||||
from .engine import get_engine
|
||||
|
||||
|
||||
@cache
|
||||
def get_session_factory(
|
||||
db_url: str,
|
||||
*,
|
||||
auto_flush: bool = True,
|
||||
) -> SessionFactory:
|
||||
return async_sessionmaker(
|
||||
bind=get_engine(db_url),
|
||||
class_=AsyncSession,
|
||||
expire_on_commit=False,
|
||||
autoflush=auto_flush,
|
||||
)
|
||||
|
||||
|
||||
def resolve_session_factory(settings: Settings | None = None) -> SessionFactory:
|
||||
settings = settings or get_settings()
|
||||
db_url = get_database_url(settings)
|
||||
return get_session_factory(db_url)
|
||||
```
|
||||
|
||||
This path lets framework-independent helpers resolve one stable factory without receiving it through every call. The tradeoff is hidden configuration resolution and a second lifecycle mechanism. `dispose_engine()` clears `get_engine`'s cache but does not clear `get_session_factory`'s cache in the template. A cached factory remains bound to the disposed engine object; SQLAlchemy can create a new pool when that engine is used again, but a later `database_scope()` for the same URL can own a different engine. Treat cached standalone resolution as process-lifetime convenience, avoid repeated application lifecycles in one process, and clear both caches together if the template evolves to support them.
|
||||
|
||||
Each call to `session_factory()` creates a distinct `AsyncSession`. The caller that invokes the factory owns that session lifetime and must close it, normally with `async with`:
|
||||
|
||||
```python
|
||||
async with session_factory() as session:
|
||||
...
|
||||
```
|
||||
|
||||
The factory can be shared across operations and tasks. Sessions produced by it cannot be shared across concurrent tasks.
|
||||
|
||||
Passing the application factory directly has three useful consequences:
|
||||
|
||||
- Lower layers do not resolve settings or global resources.
|
||||
- Tests can inject a test factory directly through `session_scope(session_factory=...)` or FastAPI state.
|
||||
- Transaction ownership remains independent of engine construction.
|
||||
|
||||
---
|
||||
|
||||
## Database and Convenience Scopes
|
||||
|
||||
The template provides three framework-independent context managers:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncGenerator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def db_session_scope(
|
||||
db_url: str | None = None,
|
||||
) -> AsyncGenerator[AsyncSession]:
|
||||
db_url = db_url or resolve_database_url()
|
||||
async with database_scope(db_url) as session_factory, session_factory() as session:
|
||||
yield session
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def db_transaction_scope(
|
||||
db_url: str | None = None,
|
||||
) -> AsyncGenerator[AsyncSession]:
|
||||
db_url = db_url or resolve_database_url()
|
||||
async with database_scope(db_url) as session_factory, session_factory.begin() as session:
|
||||
yield session
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def session_scope(
|
||||
*,
|
||||
settings: Settings | None = None,
|
||||
session_factory: SessionFactory | None = None,
|
||||
session: AsyncSession | None = None,
|
||||
) -> AsyncGenerator[AsyncSession]:
|
||||
if session is not None:
|
||||
yield session
|
||||
return
|
||||
|
||||
session_factory = session_factory or resolve_session_factory(settings=settings)
|
||||
async with session_factory() as owned_session:
|
||||
yield owned_session
|
||||
```
|
||||
|
||||
`db_session_scope()` owns a complete temporary database lifecycle and a session but does not commit. `db_transaction_scope()` owns the same resources plus a root transaction that commits on successful exit and rolls back on exception. Both initialize the schema by default because `database_scope()` enters `engine_scope()` with its default `initialize=True`. They are appropriate for scripts, commands, and isolated operations, not per-request use inside an already-running application.
|
||||
|
||||
`session_scope()` is the borrow-or-create helper. Its precedence is supplied session, supplied factory, then settings-based cached factory resolution. A supplied session remains entirely caller-owned; the helper does not require an active transaction and does not begin, commit, roll back, or close it. An owned session is closed on exit, and unfinished autobegun work rolls back.
|
||||
|
||||
Passing `session=None` is the same as omitting the session for `session_scope()` and therefore creates a session. This differs from `with_session`, which tests whether the argument name was bound rather than whether its value is non-null.
|
||||
|
||||
## Signature-Aware Session Injection
|
||||
|
||||
`with_session` allows one async function to support standalone calls and explicit composition:
|
||||
|
||||
```python
|
||||
from collections.abc import Awaitable
|
||||
from collections.abc import Callable
|
||||
from functools import wraps
|
||||
from inspect import signature
|
||||
|
||||
|
||||
def with_session[**P, R](
|
||||
func: Callable[P, Awaitable[R]],
|
||||
) -> Callable[P, Awaitable[R]]:
|
||||
sig = signature(func)
|
||||
|
||||
@wraps(func)
|
||||
async def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
|
||||
bound = sig.bind_partial(*args, **kwargs)
|
||||
|
||||
if "session" in bound.arguments:
|
||||
return await func(*args, **kwargs)
|
||||
|
||||
async with resolve_session_factory()() as session:
|
||||
bound.arguments["session"] = session
|
||||
return await func(*bound.args, **bound.kwargs)
|
||||
|
||||
return wrapper
|
||||
```
|
||||
|
||||
The function must be async and expose a parameter named exactly `session`. The decorator preserves metadata with `wraps()`, binds positional and keyword arguments through the original signature, and injects a fresh session only when the caller omitted that argument.
|
||||
|
||||
The distinction between omitted and explicit `None` is deliberate in the implementation:
|
||||
|
||||
- `await operation()` injects and owns a session.
|
||||
- `await operation(session=existing_session)` borrows the caller's session.
|
||||
- `await operation(None)` or `await operation(session=None)` forwards `None` without injection.
|
||||
|
||||
The decorated function therefore types the parameter as `AsyncSession | None = None` but should assert or guard after decoration. Explicit `None` is not a request for injection. This preserves ordinary Python call binding, but it means wrappers or callers must omit the argument instead of forwarding a nullable value.
|
||||
|
||||
`with_session` owns session lifetime only. It does not begin or commit a transaction, so it is naturally suited to reads. Decorated writes must either manage a visible transaction or be called with a session from `db_transaction_scope()` or another caller-owned transaction. Prefer explicit factory or session injection when lifecycle transparency and test substitution matter more than call-site convenience.
|
||||
|
||||
---
|
||||
|
||||
## Function and Service Boundaries
|
||||
|
||||
Template service functions support both standalone and composed use by combining `@with_session` with an optional parameter:
|
||||
|
||||
```python
|
||||
from sqlmodel import func
|
||||
from sqlmodel import select
|
||||
|
||||
|
||||
@with_session
|
||||
async def count_items(session: AsyncSession | None = None) -> int:
|
||||
assert session is not None, "Session must be provided by with_session decorator"
|
||||
result = await session.exec(select(func.count()).select_from(Item))
|
||||
return result.one()
|
||||
```
|
||||
|
||||
The standalone call injects and closes a session:
|
||||
|
||||
```python
|
||||
count = await count_items()
|
||||
```
|
||||
|
||||
A larger use case passes one caller-owned session through several decorated functions:
|
||||
|
||||
```python
|
||||
async with session_factory.begin() as session:
|
||||
count = await count_items(session)
|
||||
await create_item(payload, session=session)
|
||||
```
|
||||
|
||||
The decorator sees the bound `session` argument and leaves all ownership with the caller. It never creates a SAVEPOINT or nested transaction.
|
||||
|
||||
For low-level helpers that should never resolve settings, require a non-optional session and leave them undecorated. Application service objects may store the immutable session factory, but they must not store a mutable session:
|
||||
|
||||
```python
|
||||
class ItemService:
|
||||
def __init__(self, session_factory: SessionFactory) -> None:
|
||||
self.session_factory = session_factory
|
||||
|
||||
async def find(self, item_id: int) -> Item | None:
|
||||
async with self.session_factory() as session:
|
||||
return await find_item(session, item_id)
|
||||
```
|
||||
|
||||
Code that already owns a transaction should call the session-required function directly. Repositories should normally remain in that session-required layer; the service or use-case boundary owns standalone session creation. This avoids optional-session APIs spreading into every data-access function.
|
||||
|
||||
---
|
||||
|
||||
## SAVEPOINTs and Partial Failure
|
||||
|
||||
Use [`begin_nested()`](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#using-savepoint) only when failure inside one portion of an operation should roll back that portion while preserving the outer transaction:
|
||||
|
||||
```python
|
||||
async with db_transaction_scope() as session:
|
||||
order = await insert_order(session, payload)
|
||||
|
||||
try:
|
||||
async with session.begin_nested():
|
||||
await apply_optional_discount(session, order)
|
||||
except DiscountError:
|
||||
pass
|
||||
|
||||
await reserve_inventory(session, order)
|
||||
```
|
||||
|
||||
Important SAVEPOINT semantics:
|
||||
|
||||
- `begin_nested()` starts a root transaction if one is not already active, so call it inside a visible outer transaction when that ownership matters.
|
||||
- Entering `begin_nested()` unconditionally flushes pending session state, regardless of the `autoflush` setting.
|
||||
- Successful exit releases the SAVEPOINT; it does not commit the outer transaction.
|
||||
- Exceptional exit rolls back to the SAVEPOINT and leaves the outer transaction active.
|
||||
- In SQLAlchemy 2.x, `session.commit()` commits the outermost transaction. Never call it to release a SAVEPOINT; let the nested context manager manage its transaction handle.
|
||||
|
||||
Do not create a SAVEPOINT merely because one service calls another. SAVEPOINTs add database work and alter flush and error-recovery behavior. Use them only for explicit partial-failure requirements such as skipping one conflicting row while retaining the rest of a batch.
|
||||
|
||||
---
|
||||
|
||||
## Framework Integration
|
||||
|
||||
Keep framework adapters outside these session primitives. See [FastAPI database integration](fastapi.md) for lifespan ownership, `Annotated` dependency aliases, and read-versus-write request sessions.
|
||||
|
||||
---
|
||||
|
||||
## Configuration Guidance
|
||||
|
||||
- `expire_on_commit=False` is commonly preferred in asyncio applications to reduce accidental post-commit reload behavior.
|
||||
- `AsyncSession.refresh()` is preferred over broad expiration patterns when state refresh is needed.
|
||||
- [`async_sessionmaker.begin()`](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin) is a concise option when one scope must create a session, begin a transaction, commit on success, roll back on failure, and close. Do not use it when borrowing a caller's session.
|
||||
|
||||
## SQLModel Alignment
|
||||
|
||||
- Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per task or unit of work.
|
||||
- SQLModel does not replace SQLAlchemy async lifecycle primitives; it provides model declaration, validation, and typing ergonomics on top of them.
|
||||
- Do not mix ad hoc session construction with the canonical session factory.
|
||||
|
||||
---
|
||||
|
||||
## Concurrency Rules
|
||||
|
||||
- One session per concurrent task.
|
||||
- If work fans out into parallel tasks, each task receives its own AsyncSession.
|
||||
- Pass sessions explicitly to service functions; avoid mutable global session state.
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- A singleton/global AsyncSession reused across tasks or operations.
|
||||
- Sharing one AsyncSession across parallel tasks.
|
||||
- Passing an application-global AsyncSession to a repository constructor.
|
||||
- Creating a new `async_sessionmaker` in each operation.
|
||||
- Retaining a session factory after its bound engine scope exits.
|
||||
- Using cached standalone factory resolution when the application factory is already available.
|
||||
- Assuming `with_session` starts or commits a transaction.
|
||||
- Forwarding `session=None` to a decorated function when injection was intended.
|
||||
- Closing or committing a session supplied by the caller.
|
||||
- Silently starting or committing a transaction on a supplied session.
|
||||
- Creating a SAVEPOINT for ordinary nested service calls.
|
||||
- Hiding root transaction, joined transaction, and SAVEPOINT behavior behind one mode-driven `atomic_scope()` helper.
|
||||
- Calling `session.commit()` inside a SAVEPOINT scope.
|
||||
- Mixing commit/rollback ownership across layers without a declared boundary.
|
||||
|
||||
---
|
||||
|
||||
## Operational Checks
|
||||
|
||||
- The FastAPI application factory is created inside `database_scope()` and does not outlive its bound engine.
|
||||
- Cached standalone factories are used only where application-state injection is unavailable.
|
||||
- `session_scope()` precedence is supplied session, supplied factory, then settings-based resolution.
|
||||
- Decorated functions receive injection only when the `session` argument is omitted.
|
||||
- No code path creates AsyncSession in module import side effects.
|
||||
- Concurrent jobs and operations each create task-local sessions.
|
||||
|
||||
---
|
||||
|
||||
## Testing Checks
|
||||
|
||||
- Service constructors accept a test session factory without framework startup.
|
||||
- Session-taking access functions accept a transaction-scoped test session directly.
|
||||
- `session_scope()` tests cover supplied-session, supplied-factory, and settings-resolution precedence.
|
||||
- `db_transaction_scope()` tests verify commit on success, rollback on failure, session closure, engine disposal, and cache cleanup.
|
||||
- `with_session` tests cover omitted, positional, keyword, and explicit-`None` session arguments.
|
||||
- Composition tests verify decorated service calls borrow one caller-owned session without committing it.
|
||||
- SAVEPOINT tests verify local rollback preserves the outer transaction and successful exit does not commit it.
|
||||
- Tests that depend on SAVEPOINT timing account for `begin_nested()` flushing pending state on entry.
|
||||
- Rollback behavior is verified for failed write units.
|
||||
- Parallel-task tests verify no shared AsyncSession instances.
|
||||
- Lifecycle tests confirm schema initialization, factory availability, deterministic teardown, and expected cache behavior.
|
||||
|
||||
+6
-9
@@ -12,7 +12,7 @@
|
||||
- Status: adopted
|
||||
- Decision level: mandatory
|
||||
- Applies to: api-runtime, workers, tests
|
||||
- Last reviewed: 2026-07-26
|
||||
- Last reviewed: 2026-08-06
|
||||
|
||||
---
|
||||
|
||||
@@ -34,7 +34,7 @@ SQLModel is designed for FastAPI, built on Pydantic and SQLAlchemy, and intended
|
||||
## Rules
|
||||
|
||||
- Default to SQLModel for new table models and API data models.
|
||||
- Keep SQLAlchemy async primitives as the runtime base: `create_async_engine`, `async_sessionmaker`, and `AsyncSession`.
|
||||
- Keep SQLAlchemy engine and factory primitives as the runtime base: `create_async_engine` and `async_sessionmaker`. For SQLModel applications, use SQLModel's `AsyncSession` wrapper so its typed `exec()` API remains available.
|
||||
- Keep transaction and session ownership policies identical whether models are SQLAlchemy Declarative or SQLModel.
|
||||
- Use SQLModel inheritance to share validated fields while keeping table, create, update, and public contracts distinct where their semantics differ.
|
||||
- Use SQLAlchemy declarative models only for a concrete unsupported mapping or third-party constraint; document the reason.
|
||||
@@ -72,17 +72,14 @@ class UserRead(UserBase):
|
||||
### Pattern B: Keep SQLModel models with the async runtime
|
||||
|
||||
```python
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
||||
from sqlmodel import select
|
||||
|
||||
engine = create_async_engine(settings.database_url, pool_pre_ping=True)
|
||||
session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
||||
|
||||
async with session_factory() as session:
|
||||
users = (await session.scalars(select(User))).all()
|
||||
async with database_scope(settings.database_url) as session_factory:
|
||||
async with session_factory() as session:
|
||||
users = (await session.exec(select(User))).all()
|
||||
```
|
||||
|
||||
`sqlmodel.select()` keeps SQLModel's typing-oriented statement construction, while `AsyncSession.scalars()` and the surrounding lifecycle come from SQLAlchemy.
|
||||
`database_scope()` enters the cached engine lifecycle, initializes registered SQLModel metadata by default, and yields the application session factory while SQLModel supplies the model and statement layer. `sqlmodel.select()` keeps SQLModel's typing-oriented statement construction, and SQLModel's `AsyncSession` adds typed `exec()` results while retaining SQLAlchemy's async lifecycle and transaction behavior. Import `AsyncSession` from `sqlmodel.ext.asyncio.session` when working with SQLModel models; use SQLAlchemy's `AsyncSession` only when the code intentionally has no SQLModel dependency.
|
||||
|
||||
---
|
||||
|
||||
@@ -0,0 +1,172 @@
|
||||
# Testing Database Targets and Data
|
||||
|
||||
Use the same engine and session primitives in production and tests. Tests select a different URL and, when transaction isolation is required, bind a test session factory to one test-owned connection and outer transaction. They do not replace repositories, services, or SQLAlchemy mechanics with mocks.
|
||||
|
||||
## Decision Table
|
||||
|
||||
| Test need | Database target | Isolation approach | What it proves |
|
||||
|---|---|---|---|
|
||||
| Fast, serial application tests | `sqlite+aiosqlite://` | Per-test engine or connection-bound session factory over an outer transaction | ORM mappings and ordinary application behavior |
|
||||
| Async code using multiple simultaneous sessions | Named SQLite shared-cache URL or temporary SQLite file | Per-test schema or cleanup strategy | Concurrent-session behavior without a database server |
|
||||
| PostgreSQL-specific behavior | Dedicated PostgreSQL test database | Per-test outer transaction and SAVEPOINT | SQL, constraints, types, locking, and migrations that SQLite cannot represent |
|
||||
|
||||
SQLite is a useful fast target, not a drop-in PostgreSQL substitute. Keep a small PostgreSQL integration suite for PostgreSQL-specific queries, extensions, row locking, JSON semantics, collations, isolation, and migration validation.
|
||||
|
||||
## Shared Construction Primitives
|
||||
|
||||
Make the application factory accept a database URL or settings object. Production, workers, and ordinary integration tests enter the same [`database_scope()`](session.md#database-and-convenience-scopes). Tests enter the lower-level [`engine_scope()`](engine.md#owning-engine-scope) only when they need direct engine or connection ownership for schema setup, an outer transaction, or engine-specific assertions:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncGenerator
|
||||
|
||||
import pytest_asyncio
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||
|
||||
from .engine import engine_scope
|
||||
|
||||
|
||||
@pytest_asyncio.fixture(scope="session", loop_scope="session")
|
||||
async def test_engine(database_url: str) -> AsyncGenerator[AsyncEngine]:
|
||||
async with engine_scope(database_url) as engine:
|
||||
yield engine
|
||||
```
|
||||
|
||||
Production passes its `postgresql+asyncpg://...` URL to `database_scope()`. A local SQLite run passes `sqlite+aiosqlite:///./app.db`. Tests pass a dedicated test URL to `database_scope()` or `engine_scope()` and receive schema initialization, deterministic disposal, and engine-cache cleanup when the context exits. Do not create an engine during module import: that makes it easy for tests to retain the production URL before an override is applied.
|
||||
|
||||
Use migrations to provision an integration database when migrations are part of the release contract. `metadata.create_all()` is appropriate for focused ORM tests only when it accurately represents the schema under test. Import all table models before creating metadata; [SQLModel documents that model-registration order matters](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/#import-table-models).
|
||||
|
||||
## Transactional Async Fixture
|
||||
|
||||
For tests that exercise code which commits, start an outer transaction on one test connection. Bind a test `SessionFactory` to that connection with `join_transaction_mode="create_savepoint"`. SQLAlchemy documents this as its test-suite pattern: sessions created by the factory resolve their commits through SAVEPOINTs while fixture teardown rolls back the outer transaction.
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncGenerator
|
||||
|
||||
import pytest_asyncio
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||
|
||||
from .session import SessionFactory
|
||||
|
||||
|
||||
@pytest_asyncio.fixture(scope="function", loop_scope="session")
|
||||
async def session_factory(test_engine: AsyncEngine) -> AsyncGenerator[SessionFactory]:
|
||||
async with test_engine.connect() as connection:
|
||||
transaction = await connection.begin()
|
||||
factory = async_sessionmaker(
|
||||
bind=connection,
|
||||
class_=AsyncSession,
|
||||
expire_on_commit=False,
|
||||
join_transaction_mode="create_savepoint",
|
||||
)
|
||||
try:
|
||||
yield factory
|
||||
finally:
|
||||
await transaction.rollback()
|
||||
```
|
||||
|
||||
Each factory call still creates a distinct `AsyncSession`, matching [session factory mechanics](session.md#session-factory-mechanics). The factory belongs to the fixture's engine and outer transaction and must not escape either scope.
|
||||
|
||||
For service tests that pass a caller-owned session into decorated or undecorated service functions, derive that session from the same factory:
|
||||
|
||||
```python
|
||||
@pytest_asyncio.fixture(scope="function", loop_scope="session")
|
||||
async def session(session_factory: SessionFactory) -> AsyncGenerator[AsyncSession]:
|
||||
async with session_factory() as test_session:
|
||||
await test_session.begin()
|
||||
yield test_session
|
||||
```
|
||||
|
||||
The explicit `begin()` gives test code one visible transaction from the start. Session closure rolls back unfinished work; the outer connection transaction remains the final isolation boundary even if application code commits its SAVEPOINT.
|
||||
|
||||
For FastAPI request tests, override `_get_session_factory` so the production `SessionDep` retains its session-creation and cleanup behavior while receiving the test-bound factory. Always remove the override after the test because [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/) are stored in a mutable application-level dictionary.
|
||||
|
||||
```python
|
||||
from collections.abc import Generator
|
||||
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
|
||||
from .fastapi import _get_session_factory
|
||||
from .session import SessionFactory
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def app_with_test_database(app: FastAPI, session_factory: SessionFactory) -> Generator[FastAPI]:
|
||||
def get_test_session_factory() -> SessionFactory:
|
||||
return session_factory
|
||||
|
||||
app.dependency_overrides[_get_session_factory] = get_test_session_factory
|
||||
try:
|
||||
yield app
|
||||
finally:
|
||||
app.dependency_overrides.pop(_get_session_factory, None)
|
||||
```
|
||||
|
||||
Construct `app` with test settings before lifespan starts so startup cannot resolve the production URL. The override changes request session creation; it does not prevent lifespan from entering its configured `database_scope()`.
|
||||
|
||||
The connection-bound factory is deliberately serial even though it creates distinct sessions: those sessions still share one connection and outer transaction. A test that verifies concurrently active sessions must use independent connections and a database target that supports them.
|
||||
|
||||
## SQLite Targets
|
||||
|
||||
### Serial in-memory tests
|
||||
|
||||
Use `sqlite+aiosqlite://` for a fresh in-memory database when the test runs all database work serially. SQLAlchemy's `aiosqlite` dialect uses a single-connection `StaticPool` for this target, so all sessions share one SQLite transaction state. One session's rollback can discard another session's uncommitted work.
|
||||
|
||||
`engine_scope()` imports the model package and creates the schema by default, then disposes the engine and clears cached resolution deterministically:
|
||||
|
||||
```python
|
||||
from collections.abc import AsyncGenerator
|
||||
|
||||
import pytest_asyncio
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||
from .engine import engine_scope
|
||||
|
||||
|
||||
@pytest_asyncio.fixture(scope="session", loop_scope="session")
|
||||
async def test_engine() -> AsyncGenerator[AsyncEngine]:
|
||||
async with engine_scope("sqlite+aiosqlite://") as engine:
|
||||
yield engine
|
||||
```
|
||||
|
||||
### Concurrent in-memory tests
|
||||
|
||||
Do not use the default `:memory:` target for tests that have multiple active sessions or tasks. Use a named shared-cache database instead, with a name unique to the test process:
|
||||
|
||||
```text
|
||||
sqlite+aiosqlite:///file:test-suite?mode=memory&cache=shared&uri=true
|
||||
```
|
||||
|
||||
This lets connections share the same in-memory database while retaining independent transaction state. A temporary file URL such as `sqlite+aiosqlite:////tmp/test.db` is often simpler when test isolation or cleanup tooling already manages files.
|
||||
|
||||
For both SQLite forms, enable and test the constraints your application depends on. SQLite foreign-key enforcement is disabled by default, and its transaction behavior has driver-specific differences. Keep PostgreSQL integration coverage for behavior that SQLite cannot faithfully model.
|
||||
|
||||
## Test Data Practices
|
||||
|
||||
- Build only the data a test needs, through named factory functions or pytest fixtures rather than a large global seed.
|
||||
- Give each fixture a domain meaning, such as `active_account`, `expired_subscription`, or `admin_user`; avoid opaque rows with unexplained defaults.
|
||||
- Set values relevant to the assertion explicitly, including timestamps, permissions, statuses, and unique identifiers. Use fixed clocks or injected clock values instead of the wall clock.
|
||||
- Construct object graphs through relationships, then `await session.flush()` before reading generated identifiers or passing foreign keys onward. `flush()` exercises database constraints without ending the test transaction.
|
||||
- Seed prerequisite data before creating a client request. Let the endpoint own the mutation being asserted; do not pre-insert the row that the endpoint is supposed to create.
|
||||
- Use `commit()` in fixture setup only when the test specifically needs to prove post-commit behavior. With the transactional fixture, this remains isolated through the outer rollback.
|
||||
- Keep shared reference data immutable and explicit. If it must be reused for performance, load it once into a dedicated test database and reset all mutable tables between tests; never depend on test order.
|
||||
- Include both valid and constraint-breaking graphs where a behavior depends on foreign keys, uniqueness, nullability, or cascading deletes. SQLite-only tests should not be the sole evidence for PostgreSQL constraints.
|
||||
|
||||
## Completion Checks
|
||||
|
||||
- A test run cannot reach the production URL; production credentials are absent from the test environment.
|
||||
- Production PostgreSQL, local SQLite, and in-memory SQLite all use `database_scope()` unless a test explicitly needs lower-level engine or connection ownership.
|
||||
- Every test or fixture scope owns its override, session factory, connection, transaction, and session cleanup; the session-scoped engine fixture owns disposal and cache cleanup.
|
||||
- Request tests override `_get_session_factory`, preserving production request-session creation and cleanup behavior.
|
||||
- Test data is deterministic, minimal, and expresses the scenario under test.
|
||||
- PostgreSQL integration tests cover every PostgreSQL-specific contract and run against migrations where migrations are shipped.
|
||||
|
||||
## Sources
|
||||
|
||||
- [SQLAlchemy: joining a session into an external transaction](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites)
|
||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||
- [SQLAlchemy SQLite dialect and async in-memory pooling](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#using-a-memory-database-with-multiple-coroutines)
|
||||
- [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
||||
- [SQLModel testing with FastAPI](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/)
|
||||
- [pytest-asyncio fixtures](https://pytest-asyncio.readthedocs.io/en/stable/how-to-guides/index.html)
|
||||
+9
-28
@@ -1,23 +1,6 @@
|
||||
---
|
||||
name: copilot-customization
|
||||
description: 'Plan, create, review, and debug GitHub Copilot and VS Code agent customizations, including instructions, prompt files, skills, custom agents, hooks, MCP servers, and repo-specific personal-mcp skill integration.'
|
||||
x-personal-mcp:
|
||||
id: copilot-customization
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- copilot
|
||||
- vscode
|
||||
- customization
|
||||
- instructions
|
||||
- prompts
|
||||
- agent-skills
|
||||
- custom-agents
|
||||
- hooks
|
||||
- mcp
|
||||
- personal-mcp
|
||||
- skills
|
||||
capabilities:
|
||||
- resource://skills/copilot-customization/document
|
||||
---
|
||||
|
||||
# Copilot Customization
|
||||
@@ -78,31 +61,29 @@ Use either:
|
||||
Choose one of these patterns:
|
||||
|
||||
1. Direct URI strategy:
|
||||
- Reference known resources directly, such as:
|
||||
- `resource://catalog/skills_index`
|
||||
- `resource://catalog/skills/{skill_id}`
|
||||
- `resource://skills/<skill-id>/document`
|
||||
- `resource://skills/<skill-id>/references/<ref-id>`
|
||||
- Read `skill://<skill-name>/SKILL.md` when the required skill is known.
|
||||
- Read `skill://<skill-name>/_manifest` only when supporting material may be useful.
|
||||
- Read selected supporting files at `skill://<skill-name>/<supporting-path>`.
|
||||
2. Discovery-first strategy:
|
||||
- Start at catalog discovery (`resource://catalog/skills_index`), select the best skill match, then load the skill document and only the minimal references needed.
|
||||
- List resources, compare native main-resource names and descriptions, then load the best matching `SKILL.md`.
|
||||
|
||||
### Authoring guidance for shims
|
||||
|
||||
1. Keep shim content short and procedural; avoid copying large guidance blocks from Personal MCP.
|
||||
2. State trigger conditions clearly (for example: "when creating a new skill" or "when editing docs contracts").
|
||||
3. Specify whether to use direct URIs or discovery for that repo's common workflows.
|
||||
4. Prefer loading only the most relevant skill document first; expand to references only when needed.
|
||||
3. Specify whether to use a direct native URI or resource listing for that repo's common workflows.
|
||||
4. Prefer loading only the most relevant main file first; inspect its manifest only when needed.
|
||||
5. For stable repeated workflows, use explicit URIs. For broader or ambiguous requests, use discovery-first.
|
||||
|
||||
### Minimal shim examples
|
||||
|
||||
Instruction-style shim intent:
|
||||
|
||||
1. "For markdown edits (`applyTo: '**/*.md'`), load `resource://skills/zensical-docs/document` and apply Zensical-native documentation conventions unless they conflict with expected MkDocs compatibility."
|
||||
1. "For markdown edits (`applyTo: '**/*.md'`), load `skill://zensical-docs/SKILL.md` and apply Zensical-native documentation conventions unless they conflict with expected MkDocs compatibility."
|
||||
|
||||
Prompt-style shim intent:
|
||||
|
||||
1. "For docs authoring tasks, consult `resource://skills/zensical-docs/document`, summarize the relevant authoring constraints, then propose the smallest markdown change for this repository."
|
||||
1. "For docs authoring tasks, consult `skill://zensical-docs/SKILL.md`, summarize the relevant authoring constraints, then propose the smallest markdown change for this repository."
|
||||
|
||||
### Validation for shim implementation
|
||||
|
||||
@@ -130,7 +111,7 @@ Before finishing:
|
||||
3. Confirm names match directory names where VS Code requires it.
|
||||
4. Confirm descriptions include the phrases users are likely to ask for.
|
||||
5. Confirm extra skill resources are linked from `SKILL.md`.
|
||||
6. Confirm repo skill metadata exposes the correct `resource://skills/<skill-id>/document` capability.
|
||||
6. Confirm native discovery exposes `skill://<skill-name>/SKILL.md`, `_manifest`, and supporting-file reads.
|
||||
7. State any remaining ambiguity or user choice, such as personal vs workspace scope.
|
||||
|
||||
## Output Contract
|
||||
-11
@@ -1,17 +1,6 @@
|
||||
---
|
||||
name: fastapi-uv-docker
|
||||
description: 'Audit and migrate an existing Python project to best practices for a cloud-native ASGI FastAPI app managed with uv and run with uvicorn in Docker. Use when: conforming a project to production standards, setting up src layout, configuring pyproject.toml, writing multi-stage Dockerfiles, wiring lifespan and settings, adding health endpoints, enforcing non-root container user, migrating from requirements.txt to uv.'
|
||||
x-personal-mcp:
|
||||
id: fastapi-uv-docker
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- fastapi
|
||||
- uv
|
||||
- uvicorn
|
||||
- docker
|
||||
- architecture
|
||||
capabilities:
|
||||
- resource://skills/fastapi-uv-docker/document
|
||||
---
|
||||
|
||||
# FastAPI Project Best Practices
|
||||
@@ -1,17 +1,6 @@
|
||||
---
|
||||
name: mcp-details
|
||||
description: "Reference hub for MCP and FastMCP source documentation links. Use when you need authoritative protocol, SDK, transport, and deployment docs without loading broad implementation guidance."
|
||||
x-personal-mcp:
|
||||
id: mcp-details
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- mcp
|
||||
- model-context-protocol
|
||||
- fastmcp
|
||||
- references
|
||||
- source-docs
|
||||
capabilities:
|
||||
- resource://skills/mcp-details/document
|
||||
---
|
||||
|
||||
# MCP Details
|
||||
+2
@@ -7,6 +7,8 @@ Use this page for MCP client setup, operational tools, and integration reference
|
||||
!!! info "VS Code MCP docs"
|
||||
- [VS Code MCP servers overview](https://code.visualstudio.com/docs/agent-customization/mcp-servers)
|
||||
- [VS Code MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration)
|
||||
- [VS Code MCP developer guide](https://code.visualstudio.com/docs/agents/guides/mcp-developer-guide)
|
||||
- [VS Code MCP Apps support](https://code.visualstudio.com/blogs/2026/01/26/mcp-apps-support)
|
||||
- [VS Code Copilot customization overview](https://code.visualstudio.com/docs/copilot/customization/overview)
|
||||
|
||||
## Debugging and Inspection
|
||||
+7
@@ -13,6 +13,13 @@ Use this page for implementation-oriented links across MCP SDKs and FastMCP.
|
||||
|
||||
!!! info "FastMCP sources"
|
||||
- [FastMCP project documentation](https://gofastmcp.com/)
|
||||
- [FastMCP server identity and behavior](https://gofastmcp.com/servers/server)
|
||||
- [FastMCP tools and annotations](https://gofastmcp.com/servers/tools)
|
||||
- [FastMCP resources and templates](https://gofastmcp.com/servers/resources)
|
||||
- [FastMCP prompts](https://gofastmcp.com/servers/prompts)
|
||||
- [FastMCP argument completion](https://gofastmcp.com/servers/completions)
|
||||
- [FastMCP component icons](https://gofastmcp.com/servers/icons)
|
||||
- [FastMCP Apps](https://gofastmcp.com/apps/overview)
|
||||
- [FastMCP GitHub repository](https://github.com/jlowin/fastmcp)
|
||||
- [FastMCP examples directory](https://github.com/jlowin/fastmcp/tree/main/examples)
|
||||
- [FastMCP PyPI package](https://pypi.org/project/fastmcp/)
|
||||
@@ -1,26 +1,6 @@
|
||||
---
|
||||
name: nicegui
|
||||
description: 'Reference hub for NiceGUI and FastAPI application structure, typed configuration, ASGI and Uvicorn startup, UI composition, styling, bindable state, interactions, troubleshooting, testing, and source documentation. Use when planning, implementing, reviewing, deploying, or debugging NiceGUI applications; load only the references relevant to the task.'
|
||||
x-personal-mcp:
|
||||
id: nicegui
|
||||
version: 2.5.0
|
||||
tags:
|
||||
- nicegui
|
||||
- fastapi
|
||||
- asgi
|
||||
- uvicorn
|
||||
- pydantic-settings
|
||||
- configuration
|
||||
- deployment
|
||||
- ui
|
||||
- architecture
|
||||
- scaffolding
|
||||
- customization
|
||||
- frontend
|
||||
- testing
|
||||
- source-docs
|
||||
capabilities:
|
||||
- resource://skills/nicegui/document
|
||||
---
|
||||
|
||||
# NiceGUI Reference
|
||||
@@ -68,13 +48,14 @@ Load [FastAPI and Uvicorn startup](./references/fastapi-uvicorn-startup.md) for:
|
||||
- exposing programmatic startup through `[project.scripts]`
|
||||
- reload, worker, and process-local state constraints
|
||||
|
||||
### Components And Styling
|
||||
### Styling And Customization
|
||||
|
||||
Load [architecture and styling](./references/architecture-and-styling.md) for:
|
||||
Load [styling and customization](./references/styling-and-customization.md) for:
|
||||
|
||||
- page, component, and service boundaries
|
||||
- component extraction decisions
|
||||
- Quasar props, Tailwind utilities, and custom CSS boundaries
|
||||
- progressive discovery through NiceGUI docs, constructors, and Quasar docs
|
||||
- Quasar props, slots, events, and NiceGUI customization methods
|
||||
- app-wide and page-level color themes, dark mode, and semantic CSS tokens
|
||||
- Tailwind for structural styling and static stylesheets for fine tuning
|
||||
- responsive layout and static asset conventions
|
||||
- Tailwind and Quasar breakpoint scales, container queries, and responsive testing
|
||||
- uniformly scaling dialogs on mobile
|
||||
@@ -83,6 +64,15 @@ Load [architecture and styling](./references/architecture-and-styling.md) for:
|
||||
- sizing scrollable dialog cards under CSS `zoom`
|
||||
- validating zoomed controls with Playwright or a browser
|
||||
|
||||
### Special Component Customization
|
||||
|
||||
Load [special component customization](./references/special-component-customization.md) for:
|
||||
|
||||
- the required source-research gate before generating component customizations
|
||||
- `ui.select` constructors, Quasar props, slots, detached popups, and option caveats
|
||||
- `ui.icon` names, icon families, sizing, colors, assets, and Material Symbol variants
|
||||
- component-specific accessibility, sanitization, and validation checks
|
||||
|
||||
### Bindable State
|
||||
|
||||
Load [bindable dataclasses](./references/binding-dataclasses.md) for:
|
||||
@@ -123,12 +113,14 @@ Load [source documentation](./references/source-documentation.md) when:
|
||||
|
||||
1. Load [application architecture](./references/architecture.md).
|
||||
2. Add [FastAPI and Uvicorn startup](./references/fastapi-uvicorn-startup.md) when FastAPI owns the application or startup must be exposed as a project command.
|
||||
3. Add [architecture and styling](./references/architecture-and-styling.md) only when page and component design is in scope.
|
||||
3. Add [styling and customization](./references/styling-and-customization.md) only when page layout or visual customization is in scope.
|
||||
|
||||
### Page Or Component Work
|
||||
|
||||
1. Load [architecture and styling](./references/architecture-and-styling.md).
|
||||
2. Add [interaction patterns](./references/interaction-patterns.md) or [bindable dataclasses](./references/binding-dataclasses.md) according to the page behavior.
|
||||
1. Load [application architecture](./references/architecture.md) for page and component ownership decisions.
|
||||
2. Load [styling and customization](./references/styling-and-customization.md) for layout, responsive behavior, or visual customization.
|
||||
3. Add [special component customization](./references/special-component-customization.md) when the work targets `ui.select`, `ui.icon`, or another component with specialized Quasar behavior.
|
||||
4. Add [interaction patterns](./references/interaction-patterns.md) or [bindable dataclasses](./references/binding-dataclasses.md) according to the page behavior.
|
||||
|
||||
### Debugging Or Production Review
|
||||
|
||||
@@ -142,7 +134,9 @@ Load [source documentation](./references/source-documentation.md) when:
|
||||
- Keep business logic out of UI components and event handlers.
|
||||
- Avoid blocking I/O and CPU-heavy work in the UI event loop.
|
||||
- Prefer event-driven updates and explicit refreshes over unrelated polling.
|
||||
- Prefer Tailwind utilities, then Quasar props, then reusable component helpers; use minimal shared CSS when those are insufficient.
|
||||
- Discover component capabilities through NiceGUI docs and constructors, then the wrapped Quasar API.
|
||||
- Research the current NiceGUI and Quasar source documentation before generating component-specific code or CSS.
|
||||
- Prefer constructor arguments and native Quasar features through NiceGUI; use Tailwind for structure and scoped static CSS for stable fine tuning.
|
||||
- Provide loading, success, and failure states for user-triggered work.
|
||||
- Treat version-specific guidance as a prompt to verify the project's dependency version.
|
||||
|
||||
+8
-1
@@ -23,7 +23,6 @@ Recommended base shape:
|
||||
│ └─ app/
|
||||
│ ├─ __init__.py
|
||||
│ ├─ main.py
|
||||
│ ├─ bootstrap.py
|
||||
│ ├─ config.py
|
||||
│ ├─ logging.py
|
||||
│ ├─ api/
|
||||
@@ -69,6 +68,14 @@ Prefer:
|
||||
|
||||
Avoid imports from services back into API or UI modules.
|
||||
|
||||
## Page And Component Ownership
|
||||
|
||||
Page modules compose routes from presentation components and service calls. They should not own domain rules, persistence, or long-running synchronous work.
|
||||
|
||||
Extract a presentation pattern to `ui/components/` when it appears on two or more pages or owns a meaningful interaction boundary. Keep one-off route composition in the page module. Reusable components should accept data and event callbacks instead of importing page state or business services implicitly.
|
||||
|
||||
For page composition, responsive layout, Quasar props, and CSS customization, load [styling and customization](./styling-and-customization.md).
|
||||
|
||||
## Optional Persistence
|
||||
|
||||
Use only when the product requires durable data.
|
||||
+2
-2
@@ -126,7 +126,7 @@ def get_settings() -> Settings:
|
||||
The argument-free [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) provider is appropriate here because both the project entry point and Uvicorn's zero-argument factory need process-lifetime access. Each reload or worker process gets its own settings instance. Do not add override arguments to `get_settings()`; inject a `Settings` instance directly into `create_app()` in tests or alternate composition roots. See the [Pydantic settings implementation guide](../../pydantic-settings/SKILL.md) for source precedence, independent settings boundaries, cache clearing, and runtime reload guidance.
|
||||
|
||||
```python title="src/my_app/main.py"
|
||||
from collections.abc import AsyncIterator
|
||||
from collections.abc import AsyncGenerator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
import uvicorn
|
||||
@@ -137,7 +137,7 @@ from my_app.config import Settings, get_settings
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||
async def lifespan(app: FastAPI) -> AsyncGenerator[None]:
|
||||
app.state.ready = True
|
||||
try:
|
||||
yield
|
||||
+3
@@ -5,6 +5,9 @@ Use these links to verify framework-specific behavior before relying on version-
|
||||
## NiceGUI
|
||||
|
||||
!!! info "NiceGUI sources"
|
||||
- [Component documentation](https://nicegui.io/documentation)
|
||||
- [Element styling, props, and events](https://nicegui.io/documentation/element)
|
||||
- [NiceGUI element source](https://github.com/zauberzeug/nicegui/tree/main/nicegui/elements)
|
||||
- [Pages, routing, and FastAPI integration](https://www.nicegui.io/documentation/section_pages_routing)
|
||||
- [`ui.run_with` implementation](https://github.com/zauberzeug/nicegui/blob/main/nicegui/ui_run_with.py)
|
||||
- [FastAPI integration example](https://github.com/zauberzeug/nicegui/blob/main/examples/fastapi/main.py)
|
||||
@@ -0,0 +1,144 @@
|
||||
# NiceGUI Special Component Customization
|
||||
|
||||
Use this reference for components whose NiceGUI wrapper, Quasar implementation, popup behavior, slots, or external assets require component-specific handling. Start with [styling and customization](./styling-and-customization.md) for the general escalation workflow.
|
||||
|
||||
## Source Research Gate
|
||||
|
||||
Research the target component before generating code or CSS. Do not rely on a remembered NiceGUI or Quasar API.
|
||||
|
||||
For each component:
|
||||
|
||||
1. Read its current NiceGUI documentation page.
|
||||
2. Inspect the constructor and implementation in the target project's installed NiceGUI package.
|
||||
3. Confirm the wrapped Quasar component in the NiceGUI source.
|
||||
4. Read the matching Quasar guide and API definition for props, slots, events, and methods.
|
||||
5. Check the target project's pinned NiceGUI version before using current upstream behavior.
|
||||
6. Record which layer owns each proposed customization before writing it.
|
||||
|
||||
Use current upstream source only as a fallback when the target environment is unavailable. If installed and upstream behavior differ, follow the installed version and state the difference.
|
||||
|
||||
## `ui.select`
|
||||
|
||||
### Source Map
|
||||
|
||||
- [NiceGUI `ui.select` documentation](https://nicegui.io/documentation/select)
|
||||
- [NiceGUI `Select` source](https://github.com/zauberzeug/nicegui/blob/main/nicegui/elements/select.py)
|
||||
- [Quasar `QSelect` guide](https://quasar.dev/vue-components/select/)
|
||||
- [Quasar `QSelect` API source](https://github.com/quasarframework/quasar/blob/dev/ui/src/components/select/QSelect.json)
|
||||
|
||||
NiceGUI's `Select` wraps Quasar `QSelect` but owns important Python-side behavior. Its constructor handles options, labels, values, change callbacks, input filtering, new-value modes, multiple selection, clearing, validation, and key generation. Use those constructor parameters before adding equivalent Quasar props manually.
|
||||
|
||||
### Customization Order
|
||||
|
||||
1. Use `options`, `label`, `value`, `on_change`, `with_input`, `new_value_mode`, `multiple`, `clearable`, `validation`, and `key_generator` through the NiceGUI constructor.
|
||||
2. Use `.props()` for additional documented `QSelect` behavior such as field design, chips, option density, popup classes, popup positioning, or menu/dialog behavior.
|
||||
3. Use `.classes()` and Tailwind for the field's structural width and placement.
|
||||
4. Use named slots for prepend, append, loading, no-option, selected, or option content when props are insufficient.
|
||||
5. Use `popup-content-class` to attach an application class to the detached options popup, then fine-tune it in a static stylesheet.
|
||||
|
||||
```python
|
||||
from nicegui import ui
|
||||
|
||||
item_select = ui.select(
|
||||
options={"chair": "Chair", "desk": "Desk", "lamp": "Lamp"},
|
||||
label="Items",
|
||||
multiple=True,
|
||||
clearable=True,
|
||||
with_input=True,
|
||||
).props(
|
||||
"outlined use-chips options-dense "
|
||||
"popup-content-class=app-item-select-menu"
|
||||
).classes(
|
||||
"w-full md:max-w-md"
|
||||
)
|
||||
|
||||
with item_select.add_slot("prepend"):
|
||||
ui.icon("inventory_2")
|
||||
```
|
||||
|
||||
```css
|
||||
.app-item-select-menu {
|
||||
max-height: min(24rem, 60dvh);
|
||||
}
|
||||
```
|
||||
|
||||
### Select-Specific Caveats
|
||||
|
||||
- NiceGUI accepts a list of values or a dictionary mapping values to labels. Do not assume the Python options model is the same as Quasar's JavaScript object-array examples.
|
||||
- After mutating `options`, call `update()` or use `set_options()` so the client receives the change.
|
||||
- `new_value_mode` enables input automatically. For dictionary options with `add`, NiceGUI requires a `key_generator`.
|
||||
- A multiple select has a list value. NiceGUI normalizes a non-list initial value, but application state should still use the intended list shape.
|
||||
- `map-options` has a Quasar performance cost. Do not add it to NiceGUI's mapped options without confirming that the wrapper's value translation requires it.
|
||||
- `display-value-html` and `options-html` can create cross-site scripting risk. When using `selected`, `selected-item`, or `option` slots, the application owns sanitization.
|
||||
- Custom option slots use virtual scrolling. When one option renders multiple sibling elements, Quasar requires `q-virtual-scroll--with-prev` on every additional sibling.
|
||||
- Buttons placed in `before`, `after`, `prepend`, or `append` field slots do not propagate clicks to the parent. A submit button in one of those slots needs its own submit handler.
|
||||
- `QSelect` renders its popup outside the field. Style it through `popup-content-class`; do not assume a descendant selector beneath the field will reach it.
|
||||
- Quasar switches between menu and dialog popup behavior by platform. Verify forced `behavior=menu` carefully on iOS when input filtering is enabled.
|
||||
|
||||
Use `.on()` or `run_method()` only after confirming the event or method in the installed Quasar API. Prefer NiceGUI's `on_change`, `set_options()`, value bindings, and `is_showing_popup` when they cover the behavior.
|
||||
|
||||
## `ui.icon`
|
||||
|
||||
### Source Map
|
||||
|
||||
- [NiceGUI `ui.icon` documentation](https://nicegui.io/documentation/icon)
|
||||
- [NiceGUI `Icon` source](https://github.com/zauberzeug/nicegui/blob/main/nicegui/elements/icon.py)
|
||||
- [Quasar `QIcon` guide](https://quasar.dev/vue-components/icon/)
|
||||
- [Quasar `QIcon` API source](https://github.com/quasarframework/quasar/blob/dev/ui/src/components/icon/QIcon.json)
|
||||
- [Google Material Symbols and Icons](https://fonts.google.com/icons)
|
||||
|
||||
NiceGUI's `Icon` is a thin `QIcon` wrapper. Its constructor exposes `name`, `size`, and `color`; the source forwards these to a `q-icon` element. Use Quasar's icon naming and asset rules for anything beyond those parameters.
|
||||
|
||||
### Customization Order
|
||||
|
||||
1. Choose an icon family that is actually loaded by the application.
|
||||
2. Pass the documented icon name, size, and color to `ui.icon()`.
|
||||
3. Use `.props()` for supported `QIcon` props such as `left`, `right`, or a custom render tag.
|
||||
4. Use `.classes()` for structural placement and an application class for stable visual variants.
|
||||
5. Use a static stylesheet for Material Symbol axes, state variants, custom webfonts, or repeated effects.
|
||||
|
||||
```python
|
||||
from nicegui import ui
|
||||
|
||||
ui.icon(
|
||||
"sym_o_home",
|
||||
size="1.5rem",
|
||||
color="primary",
|
||||
).classes(
|
||||
"app-symbol-filled shrink-0"
|
||||
).tooltip(
|
||||
"Home"
|
||||
)
|
||||
```
|
||||
|
||||
```css
|
||||
.app-symbol-filled {
|
||||
font-variation-settings:
|
||||
"FILL" 1,
|
||||
"wght" 400,
|
||||
"GRAD" 0,
|
||||
"opsz" 24;
|
||||
}
|
||||
```
|
||||
|
||||
### Icon-Specific Caveats
|
||||
|
||||
- Material icon names use snake case. Material variants use prefixes such as `o_`, `r_`, `s_`, `sym_o_`, `sym_r_`, and `sym_s_`.
|
||||
- Other icon families have their own prefixes and require their webfont or stylesheet to be loaded. A valid name does not load the corresponding asset.
|
||||
- `size` accepts CSS units or Quasar sizes such as `xs`, `sm`, `md`, `lg`, and `xl`. Quasar implements icon sizing through `font-size`.
|
||||
- Icon color inherits text color unless the `color` prop or a CSS color overrides it.
|
||||
- Material Symbol variable axes apply to webfont icons, not static SVG icon exports.
|
||||
- Quasar also supports SVG path strings, `svguse:` references, and `img:` URLs. Confirm the exact `QIcon` name format and mount path before generating one of these forms.
|
||||
- For an action, use a semantic control such as `ui.button(icon=..., on_click=...)` and give it an accessible label or tooltip. Do not turn a bare decorative icon into an unlabeled control.
|
||||
- Prefer `ui.icon(...).tooltip(...)` over manually constructing tooltip slot markup when NiceGUI's method covers the requirement.
|
||||
|
||||
## Completion Check
|
||||
|
||||
Before accepting a special-component customization:
|
||||
|
||||
1. Cite the NiceGUI component page and implementation that were inspected.
|
||||
2. Cite the matching Quasar guide or API source.
|
||||
3. Identify constructor arguments, Quasar props, slots, Tailwind classes, and stylesheet rules separately.
|
||||
4. Confirm detached popup or external asset behavior where applicable.
|
||||
5. Test keyboard interaction, focus, labels, and tooltips.
|
||||
6. Test the supported mobile, landscape desktop, and portrait desktop viewports.
|
||||
@@ -0,0 +1,438 @@
|
||||
# NiceGUI Styling And Customization
|
||||
|
||||
Use this reference to discover how a NiceGUI component can be customized, apply the least invasive supported mechanism, and introduce CSS without fighting Quasar's internal geometry.
|
||||
|
||||
For package boundaries, dependency direction, and page or component ownership, load [application architecture](./architecture.md).
|
||||
|
||||
## Progressive Customization Workflow
|
||||
|
||||
Increase the customization level only when the previous source does not expose what the design requires:
|
||||
|
||||
1. Read the NiceGUI documentation page for the component.
|
||||
2. Inspect the NiceGUI element function or class constructor.
|
||||
3. Identify the wrapped Quasar component and read its documentation.
|
||||
4. Use Quasar props, slots, and events through NiceGUI's native customization APIs.
|
||||
5. Use Tailwind classes for structural layout.
|
||||
6. Add a scoped static stylesheet for stable visual fine tuning.
|
||||
|
||||
Stop as soon as the required behavior is supported. Do not begin by targeting Quasar's generated DOM or internal selectors.
|
||||
|
||||
### 1. Start With The NiceGUI Component Page
|
||||
|
||||
Find the component in the [NiceGUI documentation](https://nicegui.io/documentation). Check its examples, parameters, methods, events, bindings, and inheritance before writing CSS. The component page establishes the public NiceGUI API and often demonstrates the intended Quasar integration.
|
||||
|
||||
Confirm the target project's installed NiceGUI version because the current online documentation can differ from the pinned release.
|
||||
|
||||
### 2. Inspect The NiceGUI Constructor
|
||||
|
||||
Read the signature and implementation of the imported NiceGUI function or element class. The constructor reveals accepted Python parameters, defaults, event callbacks, validation, and values NiceGUI forwards to the frontend.
|
||||
|
||||
Use editor navigation or runtime inspection against the project's selected environment:
|
||||
|
||||
```python
|
||||
from inspect import getsource, signature
|
||||
|
||||
from nicegui import ui
|
||||
|
||||
print(signature(ui.select))
|
||||
print(getsource(ui.select))
|
||||
```
|
||||
|
||||
When `ui.<name>` is a factory or alias, follow it to the element class in the [NiceGUI element sources](https://github.com/zauberzeug/nicegui/tree/main/nicegui/elements). Prefer the installed package source when behavior may differ by version.
|
||||
|
||||
### 3. Read The Underlying Quasar Component Docs
|
||||
|
||||
NiceGUI wraps Quasar components such as [`QInput`](https://quasar.dev/vue-components/input/), [`QSelect`](https://quasar.dev/vue-components/select/), and [`QDialog`](https://quasar.dev/vue-components/dialog/). Use the matching Quasar component page to discover its complete props, slots, events, methods, and behavior notes.
|
||||
|
||||
Map Quasar's Vue API onto the NiceGUI wrapper instead of copying a Vue template. Verify that a prop or slot exists in the Quasar version used by the installed NiceGUI release.
|
||||
|
||||
### 4. Apply Native Quasar Features Through NiceGUI
|
||||
|
||||
Use the NiceGUI element customization methods to reach the supported Quasar surface:
|
||||
|
||||
- `.props(...)` for Quasar properties and boolean flags
|
||||
- `.classes(...)` for Tailwind utilities and stable application class names
|
||||
- `.style(...)` for dynamic inline values or a quick, local probe
|
||||
- `.on(...)` for events that are not represented by a constructor callback
|
||||
- slots or child elements for Quasar extension points exposed by the wrapper
|
||||
|
||||
```python
|
||||
with ui.select(
|
||||
options=items,
|
||||
label="Item",
|
||||
).props(
|
||||
"outlined clearable options-dense popup-content-class=app-item-menu"
|
||||
).classes(
|
||||
"w-full md:max-w-md"
|
||||
) as item_select:
|
||||
with item_select.add_slot("prepend"):
|
||||
ui.icon("inventory_2")
|
||||
```
|
||||
|
||||
Prefer constructor arguments when NiceGUI exposes the behavior directly. Use `.props()` for supported Quasar features that are not constructor parameters. Use slots when the Quasar docs define a semantic insertion point; do not reproduce that content with absolute positioning.
|
||||
|
||||
## Application Themes With NiceGUI And Quasar
|
||||
|
||||
Treat a theme as three related layers with different owners:
|
||||
|
||||
1. Configure Quasar's named color roles through NiceGUI.
|
||||
2. Let Quasar own light, dark, and automatic mode state.
|
||||
3. Define application semantic tokens for surfaces and content not covered by Quasar components.
|
||||
|
||||
Do not implement a parallel theme switch by replacing Quasar classes or directly restyling each component. NiceGUI's color APIs set the supported Quasar `--q-*` custom properties, so Quasar components, `color=` arguments, and classes such as `text-primary` and `bg-positive` stay aligned.
|
||||
|
||||
### Set The App-Wide Palette Once
|
||||
|
||||
Use [`app.colors()`](https://nicegui.io/documentation/colors#app-wide-colors) in the composition layer for the default palette. Prefer Quasar's semantic roles over shade names: `primary`, `secondary`, `accent`, `positive`, `negative`, `info`, and `warning`. The `dark` and `dark_page` arguments configure dark surface colors; they do not enable dark mode.
|
||||
|
||||
```python
|
||||
from nicegui import app, ui
|
||||
|
||||
app.colors(
|
||||
primary="#176b5b",
|
||||
secondary="#52645f",
|
||||
accent="#c05a32",
|
||||
dark="#202523",
|
||||
dark_page="#151917",
|
||||
positive="#2e7d32",
|
||||
negative="#b3261e",
|
||||
info="#276b8e",
|
||||
warning="#a86600",
|
||||
brand="#176b5b",
|
||||
)
|
||||
|
||||
|
||||
@ui.page("/")
|
||||
def index() -> None:
|
||||
ui.button("Save")
|
||||
ui.label("Current workspace").classes("text-brand")
|
||||
|
||||
|
||||
ui.run()
|
||||
```
|
||||
|
||||
Custom names such as `brand` become Quasar color names and can be used through `color="brand"`, `text-brand`, or `bg-brand`. Register them before any component uses them. `app.colors()` was added in NiceGUI 3.6.0; for an older pinned version, centralize the same `ui.colors(...)` call in a shared page shell.
|
||||
|
||||
Use [`ui.colors()`](https://nicegui.io/documentation/colors) only when one page intentionally overrides the app palette. It is page-scoped and takes precedence over `app.colors()`:
|
||||
|
||||
```python
|
||||
@ui.page("/operations")
|
||||
def operations_page() -> None:
|
||||
ui.colors(primary="#8f3d2c")
|
||||
ui.button("Operations action")
|
||||
```
|
||||
|
||||
Avoid scattering `ui.colors()` calls among reusable components. A component should consume semantic roles from its owning page rather than silently changing the palette for the whole page.
|
||||
|
||||
### Let Quasar Control Light And Dark Mode
|
||||
|
||||
Use [`ui.dark_mode()`](https://nicegui.io/documentation/dark_mode) for page mode. Its value is tri-state: `True` enables dark mode, `False` disables it, and `None` follows the client's `prefers-color-scheme` setting. It overrides the `dark` default supplied to `ui.run()` or `@ui.page` for that page.
|
||||
|
||||
```python
|
||||
dark_mode = ui.dark_mode(None)
|
||||
|
||||
with ui.button_group():
|
||||
ui.button("System", on_click=dark_mode.auto)
|
||||
ui.button("Light", on_click=dark_mode.disable)
|
||||
ui.button("Dark", on_click=dark_mode.enable)
|
||||
```
|
||||
|
||||
Quasar applies `body--light` or `body--dark`, updates its dark-aware components, and tracks system changes while mode is automatic. Use the NiceGUI element instead of invoking Quasar's JavaScript Dark plugin directly. Persist an explicit user preference separately when it must survive navigation or a new browser session.
|
||||
|
||||
### Add Semantic Tokens For Application Surfaces
|
||||
|
||||
Quasar's brand roles cover framework components, not every application-specific surface. Define a small set of semantic CSS variables in the static stylesheet and change their values under Quasar's documented `.body--dark` class:
|
||||
|
||||
```css
|
||||
:root {
|
||||
--app-page: #f6f8f7;
|
||||
--app-surface: #ffffff;
|
||||
--app-text: #202623;
|
||||
--app-border: #cbd4d0;
|
||||
}
|
||||
|
||||
.body--dark {
|
||||
--app-page: var(--q-dark-page);
|
||||
--app-surface: var(--q-dark);
|
||||
--app-text: #eef3f0;
|
||||
--app-border: #46504b;
|
||||
}
|
||||
|
||||
body {
|
||||
background: var(--app-page);
|
||||
color: var(--app-text);
|
||||
}
|
||||
|
||||
.app-panel {
|
||||
background: var(--app-surface);
|
||||
border: 1px solid var(--app-border);
|
||||
}
|
||||
```
|
||||
|
||||
Name tokens by purpose, such as `--app-surface` or `--app-muted-text`, rather than by a fixed color such as `--app-gray-100`. Reuse `--q-primary` and the other Quasar variables when the meaning matches. Check text, icon, border, focus, hover, disabled, positive, warning, and negative contrast in both modes; a palette is not complete merely because the page background changes.
|
||||
|
||||
## Structural Styling With Tailwind
|
||||
|
||||
Use standard [Tailwind utility classes](https://tailwindcss.com/docs/utility-first) for page and component structure:
|
||||
|
||||
- display, flex, and grid behavior
|
||||
- width, height, and maximum-width constraints
|
||||
- spacing, gaps, padding, and alignment
|
||||
- wrapping, overflow, and responsive variants
|
||||
- typography and common visual utilities when they fully express the design
|
||||
|
||||
Build the outer layout before fine-tuning individual controls:
|
||||
|
||||
1. Define the page shell and width constraints.
|
||||
2. Establish responsive rows, columns, gaps, and wrapping.
|
||||
3. Add semantic sections and repeated visual patterns.
|
||||
4. Configure component appearance and behavior with constructor arguments and Quasar props.
|
||||
5. Add stable application classes for any remaining stylesheet rules.
|
||||
|
||||
```python
|
||||
with ui.column().classes("w-full max-w-6xl mx-auto gap-6 px-4"):
|
||||
page_header(title="Inventory")
|
||||
|
||||
with ui.row().classes("w-full gap-4 flex-wrap lg:flex-nowrap items-start"):
|
||||
filters_panel().classes("w-full lg:w-72 shrink-0")
|
||||
item_grid().classes("w-full flex-1 min-w-0")
|
||||
```
|
||||
|
||||
Use stable width, minimum-width, and flex constraints so labels, icons, validation messages, and loaded content do not shift the surrounding layout.
|
||||
|
||||
## Fine Tuning With Static Stylesheets
|
||||
|
||||
Move stable fine tuning into a static stylesheet after the structure and native component configuration are correct. Static stylesheets provide reusable selectors, media queries, pseudo-classes, CSS variables, and a clear cascade that inline declarations cannot provide.
|
||||
|
||||
Attach an application-owned class with `.classes()` or a Quasar popup prop, then scope stylesheet rules beneath it:
|
||||
|
||||
```python
|
||||
ui.select(...).props("popup-content-class=app-item-menu").classes(
|
||||
"app-item-select w-full md:max-w-md"
|
||||
)
|
||||
```
|
||||
|
||||
```css
|
||||
.app-item-select {
|
||||
--app-field-accent: #176b5b;
|
||||
}
|
||||
|
||||
.app-item-select:focus-within {
|
||||
filter: drop-shadow(0 0 0.25rem rgb(23 107 91 / 20%));
|
||||
}
|
||||
|
||||
.app-item-menu {
|
||||
max-height: min(24rem, 60dvh);
|
||||
}
|
||||
```
|
||||
|
||||
Use `.style()` when a value is calculated at runtime or while testing a local hypothesis. Once a declaration becomes stable or repeated, move it to the stylesheet and keep only the application class in Python.
|
||||
|
||||
Avoid overriding Quasar internals such as `.q-field__label`, `.q-field__native`, `.q-field__control`, and `.q-field__input` unless the public props, slots, and application-level selectors cannot express the requirement.
|
||||
|
||||
Quasar coordinates field height, padding, labels, values, icons, and floating-label transforms. Changing only one internal part tends to cause clipping or overlap.
|
||||
|
||||
## Responsive Layout
|
||||
|
||||
Support these layouts only:
|
||||
|
||||
- mobile: a single-column layout with wrapping toolbars and full-width controls
|
||||
- landscape desktop: $1920 \times 1080$ with side-by-side panels where they improve scanning
|
||||
- portrait desktop: $1080 \times 1920$ with stacked panels or a narrow fixed sidebar
|
||||
|
||||
Build the mobile layout first, then add one desktop breakpoint when a row or grid needs more space. Prefer flex wrapping and fluid grids before adding another breakpoint. Use Tailwind classes for page layout and Quasar props for component behavior.
|
||||
|
||||
```python
|
||||
with ui.row().classes("w-full flex-wrap gap-4 lg:flex-nowrap items-start"):
|
||||
filters_panel().classes("w-full lg:w-72 shrink-0")
|
||||
item_grid().classes("w-full flex-1 min-w-0")
|
||||
```
|
||||
|
||||
Use `min-w-0` for flexible children, `flex-wrap` for toolbars, and `max-w-* mx-auto` to keep portrait layouts readable. Do not add device-specific component trees, container queries, or custom breakpoints unless a supported layout demonstrates a concrete failure.
|
||||
|
||||
## Loading Stylesheets And Static Assets
|
||||
|
||||
- Mount and link static stylesheets once from the composition layer rather than injecting CSS from individual pages.
|
||||
- Keep custom CSS tokenized with variables and scoped to application classes.
|
||||
- Avoid broad rules against Quasar internals.
|
||||
- Mount referenced assets in the composition layer.
|
||||
- Verify mount paths, reverse-proxy rewrites, and cache behavior.
|
||||
|
||||
```python
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
|
||||
STATIC_DIR = Path(__file__).parent / "ui" / "static"
|
||||
|
||||
app.mount("/static", StaticFiles(directory=STATIC_DIR), name="static")
|
||||
ui.add_head_html(
|
||||
'<link rel="stylesheet" href="/static/css/base.css">',
|
||||
shared=True,
|
||||
)
|
||||
```
|
||||
|
||||
## Worked Example: Responsive Dialog Customization
|
||||
|
||||
This example begins with normal field density and Quasar popup props, then uses an application class and static stylesheet for the remaining responsive fine tuning. Use whole-card scaling when a form dialog must become uniformly larger on mobile while preserving Quasar's internal proportions. Keep detached select menus unscaled and make the card itself scrollable.
|
||||
|
||||
### Use Normal Field Density
|
||||
|
||||
Normal Quasar fields are approximately `56px` high, while dense fields are approximately `40px` high. Remove `dense` when larger controls are needed.
|
||||
|
||||
```python
|
||||
ui.input("Name").props("outlined")
|
||||
ui.number("Quantity").props("outlined")
|
||||
ui.select(...).props("outlined popup-content-class=app-item-detail-menu")
|
||||
ui.textarea("Description").props("outlined autogrow")
|
||||
```
|
||||
|
||||
Add a scoped class to the dialog card:
|
||||
|
||||
```python
|
||||
ui.card().classes("app-detail-card app-item-detail-card")
|
||||
```
|
||||
|
||||
### Scale The Complete Card
|
||||
|
||||
```css
|
||||
:root {
|
||||
--item-dialog-scale: 1;
|
||||
--item-dialog-max-height: calc(100dvh - 3rem);
|
||||
}
|
||||
|
||||
.app-item-detail-card {
|
||||
width: min(50rem, 50vw);
|
||||
max-height: var(--item-dialog-max-height);
|
||||
overflow-y: auto;
|
||||
overscroll-behavior: contain;
|
||||
zoom: var(--item-dialog-scale);
|
||||
}
|
||||
|
||||
/* Restore Quasar's baseline if a global rule overrides it. */
|
||||
.app-item-detail-card .q-field,
|
||||
.app-item-detail-menu {
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
@media (max-width: 599px) {
|
||||
:root {
|
||||
--item-dialog-scale: 1.2;
|
||||
/* 75dvh becomes 90dvh after 1.2x zoom. */
|
||||
--item-dialog-max-height: 75dvh;
|
||||
}
|
||||
|
||||
.app-item-detail-card {
|
||||
width: 80vw;
|
||||
}
|
||||
|
||||
.app-item-detail-menu {
|
||||
font-size: 16.8px;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The main mobile tuning knob is:
|
||||
|
||||
```css
|
||||
--item-dialog-scale: 1.2;
|
||||
```
|
||||
|
||||
### Keep Detached Popups Unscaled
|
||||
|
||||
Do not apply `zoom` or `transform: scale()` to a `QSelect` popup menu. Quasar renders menus outside the dialog and positions them from the unscaled anchor geometry. Scaling the menu container afterward separates it from its field.
|
||||
|
||||
Avoid:
|
||||
|
||||
```css
|
||||
.app-item-detail-card,
|
||||
.app-item-detail-menu {
|
||||
zoom: 1.2;
|
||||
}
|
||||
```
|
||||
|
||||
Use:
|
||||
|
||||
```css
|
||||
.app-item-detail-card {
|
||||
zoom: 1.2;
|
||||
}
|
||||
|
||||
.app-item-detail-menu {
|
||||
font-size: 16.8px;
|
||||
}
|
||||
```
|
||||
|
||||
Use `popup-content-class=app-item-detail-menu` to target the detached menu and enlarge its text without changing its coordinate system.
|
||||
|
||||
### Account For Zoom When Scrolling
|
||||
|
||||
The card's pre-zoom maximum height must account for the scale:
|
||||
|
||||
\[
|
||||
\begin{aligned}
|
||||
h_{\mathrm{pre}} &= \frac{h_{\mathrm{visible}}}{s} \\
|
||||
\text{where } s &= \text{the zoom scale}
|
||||
\end{aligned}
|
||||
\]
|
||||
|
||||
For a desired visual height of `90dvh` at \(1.2\times\):
|
||||
|
||||
\[
|
||||
\frac{90\,\mathrm{dvh}}{1.2} = 75\,\mathrm{dvh}
|
||||
\]
|
||||
|
||||
Therefore:
|
||||
|
||||
```css
|
||||
--item-dialog-max-height: 75dvh;
|
||||
```
|
||||
|
||||
Apply scrolling to the card itself:
|
||||
|
||||
```css
|
||||
.app-item-detail-card {
|
||||
max-height: var(--item-dialog-max-height);
|
||||
overflow-y: auto;
|
||||
overscroll-behavior: contain;
|
||||
}
|
||||
```
|
||||
|
||||
This keeps the dimmed page stationary while the form scrolls.
|
||||
|
||||
### Match The Quasar Breakpoint
|
||||
|
||||
Quasar's extra-small breakpoint ends at `599.98px`. A mobile-only rule can use:
|
||||
|
||||
```css
|
||||
@media (max-width: 599px) {
|
||||
/* Mobile rules. */
|
||||
}
|
||||
```
|
||||
|
||||
Confirm custom breakpoint values against the target application's Quasar configuration.
|
||||
|
||||
## Validation Checklist
|
||||
|
||||
Check each completed page at these three viewports:
|
||||
|
||||
1. A representative mobile viewport, such as $390 \times 844$.
|
||||
2. Landscape desktop at $1920 \times 1080$.
|
||||
3. Portrait desktop at $1080 \times 1920$.
|
||||
|
||||
Confirm that page sections do not overlap, toolbars wrap on mobile, desktop panels use the available space without becoming excessively wide, and dialogs remain visible and scroll to their final field.
|
||||
|
||||
## Sources
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [NiceGUI element styling and props](https://nicegui.io/documentation/element)
|
||||
- [NiceGUI binding properties](https://nicegui.io/documentation/section_binding_properties)
|
||||
- [NiceGUI color theming](https://nicegui.io/documentation/colors)
|
||||
- [NiceGUI dark mode](https://nicegui.io/documentation/dark_mode)
|
||||
- [Quasar components](https://quasar.dev/vue-components)
|
||||
- [Quasar color palette and runtime brand variables](https://quasar.dev/style/color-palette)
|
||||
- [Quasar dark mode](https://quasar.dev/style/dark-mode)
|
||||
- [Quasar field](https://quasar.dev/vue-components/field/)
|
||||
- [Quasar select](https://quasar.dev/vue-components/select/)
|
||||
- [Tailwind responsive design](https://tailwindcss.com/docs/responsive-design)
|
||||
- [MDN `zoom`](https://developer.mozilla.org/en-US/docs/Web/CSS/zoom)
|
||||
+114
-16
@@ -1,22 +1,6 @@
|
||||
---
|
||||
name: pydantic-settings
|
||||
description: "Practical guide for implementing typed application configuration with pydantic-settings. Use when designing BaseSettings models, choosing nested or independent settings boundaries, managing settings lifecycles, configuring dotenv or secrets, and customizing source priority safely."
|
||||
x-personal-mcp:
|
||||
id: pydantic-settings
|
||||
version: 1.1.0
|
||||
tags:
|
||||
- python
|
||||
- pydantic
|
||||
- pydantic-settings
|
||||
- configuration
|
||||
- env-vars
|
||||
- secrets
|
||||
- dotenv
|
||||
- source-priority
|
||||
- caching
|
||||
- lifecycle
|
||||
capabilities:
|
||||
- resource://skills/pydantic-settings/document
|
||||
---
|
||||
|
||||
# Pydantic Settings Implementation Guide
|
||||
@@ -203,11 +187,125 @@ Use independent `BaseSettings` classes when the objects have genuinely independe
|
||||
|
||||
Construct independent objects explicitly at the composition root and inject each dependency. Do not nest one `BaseSettings` class inside another merely to reuse its fields. Extract a shared `BaseModel` schema when models need common structure.
|
||||
|
||||
### Alternative Database Backends
|
||||
|
||||
When one application can run against one of several database backends, model the selected backend as a [discriminated union](https://docs.pydantic.dev/latest/concepts/unions/#discriminated-unions). Pydantic validates only the variant selected by `driver`, so required PostgreSQL values do not make a SQLite configuration fail, and vice versa.
|
||||
|
||||
```python
|
||||
from typing import Annotated, Literal
|
||||
|
||||
from pydantic import BaseModel, Field, SecretStr
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
class SqliteSettings(BaseModel):
|
||||
driver: Literal["sqlite"] = "sqlite"
|
||||
path: str = "app.db"
|
||||
|
||||
|
||||
class PostgresSettings(BaseModel):
|
||||
driver: Literal["postgres"] = "postgres"
|
||||
host: str
|
||||
port: int = 5432
|
||||
database: str
|
||||
user: str
|
||||
password: SecretStr
|
||||
|
||||
|
||||
DatabaseSettings = Annotated[
|
||||
SqliteSettings | PostgresSettings,
|
||||
Field(discriminator="driver"),
|
||||
]
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(
|
||||
env_prefix="APP_",
|
||||
env_nested_delimiter="__",
|
||||
env_file=".env",
|
||||
extra="ignore",
|
||||
frozen=True,
|
||||
)
|
||||
|
||||
database: DatabaseSettings
|
||||
```
|
||||
|
||||
Choose one configuration. A SQLite deployment requires no PostgreSQL variables:
|
||||
|
||||
```dotenv
|
||||
APP_DATABASE__DRIVER=sqlite
|
||||
APP_DATABASE__PATH=./data/app.db
|
||||
```
|
||||
|
||||
A PostgreSQL deployment requires only the PostgreSQL branch:
|
||||
|
||||
```dotenv
|
||||
APP_DATABASE__DRIVER=postgres
|
||||
APP_DATABASE__HOST=db.internal
|
||||
APP_DATABASE__PORT=5432
|
||||
APP_DATABASE__DATABASE=app
|
||||
APP_DATABASE__USER=app_user
|
||||
APP_DATABASE__PASSWORD=provided-by-the-runtime
|
||||
```
|
||||
|
||||
After settings validation, select an async SQLAlchemy driver URL. This is a pure configuration step; create the engine, session factory, and sessions in their own lifecycle-managed providers:
|
||||
|
||||
```python
|
||||
from functools import cache
|
||||
|
||||
from sqlalchemy import URL
|
||||
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||
from sqlalchemy.ext.asyncio import create_async_engine
|
||||
|
||||
|
||||
def get_database_url(settings: Settings) -> str:
|
||||
match settings.database:
|
||||
case SqliteSettings(path=path):
|
||||
url = URL.create(
|
||||
drivername="sqlite+aiosqlite",
|
||||
database=path,
|
||||
)
|
||||
case PostgresSettings() as database:
|
||||
url = URL.create(
|
||||
drivername="postgresql+asyncpg",
|
||||
host=database.host,
|
||||
port=database.port,
|
||||
database=database.database,
|
||||
username=database.user,
|
||||
password=database.password.get_secret_value(),
|
||||
)
|
||||
return url.render_as_string(hide_password=False)
|
||||
|
||||
|
||||
@cache
|
||||
def get_engine(database_url: str) -> AsyncEngine:
|
||||
return create_async_engine(database_url, pool_pre_ping=True)
|
||||
|
||||
|
||||
async def dispose_engine(database_url: str) -> None:
|
||||
engine = get_engine(database_url)
|
||||
try:
|
||||
await engine.dispose()
|
||||
finally:
|
||||
get_engine.cache_clear()
|
||||
|
||||
|
||||
async def refresh_engine(database_url: str) -> AsyncEngine:
|
||||
await dispose_engine(database_url)
|
||||
return get_engine(database_url)
|
||||
```
|
||||
|
||||
At the composition boundary, resolve the URL once with `get_database_url(settings)` and use it to retrieve the cached engine. In FastAPI, expose that engine through lifespan and build one `async_sessionmaker` from it; each request or unit of work then creates its own `AsyncSession`. Do not call `aiosqlite.connect()` or `asyncpg.create_pool()` directly: `aiosqlite` and `asyncpg` are selected as SQLAlchemy drivers by the URL, while SQLAlchemy owns pooling, disposal, and session integration.
|
||||
|
||||
The nested variants remain `BaseModel` classes. `Settings` is the only `BaseSettings` model and therefore the only object that reads environment variables, dotenv files, or secrets. This keeps one source policy and validated configuration snapshot while keeping the engine, session factory, and sessions in their distinct lifecycles. See the [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html), the [engine lifecycle guidance](../async-fastapi-sqlmodel/references/engine.md), and the [session lifecycle guidance](../async-fastapi-sqlmodel/references/session.md).
|
||||
|
||||
Quality gate:
|
||||
|
||||
1. Nested sections share one source policy and lifecycle.
|
||||
2. Independent settings have distinct owners, prefixes, or lifecycles.
|
||||
3. The application does not repeatedly scan the same sources through accidental nested `BaseSettings` construction.
|
||||
4. Each backend configuration validates without values required only by another backend.
|
||||
5. One cached `AsyncEngine` exists per configured driver URL, while each request or unit of work receives a new `AsyncSession`.
|
||||
|
||||
### 7. Own The Settings Lifecycle
|
||||
|
||||
@@ -1,19 +1,6 @@
|
||||
---
|
||||
name: pytesting
|
||||
description: "Reference hub for pytest suite structure, naming, markers, and stack-specific testing patterns. Optimized for progressive discovery so naming and hierarchy guidance are loaded first when shaping or reorganizing tests."
|
||||
x-personal-mcp:
|
||||
id: pytesting
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- pytest
|
||||
- testing
|
||||
- python
|
||||
- fastapi
|
||||
- asyncio
|
||||
- anyio
|
||||
- deterministic
|
||||
capabilities:
|
||||
- resource://skills/pytesting/document
|
||||
---
|
||||
|
||||
# Pytesting
|
||||
-9
@@ -1,15 +1,6 @@
|
||||
---
|
||||
name: python-logging
|
||||
description: 'Design, review, or refactor Python logging. Use when choosing logger names, levels, handlers, library/application boundaries, basicConfig, dictConfig, structured logs, or operational logging defaults.'
|
||||
x-personal-mcp:
|
||||
id: python-logging
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- logging
|
||||
- python
|
||||
- observability
|
||||
capabilities:
|
||||
- resource://skills/python-logging/document
|
||||
---
|
||||
|
||||
# Python Logging
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user