Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2a2700b78c | ||
|
|
7162645ff8 | ||
|
|
44edffb8b7 | ||
|
|
c6817a074e | ||
|
|
7ac90d29dd | ||
|
|
0dc06f72ca | ||
|
|
cd11ea8255 | ||
|
|
bc21643e8c | ||
|
|
1ed5856db0 | ||
|
|
70695ff218 | ||
|
|
a238fb4dc3 | ||
|
|
b6f109cf91 |
@@ -14,4 +14,6 @@ Primary references:
|
|||||||
- [Prompt contract](../../docs/contracts/prompt.md)
|
- [Prompt contract](../../docs/contracts/prompt.md)
|
||||||
- [Frontmatter contract](../../docs/contracts/frontmatter.md)
|
- [Frontmatter contract](../../docs/contracts/frontmatter.md)
|
||||||
- [URI contract](../../docs/contracts/uris.md)
|
- [URI contract](../../docs/contracts/uris.md)
|
||||||
- [Zensical documentation authoring skill](../../docs/skills/zensical-docs/SKILL.md)
|
- `skill://zensical-docs/SKILL.md`
|
||||||
|
|
||||||
|
Inspect `skill://zensical-docs/_manifest` only when a supporting documentation reference is needed.
|
||||||
|
|||||||
@@ -0,0 +1,15 @@
|
|||||||
|
---
|
||||||
|
name: FastMCP Python Guidance
|
||||||
|
description: Route FastMCP Python changes to the Personal MCP source-reference skill.
|
||||||
|
applyTo: '**/*.py'
|
||||||
|
---
|
||||||
|
|
||||||
|
A core part of this repository is [FastMCP](https://gofastmcp.com/servers/server).
|
||||||
|
|
||||||
|
For FastMCP implementation or protocol questions, load `skill://mcp-details/SKILL.md` first.
|
||||||
|
|
||||||
|
Inspect `skill://mcp-details/_manifest` only when a source reference is needed, then read the relevant supporting path. For FastMCP Python APIs, prefer the supporting reference that covers SDKs and FastMCP.
|
||||||
|
|
||||||
|
This repository is resource-first and exposes resources as tools only as a fallback. In tool-only clients, use `list_resources` and `read_resource` with the same `skill://` URIs.
|
||||||
|
|
||||||
|
Reconcile the skill guidance with the installed FastMCP version and the repository's existing implementation before editing.
|
||||||
@@ -4,14 +4,15 @@ description: Route tests edits to the Personal MCP pytesting resource.
|
|||||||
applyTo: 'tests/**'
|
applyTo: 'tests/**'
|
||||||
---
|
---
|
||||||
|
|
||||||
When editing files under `tests/`, use `resource://skills/pytesting/document` as the primary guidance source for test scaffolding and pytest authoring decisions.
|
When editing files under `tests/`, use `skill://pytesting/SKILL.md` as the primary guidance source for test scaffolding and pytest authoring decisions.
|
||||||
|
|
||||||
Execution pattern:
|
Execution pattern:
|
||||||
|
|
||||||
1. Load `resource://skills/pytesting/document` first.
|
1. Load `skill://pytesting/SKILL.md` first.
|
||||||
2. Apply only the portions relevant to the file being edited.
|
2. Inspect `skill://pytesting/_manifest` only when a supporting reference is needed.
|
||||||
3. Keep tests focused, deterministic, and aligned with repository conventions.
|
3. Apply only the portions relevant to the file being edited.
|
||||||
4. Include source-document links for any feature-level recommendation.
|
4. Keep tests focused, deterministic, and aligned with repository conventions.
|
||||||
|
5. Include source-document links for any feature-level recommendation.
|
||||||
|
|
||||||
If task intent is ambiguous, ask one clarifying question before editing.
|
If task intent is ambiguous, ask one clarifying question before editing.
|
||||||
|
|
||||||
|
|||||||
@@ -4,12 +4,12 @@ description: Route .vscode edits to the Personal MCP VS Code configuration skill
|
|||||||
applyTo: '.vscode/**'
|
applyTo: '.vscode/**'
|
||||||
---
|
---
|
||||||
|
|
||||||
When editing files under `.vscode/`, use `resource://skills/vscode-configuration/document` as the primary guidance source.
|
When editing files under `.vscode/`, use `skill://vscode-configuration/SKILL.md` as the primary guidance source.
|
||||||
|
|
||||||
Execution pattern:
|
Execution pattern:
|
||||||
|
|
||||||
1. Load `resource://skills/vscode-configuration/document` first.
|
1. Load `skill://vscode-configuration/SKILL.md` first.
|
||||||
2. Select only the matching reference page for the current file type:
|
2. Inspect `skill://vscode-configuration/_manifest` and select only the matching reference page for the current file type:
|
||||||
- `launch.json` -> debug launch configurations.
|
- `launch.json` -> debug launch configurations.
|
||||||
- `tasks.json` -> tasks.json project tasks.
|
- `tasks.json` -> tasks.json project tasks.
|
||||||
- `mcp.json` -> mcp.json MCP server configuration.
|
- `mcp.json` -> mcp.json MCP server configuration.
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ description: Use the Zensical docs MCP resource when editing Markdown documentat
|
|||||||
applyTo: '**/*.md'
|
applyTo: '**/*.md'
|
||||||
---
|
---
|
||||||
|
|
||||||
When editing Markdown files in this repository, use the Zensical docs resource `resource://skills/zensical-docs/document` for relevant documentation authoring guidance.
|
When editing Markdown files in this repository, use `skill://zensical-docs/SKILL.md` for relevant documentation authoring guidance. Inspect `skill://zensical-docs/_manifest` only when a supporting reference is needed.
|
||||||
|
|
||||||
Prefer Zensical-native documentation conventions when they cover the need cleanly, while preserving expected MkDocs compatibility unless the Zensical guidance intentionally diverges.
|
Prefer Zensical-native documentation conventions when they cover the need cleanly, while preserving expected MkDocs compatibility unless the Zensical guidance intentionally diverges.
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -47,7 +47,7 @@ RUN --mount=type=cache,target=/root/.cache/uv \
|
|||||||
--mount=type=bind,source=uv.lock,target=uv.lock \
|
--mount=type=bind,source=uv.lock,target=uv.lock \
|
||||||
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
|
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
|
||||||
--mount=type=bind,source=src/,target=src/ \
|
--mount=type=bind,source=src/,target=src/ \
|
||||||
uv sync --no-editable
|
uv sync --no-editable --refresh-package prompts
|
||||||
|
|
||||||
USER appuser
|
USER appuser
|
||||||
|
|
||||||
|
|||||||
@@ -6,5 +6,3 @@ services:
|
|||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
ports:
|
ports:
|
||||||
- "8765:8765"
|
- "8765:8765"
|
||||||
volumes:
|
|
||||||
- ./docs:/app/src/personal_mcp/docs
|
|
||||||
|
|||||||
+47
-59
@@ -21,8 +21,8 @@ Prompt documents under `docs/prompts/` are also indexed and exposed as first-cla
|
|||||||
This architecture is anchored by three contracts:
|
This architecture is anchored by three contracts:
|
||||||
|
|
||||||
1. Docs-first authored content contract under `docs/` with strict per-skill ownership.
|
1. Docs-first authored content contract under `docs/` with strict per-skill ownership.
|
||||||
2. `SKILL.md` frontmatter contract with Anthropic fields plus `x-personal-mcp` metadata.
|
2. Standard `SKILL.md` frontmatter consumed directly by FastMCP.
|
||||||
3. Canonical resource URI contract with break-and-replace policy for contract changes.
|
3. Native `skill://` resource URIs with break-and-replace policy for contract changes.
|
||||||
|
|
||||||
Detailed contract pages:
|
Detailed contract pages:
|
||||||
|
|
||||||
@@ -51,11 +51,13 @@ Each skill encapsulates one methodology domain in a docs-owned directory:
|
|||||||
|
|
||||||
The skill document and references are the authored source of truth; runtime code indexes and serves these files without becoming a second authored source.
|
The skill document and references are the authored source of truth; runtime code indexes and serves these files without becoming a second authored source.
|
||||||
|
|
||||||
Each skill publishes resource families:
|
Each skill publishes three native resource families:
|
||||||
|
|
||||||
1. document
|
1. `skill://<name>/SKILL.md` for primary instructions
|
||||||
|
2. `skill://<name>/_manifest` for file discovery and integrity metadata
|
||||||
|
3. `skill://<name>/{path*}` for supporting files
|
||||||
|
|
||||||
The document resource returns canonical Markdown, while clients can perform any downstream section extraction they need.
|
The main resource returns canonical Markdown. The generated manifest lists real relative paths, sizes, and SHA256 hashes so clients can load supporting material selectively.
|
||||||
|
|
||||||
### Prompt Modules
|
### Prompt Modules
|
||||||
|
|
||||||
@@ -70,29 +72,27 @@ This keeps authored markdown as source-of-truth while allowing clients to discov
|
|||||||
|
|
||||||
### Catalog Module
|
### Catalog Module
|
||||||
|
|
||||||
The catalog is the canonical discovery layer and publishes normalized records for all modules. It may also expose a minimal set of read-only discovery tools that resolve back to the same canonical markdown content when a client chat surface does not expose MCP resource attachment.
|
The catalog publishes normalized records for prompts. Skills use FastMCP's native resource discovery and client utilities instead of a parallel catalog.
|
||||||
|
|
||||||
Typical catalog resources:
|
Typical catalog resources:
|
||||||
|
|
||||||
1. resource://catalog/skills_index
|
1. resource://catalog/prompts_index
|
||||||
2. resource://catalog/skills_index{?q,tag,capability,cursor,limit}
|
2. resource://catalog/prompts_index{?q,tag,cursor,limit}
|
||||||
3. resource://catalog/skills/{skill_id}
|
3. resource://catalog/prompts/{prompt_id}
|
||||||
4. resource://catalog/prompts_index
|
|
||||||
5. resource://catalog/prompts_index{?q,tag,cursor,limit}
|
|
||||||
6. resource://catalog/prompts/{prompt_id}
|
|
||||||
|
|
||||||
Only canonical catalog resources are part of the runtime contract in this phase.
|
Only canonical catalog resources are part of the runtime contract in this phase.
|
||||||
|
|
||||||
### Registry Loader
|
### Registry Loader
|
||||||
|
|
||||||
Importing the package does not read or parse documentation. The MCP server and FastAPI application factories request the registry when constructing a runnable server, using packaged resources through `importlib.resources.files(...)` and `Traversable` APIs.
|
Importing the package does not read or parse documentation. The MCP server and FastAPI application factories initialize content when constructing a runnable server. The prompt/docs registry reads packaged resources through `importlib.resources.files(...)` and `Traversable` APIs; the native skills provider receives the packaged `personal_mcp/docs/skills` filesystem path.
|
||||||
|
|
||||||
Loader responsibilities:
|
Loader responsibilities:
|
||||||
|
|
||||||
1. Parse SKILL.md frontmatter for each skill.
|
1. Parse and validate prompt frontmatter.
|
||||||
2. Validate schema and cross-field constraints before any resource registration.
|
2. Build the prompt catalog and MCP prompt objects.
|
||||||
3. Build an in-memory registry keyed by `skill_id`.
|
3. Index authored Markdown for `resource://docs/{path*}`.
|
||||||
4. Fail fast for duplicate ids, missing markdown files, and broken reference mappings.
|
|
||||||
|
Skill loading is owned by `SkillsDirectoryProvider`, which scans the packaged skills directory and constructs native resources before the server starts serving requests.
|
||||||
|
|
||||||
The immutable registry is cached for the process lifetime. Each Uvicorn worker constructs and retains its own registry because worker processes do not share Python objects. Registry load failure is a server-factory startup error, not a package-import error or partial runtime warning.
|
The immutable registry is cached for the process lifetime. Each Uvicorn worker constructs and retains its own registry because worker processes do not share Python objects. Registry load failure is a server-factory startup error, not a package-import error or partial runtime warning.
|
||||||
|
|
||||||
@@ -102,7 +102,7 @@ Content is authored in markdown under `docs/` and managed as long-form reference
|
|||||||
|
|
||||||
The repository root `docs/` directory is the only authored source. The `src/personal_mcp/docs` path is a relative symlink to that directory for source-checkout and editable-install workflows; it is not a second content tree and packaging does not depend on traversing it.
|
The repository root `docs/` directory is the only authored source. The `src/personal_mcp/docs` path is a relative symlink to that directory for source-checkout and editable-install workflows; it is not a second content tree and packaging does not depend on traversing it.
|
||||||
|
|
||||||
For wheel builds, [Hatchling forced inclusion](https://hatch.pypa.io/latest/config/build/#forced-inclusion) maps the root `docs/` tree to `personal_mcp/docs/`. The wheel therefore contains regular resource files at that destination rather than a symlink. Runtime registry loading uses [`importlib.resources.files`](https://docs.python.org/3/library/importlib.resources.html#importlib.resources.files) and `Traversable` operations from the `personal_mcp` package anchor, so it does not depend on the repository layout or current working directory.
|
For wheel builds, Hatchling's normal `src/personal_mcp` package traversal follows the relative `docs` symlink and archives its targets as regular files under `personal_mcp/docs/`. No `force-include` mapping is used because that would add the same archive paths twice. The prompt/docs registry uses [`importlib.resources.files`](https://docs.python.org/3/library/importlib.resources.html#importlib.resources.files), while `SkillsDirectoryProvider` scans the package-relative filesystem path. Neither path depends on the current working directory.
|
||||||
|
|
||||||
### Static Docs Surface
|
### Static Docs Surface
|
||||||
|
|
||||||
@@ -119,36 +119,29 @@ Generated `site/` files are deployment assets for the human-facing static site.
|
|||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart TD
|
flowchart TD
|
||||||
A[Authored Markdown] --> C[Resource Handlers]
|
A[Authored Skill Directories] --> B[SkillsDirectoryProvider]
|
||||||
B[Pattern Metadata] --> D[Catalog Resources]
|
B --> C[Native Skill Resources]
|
||||||
A --> E[Zensical Static Build]
|
D[Authored Prompts and Docs] --> E[Prompt and Docs Registry]
|
||||||
E --> H[FastAPI Static Mount]
|
E --> F[Prompt Catalog and Docs Resources]
|
||||||
H --> I[Served Docs Site]
|
A --> G[Zensical Static Build]
|
||||||
D --> I
|
D --> G
|
||||||
|
G --> H[FastAPI Static Mount]
|
||||||
```
|
```
|
||||||
|
|
||||||
## Contracts
|
## Contracts
|
||||||
|
|
||||||
### Metadata Contract
|
### Metadata Contract
|
||||||
|
|
||||||
Each skill declares frontmatter in `docs/skills/<skill-id>/SKILL.md`.
|
Each skill declares standard frontmatter in `docs/skills/<skill-id>/SKILL.md`.
|
||||||
|
|
||||||
For the full field-level contract, validation model, and FastMCP metadata mapping, see [Frontmatter Contract](./contracts/frontmatter.md).
|
For the full field-level contract, validation model, and FastMCP metadata mapping, see [Frontmatter Contract](./contracts/frontmatter.md).
|
||||||
|
|
||||||
Anthropic-facing required fields:
|
Required fields:
|
||||||
|
|
||||||
1. name
|
1. name
|
||||||
2. description
|
2. description
|
||||||
|
|
||||||
Repository indexing metadata is declared in `x-personal-mcp`:
|
The directory name is the provider identity and must match `name`. There is no skill catalog metadata or sidecar.
|
||||||
|
|
||||||
1. id
|
|
||||||
2. version
|
|
||||||
3. tags
|
|
||||||
4. capabilities
|
|
||||||
5. optional references map (for nested entries, overrides, and aliases)
|
|
||||||
|
|
||||||
No `metadata.yaml` sidecar is part of the end-state contract.
|
|
||||||
|
|
||||||
### URI Contract
|
### URI Contract
|
||||||
|
|
||||||
@@ -156,28 +149,24 @@ Canonical resource URIs are:
|
|||||||
|
|
||||||
For the full URI semantics, parameter validation rules, and compatibility policy, see [URI Contract](./contracts/uris.md).
|
For the full URI semantics, parameter validation rules, and compatibility policy, see [URI Contract](./contracts/uris.md).
|
||||||
|
|
||||||
1. resource://skills/<skill_id>/document
|
1. skill://<skill_name>/SKILL.md
|
||||||
2. resource://skills/<skill_id>/references/<ref_id>
|
2. skill://<skill_name>/_manifest
|
||||||
3. resource://catalog/skills_index
|
3. skill://<skill_name>/<supporting_path>
|
||||||
4. resource://catalog/skills_index{?q,tag,capability,cursor,limit}
|
4. resource://docs/{path*}
|
||||||
5. resource://catalog/skills/{skill_id}
|
5. resource://catalog/prompts_index
|
||||||
6. resource://docs/{path*}
|
6. resource://catalog/prompts_index{?q,tag,cursor,limit}
|
||||||
7. resource://catalog/prompts_index
|
7. resource://catalog/prompts/{prompt_id}
|
||||||
8. resource://catalog/prompts_index{?q,tag,cursor,limit}
|
8. resource://prompts/{prompt_id}/document
|
||||||
9. resource://catalog/prompts/{prompt_id}
|
|
||||||
10. resource://prompts/{prompt_id}/document
|
|
||||||
|
|
||||||
Validation rules:
|
Validation rules:
|
||||||
|
|
||||||
1. `skill_id` is lowercase kebab-case and must satisfy the stable skill id contract.
|
1. `skill_name` is the lowercase kebab-case skill directory name.
|
||||||
2. `ref_id` is lowercase kebab-case and must resolve from either:
|
2. `supporting_path` is a provider-validated relative path within that skill.
|
||||||
- top-level auto-discovery of `references/*.md` filename stems, or
|
3. Docs `path*` resolves only to normalized Markdown paths under `docs/`.
|
||||||
- an explicit `x-personal-mcp.references` entry.
|
|
||||||
3. `path*` resolves only to normalized markdown paths under `docs/`.
|
|
||||||
|
|
||||||
### Resource Registration Contract
|
### Resource Registration Contract
|
||||||
|
|
||||||
Resources are registered from the validated registry, not by ad hoc per-skill hardcoding.
|
Skill resources are registered by one `SkillsDirectoryProvider`; prompt and docs resources remain registered from the validated registry.
|
||||||
|
|
||||||
Registration rules:
|
Registration rules:
|
||||||
|
|
||||||
@@ -232,16 +221,17 @@ Clients can use Ask, Edit, or Agent modes without requiring prompt-first orchest
|
|||||||
## Authoring and Publishing Lifecycle
|
## Authoring and Publishing Lifecycle
|
||||||
|
|
||||||
1. Update markdown reference content.
|
1. Update markdown reference content.
|
||||||
2. Update metadata if capability surface changes.
|
2. Keep skill `name` and directory identity aligned.
|
||||||
3. Build static docs with Zensical.
|
3. Build static docs with Zensical and run provider tests.
|
||||||
4. Serve built output through FastAPI static mount.
|
4. Package authored docs into `personal_mcp/docs/`.
|
||||||
|
5. Serve native MCP resources and the static docs mount.
|
||||||
|
|
||||||
## Scope and Non-Goals
|
## Scope and Non-Goals
|
||||||
|
|
||||||
In-scope:
|
In-scope:
|
||||||
|
|
||||||
1. Resource-first methodology delivery
|
1. Resource-first methodology delivery
|
||||||
2. Catalog-based discovery
|
2. Native FastMCP skill discovery
|
||||||
3. Pre-built static docs hosting in app runtime
|
3. Pre-built static docs hosting in app runtime
|
||||||
|
|
||||||
Out-of-scope:
|
Out-of-scope:
|
||||||
@@ -250,9 +240,7 @@ Out-of-scope:
|
|||||||
2. Large tool inventories duplicating static guidance across skill modules
|
2. Large tool inventories duplicating static guidance across skill modules
|
||||||
3. Separate dynamic docs service at runtime
|
3. Separate dynamic docs service at runtime
|
||||||
|
|
||||||
Allowed exception:
|
The prompt catalog remains an independent surface. Tool-only skill clients use generic resource tools rather than a skill-specific compatibility layer.
|
||||||
|
|
||||||
1. A small catalog-level tool layer is acceptable when it improves client interoperability without creating a second source of truth for skill content.
|
|
||||||
|
|
||||||
## Example Content Inputs
|
## Example Content Inputs
|
||||||
|
|
||||||
@@ -263,4 +251,4 @@ Existing markdown reference sets are valid examples of authored source material
|
|||||||
3. docs/skills/python-logging/references/json-file-logging.md
|
3. docs/skills/python-logging/references/json-file-logging.md
|
||||||
4. docs/skills/fastapi-uv-docker/references/fastapi-best-practices.md
|
4. docs/skills/fastapi-uv-docker/references/fastapi-best-practices.md
|
||||||
|
|
||||||
These inputs are treated as content sources, while resource URIs and catalog payloads remain the machine-facing contracts.
|
These inputs are treated as content sources, while native skill URIs and generated manifests form the machine-facing skill contract.
|
||||||
|
|||||||
+81
-213
@@ -4,259 +4,127 @@ icon: lucide/pencil
|
|||||||
|
|
||||||
# Authoring Guide
|
# Authoring Guide
|
||||||
|
|
||||||
This page defines the practical authoring workflow for this repository so Markdown remains the single source of truth for both published docs and MCP resources.
|
This page defines the practical workflow for maintaining skills, prompts, and project documentation while keeping root `docs/` as the only authored source.
|
||||||
|
|
||||||
Primary references:
|
Primary references:
|
||||||
- [Skill contract](./contracts/skill_contract.md)
|
|
||||||
- [Prompt contract](./contracts/prompt.md)
|
|
||||||
- [Frontmatter contract](./contracts/frontmatter.md)
|
|
||||||
- [URI contract](./contracts/uris.md)
|
|
||||||
- [Zensical documentation authoring skill](./skills/zensical-docs/SKILL.md)
|
|
||||||
|
|
||||||
## What You Author
|
1. [Skill Contract](./contracts/skill_contract.md)
|
||||||
|
2. [Prompt Contract](./contracts/prompt.md)
|
||||||
This repository has two primary authored content types:
|
3. [Frontmatter Contract](./contracts/frontmatter.md)
|
||||||
|
4. [URI Contract](./contracts/uris.md)
|
||||||
1. Skills under `docs/skills/<skill-id>/`.
|
5. [Zensical documentation skill](./skills/zensical-docs/SKILL.md)
|
||||||
2. Prompts under `docs/prompts/<prompt-id>/`.
|
|
||||||
|
|
||||||
Each module keeps one canonical document plus optional references:
|
|
||||||
|
|
||||||
```text
|
|
||||||
docs/
|
|
||||||
skills/<skill-id>/
|
|
||||||
SKILL.md
|
|
||||||
references/
|
|
||||||
*.md
|
|
||||||
|
|
||||||
prompts/<prompt-id>/
|
|
||||||
PROMPT.md
|
|
||||||
references/
|
|
||||||
*.md
|
|
||||||
```
|
|
||||||
|
|
||||||
## Source Tree Ownership
|
## Source Tree Ownership
|
||||||
|
|
||||||
Edit content only under the repository root `docs/` directory. The `src/personal_mcp/docs` path is a relative symlink provided so package-oriented tooling and editable installs see the same files; do not replace it with copied content or author files through a second tree.
|
Edit content only under root `docs/`. The `src/personal_mcp/docs` path is a relative symlink for editable installs; do not author through a copied package tree.
|
||||||
|
|
||||||
[Hatchling forced inclusion](https://hatch.pypa.io/latest/config/build/#forced-inclusion) projects root `docs/` into `personal_mcp/docs/` when building the wheel. Installed code reads that destination through [`importlib.resources`](https://docs.python.org/3/library/importlib.resources.html), while Zensical continues to build the human-facing site directly from root `docs/`.
|
Hatchling's normal package traversal follows `src/personal_mcp/docs` during wheel builds and archives the linked targets as regular files under `personal_mcp/docs/`. Do not add a `force-include` entry for root `docs/`; it duplicates those wheel paths. The installed package therefore gives `SkillsDirectoryProvider` a regular filesystem directory while Zensical builds the human site directly from root `docs/`.
|
||||||
|
|
||||||
Package import does not load these resources. A runnable MCP or FastAPI server loads and validates them when its factory runs, then caches the immutable registry for that process. Restart initialized development or worker processes after changing authored Markdown.
|
Generated `site/` content is a build artifact and must not be edited by hand.
|
||||||
|
|
||||||
## Authoring Principles
|
## Content Layout
|
||||||
|
|
||||||
1. Keep Markdown as the canonical source and avoid duplicating content into alternate metadata files.
|
```text
|
||||||
2. Prefer resource-first discovery paths (`resource://catalog/...` then `resource://skills/...` or `resource://prompts/...`).
|
docs/
|
||||||
3. Keep pages focused and composable: overview in the primary doc, details in `references/`.
|
*.md
|
||||||
4. Use descriptive inline links for external sources instead of bare URLs.
|
contracts/
|
||||||
5. Use stable ids and slugs; renames are breaking changes and should be intentional.
|
prompts/<prompt-id>/
|
||||||
|
PROMPT.md
|
||||||
|
references/
|
||||||
|
skills/<skill-name>/
|
||||||
|
SKILL.md
|
||||||
|
references/
|
||||||
|
```
|
||||||
|
|
||||||
## Skill Authoring Workflow
|
Keep skill and prompt files inside their owning directories. Relative links may cross sections, but content ownership should remain clear.
|
||||||
|
|
||||||
When creating or updating a skill:
|
## Skill Authoring
|
||||||
|
|
||||||
1. Confirm slug format is lowercase kebab-case.
|
A skill is discovered when a direct child of `docs/skills/` contains `SKILL.md`.
|
||||||
2. Keep directory name, `name`, and `x-personal-mcp.id` aligned.
|
|
||||||
3. Ensure capabilities include `resource://skills/<skill-id>/document`.
|
Required frontmatter:
|
||||||
4. Place supporting material under `references/`.
|
|
||||||
5. Use explicit frontmatter reference entries only when you need overrides or nested mappings.
|
```yaml
|
||||||
|
---
|
||||||
|
name: <skill-name>
|
||||||
|
description: <what the skill does and when to use it>
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
1. Use lowercase kebab-case for the directory and `name`.
|
||||||
|
2. Keep `name` exactly equal to the directory name.
|
||||||
|
3. Write a specific description because clients use it for discovery.
|
||||||
|
4. Do not add `x-personal-mcp`, versions, tags, capabilities, or reference mappings.
|
||||||
|
5. Put supporting material anywhere beneath the skill directory, normally under `references/`.
|
||||||
|
6. Link supporting files from `SKILL.md` so humans and agents understand when to load them.
|
||||||
|
|
||||||
|
FastMCP recursively scans every skill file and generates `skill://<name>/_manifest`. Supporting-resource identity is the real relative path, not a synthetic reference id.
|
||||||
|
|
||||||
Recommended sequence:
|
Recommended sequence:
|
||||||
|
|
||||||
1. Draft `SKILL.md` intent and routing sections.
|
1. Draft or revise `SKILL.md` routing guidance.
|
||||||
2. Add or refine `references/*.md`.
|
2. Add focused supporting files.
|
||||||
3. Verify links and example commands.
|
3. Verify relative links.
|
||||||
4. Run docs build and tests.
|
4. Run the provider tests and docs build.
|
||||||
|
5. Restart running servers because production uses `reload=False`.
|
||||||
|
|
||||||
For exact metadata rules, see [Frontmatter contract](./contracts/frontmatter.md) and [Skill contract](./contracts/skill_contract.md).
|
## Prompt Authoring
|
||||||
|
|
||||||
## Prompt Authoring Workflow
|
Prompts remain registry-backed:
|
||||||
|
|
||||||
When creating or updating a prompt module:
|
|
||||||
|
|
||||||
1. Keep one canonical `PROMPT.md`.
|
1. Keep one canonical `PROMPT.md`.
|
||||||
2. Keep `name`, `x-personal-mcp.id`, and directory slug aligned.
|
2. Align directory name, `name`, and `x-personal-mcp.id`.
|
||||||
3. Include `resource://prompts/<prompt-id>/document` in capabilities.
|
3. Include `resource://prompts/<prompt-id>/document` in capabilities.
|
||||||
4. Define prompt arguments in `x-personal-mcp.arguments` when inputs are required.
|
4. Define arguments beneath `x-personal-mcp.arguments`.
|
||||||
5. Keep long rationale and source notes in `references/` to preserve prompt clarity.
|
5. Keep long rationale and sources in `references/`.
|
||||||
|
|
||||||
For exact structure, see [Prompt contract](./contracts/prompt.md).
|
Prompt argument names must be valid Python identifiers. Each argument accepts optional `title`, `description`, and `required`; unknown fields fail strict validation.
|
||||||
|
|
||||||
## Prompt Argument Mechanics
|
## Frontmatter Safety
|
||||||
|
|
||||||
When defining prompt inputs, keep argument metadata aligned with the prompt contract and runtime behavior.
|
1. Quote scalar values containing `:`.
|
||||||
|
2. Quote values with reserved YAML characters such as `#`, `{}`, `[]`, or leading `*`.
|
||||||
|
3. Use block scalars for punctuation-heavy multiline text.
|
||||||
|
4. Keep fields within the applicable skill or prompt contract.
|
||||||
|
|
||||||
1. Define arguments under `x-personal-mcp.arguments` as a map keyed by argument name.
|
## Writing Quality
|
||||||
2. Argument names must match Python identifier format: `^[A-Za-z_][A-Za-z0-9_]*$`.
|
|
||||||
3. Each argument entry supports only:
|
|
||||||
- `title` (optional)
|
|
||||||
- `description` (optional)
|
|
||||||
- `required` (optional, defaults to `false`)
|
|
||||||
4. Unknown argument fields are rejected by strict frontmatter validation.
|
|
||||||
5. Prompt argument metadata appears in `resource://catalog/prompts/{prompt_id}`, and MCP prompt objects expose the same arguments for prompt-list/get-prompt workflows.
|
|
||||||
6. Enum-like constraints are not a native argument field; encode allowed values in `description`.
|
|
||||||
|
|
||||||
### Frontmatter Safety Rules
|
1. Prefer focused sections and descriptive headings.
|
||||||
|
2. Link feature-level claims to authoritative sources.
|
||||||
|
3. Use relative links for internal pages.
|
||||||
|
4. Keep code examples minimal and actionable.
|
||||||
|
5. Avoid bare URLs in prose.
|
||||||
|
6. Load only supporting material relevant to the immediate task.
|
||||||
|
|
||||||
Use these rules to avoid YAML parse failures in prompt and skill frontmatter:
|
## Copilot Routing
|
||||||
|
|
||||||
1. Quote any scalar value that contains `:` (for example, `description: "Enum: skill | prompt | shim"`).
|
Active instructions should point directly to native main resources:
|
||||||
2. Prefer quoted scalars for values with reserved YAML characters such as `#`, `{}`, `[]`, or leading `*`.
|
|
||||||
3. If a description needs multiple lines, use a block scalar (`|`) instead of packing punctuation-heavy text into one line.
|
|
||||||
4. Keep frontmatter keys simple and contract-bound; do not add undeclared argument fields.
|
|
||||||
|
|
||||||
### Validation Timing
|
1. `skill://zensical-docs/SKILL.md`
|
||||||
|
2. `skill://pytesting/SKILL.md`
|
||||||
|
3. `skill://vscode-configuration/SKILL.md`
|
||||||
|
|
||||||
Run validation immediately after frontmatter edits, not only at the end of a task:
|
When deeper guidance is needed, read the selected skill's `_manifest` and fetch supporting files by their listed path. Tool-only clients use `list_resources` and `read_resource` over the same URIs.
|
||||||
|
|
||||||
1. First pass after metadata changes: `uv run zensical build`
|
|
||||||
2. Prompt/skill load verification: `uv run pytest -q`
|
|
||||||
3. Final full pass before completion: run the full checklist in [Validation Checklist](#validation-checklist)
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
x-personal-mcp:
|
|
||||||
arguments:
|
|
||||||
artifact_type:
|
|
||||||
title: Artifact type
|
|
||||||
description: Allowed values are skill, prompt, or shim.
|
|
||||||
required: true
|
|
||||||
scope_glob:
|
|
||||||
title: Scope glob
|
|
||||||
description: Optional applyTo glob for shim outputs.
|
|
||||||
required: false
|
|
||||||
```
|
|
||||||
|
|
||||||
References:
|
|
||||||
|
|
||||||
1. [Frontmatter contract](./contracts/frontmatter.md)
|
|
||||||
2. [URI contract](./contracts/uris.md)
|
|
||||||
3. [Resource-First Pattern Module Architecture](./architecture.md)
|
|
||||||
4. [Prompt objects concept docs](https://modelcontextprotocol.io/docs/learn/server-concepts#prompts)
|
|
||||||
|
|
||||||
## Writing Quality Rules
|
|
||||||
|
|
||||||
Apply these defaults to all docs pages:
|
|
||||||
|
|
||||||
1. Prefer short sections with strong headings over long unbroken prose.
|
|
||||||
2. Keep claims source-linked, especially for MCP, FastMCP, pytest, FastAPI, SQLAlchemy, and Zensical behavior.
|
|
||||||
3. Prefer relative links for internal docs paths.
|
|
||||||
4. Use code blocks for commands and configuration snippets.
|
|
||||||
5. Keep examples minimal and actionable.
|
|
||||||
|
|
||||||
Source examples:
|
|
||||||
- [Model Context Protocol docs](https://modelcontextprotocol.io/docs/getting-started/intro)
|
|
||||||
- [FastMCP docs](https://gofastmcp.com/getting-started/welcome)
|
|
||||||
- [Zensical docs](https://zensical.org/docs/)
|
|
||||||
|
|
||||||
## Authoring for GitHub Copilot
|
|
||||||
|
|
||||||
For resource selection or tool-based matching to work well, each skill should have:
|
|
||||||
|
|
||||||
1. precise `description`
|
|
||||||
2. focused `tags`
|
|
||||||
3. explicit `capabilities`
|
|
||||||
4. stable `id` and slug naming
|
|
||||||
|
|
||||||
Weak metadata reduces Copilot match quality and increases wrong context injection.
|
|
||||||
|
|
||||||
### Copilot Instruction Authoring Pattern
|
|
||||||
|
|
||||||
If you want Copilot to use `personal-mcp` skill content more reliably, instruction files should describe three things clearly:
|
|
||||||
|
|
||||||
1. when MCP-backed skill guidance is relevant
|
|
||||||
2. which retrieval path Copilot should prefer first
|
|
||||||
3. how much skill context it should load before answering
|
|
||||||
|
|
||||||
Instructions strongly steer discovery behavior, but they do not force VS Code to auto-attach MCP resources. Keep wording explicit about preferred path and fallback path.
|
|
||||||
|
|
||||||
Repository policy:
|
|
||||||
|
|
||||||
1. start from catalog discovery
|
|
||||||
2. prefer MCP resources when the current chat surface exposes resource attachment
|
|
||||||
3. fall back to catalog tools when resource attachment is unavailable
|
|
||||||
4. keep loaded skill context bounded
|
|
||||||
|
|
||||||
Suggested instruction text:
|
|
||||||
|
|
||||||
```md
|
|
||||||
When a task may match a documented implementation pattern from `personal-mcp`:
|
|
||||||
|
|
||||||
1. Start with catalog-first discovery.
|
|
||||||
2. Prefer MCP resources when the chat surface exposes resource attachment.
|
|
||||||
3. If MCP resource attachment is unavailable, use `list_resources`/`read_resource` first, then thin catalog tools if needed.
|
|
||||||
4. Load only the most relevant skill document, or at most 2 skill documents.
|
|
||||||
5. Reconcile loaded skill guidance with the actual repository code before making changes.
|
|
||||||
|
|
||||||
Preferred resource order:
|
|
||||||
|
|
||||||
1. `resource://catalog/skills_index`
|
|
||||||
2. `resource://catalog/skills/{skill_id}`
|
|
||||||
3. `resource://skills/<skill-id>/document`
|
|
||||||
4. `resource://skills/<skill-id>/references/<ref-id>` when needed
|
|
||||||
|
|
||||||
Preferred tool fallback order:
|
|
||||||
|
|
||||||
1. `list_resources`
|
|
||||||
2. `read_resource`
|
|
||||||
3. `search_patterns`
|
|
||||||
4. `get_pattern_by_id`
|
|
||||||
5. `get_skill_document_by_id`
|
|
||||||
|
|
||||||
Compatibility aliases for clients that use `catalog_*` naming are also available:
|
|
||||||
|
|
||||||
1. `catalog_search_patterns`
|
|
||||||
2. `catalog_get_pattern_by_id`
|
|
||||||
3. `catalog_get_skill_document_by_id`
|
|
||||||
4. `catalog_search_prompts`
|
|
||||||
5. `catalog_get_prompt_by_id`
|
|
||||||
|
|
||||||
Use canonical names first; aliases exist only to preserve interoperability when a client emits non-canonical names.
|
|
||||||
|
|
||||||
If confidence is low after discovery, ask one clarifying question before loading more context.
|
|
||||||
```
|
|
||||||
|
|
||||||
This is guidance, not a guarantee. It defines a reliable policy while preserving the resource-first architecture.
|
|
||||||
|
|
||||||
Thin shim path binding guidance for MCP consumers is covered in [Skill Usage Mechanics](./usage.md).
|
|
||||||
|
|
||||||
## Zensical Details
|
|
||||||
|
|
||||||
When adding or restructuring pages:
|
|
||||||
|
|
||||||
1. Update navigation in `zensical.toml`.
|
|
||||||
2. Ensure top-level pages include frontmatter with an icon.
|
|
||||||
3. Keep naming and labels concise so navigation remains scannable.
|
|
||||||
|
|
||||||
Top-level page pattern:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
---
|
|
||||||
icon: lucide/pencil
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
## Validation Checklist
|
## Validation Checklist
|
||||||
|
|
||||||
Run these checks before considering authoring changes complete:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
uv run pytest tests/skills/test_provider.py tests/web/test_mcp_skills.py -q
|
||||||
uv run zensical build
|
uv run zensical build
|
||||||
uv run ruff check .
|
uv run ruff check .
|
||||||
uv run ty check
|
uv run ty check
|
||||||
uv run pytest
|
uv run pytest
|
||||||
```
|
```
|
||||||
|
|
||||||
Address any errors or warnings that result.
|
For packaging changes, also build and inspect an installed wheel so provider path resolution is verified outside the editable checkout.
|
||||||
|
|
||||||
If a change only affects docs content, `uv run zensical build` is still required.
|
## Navigation
|
||||||
|
|
||||||
## Quick Authoring Checklist
|
When adding or moving pages:
|
||||||
|
|
||||||
1. Correct location (`skills/` or `prompts/`).
|
1. update `zensical.toml`
|
||||||
2. Frontmatter id and slug alignment.
|
2. keep top-level page icons in frontmatter
|
||||||
3. Capability URI present.
|
3. rebuild the site
|
||||||
4. Links valid and descriptive.
|
4. verify internal links and navigation labels
|
||||||
5. Navigation updated when needed.
|
|
||||||
6. Validation commands passed.
|
|
||||||
|
|||||||
+50
-204
@@ -4,231 +4,77 @@ icon: lucide/braces
|
|||||||
|
|
||||||
# Frontmatter Contract
|
# Frontmatter Contract
|
||||||
|
|
||||||
This page defines the `SKILL.md` frontmatter and FastMCP metadata contract.
|
This page defines the authored frontmatter contracts for native FastMCP skills and registry-backed prompts.
|
||||||
|
|
||||||
Prompt modules use the same contract style in `docs/prompts/<prompt-id>/PROMPT.md` with prompt-specific capability and MCP-aligned prompt argument metadata.
|
## Skill Frontmatter
|
||||||
|
|
||||||
## Validated Frontmatter Surface
|
Skills use the standard Agent Skills fields consumed by the [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills):
|
||||||
|
|
||||||
The registry runtime validates a strict, standard-only frontmatter surface:
|
|
||||||
|
|
||||||
1. Top-level fields accepted for skills: `name`, `description`, `x-personal-mcp`.
|
|
||||||
2. Top-level fields accepted for prompts: `name`, `description`, `x-personal-mcp`.
|
|
||||||
3. Unknown top-level fields are rejected during registry load.
|
|
||||||
|
|
||||||
Skill and prompt identifier rules:
|
|
||||||
|
|
||||||
1. `name` is required, 1-64 chars, lowercase kebab-case, and must not contain `anthropic` or `claude`.
|
|
||||||
2. `description` is required, 1-1024 chars.
|
|
||||||
3. `x-personal-mcp.id` must exactly match `name`.
|
|
||||||
4. Directory slug must exactly match `name`.
|
|
||||||
|
|
||||||
Capability invariants:
|
|
||||||
|
|
||||||
1. Skill capabilities must include `resource://skills/<skill-id>/document`.
|
|
||||||
2. Prompt capabilities must include `resource://prompts/<prompt-id>/document`.
|
|
||||||
|
|
||||||
Repository contract decisions:
|
|
||||||
|
|
||||||
1. Treat `name` and `description` as required in all `SKILL.md` files.
|
|
||||||
2. Keep only validated standard fields at top level.
|
|
||||||
3. Keep MCP indexing metadata in a namespaced extension block.
|
|
||||||
4. Reject unsupported optional top-level fields until explicit model support is added.
|
|
||||||
|
|
||||||
Reference specs:
|
|
||||||
|
|
||||||
1. MCP prompts data types: [Prompts](https://modelcontextprotocol.io/specification/latest/server/prompts)
|
|
||||||
2. MCP schema reference for `Prompt` and `PromptArgument`: [Schema](https://modelcontextprotocol.io/specification/latest/schema)
|
|
||||||
|
|
||||||
## Canonical Frontmatter Schema
|
|
||||||
|
|
||||||
Use this two-layer pattern:
|
|
||||||
|
|
||||||
1. Anthropic layer: top-level fields intended for Anthropic and Agent Skills behavior.
|
|
||||||
2. Repository layer: one namespaced block, `x-personal-mcp`, for MCP catalog and routing metadata.
|
|
||||||
|
|
||||||
Canonical shape:
|
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
---
|
---
|
||||||
name: <skill-id>
|
name: <skill-id>
|
||||||
description: <what this skill does and when to use it>
|
description: <what the skill does and when to use it>
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
# Repository-specific metadata
|
Rules:
|
||||||
|
|
||||||
|
1. `name` and `description` are required.
|
||||||
|
2. `name` must equal the skill directory name.
|
||||||
|
3. The repository uses lowercase kebab-case directory names.
|
||||||
|
4. Skill frontmatter contains no `x-personal-mcp` catalog metadata.
|
||||||
|
5. Supporting files require no frontmatter manifest. The provider discovers files recursively and generates `_manifest` with relative paths, byte sizes, and SHA256 hashes.
|
||||||
|
|
||||||
|
The provider uses the directory name as the URI identity and the frontmatter `description` as the main resource description. Repository tests enforce directory/name parity and reject extra skill frontmatter fields.
|
||||||
|
|
||||||
|
## Prompt Frontmatter
|
||||||
|
|
||||||
|
Prompts remain registry-backed and retain repository metadata:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
name: <prompt-id>
|
||||||
|
description: <what the prompt does and when to use it>
|
||||||
x-personal-mcp:
|
x-personal-mcp:
|
||||||
id: <skill-id>
|
id: <prompt-id>
|
||||||
version: <semver>
|
version: <semver>
|
||||||
tags:
|
tags:
|
||||||
- <tag>
|
- <tag>
|
||||||
capabilities:
|
capabilities:
|
||||||
- resource://skills/<skill-id>/document
|
- resource://prompts/<prompt-id>/document
|
||||||
# Optional: overrides and nested references only.
|
arguments:
|
||||||
# Top-level references/*.md are auto-discovered.
|
<argument-name>:
|
||||||
references:
|
title: <display title>
|
||||||
<ref-id>:
|
description: <input guidance>
|
||||||
path: references/<file>.md
|
required: true
|
||||||
mime_type: text/markdown
|
|
||||||
title: <short title>
|
|
||||||
---
|
---
|
||||||
```
|
```
|
||||||
|
|
||||||
## Repository Metadata Field Rules
|
Prompt rules:
|
||||||
|
|
||||||
Rules for `x-personal-mcp`:
|
1. `name`, `description`, and `x-personal-mcp` are required.
|
||||||
|
2. `x-personal-mcp.id`, `name`, and the prompt directory name must match.
|
||||||
|
3. `version` must be semantic version text.
|
||||||
|
4. `capabilities` must include `resource://prompts/<prompt-id>/document`.
|
||||||
|
5. Argument names must be valid Python identifiers.
|
||||||
|
6. Argument entries accept optional `title`, `description`, and `required` fields.
|
||||||
|
7. Unknown prompt fields are rejected by the strict Pydantic registry models.
|
||||||
|
|
||||||
1. `id` is required, must follow the skill id rules from the content contract, and must equal the directory name.
|
See the MCP [prompts concept documentation](https://modelcontextprotocol.io/docs/learn/server-concepts#prompts) and [schema reference](https://modelcontextprotocol.io/specification/latest/schema) for the protocol-level prompt shape.
|
||||||
2. `version` is required and must be a semantic version string.
|
|
||||||
3. `tags` is optional and should be a list of kebab-case discovery labels.
|
|
||||||
4. `capabilities` is required and lists the MCP URIs the skill publishes.
|
|
||||||
5. `references` is an optional map keyed by `ref-id` for overrides and nested entries.
|
|
||||||
|
|
||||||
Prompt-specific additions:
|
## Validation Timing
|
||||||
|
|
||||||
1. `arguments` is an optional map keyed by argument name.
|
Skill validation is file- and provider-oriented:
|
||||||
2. Each argument supports optional `title`, optional `description`, and optional `required`.
|
|
||||||
3. This aligns with MCP `PromptArgument` shape (`name`, optional `title`, optional `description`, optional `required`) where `name` is represented by the map key.
|
|
||||||
4. Prompt `capabilities` must include `resource://prompts/<prompt-id>/document`.
|
|
||||||
|
|
||||||
Example prompt frontmatter:
|
1. `SkillsDirectoryProvider` discovers each directory containing `SKILL.md`.
|
||||||
|
2. FastMCP parses the description and scans all files when the provider is created.
|
||||||
|
3. Repository tests enforce the stricter standard-only frontmatter and directory/name rules.
|
||||||
|
|
||||||
```yaml
|
Prompt validation remains registry-oriented and fails server startup for invalid metadata, duplicate prompt ids, or malformed arguments.
|
||||||
---
|
|
||||||
name: initial-test-structure
|
|
||||||
description: Generate a baseline pytest test layout for a target scope.
|
|
||||||
x-personal-mcp:
|
|
||||||
id: initial-test-structure
|
|
||||||
version: 1.0.0
|
|
||||||
tags:
|
|
||||||
- pytest
|
|
||||||
- testing
|
|
||||||
capabilities:
|
|
||||||
- resource://prompts/initial-test-structure/document
|
|
||||||
arguments:
|
|
||||||
target_scope:
|
|
||||||
title: Target scope
|
|
||||||
description: Target package or module under test.
|
|
||||||
required: true
|
|
||||||
---
|
|
||||||
```
|
|
||||||
|
|
||||||
Reference entry rules:
|
|
||||||
|
|
||||||
1. `ref-id` is lowercase kebab-case.
|
|
||||||
2. `path` is a skill-relative markdown path and must stay inside the same skill directory.
|
|
||||||
3. Top-level files under `references/*.md` are auto-discovered with `ref-id` derived from a normalized filename stem (lowercase kebab-case).
|
|
||||||
4. Nested folders under `references/` are not auto-discovered and must be declared explicitly.
|
|
||||||
5. `mime_type` defaults to `text/markdown` when omitted.
|
|
||||||
6. `title` is an optional display label.
|
|
||||||
7. Renaming `ref-id` values is allowed when needed; optional aliases may be used during transitions.
|
|
||||||
|
|
||||||
## Auto-Generated Reference IDs
|
|
||||||
|
|
||||||
Top-level markdown files directly under `references/` are auto-registered as MCP references even when `x-personal-mcp.references` is empty.
|
|
||||||
|
|
||||||
How `ref-id` is derived:
|
|
||||||
|
|
||||||
1. Start from the filename stem (without `.md`).
|
|
||||||
2. Normalize to lowercase kebab-case.
|
|
||||||
3. Publish at `resource://skills/<skill-id>/references/<ref-id>`.
|
|
||||||
|
|
||||||
Examples:
|
|
||||||
|
|
||||||
1. `references/ruff-docs.md` -> `ref-id: ruff-docs`
|
|
||||||
2. `references/Ruff Integrations.md` -> `ref-id: ruff-integrations`
|
|
||||||
3. `references/python_logging_docs.md` -> `ref-id: python-logging-docs`
|
|
||||||
|
|
||||||
When to use explicit `x-personal-mcp.references` entries:
|
|
||||||
|
|
||||||
1. The file is nested, for example `references/guides/ci.md`.
|
|
||||||
2. You need to override defaults (`title`, `mime_type`, or custom `ref-id`).
|
|
||||||
3. You need compatibility aliases during a rename.
|
|
||||||
|
|
||||||
## Validation Models
|
|
||||||
|
|
||||||
The normative runtime model uses strict Pydantic v2 validation:
|
|
||||||
|
|
||||||
1. Models are immutable (`frozen=True`) and reject unknown fields (`extra="forbid"`).
|
|
||||||
2. `SkillFrontmatter` accepts only `name`, `description`, and `x-personal-mcp`.
|
|
||||||
3. `PromptFrontmatter` accepts only `name`, `description`, and `x-personal-mcp`.
|
|
||||||
4. `PromptArgumentEntry` accepts only optional `title`, optional `description`, and optional `required`.
|
|
||||||
5. Skill and prompt metadata enforce semver, kebab-case ids, capability requirements, and id/name/directory consistency.
|
|
||||||
6. Reference paths are validated as markdown files under `references/`.
|
|
||||||
|
|
||||||
Validation behavior contract:
|
|
||||||
|
|
||||||
1. Validate required core fields and relationships during registry load before FastMCP resource or tool registration.
|
|
||||||
2. Reject unknown or unsupported fields at parse and model-validation time.
|
|
||||||
3. Treat hard contract violations, including missing required fields, invalid ids, and broken required mappings, as startup errors.
|
|
||||||
4. Keep failure messages path-aware and field-specific for CI readability.
|
|
||||||
|
|
||||||
Projection mode contract for Anthropic API upload pipelines:
|
|
||||||
|
|
||||||
1. Parse with `SkillFrontmatter` first.
|
|
||||||
2. Emit Anthropic-safe frontmatter with standard fields only.
|
|
||||||
3. Preserve `x-personal-mcp` in source-of-truth documents; projection output is a build artifact.
|
|
||||||
|
|
||||||
## Anthropic Upload Compatibility Rule
|
|
||||||
|
|
||||||
1. Anthropic documentation guarantees behavior for standard frontmatter fields but does not explicitly guarantee handling of arbitrary unknown top-level keys.
|
|
||||||
2. Publishing pipelines that target strict API compatibility should support a projection mode that emits only standard frontmatter fields for upload.
|
|
||||||
3. Source-of-truth authoring remains in `x-personal-mcp`; upload payload shape is an explicit build concern.
|
|
||||||
|
|
||||||
## FastMCP Native Metadata Surfaces
|
|
||||||
|
|
||||||
Resources support native definition metadata:
|
|
||||||
|
|
||||||
1. `name`
|
|
||||||
2. `description`
|
|
||||||
3. `mime_type`
|
|
||||||
4. `tags`
|
|
||||||
5. `annotations`, including `readOnlyHint` and `idempotentHint`
|
|
||||||
6. `icons`
|
|
||||||
7. `meta`
|
|
||||||
8. `version`
|
|
||||||
9. `enabled`, which is deprecated in FastMCP v3 in favor of server-level enable and disable controls
|
|
||||||
|
|
||||||
Resources also support runtime metadata through `ResourceContent.meta` and `ResourceResult.meta`.
|
|
||||||
|
|
||||||
Tools support native definition metadata:
|
|
||||||
|
|
||||||
1. `name`
|
|
||||||
2. `description`
|
|
||||||
3. `tags`
|
|
||||||
4. `annotations`, including `title`, `readOnlyHint`, `destructiveHint`, `idempotentHint`, and `openWorldHint`
|
|
||||||
5. `icons`
|
|
||||||
6. `meta`
|
|
||||||
7. `version`
|
|
||||||
8. `timeout`
|
|
||||||
9. `output_schema`
|
|
||||||
10. `run_in_thread`
|
|
||||||
11. `enabled`, which is deprecated in FastMCP v3 in favor of server-level enable and disable controls
|
|
||||||
|
|
||||||
Tools also support runtime metadata through `ToolResult.meta`.
|
|
||||||
|
|
||||||
## Frontmatter To FastMCP Mapping Contract
|
|
||||||
|
|
||||||
At server startup, map `x-personal-mcp` into FastMCP registration as follows:
|
|
||||||
|
|
||||||
1. `x-personal-mcp.id` defines the canonical URI namespace and identity checks.
|
|
||||||
2. `description` becomes the default description for the primary skill document resource.
|
|
||||||
3. `x-personal-mcp.tags` maps to resource and tool tags.
|
|
||||||
4. `x-personal-mcp.version` maps to resource and tool version metadata.
|
|
||||||
5. `x-personal-mcp.capabilities` becomes the registered URI list and catalog exposure.
|
|
||||||
6. `x-personal-mcp.references[*]` becomes resource templates or concrete resources with `mime_type`, read-only annotations, and `meta` that includes `skill_id`, `ref_id`, and source `path`.
|
|
||||||
|
|
||||||
## Invariants
|
## Invariants
|
||||||
|
|
||||||
This contract guarantees:
|
1. Skills remain directly portable to tools that understand standard Agent Skills directories.
|
||||||
|
2. Native skill discovery has no parallel catalog metadata source.
|
||||||
1. Anthropic-required frontmatter stays valid for custom skill upload and Claude Code loading.
|
3. Prompts retain the richer metadata required by their catalog and MCP prompt-object surfaces.
|
||||||
2. MCP-specific metadata remains embedded in `SKILL.md` frontmatter, with no `metadata.yaml` sidecar.
|
4. All authored content remains under `docs/`.
|
||||||
3. FastMCP registration uses native metadata fields for resources and tools.
|
|
||||||
4. Reference ids and metadata can evolve with low-friction updates while internal file layout under `references/` stays refactor-friendly.
|
|
||||||
|
|
||||||
## Non-Goals
|
|
||||||
|
|
||||||
This contract does not define:
|
|
||||||
|
|
||||||
1. URI versioning and deprecation rollout policy details.
|
|
||||||
2. Migration script design from existing `metadata.yaml` files.
|
|
||||||
3. Runtime caching and indexing performance tuning.
|
|
||||||
|
|||||||
@@ -40,9 +40,9 @@ Rules:
|
|||||||
|
|
||||||
## Metadata Location Constraint
|
## Metadata Location Constraint
|
||||||
|
|
||||||
1. Skill metadata is embedded in YAML frontmatter in `SKILL.md`.
|
1. `SKILL.md` frontmatter contains only standard `name` and `description` fields.
|
||||||
2. No `metadata.yaml` sidecar exists in the end state.
|
2. No `metadata.yaml` sidecar or repository-specific skill metadata block exists.
|
||||||
3. Reference lookup metadata is documented and explicit: top-level `references/*.md` are auto-discovered from filenames, while `SKILL.md` frontmatter declares overrides and nested mappings when needed.
|
3. The provider discovers supporting files recursively; their real relative paths are published in the generated `_manifest`.
|
||||||
|
|
||||||
## Skill Id Contract
|
## Skill Id Contract
|
||||||
|
|
||||||
@@ -52,8 +52,8 @@ Rules:
|
|||||||
2. Character set: `a-z`, `0-9`, and `-`.
|
2. Character set: `a-z`, `0-9`, and `-`.
|
||||||
3. Must start with a letter.
|
3. Must start with a letter.
|
||||||
4. No underscores, spaces, dots, or uppercase characters.
|
4. No underscores, spaces, dots, or uppercase characters.
|
||||||
5. Directory name should equal `skill-id` in each committed revision.
|
5. Directory name equals `skill-id` in each committed revision.
|
||||||
6. Frontmatter `id` should equal directory name in each committed revision.
|
6. Frontmatter `name` equals the directory name.
|
||||||
7. Treat `skill-id` as immutable after release; any rename is a breaking replacement and clients must move to the new id.
|
7. Treat `skill-id` as immutable after release; any rename is a breaking replacement and clients must move to the new id.
|
||||||
|
|
||||||
Valid examples:
|
Valid examples:
|
||||||
@@ -68,6 +68,16 @@ Invalid examples:
|
|||||||
2. `Zensical-Docs`
|
2. `Zensical-Docs`
|
||||||
3. `docs.zensical`
|
3. `docs.zensical`
|
||||||
|
|
||||||
|
## Provider Publication
|
||||||
|
|
||||||
|
[`SkillsDirectoryProvider`](https://gofastmcp.com/servers/providers/skills) scans `docs/skills/` with `supporting_files="template"` and publishes:
|
||||||
|
|
||||||
|
1. `skill://<skill-id>/SKILL.md`
|
||||||
|
2. `skill://<skill-id>/_manifest`
|
||||||
|
3. `skill://<skill-id>/{path*}` for supporting files
|
||||||
|
|
||||||
|
Only the main file and manifest appear in `resources/list`. Clients inspect the manifest before reading supporting paths.
|
||||||
|
|
||||||
## Direct Documentation Inclusion
|
## Direct Documentation Inclusion
|
||||||
|
|
||||||
1. For direct API documentation, use mkdocstrings directives rather than pasting large code blocks.
|
1. For direct API documentation, use mkdocstrings directives rather than pasting large code blocks.
|
||||||
|
|||||||
+58
-162
@@ -4,183 +4,79 @@ icon: lucide/link
|
|||||||
|
|
||||||
# URI Contract
|
# URI Contract
|
||||||
|
|
||||||
This page defines the canonical resource URI contract, template parameter rules, and compatibility policy.
|
This page defines the public resource URI contract for native skills, registry-backed prompts, and general authored documentation.
|
||||||
|
|
||||||
Conventions in this document follow [MCP resource semantics](https://modelcontextprotocol.io/docs/learn/server-concepts#resources), [URI generic syntax (RFC3986)](https://www.rfc-editor.org/rfc/rfc3986), and [URI templates (RFC6570)](https://www.rfc-editor.org/rfc/rfc6570).
|
## Native Skill URIs
|
||||||
|
|
||||||
## Canonical URI Surface
|
The [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills) publishes each skill through the `skill://` scheme:
|
||||||
|
|
||||||
The public, preferred direct resource URIs are:
|
1. `skill://<skill-name>/SKILL.md`
|
||||||
|
2. `skill://<skill-name>/_manifest`
|
||||||
|
3. `skill://<skill-name>/<supporting-path>`
|
||||||
|
|
||||||
1. `resource://catalog/skills_index`
|
The first two are concrete resources returned by `resources/list`. Supporting files use a per-skill wildcard resource template when the provider is configured with `supporting_files="template"`:
|
||||||
2. `resource://catalog/skills/{skill_id}`
|
|
||||||
3. `resource://skills/{skill_id}/document`
|
|
||||||
4. `resource://skills/{skill_id}/references/{ref_id}`
|
|
||||||
5. `resource://docs/{path*}`
|
|
||||||
6. `resource://catalog/prompts_index`
|
|
||||||
7. `resource://catalog/prompts/{prompt_id}`
|
|
||||||
8. `resource://prompts/{prompt_id}/document`
|
|
||||||
|
|
||||||
The public, preferred resource template URIs are:
|
```text
|
||||||
|
skill://<skill-name>/{path*}
|
||||||
|
```
|
||||||
|
|
||||||
1. `resource://catalog/skills_index{?q,tag,capability,cursor,limit}`
|
### Main File
|
||||||
|
|
||||||
|
`skill://<skill-name>/SKILL.md` returns the canonical authored skill document. The skill directory name supplies `<skill-name>`, and the resource description comes from `SKILL.md` frontmatter.
|
||||||
|
|
||||||
|
### Manifest
|
||||||
|
|
||||||
|
`skill://<skill-name>/_manifest` returns JSON containing the skill name and every file beneath its directory. Each file entry includes:
|
||||||
|
|
||||||
|
1. relative POSIX path
|
||||||
|
2. byte size
|
||||||
|
3. SHA256 hash
|
||||||
|
|
||||||
|
Clients read the manifest before requesting supporting files. FastMCP client utilities such as `list_skills()` and `get_skill_manifest()` understand this contract directly.
|
||||||
|
|
||||||
|
### Supporting Files
|
||||||
|
|
||||||
|
Supporting files retain their real skill-relative paths. For example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
skill://pytesting/references/pytest-docs.md
|
||||||
|
```
|
||||||
|
|
||||||
|
FastMCP confines reads to the selected skill directory. Absolute paths, traversal outside the directory, missing files, directories, and symlinks that resolve outside the skill root are rejected.
|
||||||
|
|
||||||
|
## Prompt And Docs URIs
|
||||||
|
|
||||||
|
Prompts and general documentation retain the existing registry-backed resource surface:
|
||||||
|
|
||||||
|
1. `resource://catalog/prompts_index`
|
||||||
2. `resource://catalog/prompts_index{?q,tag,cursor,limit}`
|
2. `resource://catalog/prompts_index{?q,tag,cursor,limit}`
|
||||||
|
3. `resource://catalog/prompts/{prompt_id}`
|
||||||
|
4. `resource://prompts/{prompt_id}/document`
|
||||||
|
5. `resource://docs/{path*}`
|
||||||
|
|
||||||
Contract intent:
|
Prompt ids remain lowercase kebab-case. The docs wildcard accepts normalized relative POSIX Markdown paths beneath `docs/` and rejects absolute paths, traversal segments, backslashes, and non-Markdown targets.
|
||||||
|
|
||||||
1. Catalog URIs are discovery surfaces.
|
## Discovery Order
|
||||||
2. Skill URIs are the primary per-skill guidance surfaces.
|
|
||||||
3. Catalog query templates are additive discovery helpers for filtering and pagination.
|
|
||||||
4. The docs wildcard URI is a direct authored-markdown access surface under `docs/`.
|
|
||||||
|
|
||||||
Best-practice alignment:
|
For skills:
|
||||||
|
|
||||||
1. Resource identifiers are stable and noun-oriented.
|
1. list resources or call FastMCP `list_skills()`
|
||||||
2. Dynamic lookup variants are represented as RFC6570 templates.
|
2. select a skill by name and description
|
||||||
3. Resources remain read-oriented and are described with explicit MIME types.
|
3. read `skill://<skill-name>/SKILL.md`
|
||||||
|
4. read `_manifest` when supporting material may be needed
|
||||||
|
5. fetch only the supporting paths relevant to the task
|
||||||
|
|
||||||
## URI Semantics
|
For prompts, use the prompt catalog or MCP prompt-object APIs.
|
||||||
|
|
||||||
### `resource://catalog/skills_index`
|
## Compatibility Policy
|
||||||
|
|
||||||
1. Returns a compact list of skill records for discovery.
|
The native `skill://` family directly replaces the repository's former custom skill URI and catalog surfaces. No compatibility aliases or dual registrations are maintained. Prompt and general-doc URIs are unaffected.
|
||||||
2. Contains one entry per `skill_id`.
|
|
||||||
3. Includes enough metadata for client-side selection, at minimum `id`, `name`, `description`, `tags`, and `capabilities`.
|
|
||||||
|
|
||||||
### `resource://catalog/skills/{skill_id}`
|
Skill renames are breaking because the directory name is part of every native skill URI. Supporting-file renames change the corresponding manifest path and URI.
|
||||||
|
|
||||||
1. Returns one normalized record for `skill_id`.
|
|
||||||
2. Includes the canonical document URI and declared reference ids.
|
|
||||||
3. Returns not found when `skill_id` does not exist.
|
|
||||||
|
|
||||||
### `resource://skills/{skill_id}/document`
|
|
||||||
|
|
||||||
1. Returns the canonical `SKILL.md` authored content for that skill.
|
|
||||||
2. `skill_id` must satisfy the stable skill id rules from the content contract.
|
|
||||||
|
|
||||||
### `resource://skills/{skill_id}/references/{ref_id}`
|
|
||||||
|
|
||||||
1. Returns one reference document declared in the skill frontmatter references manifest.
|
|
||||||
2. `ref_id` is the stable public handle for that reference document.
|
|
||||||
|
|
||||||
### `resource://docs/{path*}`
|
|
||||||
|
|
||||||
1. Returns authored markdown at a normalized relative path under `docs/`.
|
|
||||||
2. Supports nested paths via [RFC6570 wildcard expansion](https://www.rfc-editor.org/rfc/rfc6570).
|
|
||||||
3. Typical examples include `index.md`, `usage.md`, `skills/<skill-id>/SKILL.md`, and `skills/<skill-id>/references/<file>.md`.
|
|
||||||
|
|
||||||
### `resource://catalog/prompts_index`
|
|
||||||
|
|
||||||
1. Returns a compact list of prompt records for discovery.
|
|
||||||
2. Contains one entry per `prompt_id`.
|
|
||||||
3. Includes `id`, `name`, `description`, `tags`, `version`, and canonical document URI.
|
|
||||||
|
|
||||||
### `resource://catalog/skills_index{?q,tag,capability,cursor,limit}`
|
|
||||||
|
|
||||||
1. Returns the same record family as `resource://catalog/skills_index` with optional filtering and pagination.
|
|
||||||
2. Query parameters are optional and composable.
|
|
||||||
3. Unknown query keys are ignored or rejected deterministically by server policy.
|
|
||||||
|
|
||||||
### `resource://catalog/prompts_index{?q,tag,cursor,limit}`
|
|
||||||
|
|
||||||
1. Returns the same record family as `resource://catalog/prompts_index` with optional filtering and pagination.
|
|
||||||
2. Query parameters are optional and composable.
|
|
||||||
3. Unknown query keys are ignored or rejected deterministically by server policy.
|
|
||||||
|
|
||||||
### `resource://catalog/prompts/{prompt_id}`
|
|
||||||
|
|
||||||
1. Returns one normalized record for `prompt_id`.
|
|
||||||
2. Includes prompt argument metadata when declared in frontmatter.
|
|
||||||
3. Returns not found when `prompt_id` does not exist.
|
|
||||||
|
|
||||||
### `resource://prompts/{prompt_id}/document`
|
|
||||||
|
|
||||||
1. Returns the canonical prompt markdown document.
|
|
||||||
2. `prompt_id` must satisfy lowercase kebab-case rules.
|
|
||||||
|
|
||||||
## Template Parameter And Validation Rules
|
|
||||||
|
|
||||||
### `skill_id`
|
|
||||||
|
|
||||||
1. Lowercase kebab-case.
|
|
||||||
2. Must satisfy the stable skill id rules from the content contract.
|
|
||||||
|
|
||||||
### `ref_id`
|
|
||||||
|
|
||||||
1. Lowercase kebab-case.
|
|
||||||
2. Must be declared in the skill's references manifest.
|
|
||||||
|
|
||||||
### `path*`
|
|
||||||
|
|
||||||
1. Relative POSIX path only, expressed as URI path segments under [RFC3986 path syntax](https://www.rfc-editor.org/rfc/rfc3986#section-3.3).
|
|
||||||
2. No leading slash.
|
|
||||||
3. No `..` traversal segments.
|
|
||||||
4. Resolves only inside `docs/`.
|
|
||||||
5. Markdown-only in the end state, meaning `.md` files.
|
|
||||||
6. Any reserved URI characters in path segments must be [percent-encoded](https://www.rfc-editor.org/rfc/rfc3986#section-2.1).
|
|
||||||
|
|
||||||
### `prompt_id`
|
|
||||||
|
|
||||||
1. Lowercase kebab-case.
|
|
||||||
2. Must be unique across prompt ids and must not collide with skill ids.
|
|
||||||
|
|
||||||
## URI Hygiene Rules
|
|
||||||
|
|
||||||
1. Use lowercase, human-readable path segments for stable discoverability.
|
|
||||||
2. Keep identifiers immutable once public whenever practical.
|
|
||||||
3. Keep template variables semantic (`skill_id`, `prompt_id`, `ref_id`, `path*`) and avoid overloading one variable for unrelated meanings.
|
|
||||||
4. Do not include secrets, tokens, or user-identifying data in URI paths or query strings.
|
|
||||||
5. Prefer additive query parameters for discovery over introducing parallel URI families, matching [MCP resource-template discovery patterns](https://modelcontextprotocol.io/docs/learn/server-concepts#resources).
|
|
||||||
6. Return clear not-found semantics for unknown ids and invalid template resolution.
|
|
||||||
|
|
||||||
## URI Versioning Policy
|
|
||||||
|
|
||||||
Default rule:
|
|
||||||
|
|
||||||
1. Keep URIs unversioned by default.
|
|
||||||
2. Allow URI and payload updates when they improve clarity or implementation simplicity.
|
|
||||||
|
|
||||||
Breaking-change rule:
|
|
||||||
|
|
||||||
1. Breaking changes use direct replacement of the canonical URI family.
|
|
||||||
2. No compatibility aliases or dual URI families are maintained.
|
|
||||||
|
|
||||||
FastMCP version metadata usage:
|
|
||||||
|
|
||||||
1. Resource `version` metadata may be used for implementation and version discovery.
|
|
||||||
2. URI readability and maintainability remain the primary contract.
|
|
||||||
|
|
||||||
## Reference Id Compatibility Policy
|
|
||||||
|
|
||||||
`ref_id` is the public identifier for a reference document, separate from file path.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
1. Prefer keeping `ref_id` stable when practical.
|
|
||||||
2. File paths may change without URI churn as long as the mapped `ref_id` still resolves.
|
|
||||||
3. If a reference is renamed, introduce a new `ref_id` and treat the old one as retired.
|
|
||||||
4. Avoid reusing retired `ref_id` values for unrelated content.
|
|
||||||
|
|
||||||
## Invariants
|
|
||||||
|
|
||||||
This contract guarantees:
|
|
||||||
|
|
||||||
1. One canonical URI pattern per core capability surface.
|
|
||||||
2. Fast, low-friction URI evolution through direct replacement of canonical URIs.
|
|
||||||
3. A single canonical catalog URI family with no alias maintenance overhead.
|
|
||||||
4. Reference mappings can evolve with minimal churn.
|
|
||||||
|
|
||||||
## Non-Goals
|
|
||||||
|
|
||||||
This contract does not define:
|
|
||||||
|
|
||||||
1. Implementation-specific transform wiring details, such as `VersionFilter`, mounts, or provider composition.
|
|
||||||
2. Migration script mechanics for auto-generating aliases.
|
|
||||||
3. Authorization policy design for URI-level access control.
|
|
||||||
|
|
||||||
## Sources
|
## Sources
|
||||||
|
|
||||||
1. [MCP Server Concepts: Resources](https://modelcontextprotocol.io/docs/learn/server-concepts#resources)
|
1. [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills)
|
||||||
2. [MCP Architecture Overview](https://modelcontextprotocol.io/docs/learn/architecture)
|
2. [MCP resources](https://modelcontextprotocol.io/specification/latest/server/resources)
|
||||||
3. [MCP Specification Repository](https://github.com/modelcontextprotocol/spec)
|
3. [RFC 3986 URI syntax](https://www.rfc-editor.org/rfc/rfc3986)
|
||||||
4. [RFC6570 URI Template](https://www.rfc-editor.org/rfc/rfc6570)
|
4. [RFC 6570 URI templates](https://www.rfc-editor.org/rfc/rfc6570)
|
||||||
|
|||||||
+61
-147
@@ -6,199 +6,113 @@ icon: lucide/bot
|
|||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
This page explains how the GitHub Copilot extension in VS Code behaves as an MCP client when connected to `personal-mcp`, including why tools can appear while resource attachment appears unavailable.
|
This page explains how GitHub Copilot in VS Code consumes native skill resources from `personal-mcp`, including sessions where tools are visible but resource attachment is not.
|
||||||
|
|
||||||
## Core Model
|
## Capability Lanes
|
||||||
|
|
||||||
Copilot interacts with MCP servers through separate capability lanes:
|
Copilot interacts with MCP servers through independently exposed lanes:
|
||||||
|
|
||||||
1. tools (invoked by the model during execution)
|
1. tools invoked during execution
|
||||||
2. resources (attached as read-only context)
|
2. resources attached as read-only context
|
||||||
3. prompts (server-provided prompt templates)
|
3. server-provided prompts
|
||||||
|
|
||||||
These lanes are related but independently gated in the client.
|
This server publishes skills as native `skill://` resources, prompts through registry-backed resources and MCP prompt objects, and generic resource fallback tools through FastMCP.
|
||||||
|
|
||||||
Reliable paths are:
|
## Native Skill Resources
|
||||||
|
|
||||||
1. attach MCP resources explicitly through `Add Context > MCP Resources` or `MCP: Browse Resources`
|
For every skill, Copilot can discover:
|
||||||
2. let Copilot invoke MCP tools when the task and tool descriptions make that relevant
|
|
||||||
3. invoke MCP prompts explicitly with `/server.prompt` when your server exposes them
|
|
||||||
|
|
||||||
## What Actually Happens In VS Code
|
1. `skill://<name>/SKILL.md`
|
||||||
|
2. `skill://<name>/_manifest`
|
||||||
|
3. `skill://<name>/{path*}` supporting-file template
|
||||||
|
|
||||||
### MCP server side
|
The main resource description comes from `SKILL.md`. The manifest discloses supporting paths, sizes, and SHA256 hashes. This is the only skill discovery contract; there is no parallel skill catalog.
|
||||||
|
|
||||||
Your server can advertise resources and serve them correctly. In this project that includes catalog resources and skill document resources.
|
## Resource Picker Availability
|
||||||
|
|
||||||
### Copilot session side
|
`MCP Resources...` in Add Context requires both:
|
||||||
|
|
||||||
The chat surface exposes tools, resources, and prompts through different UI paths. In practice, you can encounter sessions where tool use is available but MCP resource attachment is not exposed in `Add Context`.
|
1. a connected server advertising resource capability
|
||||||
|
2. a chat surface that exposes MCP resource attachment
|
||||||
|
|
||||||
That is why you can sometimes see MCP tools before you see `Add Context > MCP Resources`.
|
A successful `resources/list` response does not guarantee the picker appears in every session type. Use `MCP: Browse Resources` to distinguish server availability from chat UI availability.
|
||||||
|
|
||||||
## Why The Picker Sometimes Shows Only Tools
|
## Recommended Workflow
|
||||||
|
|
||||||
`MCP Resources...` in Add Context requires at least:
|
When resource attachment is available:
|
||||||
|
|
||||||
1. at least one connected MCP server advertises resource capability
|
1. browse the server's resources
|
||||||
2. the current chat surface exposes MCP resource attachment
|
2. attach one relevant `skill://<name>/SKILL.md`
|
||||||
|
3. attach `_manifest` only if supporting detail may be needed
|
||||||
|
4. attach only selected supporting files
|
||||||
|
|
||||||
If the second condition is not met, resources can be available on the server while still being absent from the picker.
|
When only tools are available:
|
||||||
|
|
||||||
## Practical Workflow
|
1. call `list_resources`
|
||||||
|
2. select a native main skill URI by name and description
|
||||||
|
3. call `read_resource` for that URI
|
||||||
|
4. read `_manifest` and supporting files only as needed
|
||||||
|
|
||||||
Use this sequence to confirm behavior:
|
Both paths resolve through the same FastMCP provider.
|
||||||
|
|
||||||
1. run `MCP: Browse Resources` and verify resources exist
|
## Prompt Examples
|
||||||
2. use `MCP: List Servers` to verify the server is enabled and running
|
|
||||||
3. open Copilot Chat
|
|
||||||
4. check `Add Context` for `MCP Resources...`
|
|
||||||
5. if still missing, restart the server and reload VS Code window
|
|
||||||
|
|
||||||
## Recommended Usage Pattern
|
Resource attachment:
|
||||||
|
|
||||||
1. rely on canonical catalog resources for discovery (`skills_index`, then `skills/{skill_id}`)
|
|
||||||
2. fetch only selected skill documents for context
|
|
||||||
3. keep slash commands for deterministic fallback flows
|
|
||||||
|
|
||||||
When resource attachment is unavailable in the active session, use ResourcesAsTools first, then thin catalog discovery tools as parity fallback:
|
|
||||||
|
|
||||||
1. `list_resources`
|
|
||||||
2. `read_resource`
|
|
||||||
3. `search_patterns`
|
|
||||||
4. `get_pattern_by_id`
|
|
||||||
5. `get_skill_document_by_id`
|
|
||||||
|
|
||||||
Canonical naming policy:
|
|
||||||
|
|
||||||
1. Prefer the five canonical tool names above in prompts and instructions.
|
|
||||||
2. For compatibility with clients that emit `catalog_*` naming, the server also exposes:
|
|
||||||
- `catalog_search_patterns`
|
|
||||||
- `catalog_get_pattern_by_id`
|
|
||||||
- `catalog_get_skill_document_by_id`
|
|
||||||
3. Canonical and compatibility alias tools return equivalent payloads for the same input.
|
|
||||||
|
|
||||||
The first two are generated from the canonical resource surface and should be preferred in tool-only clients.
|
|
||||||
|
|
||||||
These should stay read-only, minimal, and schema-aligned with catalog resources.
|
|
||||||
|
|
||||||
For very large tool catalogs, server operators can optionally enable tool search mode (`regex` or `bm25`) while keeping `list_resources` and `read_resource` pinned as always-visible fallback tools.
|
|
||||||
|
|
||||||
## What To Type In Copilot Chat
|
|
||||||
|
|
||||||
Use prompts that tell Copilot which MCP feature path to take.
|
|
||||||
|
|
||||||
### If `MCP Resources...` is available
|
|
||||||
|
|
||||||
Use the resource attachment UI first, then ask Copilot to work from the attached material.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
```text
|
```text
|
||||||
I attached the catalog resources and the FastAPI async SQLAlchemy modernization skill document. Use that context to propose a migration plan for this repo.
|
Use the attached personal-mcp skill as guidance, then reconcile it with the repository before proposing changes.
|
||||||
```
|
```
|
||||||
|
|
||||||
If you want to keep the attachment sequence explicit, use:
|
Tool-only discovery:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
I attached personal-mcp catalog resources first. Use them to identify the best matching skill, then work only from the selected skill document.
|
Call list_resources, choose the best matching skill://.../SKILL.md resource, and read it. Inspect its _manifest only if a supporting file is needed. Load at most two candidate skills.
|
||||||
```
|
```
|
||||||
|
|
||||||
### If only tools are available
|
Direct loading:
|
||||||
|
|
||||||
Ask Copilot to explicitly use resource-backed tools first.
|
|
||||||
|
|
||||||
Example resource-backed prompt:
|
|
||||||
|
|
||||||
```text
|
```text
|
||||||
Use personal-mcp tool fallback by first calling list_resources, then read_resource for resource://catalog/skills_index and the selected resource://skills/<skill-id>/document URI. Use only that loaded skill context in your answer.
|
Read skill://async-fastapi-sqlmodel/SKILL.md and apply only the sections relevant to this repository.
|
||||||
```
|
```
|
||||||
|
|
||||||
If needed, use the thin catalog tools.
|
Supporting material:
|
||||||
|
|
||||||
Example discovery prompt:
|
|
||||||
|
|
||||||
```text
|
```text
|
||||||
Use the personal-mcp catalog tools to search for the most relevant skill for FastAPI async SQLAlchemy modernization. Then load the selected skill document and use it as context for your answer.
|
Read skill://pytesting/_manifest, select the one reference relevant to async test lifecycle, and use that file with the main skill instructions.
|
||||||
```
|
```
|
||||||
|
|
||||||
Example direct-load prompt:
|
## Repository Instruction Pattern
|
||||||
|
|
||||||
```text
|
A repo-level instruction should name the native retrieval order and context budget:
|
||||||
Call get_skill_document_by_id for async-fastapi-sqlmodel and use that document as the main context for this task.
|
|
||||||
```
|
|
||||||
|
|
||||||
Example bounded-selection prompt:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Search personal-mcp skills for NiceGUI UI customization, select at most 2 strong matches, load the best skill document, and answer using only that material plus the workspace code.
|
|
||||||
```
|
|
||||||
|
|
||||||
## Repo Instructions Example
|
|
||||||
|
|
||||||
Repo instructions are the best place to teach Copilot when MCP content is relevant and which path to prefer.
|
|
||||||
|
|
||||||
If you add a repo-level `copilot-instructions.md`, keep the rule simple: prefer catalog-first discovery, keep loaded skill context small, and fall back to tools when resource attachment is unavailable.
|
|
||||||
|
|
||||||
Instructions can strongly steer behavior, but they do not guarantee that VS Code will auto-attach MCP resources for a request. For reliable resource use, either attach resources explicitly or prompt Copilot to use the fallback tools.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
```md
|
```md
|
||||||
# MCP Usage
|
When a task matches a personal-mcp skill:
|
||||||
|
|
||||||
When a task may benefit from personal-mcp skills, use this sequence:
|
1. Prefer an already attached native skill resource.
|
||||||
|
2. Otherwise use `list_resources` and select one `skill://<name>/SKILL.md` resource by description.
|
||||||
1. Start with personal-mcp catalog discovery when the task appears to match documented implementation patterns.
|
3. Read `_manifest` only when supporting material is needed.
|
||||||
2. Prefer MCP resources when the chat surface exposes resource attachment.
|
4. Load at most two candidate main files and only the relevant supporting paths.
|
||||||
3. If MCP resource attachment is unavailable, use `list_resources`/`read_resource` first, then thin catalog tools if needed.
|
5. Reconcile guidance with the current repository before editing.
|
||||||
4. Load only the most relevant skill document or at most 2 skill documents.
|
|
||||||
5. Treat skill documents as guidance, then reconcile them with the actual repository code before making changes.
|
|
||||||
|
|
||||||
Preferred discovery order:
|
|
||||||
|
|
||||||
1. `resource://catalog/skills_index`
|
|
||||||
2. `resource://catalog/skills/{skill_id}`
|
|
||||||
3. `resource://skills/<skill-id>/document`
|
|
||||||
4. `resource://skills/<skill-id>/references/<ref-id>` when needed
|
|
||||||
|
|
||||||
Tool fallback order:
|
|
||||||
|
|
||||||
1. `list_resources`
|
|
||||||
2. `read_resource`
|
|
||||||
3. `search_patterns`
|
|
||||||
4. `get_pattern_by_id`
|
|
||||||
5. `get_skill_document_by_id`
|
|
||||||
|
|
||||||
If confidence is low after catalog discovery, ask one clarifying question before loading more skill documents.
|
|
||||||
```
|
```
|
||||||
|
|
||||||
That instruction style does two useful things:
|
Instructions steer behavior but do not force VS Code to attach resources automatically.
|
||||||
|
|
||||||
1. it tells Copilot to prefer the MCP server when relevant without forcing it on every prompt
|
## Prompt Objects
|
||||||
2. it keeps context size bounded so skill loading does not become noisy or expensive
|
|
||||||
|
|
||||||
If you want stronger behavior, add one more line that names the MCP server directly:
|
Prompt modules remain separate from skills. When the client supports MCP prompt APIs, use prompt listing and `get_prompt` for parameterized workflows. Authored `PROMPT.md` remains the source of truth for each prompt.
|
||||||
|
|
||||||
```md
|
## Troubleshooting
|
||||||
Use the `personal-mcp` server for skill discovery whenever the task involves documented implementation patterns available from the catalog.
|
|
||||||
```
|
|
||||||
|
|
||||||
## Known Gotcha
|
1. Use `MCP: List Servers` to confirm the server is enabled.
|
||||||
|
2. Use `MCP: Browse Resources` to confirm native skill resources exist.
|
||||||
A successful `resources/list` response from the server does not guarantee the resource picker appears in every Copilot session type. UI availability is session-capability-dependent.
|
3. Restart the MCP server after changing skill files because production uses `reload=False`.
|
||||||
|
4. Reload the VS Code window if the server is healthy but the resource picker remains stale.
|
||||||
|
5. In tool-only sessions, verify `list_resources` and `read_resource` are visible.
|
||||||
|
|
||||||
## Further Reading
|
## Further Reading
|
||||||
|
|
||||||
### VS Code docs
|
1. [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills)
|
||||||
|
2. [Add and manage MCP servers](https://code.visualstudio.com/docs/agent-customization/mcp-servers)
|
||||||
1. [Add and manage MCP servers](https://code.visualstudio.com/docs/agent-customization/mcp-servers)
|
3. [MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration)
|
||||||
2. [MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration)
|
4. [Manage context for AI](https://code.visualstudio.com/docs/chat/copilot-chat-context)
|
||||||
3. [Manage context for AI](https://code.visualstudio.com/docs/chat/copilot-chat-context)
|
5. [Skill Usage Mechanics](./usage.md)
|
||||||
4. [AI features cheat sheet](https://code.visualstudio.com/docs/agents/reference/ai-features-cheat-sheet)
|
|
||||||
|
|
||||||
### Project docs
|
|
||||||
|
|
||||||
1. [Resource-First Pattern Module Architecture](./architecture.md)
|
|
||||||
2. [Static Docs Hosting Pattern](./mcp_layout.md)
|
|
||||||
3. [Skill Usage Mechanics](./usage.md)
|
|
||||||
|
|||||||
+22
-23
@@ -80,20 +80,22 @@ The runtime process serves two surfaces:
|
|||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
flowchart TD
|
flowchart TD
|
||||||
A[Docs Registry Loader] --> B[Validated In-Memory Registry]
|
A[Packaged Skill Directory] --> B[SkillsDirectoryProvider]
|
||||||
B --> C[FastMCP Resource Registration]
|
C[Packaged Prompts and Docs] --> D[Validated Registry]
|
||||||
C --> D[MCP Transport]
|
B --> E[FastMCP Server]
|
||||||
C --> E[FastAPI Application]
|
D --> E
|
||||||
E --> F[Static Mount /docs]
|
E --> F[MCP Transport]
|
||||||
F --> G[Zensical site output directory]
|
E --> G[FastAPI Application]
|
||||||
|
G --> H[Static Mount /docs]
|
||||||
|
H --> I[Zensical Site Output]
|
||||||
```
|
```
|
||||||
|
|
||||||
Runtime guarantees:
|
Runtime guarantees:
|
||||||
|
|
||||||
1. Docs registry load and validation happen before resource exposure.
|
1. The skills provider and prompt/docs registry initialize before resource exposure.
|
||||||
2. Duplicate resource and template registration fails startup (`on_duplicate="error"`).
|
2. Duplicate resource and template registration fails startup (`on_duplicate="error"`).
|
||||||
3. Resource registration is metadata-driven from SKILL frontmatter and reference manifests.
|
3. Skill resources come directly from `SkillsDirectoryProvider` directory discovery.
|
||||||
4. Legacy per-skill Python servers and `metadata.yaml` sidecars are not part of the runtime.
|
4. Legacy per-skill Python servers, custom skill catalogs, and metadata sidecars are not part of the runtime.
|
||||||
|
|
||||||
## Build and Publish Flow
|
## Build and Publish Flow
|
||||||
|
|
||||||
@@ -120,27 +122,24 @@ MCP resources map directly to canonical Markdown documents.
|
|||||||
|
|
||||||
Example mapping model:
|
Example mapping model:
|
||||||
|
|
||||||
1. docs/skills/<skill-id>/SKILL.md -> resource://skills/<skill_id>/document
|
1. docs/skills/<skill-id>/SKILL.md -> skill://<skill-id>/SKILL.md
|
||||||
2. docs/skills/<skill-id>/references/<file>.md -> resource://skills/<skill_id>/references/<ref_id> (via frontmatter references manifest)
|
2. docs/skills/<skill-id>/<path> -> skill://<skill-id>/<path>
|
||||||
3. docs/<path>.md -> resource://docs/{path*}
|
3. docs/<path>.md -> resource://docs/{path*}
|
||||||
|
|
||||||
Catalog discovery resources are:
|
Catalog discovery resources are:
|
||||||
|
|
||||||
1. resource://catalog/skills_index
|
1. resource://catalog/prompts_index
|
||||||
2. resource://catalog/skills_index{?q,tag,capability,cursor,limit}
|
2. resource://catalog/prompts_index{?q,tag,cursor,limit}
|
||||||
3. resource://catalog/skills/{skill_id}
|
3. resource://catalog/prompts/{prompt_id}
|
||||||
4. resource://catalog/prompts_index
|
|
||||||
5. resource://catalog/prompts_index{?q,tag,cursor,limit}
|
|
||||||
6. resource://catalog/prompts/{prompt_id}
|
|
||||||
|
|
||||||
Registry-backed registration details:
|
Resource registration details:
|
||||||
|
|
||||||
1. `resource://skills/{skill_id}/document` resolves to each skill's SKILL.md.
|
1. `skill://<skill-id>/SKILL.md` resolves to each skill's main instructions.
|
||||||
2. `resource://skills/{skill_id}/references/{ref_id}` resolves through frontmatter reference manifests.
|
2. `skill://<skill-id>/_manifest` lists every skill file with size and SHA256 hash.
|
||||||
3. `resource://docs/{path*}` resolves normalized markdown paths under `docs/`.
|
3. Per-skill wildcard templates resolve validated supporting-file paths.
|
||||||
4. Resource metadata includes explicit mime type and read-only/idempotent annotations.
|
4. `resource://docs/{path*}` resolves normalized Markdown paths under `docs/`.
|
||||||
|
|
||||||
When clients cannot attach MCP resources directly, thin catalog tools may retrieve the same underlying skill documents indirectly. This does not create a second content source.
|
When clients cannot attach MCP resources directly, `ResourcesAsTools` exposes generic `list_resources` and `read_resource` tools over the same provider resources.
|
||||||
|
|
||||||
## URI Compatibility Policy
|
## URI Compatibility Policy
|
||||||
|
|
||||||
|
|||||||
@@ -69,14 +69,12 @@ Load only what matches the requested artifact:
|
|||||||
7. Produce only the requested artifact type.
|
7. Produce only the requested artifact type.
|
||||||
8. Keep guidance deterministic and minimal, with explicit references to source docs.
|
8. Keep guidance deterministic and minimal, with explicit references to source docs.
|
||||||
9. If artifact_type is shim:
|
9. If artifact_type is shim:
|
||||||
- bind one applyTo scope to one primary skill resource URI
|
- bind one applyTo scope to one `skill://<name>/SKILL.md` resource URI
|
||||||
- prefer MCP resource attachment first
|
- prefer MCP resource attachment first
|
||||||
- if resource attachment is unavailable, use fallback tool order:
|
- inspect the selected skill's `_manifest` only when supporting material is needed
|
||||||
|
- if resource attachment is unavailable, use the generic fallback tools:
|
||||||
1. list_resources
|
1. list_resources
|
||||||
2. read_resource
|
2. read_resource
|
||||||
3. search_patterns
|
|
||||||
4. get_pattern_by_id
|
|
||||||
5. get_skill_document_by_id
|
|
||||||
10. Return created or updated file paths and any validation commands that should be run.
|
10. Return created or updated file paths and any validation commands that should be run.
|
||||||
|
|
||||||
## Output Contract
|
## Output Contract
|
||||||
|
|||||||
@@ -0,0 +1,66 @@
|
|||||||
|
---
|
||||||
|
name: jsfiddle-page-layout
|
||||||
|
description: Create a responsive sample page layout for a user-supplied domain and return paste-ready HTML and CSS for JSFiddle.
|
||||||
|
x-personal-mcp:
|
||||||
|
id: jsfiddle-page-layout
|
||||||
|
version: 1.1.0
|
||||||
|
tags:
|
||||||
|
- frontend
|
||||||
|
- html
|
||||||
|
- css
|
||||||
|
- jsfiddle
|
||||||
|
- layout
|
||||||
|
- prototyping
|
||||||
|
- prompts
|
||||||
|
capabilities:
|
||||||
|
- resource://prompts/jsfiddle-page-layout/document
|
||||||
|
---
|
||||||
|
|
||||||
|
# JSFiddle Page Layout
|
||||||
|
|
||||||
|
Create a polished sample page layout for the supplied domain. The result must run by pasting the markup and styles into the [JSFiddle](https://jsfiddle.net/) HTML and CSS panes.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
1. `domain`: the product, service, organization, or subject represented by the page, including its intended audience when known
|
||||||
|
2. `layout_brief`: optional page type, required sections, content priorities, visual direction, or constraints
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. Infer the page's primary purpose, audience, content hierarchy, and most important user action from the inputs.
|
||||||
|
2. If the domain does not provide enough information to choose a useful page type or primary action, ask one concise clarification question before generating code.
|
||||||
|
3. Choose a visual direction and information density appropriate to the domain. Build the usable page itself, not a marketing explanation of the page.
|
||||||
|
4. Write semantic HTML with realistic domain-specific sample content. Do not use placeholder text such as lorem ipsum.
|
||||||
|
5. Build the layout with modern CSS, using [CSS Grid](https://css-tricks.com/complete-guide-css-grid-layout/) for two-dimensional page structure and [Flexbox](https://css-tricks.com/snippets/css/a-guide-to-flexbox/) for one-dimensional alignment where each fits naturally.
|
||||||
|
6. Make the page responsive at narrow mobile and desktop widths without horizontal overflow, overlapping content, or clipped text.
|
||||||
|
7. Keep the example self-contained. Use no JavaScript, build tools, external stylesheets, images, or icon libraries unless the layout brief explicitly requires them.
|
||||||
|
8. Include accessible landmarks, heading order, labels, focus styles, color contrast, and reduced-motion handling when animation is present.
|
||||||
|
9. Use CSS custom properties for the color, typography, spacing, border, and shadow system. Avoid generic framework styling and tailor the visual language to the domain.
|
||||||
|
|
||||||
|
## Design References
|
||||||
|
|
||||||
|
Use these references as comparative guidance, not as templates to copy. Select principles that fit the domain and layout brief, and do not reproduce a vendor's visual language unless the user requests it.
|
||||||
|
|
||||||
|
1. [Material Design 3 foundations](https://m3.material.io/foundations) for current approaches to layout, interaction states, design tokens, and adaptable UI systems.
|
||||||
|
2. [Apple Human Interface Guidelines](https://developer.apple.com/design/human-interface-guidelines/) for contemporary principles covering hierarchy, typography, controls, and platform-aware interaction.
|
||||||
|
3. [web.dev responsive web design basics](https://web.dev/articles/responsive-web-design-basics) for content-led breakpoints, flexible layouts, and input-aware responsiveness.
|
||||||
|
4. [Web Content Accessibility Guidelines (WCAG) 2.2](https://www.w3.org/TR/WCAG22/) as the accessibility baseline for structure, contrast, focus, reflow, and target sizing.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
Return exactly two fenced code blocks in this order:
|
||||||
|
|
||||||
|
1. An `html` block containing only the content for JSFiddle's HTML pane.
|
||||||
|
2. A `css` block containing only the content for JSFiddle's CSS pane.
|
||||||
|
|
||||||
|
Do not include setup instructions, design commentary, JavaScript, or prose outside the two code blocks.
|
||||||
|
|
||||||
|
## Quality Rules
|
||||||
|
|
||||||
|
1. Prefer semantic elements such as `header`, `nav`, `main`, `section`, `article`, `aside`, and `footer` when they match the content.
|
||||||
|
2. Reserve large display type for a true hero or primary page title; keep operational interfaces compact and easy to scan.
|
||||||
|
3. Use cards only for repeated items or genuinely framed tools. Do not place cards inside cards.
|
||||||
|
4. Use stable responsive constraints for grids, controls, media, and navigation so dynamic content does not shift the layout unexpectedly.
|
||||||
|
5. Avoid decorative gradients, floating color blobs, excessive rounding, and one-note palettes unless they are explicitly appropriate to the domain.
|
||||||
|
6. Ensure controls look and behave like their purpose, with visible hover and keyboard-focus states.
|
||||||
|
7. Keep all visible copy relevant to the fictional domain rather than describing the mockup or its implementation.
|
||||||
@@ -17,7 +17,7 @@ x-personal-mcp:
|
|||||||
description: File glob scope for the shim applyTo field, such as tests/** or **/*.md.
|
description: File glob scope for the shim applyTo field, such as tests/** or **/*.md.
|
||||||
required: true
|
required: true
|
||||||
primary_skill_resource:
|
primary_skill_resource:
|
||||||
description: Primary skill resource URI, usually resource://skills/<skill-id>/document.
|
description: Primary native skill resource URI in the form skill://<skill-name>/SKILL.md.
|
||||||
required: true
|
required: true
|
||||||
shim_title:
|
shim_title:
|
||||||
description: Human-readable name for the instruction shim frontmatter.
|
description: Human-readable name for the instruction shim frontmatter.
|
||||||
@@ -52,23 +52,22 @@ Load only sections relevant to the requested shim:
|
|||||||
## Workflow
|
## Workflow
|
||||||
|
|
||||||
1. Validate that apply_to_glob and primary_skill_resource are present.
|
1. Validate that apply_to_glob and primary_skill_resource are present.
|
||||||
2. If either value is missing or ambiguous, ask exactly one clarifying question before generating output.
|
2. Validate that primary_skill_resource uses the `skill://<skill-name>/SKILL.md` form.
|
||||||
3. Generate one .instructions.md file content block only.
|
3. If either value is missing or ambiguous, ask exactly one clarifying question before generating output.
|
||||||
4. Keep the shim concise and deterministic:
|
4. Generate one .instructions.md file content block only.
|
||||||
|
5. Keep the shim concise and deterministic:
|
||||||
- include YAML frontmatter with name, description, and applyTo
|
- include YAML frontmatter with name, description, and applyTo
|
||||||
- include a primary rule that uses the selected primary_skill_resource first
|
- include a primary rule that uses the selected primary_skill_resource first
|
||||||
- include a bounded execution pattern (load primary doc, apply only relevant sections, keep edits minimal)
|
- include a bounded execution pattern (load primary doc, apply only relevant sections, keep edits minimal)
|
||||||
5. Include VS Code/Copilot integration mechanics in the shim body:
|
6. Include VS Code/Copilot integration mechanics in the shim body:
|
||||||
- prefer MCP resource attachment when available
|
- prefer MCP resource attachment when available
|
||||||
- if attachment is unavailable, use tool fallback order:
|
- inspect `_manifest` only when the task needs supporting material
|
||||||
|
- if attachment is unavailable, use the generic fallback tools:
|
||||||
1. list_resources
|
1. list_resources
|
||||||
2. read_resource
|
2. read_resource
|
||||||
3. search_patterns
|
|
||||||
4. get_pattern_by_id
|
|
||||||
5. get_skill_document_by_id
|
|
||||||
- ask one clarifying question when confidence is low
|
- ask one clarifying question when confidence is low
|
||||||
6. If companion_docs_page is provided, include it as a companion docs link line.
|
7. If companion_docs_page is provided, include it as a companion docs link line.
|
||||||
7. Do not generate additional files, code changes, or batch shim packs.
|
8. Do not generate additional files, code changes, or batch shim packs.
|
||||||
|
|
||||||
## Output Format
|
## Output Format
|
||||||
|
|
||||||
@@ -98,13 +97,9 @@ Execution pattern:
|
|||||||
2. Apply only sections relevant to the file being edited.
|
2. Apply only sections relevant to the file being edited.
|
||||||
3. Keep edits minimal and aligned with repository conventions.
|
3. Keep edits minimal and aligned with repository conventions.
|
||||||
4. Prefer MCP resource attachment when available in the current chat surface.
|
4. Prefer MCP resource attachment when available in the current chat surface.
|
||||||
5. If MCP resource attachment is unavailable, use tool fallback in this order:
|
5. Read the selected skill's `_manifest` only when supporting material is needed.
|
||||||
1. list_resources
|
6. If MCP resource attachment is unavailable, use `list_resources` and `read_resource`.
|
||||||
2. read_resource
|
7. If confidence is low, ask one clarifying question before editing.
|
||||||
3. search_patterns
|
|
||||||
4. get_pattern_by_id
|
|
||||||
5. get_skill_document_by_id
|
|
||||||
6. If confidence is low, ask one clarifying question before editing.
|
|
||||||
|
|
||||||
Companion docs page: <optional-relative-doc-link>
|
Companion docs page: <optional-relative-doc-link>
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -0,0 +1,95 @@
|
|||||||
|
---
|
||||||
|
name: nicegui-component-extraction
|
||||||
|
description: Extract a user-selected component from a JSFiddle page layout and implement it as a reusable NiceGUI render function with responsive styling and typed bindable state where needed.
|
||||||
|
x-personal-mcp:
|
||||||
|
id: nicegui-component-extraction
|
||||||
|
version: 1.0.0
|
||||||
|
tags:
|
||||||
|
- nicegui
|
||||||
|
- components
|
||||||
|
- frontend
|
||||||
|
- refactoring
|
||||||
|
- jsfiddle
|
||||||
|
- prompts
|
||||||
|
capabilities:
|
||||||
|
- resource://prompts/nicegui-component-extraction/document
|
||||||
|
arguments:
|
||||||
|
component:
|
||||||
|
title: Component
|
||||||
|
description: Component or page region to extract, identified by its visible label, semantic role, or selector.
|
||||||
|
required: true
|
||||||
|
source_layout:
|
||||||
|
title: Source layout
|
||||||
|
description: Optional HTML and CSS from the JSFiddle page layout prompt; when omitted, use the latest applicable output in the conversation.
|
||||||
|
required: false
|
||||||
|
target_location:
|
||||||
|
title: Target location
|
||||||
|
description: Optional target NiceGUI page, module, or package in which to create and integrate the component.
|
||||||
|
required: false
|
||||||
|
behavior_requirements:
|
||||||
|
title: Behavior requirements
|
||||||
|
description: Optional interactions, state, callbacks, or content variations the extracted component must support.
|
||||||
|
required: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# NiceGUI Component Extraction
|
||||||
|
|
||||||
|
Extract one user-selected component from the output of the [JSFiddle Page Layout](../jsfiddle-page-layout/PROMPT.md) prompt and implement it as a reusable NiceGUI component in the target repository.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
1. `component`: required visible label, semantic role, or selector identifying the component to extract
|
||||||
|
2. `source_layout`: optional HTML and CSS; when omitted, use the latest applicable JSFiddle page layout output in the conversation
|
||||||
|
3. `target_location`: optional target page, module, or package; infer it from the repository when omitted
|
||||||
|
4. `behavior_requirements`: optional interactions, state, callbacks, or content variations
|
||||||
|
|
||||||
|
If the selected component or source layout cannot be identified unambiguously, ask one concise clarification question before editing.
|
||||||
|
|
||||||
|
## Required References
|
||||||
|
|
||||||
|
Apply both references before implementation:
|
||||||
|
|
||||||
|
1. Component boundaries, responsive layout, Quasar props, Tailwind utilities, and shared CSS: [NiceGUI Page Layout and Styling](../../skills/nicegui/references/architecture-and-styling.md)
|
||||||
|
2. Typed UI state, propagation, mutable defaults, binding strictness, and version checks: [Binding Dataclasses Deep Dive](../../skills/nicegui/references/binding-dataclasses.md)
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. Locate the selected region in the source HTML and CSS, including its responsive rules, states, and dependencies on surrounding layout.
|
||||||
|
2. Inspect the target repository's NiceGUI version, package structure, component conventions, shared CSS loading, and nearest page call site.
|
||||||
|
3. Define the smallest reusable API for the component:
|
||||||
|
- name the public function `render_<component_name>` using snake_case
|
||||||
|
- accept content, typed state, and event callbacks as explicit parameters
|
||||||
|
- keep business rules, persistence, and service access outside the component
|
||||||
|
- preserve an established return-value convention; otherwise return the component's root NiceGUI element
|
||||||
|
4. Translate semantic HTML into native NiceGUI and Quasar elements. Do not embed the original page wholesale with `ui.html` when standard components express the structure.
|
||||||
|
5. Recreate only the CSS needed by the extracted component:
|
||||||
|
- use Quasar props for component appearance and behavior
|
||||||
|
- use NiceGUI classes and Tailwind utilities for spacing, sizing, alignment, and responsive layout
|
||||||
|
- use scoped shared CSS only where props and utilities are insufficient
|
||||||
|
- do not override Quasar field internals or duplicate globally loaded styles
|
||||||
|
6. Model editable or shared component state with a typed `@binding.bindable_dataclass` only when binding improves the interaction:
|
||||||
|
- use `field(default_factory=...)` for mutable defaults
|
||||||
|
- scope state to the appropriate page, client, or user
|
||||||
|
- keep binding transforms pure and inexpensive
|
||||||
|
- assign updated collections back to bound fields instead of relying on in-place mutation
|
||||||
|
7. Integrate the render function at the nearest target page or call site without moving unrelated page composition or domain logic into the component.
|
||||||
|
8. Preserve accessibility, focus behavior, text wrapping, stable dimensions, and the source layout's visual hierarchy.
|
||||||
|
9. Run the narrowest available tests, lint, and type checks for the changed files. For visual components, verify representative mobile, landscape desktop, and portrait desktop viewports when browser tooling is available.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
Complete the implementation in the target repository, then report:
|
||||||
|
|
||||||
|
1. Files created or updated.
|
||||||
|
2. The `render_*` function signature and its state or callback contract.
|
||||||
|
3. Any deliberate visual or interaction differences from the JSFiddle source.
|
||||||
|
4. Validation commands and outcomes, including viewport checks when performed.
|
||||||
|
|
||||||
|
## Quality Rules
|
||||||
|
|
||||||
|
1. Extract exactly the requested component and its necessary local dependencies.
|
||||||
|
2. Prefer the target repository's established patterns over introducing a new abstraction style.
|
||||||
|
3. Keep the component presentation-focused and reusable across pages with compatible data.
|
||||||
|
4. Do not add a bindable dataclass for static content or event-local state that is clearer as ordinary parameters.
|
||||||
|
5. Do not create a second component tree for mobile; use responsive classes and stable layout constraints.
|
||||||
|
6. Keep custom CSS tokenized, scoped to the component, and loaded once by the application's composition layer.
|
||||||
@@ -1,21 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: async-fastapi-sqlmodel
|
name: async-fastapi-sqlmodel
|
||||||
description: 'Explain and apply async database principles for FastAPI, SQLAlchemy 2.x, and SQLModel. Use when: learning or reviewing AsyncEngine and AsyncSession lifecycles, FastAPI lifespan and yield dependencies, transaction boundaries, concurrency safety, implicit ORM I/O, AsyncExitStack, pooling, testing, or SQLModel integration.'
|
description: 'Explain and apply async database principles for FastAPI, SQLAlchemy 2.x, and SQLModel. Use when: learning or reviewing cached AsyncEngine and session-factory lifecycles, AsyncSession scopes and injection, FastAPI lifespan and yield dependencies, transaction boundaries, concurrency safety, implicit ORM I/O, pooling, testing, or SQLModel integration.'
|
||||||
x-personal-mcp:
|
|
||||||
id: async-fastapi-sqlmodel
|
|
||||||
version: 1.1.0
|
|
||||||
tags:
|
|
||||||
- fastapi
|
|
||||||
- sqlalchemy
|
|
||||||
- sqlmodel
|
|
||||||
- async
|
|
||||||
- asyncio
|
|
||||||
- database
|
|
||||||
- transactions
|
|
||||||
- resource-lifecycle
|
|
||||||
- architecture
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/async-fastapi-sqlmodel/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Async FastAPI, SQLAlchemy, and SQLModel
|
# Async FastAPI, SQLAlchemy, and SQLModel
|
||||||
@@ -24,6 +9,8 @@ Use this skill to explain how an async database layer works, why the recommended
|
|||||||
|
|
||||||
Primary targets: PostgreSQL with asyncpg and SQLite with aiosqlite.
|
Primary targets: PostgreSQL with asyncpg and SQLite with aiosqlite.
|
||||||
|
|
||||||
|
Engine and session mechanics mirror the [`nicegui-db` template repository](https://forgejo.john-stream.com/john/nicegui-db). Treat that template as the implementation baseline, then explain the rationale, lifecycle constraints, and tradeoffs behind its cached engines, session factories, context managers, dependency wiring, and `with_session` decorator. Source-specific claims in the references link to the reviewed template commit so behavior remains auditable as the template evolves.
|
||||||
|
|
||||||
## When to Use
|
## When to Use
|
||||||
|
|
||||||
- Explain an async engine, session factory, session, connection, or transaction.
|
- Explain an async engine, session factory, session, connection, or transaction.
|
||||||
@@ -51,7 +38,7 @@ Keep three ownership scopes distinct:
|
|||||||
|
|
||||||
| Scope | Object | Purpose | Typical owner |
|
| Scope | Object | Purpose | Typical owner |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| Application process | `AsyncEngine` and `async_sessionmaker` | Dialect, connection pool, and repeatable session configuration | FastAPI lifespan |
|
| Application process | Cached `AsyncEngine` and lifespan-owned `async_sessionmaker` | Dialect, connection pool, schema initialization, and repeatable session configuration | FastAPI lifespan |
|
||||||
| Request or concurrent task | `AsyncSession` | Mutable ORM identity map and transactional state | A `yield` dependency or explicit unit of work |
|
| Request or concurrent task | `AsyncSession` | Mutable ORM identity map and transactional state | A `yield` dependency or explicit unit of work |
|
||||||
| Atomic operation | `SessionTransaction` | Commit all changes together or roll them back together | Service or use-case boundary |
|
| Atomic operation | `SessionTransaction` | Commit all changes together or roll them back together | Service or use-case boundary |
|
||||||
|
|
||||||
@@ -61,16 +48,16 @@ The engine is a long-lived factory and pool, not a single database connection. T
|
|||||||
|
|
||||||
### Match lifetime to ownership
|
### Match lifetime to ownership
|
||||||
|
|
||||||
- Create one `AsyncEngine` per process and database configuration in the normal case.
|
- Resolve one cached `AsyncEngine` per database URL during the active application lifecycle.
|
||||||
- Dispose it explicitly in an awaitable shutdown path; garbage collection cannot reliably await async driver cleanup.
|
- Enter one owning engine scope per URL; initialize registered SQLModel metadata by default, then dispose the engine and clear cached resolution on exit.
|
||||||
- Configure `async_sessionmaker` once and call it to create short-lived sessions.
|
- Configure the application `async_sessionmaker` inside the engine lifecycle; use the template's cached factory resolver only for standalone helpers that cannot receive the application factory.
|
||||||
- Close each session deterministically with `async with` or a FastAPI dependency that yields once.
|
- Close each session deterministically with `async with` or a FastAPI dependency that yields once.
|
||||||
|
|
||||||
See [engine lifecycle](references/engine.md) and [session management](references/session.md).
|
See [engine lifecycle](references/engine.md) and [session management](references/session.md).
|
||||||
|
|
||||||
### Isolate mutable session state
|
### Isolate mutable session state
|
||||||
|
|
||||||
An `AsyncSession` represents one stateful transaction in progress. Never use one session in multiple concurrent tasks, including branches of `asyncio.gather()`. Give each task its own session and pass sessions explicitly rather than relying on mutable scoped globals.
|
An `AsyncSession` represents one stateful transaction in progress. Never use one session in multiple concurrent tasks, including branches of `asyncio.gather()`. Give each task its own session. Template-style `@with_session` functions inject one only when the `session` argument is omitted; a supplied session remains caller-owned, and explicit `None` is forwarded unchanged.
|
||||||
|
|
||||||
See [session management](references/session.md).
|
See [session management](references/session.md).
|
||||||
|
|
||||||
@@ -92,9 +79,9 @@ See [transaction boundaries](references/transactions.md).
|
|||||||
|
|
||||||
FastAPI lifespan owns resources shared by many requests. A dependency with one `yield` owns request-scoped resources and runs cleanup after use. These are related context-manager mechanisms but solve different lifetime problems.
|
FastAPI lifespan owns resources shared by many requests. A dependency with one `yield` owns request-scoped resources and runs cleanup after use. These are related context-manager mechanisms but solve different lifetime problems.
|
||||||
|
|
||||||
Use `AsyncExitStack` when lifespan acquires a variable, conditional, or mixed collection of context-managed resources. It records cleanup as resources are acquired and unwinds callbacks in reverse order. A single engine with one cleanup callback can use a plain `try/finally`; `AsyncExitStack` is a composition tool, not a requirement.
|
Use `AsyncExitStack` when lifespan acquires a variable, conditional, or mixed collection of context-managed resources. It records cleanup as resources are acquired and unwinds callbacks in reverse order. A single engine should use the direct engine context manager; `AsyncExitStack` is a composition tool, not a requirement.
|
||||||
|
|
||||||
See [engine lifecycle](references/engine.md).
|
See [FastAPI database integration](references/fastapi.md).
|
||||||
|
|
||||||
### Use SQLModel as the primary modeling layer
|
### Use SQLModel as the primary modeling layer
|
||||||
|
|
||||||
@@ -110,6 +97,14 @@ Pool sizing, overflow, recycle, pre-ping, isolation, statement timeouts, and hea
|
|||||||
|
|
||||||
See [observability and resilience](references/observability.md).
|
See [observability and resilience](references/observability.md).
|
||||||
|
|
||||||
|
### Test through the production seam
|
||||||
|
|
||||||
|
Keep the production engine and session-factory construction path intact in tests. Select a dedicated PostgreSQL, local SQLite, or in-memory SQLite URL at that seam, then override the request-session dependency only for the test lifetime. Use a test-scoped outer transaction with SAVEPOINT-backed session commits when application code calls `commit()`; it exercises normal transaction behavior while cleanup remains deterministic.
|
||||||
|
|
||||||
|
In-memory SQLite is suitable for serial tests. For multiple simultaneous sessions, use a named shared-cache SQLite URL or a temporary file, and retain PostgreSQL integration coverage for PostgreSQL-specific behavior.
|
||||||
|
|
||||||
|
See [database testing and fixture data](references/testing.md).
|
||||||
|
|
||||||
## Reference Map
|
## Reference Map
|
||||||
|
|
||||||
| Concept | Reference |
|
| Concept | Reference |
|
||||||
@@ -117,41 +112,19 @@ See [observability and resilience](references/observability.md).
|
|||||||
| Engine lifecycle and ownership | [Engine lifecycle reference](references/engine.md) |
|
| Engine lifecycle and ownership | [Engine lifecycle reference](references/engine.md) |
|
||||||
| Session factory and scope | [Session management reference](references/session.md) |
|
| Session factory and scope | [Session management reference](references/session.md) |
|
||||||
| Transaction boundaries | [Transaction boundaries reference](references/transactions.md) |
|
| Transaction boundaries | [Transaction boundaries reference](references/transactions.md) |
|
||||||
| Lifespan composition | [Engine lifecycle reference](references/engine.md) |
|
| FastAPI lifespan composition | [FastAPI integration reference](references/fastapi.md) |
|
||||||
| Dependency injection | [Session management reference](references/session.md) |
|
| FastAPI dependency injection | [FastAPI integration reference](references/fastapi.md) |
|
||||||
| Implicit I/O control in ORM | [Implicit I/O reference](references/implicit_io.md) |
|
| Implicit I/O control in ORM | [Implicit I/O reference](references/implicit_io.md) |
|
||||||
| Observability and resilience | [Observability reference](references/observability.md) |
|
| Observability and resilience | [Observability reference](references/observability.md) |
|
||||||
| SQLModel-first modeling | [SQLModel integration reference](references/sqlmodel.md) |
|
| SQLModel-first modeling | [SQLModel integration reference](references/sqlmodel.md) |
|
||||||
| CRUD repository and standalone functions | [Basic CRUD reference](references/crud.md) |
|
| CRUD repository and standalone functions | [Basic CRUD reference](references/crud.md) |
|
||||||
|
| Test database selection and fixture data | [Database testing reference](references/testing.md) |
|
||||||
|
|
||||||
## Canonical Composition Pattern
|
## Canonical Composition Pattern
|
||||||
|
|
||||||
This example shows the ownership boundaries. Adapt state storage and dependency wiring to the application's conventions.
|
The framework-independent primitives live in [engine lifecycle](references/engine.md), [session management](references/session.md), and [transaction boundaries](references/transactions.md). Their canonical FastAPI adaptation, including lifespan state and `Annotated` dependencies, lives in [FastAPI database integration](references/fastapi.md).
|
||||||
|
|
||||||
```python
|
For background work that outlives a request, inject the shared factory and create a new session inside that task instead of retaining the request's session.
|
||||||
from contextlib import AsyncExitStack, asynccontextmanager
|
|
||||||
from collections.abc import AsyncIterator
|
|
||||||
|
|
||||||
from fastapi import FastAPI
|
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
|
||||||
|
|
||||||
@asynccontextmanager
|
|
||||||
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
|
||||||
async with AsyncExitStack() as stack:
|
|
||||||
engine = create_async_engine(settings.database_url)
|
|
||||||
stack.push_async_callback(engine.dispose)
|
|
||||||
|
|
||||||
session_factory = async_sessionmaker(engine, expire_on_commit=False)
|
|
||||||
app.state.session_factory = session_factory
|
|
||||||
yield
|
|
||||||
|
|
||||||
|
|
||||||
async def get_session() -> AsyncIterator[AsyncSession]:
|
|
||||||
async with app.state.session_factory() as session:
|
|
||||||
yield session
|
|
||||||
```
|
|
||||||
|
|
||||||
For direct construction without `AsyncExitStack`, put `await engine.dispose()` in a `finally` block. For background work that outlives a request, create a new session inside that task instead of retaining the request's session.
|
|
||||||
|
|
||||||
## Explanation Procedure
|
## Explanation Procedure
|
||||||
|
|
||||||
@@ -173,6 +146,7 @@ When reviewing code, verify:
|
|||||||
- Relationship and deferred-column access cannot surprise the event loop with implicit I/O.
|
- Relationship and deferred-column access cannot surprise the event loop with implicit I/O.
|
||||||
- Pool and timeout settings are justified by deployment behavior.
|
- Pool and timeout settings are justified by deployment behavior.
|
||||||
- Tests exercise rollback, cleanup, concurrency, and lifespan behavior where relevant.
|
- Tests exercise rollback, cleanup, concurrency, and lifespan behavior where relevant.
|
||||||
|
- Tests use a dedicated database target and preserve production session mechanics.
|
||||||
|
|
||||||
## Anti-Patterns to Flag
|
## Anti-Patterns to Flag
|
||||||
|
|
||||||
|
|||||||
@@ -5,12 +5,13 @@
|
|||||||
- [SQLModel update-data tutorial](https://sqlmodel.tiangolo.com/tutorial/fastapi/update-extra-data/)
|
- [SQLModel update-data tutorial](https://sqlmodel.tiangolo.com/tutorial/fastapi/update-extra-data/)
|
||||||
- [SQLModel select tutorial](https://sqlmodel.tiangolo.com/tutorial/select/)
|
- [SQLModel select tutorial](https://sqlmodel.tiangolo.com/tutorial/select/)
|
||||||
- [SQLAlchemy `AsyncSession` API](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.AsyncSession)
|
- [SQLAlchemy `AsyncSession` API](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.AsyncSession)
|
||||||
|
- [`nicegui-db` service functions](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/services/my_table.py)
|
||||||
|
|
||||||
??? abstract "Decision metadata"
|
??? abstract "Decision metadata"
|
||||||
- Status: adopted
|
- Status: adopted
|
||||||
- Decision level: advisory
|
- Decision level: advisory
|
||||||
- Applies to: api-runtime, workers, tests
|
- Applies to: api-runtime, workers, tests
|
||||||
- Last reviewed: 2026-07-26
|
- Last reviewed: 2026-08-06
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -21,17 +22,17 @@ Show a small SQLModel CRUD layer in two forms:
|
|||||||
- independent functions for convenient standalone or composed operations;
|
- independent functions for convenient standalone or composed operations;
|
||||||
- a repository object that groups those functions behind one domain-oriented interface.
|
- a repository object that groups those functions behind one domain-oriented interface.
|
||||||
|
|
||||||
Every public operation accepts an optional `AsyncSession`. When omitted, reads resolve the cached session factory and own a short-lived session, while writes resolve the same factory and own a complete session-and-transaction scope. When supplied, reads borrow the session and writes borrow its already-active caller-owned transaction. The repository stores configuration and delegates to the same functions without changing those semantics.
|
Template-style public functions use `@with_session` and accept an optional `AsyncSession`. When the argument is omitted, the decorator resolves the cached session factory and owns a short-lived session. When supplied, the function borrows the session without controlling its lifetime or transaction. The decorator does not commit, so standalone writes need a visible transaction strategy; repository methods remain explicit-session operations for predictable composition.
|
||||||
|
|
||||||
Use the same vocabulary at every layer:
|
Use the same vocabulary at every layer:
|
||||||
|
|
||||||
| Operation | Function | Repository method | Scope when session is omitted | Missing-row result |
|
| Operation | Function | Repository method | Scope when session is omitted | Missing-row result |
|
||||||
|---|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
| Create | `create_widget()` | `create()` | Owned transaction | Not applicable |
|
| Create | `create_widget()` | `create()` | Owned session; no implicit commit | Not applicable |
|
||||||
| Read one | `get_widget()` | `get()` | Owned session | `None` |
|
| Read one | `get_widget()` | `get()` | Owned session | `None` |
|
||||||
| Read many | `list_widgets()` | `list()` | Owned session | Empty list |
|
| Read many | `list_widgets()` | `list()` | Owned session | Empty list |
|
||||||
| Update | `update_widget()` | `update()` | Owned transaction | `None` |
|
| Update | `update_widget()` | `update()` | Owned session; no implicit commit | `None` |
|
||||||
| Delete | `delete_widget()` | `delete()` | Owned transaction | `None` |
|
| Delete | `delete_widget()` | `delete()` | Owned session; no implicit commit | `None` |
|
||||||
|
|
||||||
Functions and repository methods both put domain arguments first. Database configuration and sessions are keyword-only infrastructure arguments. This keeps call sites analogous and makes ownership choices visible.
|
Functions and repository methods both put domain arguments first. Database configuration and sessions are keyword-only infrastructure arguments. This keeps call sites analogous and makes ownership choices visible.
|
||||||
|
|
||||||
@@ -58,159 +59,132 @@ This reference uses direct field arguments and full-update semantics to keep the
|
|||||||
|
|
||||||
## Independent CRUD Functions
|
## Independent CRUD Functions
|
||||||
|
|
||||||
Functions are the simplest default when grouping state or behavior in an object adds no value. Each function is a complete operation boundary: it can run standalone by resolving the cached factory from `database_url`, or compose into a caller-owned scope through `session`.
|
Functions are the simplest default when grouping state or behavior in an object adds no value. Decorate public service functions when both standalone reads and explicit composition are useful. The assertion narrows the optional type after decorator injection and catches accidental explicit `None` calls.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession
|
|
||||||
from sqlmodel import select
|
from sqlmodel import select
|
||||||
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
from .session import session_scope
|
from .session import with_session
|
||||||
from .session import transaction_scope
|
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
async def create_widget(
|
async def create_widget(
|
||||||
name: str,
|
name: str,
|
||||||
description: str | None = None,
|
description: str | None = None,
|
||||||
*,
|
*,
|
||||||
database_url: str,
|
|
||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> Widget:
|
) -> Widget:
|
||||||
async with transaction_scope(
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
database_url=database_url,
|
widget = Widget(name=name, description=description)
|
||||||
session=session,
|
session.add(widget)
|
||||||
) as active_session:
|
await session.flush()
|
||||||
widget = Widget(name=name, description=description)
|
return widget
|
||||||
active_session.add(widget)
|
|
||||||
await active_session.flush()
|
|
||||||
return widget
|
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
async def get_widget(
|
async def get_widget(
|
||||||
widget_id: int,
|
widget_id: int,
|
||||||
*,
|
*,
|
||||||
database_url: str,
|
|
||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> Widget | None:
|
) -> Widget | None:
|
||||||
async with session_scope(
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
database_url=database_url,
|
return await session.get(Widget, widget_id)
|
||||||
session=session,
|
|
||||||
) as active_session:
|
|
||||||
return await active_session.get(Widget, widget_id)
|
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
async def list_widgets(
|
async def list_widgets(
|
||||||
*,
|
*,
|
||||||
database_url: str,
|
|
||||||
offset: int = 0,
|
offset: int = 0,
|
||||||
limit: int = 100,
|
limit: int = 100,
|
||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> list[Widget]:
|
) -> list[Widget]:
|
||||||
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
if offset < 0:
|
if offset < 0:
|
||||||
raise ValueError("offset must be non-negative")
|
raise ValueError("offset must be non-negative")
|
||||||
if not 1 <= limit <= 100:
|
if not 1 <= limit <= 100:
|
||||||
raise ValueError("limit must be between 1 and 100")
|
raise ValueError("limit must be between 1 and 100")
|
||||||
|
|
||||||
async with session_scope(
|
statement = select(Widget).order_by(Widget.id).offset(offset).limit(limit)
|
||||||
database_url=database_url,
|
return list(await session.scalars(statement))
|
||||||
session=session,
|
|
||||||
) as active_session:
|
|
||||||
statement = select(Widget).order_by(Widget.id).offset(offset).limit(limit)
|
|
||||||
return list(await active_session.scalars(statement))
|
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
async def update_widget(
|
async def update_widget(
|
||||||
widget_id: int,
|
widget_id: int,
|
||||||
name: str,
|
name: str,
|
||||||
description: str | None,
|
description: str | None,
|
||||||
*,
|
*,
|
||||||
database_url: str,
|
|
||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> Widget | None:
|
) -> Widget | None:
|
||||||
async with transaction_scope(
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
database_url=database_url,
|
widget = await session.get(Widget, widget_id)
|
||||||
session=session,
|
if widget is None:
|
||||||
) as active_session:
|
return None
|
||||||
widget = await active_session.get(Widget, widget_id)
|
|
||||||
if widget is None:
|
|
||||||
return None
|
|
||||||
|
|
||||||
widget.name = name
|
widget.name = name
|
||||||
widget.description = description
|
widget.description = description
|
||||||
await active_session.flush()
|
await session.flush()
|
||||||
return widget
|
return widget
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
async def delete_widget(
|
async def delete_widget(
|
||||||
widget_id: int,
|
widget_id: int,
|
||||||
*,
|
*,
|
||||||
database_url: str,
|
|
||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> Widget | None:
|
) -> Widget | None:
|
||||||
async with transaction_scope(
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
database_url=database_url,
|
widget = await session.get(Widget, widget_id)
|
||||||
session=session,
|
if widget is None:
|
||||||
) as active_session:
|
return None
|
||||||
widget = await active_session.get(Widget, widget_id)
|
|
||||||
if widget is None:
|
|
||||||
return None
|
|
||||||
|
|
||||||
await active_session.delete(widget)
|
await session.delete(widget)
|
||||||
await active_session.flush()
|
await session.flush()
|
||||||
return widget
|
return widget
|
||||||
```
|
```
|
||||||
|
|
||||||
Update and delete load the row through the same session that mutates it. This avoids accepting detached instances from an earlier standalone read and gives both operations an explicit `None` result that the application layer can map to a domain or HTTP error. Delete returns the loaded object for callers that need its values, but that object represents a row scheduled for deletion and must not be reused as persistent state. List operations validate their bounds and order by the primary key so pagination is deterministic. Add a unique tiebreaker whenever ordering by a non-unique field.
|
Update and delete load the row through the same session that mutates it. This avoids accepting detached instances from an earlier standalone read and gives both operations an explicit `None` result that the application layer can map to a domain or HTTP error. Delete returns the loaded object for callers that need its values, but that object represents a row scheduled for deletion and must not be reused as persistent state. List operations validate their bounds and order by the primary key so pagination is deterministic. Add a unique tiebreaker whenever ordering by a non-unique field.
|
||||||
|
|
||||||
`flush()` sends pending writes and populates ordinary generated primary keys. It does not itself commit. For a standalone write, the surrounding owned `transaction_scope()` commits after the function body succeeds. For a supplied session, the caller's outer transaction retains commit and rollback ownership. Use `await active_session.refresh(widget)` only when the operation deliberately needs database-generated state that was not returned during the flush; an unconditional refresh adds another query.
|
`flush()` sends pending writes and populates ordinary generated primary keys. It does not itself commit. A decorated write called without a session will therefore roll back when its owned session closes unless the function explicitly commits. Prefer passing a transaction-scoped session so several writes compose atomically. Use `await session.refresh(widget)` only when the operation deliberately needs database-generated state that was not returned during the flush; an unconditional refresh adds another query.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Repository Object
|
## Repository Object
|
||||||
|
|
||||||
A repository can provide a stable domain-facing interface when several callers need the same grouped operations. It stores repeatable database configuration, never a mutable session. Every method delegates to the analogous function and exposes the same optional-session contract.
|
A repository can provide a stable domain-facing interface when several callers need the same grouped operations. It remains stateless here: every method requires a session and delegates to the analogous function.
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
class WidgetRepository:
|
class WidgetRepository:
|
||||||
def __init__(self, database_url: str) -> None:
|
|
||||||
self.database_url = database_url
|
|
||||||
|
|
||||||
async def create(
|
async def create(
|
||||||
self,
|
self,
|
||||||
|
session: AsyncSession,
|
||||||
name: str,
|
name: str,
|
||||||
description: str | None = None,
|
description: str | None = None,
|
||||||
*,
|
|
||||||
session: AsyncSession | None = None,
|
|
||||||
) -> Widget:
|
) -> Widget:
|
||||||
return await create_widget(
|
return await create_widget(
|
||||||
name,
|
name,
|
||||||
description,
|
description,
|
||||||
database_url=self.database_url,
|
|
||||||
session=session,
|
session=session,
|
||||||
)
|
)
|
||||||
|
|
||||||
async def get(
|
async def get(
|
||||||
self,
|
self,
|
||||||
|
session: AsyncSession,
|
||||||
widget_id: int,
|
widget_id: int,
|
||||||
*,
|
|
||||||
session: AsyncSession | None = None,
|
|
||||||
) -> Widget | None:
|
) -> Widget | None:
|
||||||
return await get_widget(
|
return await get_widget(widget_id, session=session)
|
||||||
widget_id,
|
|
||||||
database_url=self.database_url,
|
|
||||||
session=session,
|
|
||||||
)
|
|
||||||
|
|
||||||
async def list(
|
async def list(
|
||||||
self,
|
self,
|
||||||
|
session: AsyncSession,
|
||||||
*,
|
*,
|
||||||
offset: int = 0,
|
offset: int = 0,
|
||||||
limit: int = 100,
|
limit: int = 100,
|
||||||
session: AsyncSession | None = None,
|
|
||||||
) -> list[Widget]:
|
) -> list[Widget]:
|
||||||
return await list_widgets(
|
return await list_widgets(
|
||||||
database_url=self.database_url,
|
|
||||||
offset=offset,
|
offset=offset,
|
||||||
limit=limit,
|
limit=limit,
|
||||||
session=session,
|
session=session,
|
||||||
@@ -218,34 +192,27 @@ class WidgetRepository:
|
|||||||
|
|
||||||
async def update(
|
async def update(
|
||||||
self,
|
self,
|
||||||
|
session: AsyncSession,
|
||||||
widget_id: int,
|
widget_id: int,
|
||||||
name: str,
|
name: str,
|
||||||
description: str | None,
|
description: str | None,
|
||||||
*,
|
|
||||||
session: AsyncSession | None = None,
|
|
||||||
) -> Widget | None:
|
) -> Widget | None:
|
||||||
return await update_widget(
|
return await update_widget(
|
||||||
widget_id,
|
widget_id,
|
||||||
name,
|
name,
|
||||||
description,
|
description,
|
||||||
database_url=self.database_url,
|
|
||||||
session=session,
|
session=session,
|
||||||
)
|
)
|
||||||
|
|
||||||
async def delete(
|
async def delete(
|
||||||
self,
|
self,
|
||||||
|
session: AsyncSession,
|
||||||
widget_id: int,
|
widget_id: int,
|
||||||
*,
|
|
||||||
session: AsyncSession | None = None,
|
|
||||||
) -> Widget | None:
|
) -> Widget | None:
|
||||||
return await delete_widget(
|
return await delete_widget(widget_id, session=session)
|
||||||
widget_id,
|
|
||||||
database_url=self.database_url,
|
|
||||||
session=session,
|
|
||||||
)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
The object is intentionally thin. Tests can construct it with a test database URL or pass a transaction-scoped test session to individual methods. A caller-provided session always wins and remains open after the method returns. A standalone operation closes its owned session before returning, so returned objects are detached; load every required scalar, deferred column, and relationship explicitly before the scope exits, and do not mutate those objects expecting persistence.
|
The object is intentionally thin. Tests pass a transaction-scoped test session directly. The caller always owns that session and its transaction, and the repository never closes or commits it.
|
||||||
|
|
||||||
If a read participates in a later write, pass the same session and place both operations inside the explicit transaction. This avoids splitting one use case across sessions and keeps SQLAlchemy's autobegin behavior from obscuring transaction ownership. Add a repository only when its naming, shared query policy, dependency substitution, or domain boundary improves the application. Independent functions remain a valid and often clearer design.
|
If a read participates in a later write, pass the same session and place both operations inside the explicit transaction. This avoids splitting one use case across sessions and keeps SQLAlchemy's autobegin behavior from obscuring transaction ownership. Add a repository only when its naming, shared query policy, dependency substitution, or domain boundary improves the application. Independent functions remain a valid and often clearer design.
|
||||||
|
|
||||||
@@ -253,12 +220,10 @@ If a read participates in a later write, pass the same session and place both op
|
|||||||
|
|
||||||
## Transaction Ownership
|
## Transaction Ownership
|
||||||
|
|
||||||
Compose multiple calls under one use-case transaction. At this boundary, a supplied session joins its already-active caller-owned transaction, while omitting the session creates a standalone session and transaction. Each nested CRUD write receives `active_session`, detects that transaction, and borrows it instead of committing independently.
|
Compose multiple calls under one use-case transaction. `db_transaction_scope()` owns the standalone engine, factory, session, and transaction lifetimes. Decorated CRUD functions detect the supplied session and borrow it; repository methods receive it directly.
|
||||||
|
|
||||||
The scope names describe exactly what they own: `session_scope()` manages session lifetime but never commits, while `transaction_scope()` manages a complete transaction only when it also creates the session. Both yield the name `active_session` because downstream CRUD code does not need to know whether the session was borrowed or owned.
|
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from .session import transaction_scope
|
from .session import db_transaction_scope
|
||||||
|
|
||||||
|
|
||||||
async def replace_widget(
|
async def replace_widget(
|
||||||
@@ -266,38 +231,33 @@ async def replace_widget(
|
|||||||
widget_id: int,
|
widget_id: int,
|
||||||
replacement_name: str,
|
replacement_name: str,
|
||||||
replacement_description: str | None = None,
|
replacement_description: str | None = None,
|
||||||
*,
|
|
||||||
session: AsyncSession | None = None,
|
|
||||||
) -> Widget | None:
|
) -> Widget | None:
|
||||||
async with transaction_scope(
|
async with db_transaction_scope() as active_session:
|
||||||
database_url=repository.database_url,
|
|
||||||
session=session,
|
|
||||||
) as active_session:
|
|
||||||
deleted_widget = await repository.delete(
|
deleted_widget = await repository.delete(
|
||||||
|
active_session,
|
||||||
widget_id,
|
widget_id,
|
||||||
session=active_session,
|
|
||||||
)
|
)
|
||||||
if deleted_widget is None:
|
if deleted_widget is None:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
return await repository.create(
|
return await repository.create(
|
||||||
|
active_session,
|
||||||
replacement_name,
|
replacement_name,
|
||||||
replacement_description,
|
replacement_description,
|
||||||
session=active_session,
|
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
If creation fails, deletion rolls back with it. For a caller-owned transaction, wrap the call in `async with session.begin():` and pass that session. For a standalone use case, omit the session; the outer `transaction_scope()` commits on successful exit, rolls back on exception, and closes its owned session. Do not add direct `commit()` calls to CRUD functions or repository methods because that prevents callers from composing several operations atomically. See [transaction boundaries](transactions.md) and [session management](session.md) for ownership details.
|
If creation fails, deletion rolls back with it. Inside an already-running application, prefer `async with session_factory.begin()` or `async with session.begin()` over `db_transaction_scope()` so the application-owned engine and factory remain in use. Do not add direct `commit()` calls to CRUD functions or repository methods because that prevents callers from composing several operations atomically. See [transaction boundaries](transactions.md) and [session management](session.md) for ownership details.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Anti-Patterns
|
## Anti-Patterns
|
||||||
|
|
||||||
- Storing one mutable `AsyncSession` on a long-lived repository object.
|
- Storing one mutable `AsyncSession` on a long-lived repository object.
|
||||||
- Constructing ad hoc factories or sessions instead of resolving the cached factory through the scope helpers.
|
- Creating sessions manually inside functions already using `@with_session`.
|
||||||
- Using `session_scope()` for an optional write, which would close an owned session without committing.
|
- Passing database configuration through every CRUD call instead of injecting a session at the data-access boundary.
|
||||||
- Accepting a supplied session for a write without requiring an active caller-owned transaction.
|
- Assuming decorator-owned write sessions commit on close.
|
||||||
- Calling `commit()` or `rollback()` directly instead of expressing ownership through `transaction_scope()`.
|
- Forwarding explicit `session=None` when decorator injection was intended.
|
||||||
- Accepting unbounded list queries.
|
- Accepting unbounded list queries.
|
||||||
- Accepting detached ORM instances for update or delete when an identifier can be resolved in the active session.
|
- Accepting detached ORM instances for update or delete when an identifier can be resolved in the active session.
|
||||||
- Accessing unloaded attributes after a standalone repository read has closed its owned session.
|
- Accessing unloaded attributes after a standalone repository read has closed its owned session.
|
||||||
@@ -307,16 +267,16 @@ If creation fails, deletion rolls back with it. For a caller-owned transaction,
|
|||||||
## Operational Checks
|
## Operational Checks
|
||||||
|
|
||||||
- Every CRUD call receives a task-local `AsyncSession`.
|
- Every CRUD call receives a task-local `AsyncSession`.
|
||||||
- Standalone reads resolve the cached factory by database URL and close their owned session.
|
- Standalone reads create and close a session at the service or application boundary.
|
||||||
- Standalone writes resolve the cached factory and own commit, rollback, and session cleanup through `transaction_scope()`.
|
- Standalone reads may omit `session`; decorated writes receive a transaction-scoped session or explicitly own their commit policy.
|
||||||
- Supplied write sessions already have an active caller-owned transaction.
|
- Supplied write sessions remain caller-owned.
|
||||||
- Each complete operation, service, or use-case boundary borrows an active transaction or owns a complete session-and-transaction scope.
|
- Each complete write operation declares a visible transaction boundary.
|
||||||
- List operations have pagination and deterministic ordering where required.
|
- List operations have pagination and deterministic ordering where required.
|
||||||
- Update requires values for both mutable fields; passing `None` explicitly clears the nullable description.
|
- Update requires values for both mutable fields; passing `None` explicitly clears the nullable description.
|
||||||
- Get, update, and delete use the same identifier and missing-row semantics.
|
- Get, update, and delete use the same identifier and missing-row semantics.
|
||||||
- Functions and repository methods use domain arguments first and keyword-only infrastructure arguments consistently.
|
- Decorated functions accept an optional keyword-only session; repository methods require one explicitly.
|
||||||
- Standalone reads load all state needed after their owned session closes.
|
- Standalone service reads load all state needed after their owned session closes.
|
||||||
- Repository objects hold configuration or policy, never request-scoped session state.
|
- Repository objects hold query policy when useful, never database configuration or request-scoped session state.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -330,5 +290,5 @@ If creation fails, deletion rolls back with it. For a caller-owned transaction,
|
|||||||
- Delete tests verify the returned row and its absence after commit.
|
- Delete tests verify the returned row and its absence after commit.
|
||||||
- Failure tests verify that a surrounding transaction rolls back all composed CRUD calls.
|
- Failure tests verify that a surrounding transaction rolls back all composed CRUD calls.
|
||||||
- Optional-session read tests verify borrowed sessions remain open and owned sessions close without committing.
|
- Optional-session read tests verify borrowed sessions remain open and owned sessions close without committing.
|
||||||
- Optional-session write tests verify supplied transactions remain caller-owned and standalone transactions commit or roll back before closing.
|
- Decorated write tests verify supplied transactions remain caller-owned and omitted sessions do not imply a commit.
|
||||||
- Composition tests pass one active session through several CRUD calls and verify one atomic commit or rollback.
|
- Composition tests pass one active session through several CRUD calls and verify one atomic commit or rollback.
|
||||||
@@ -1,45 +1,92 @@
|
|||||||
# Async SQLAlchemy Engine
|
# Async SQLAlchemy Engine
|
||||||
|
|
||||||
!!! info "Primary sources"
|
!!! info "Primary sources"
|
||||||
|
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
|
||||||
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||||
- [SQLAlchemy connections](https://docs.sqlalchemy.org/en/21/core/connections.html)
|
- [SQLAlchemy connections](https://docs.sqlalchemy.org/en/21/core/connections.html)
|
||||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
- [SQLAlchemy pooling and multiprocessing](https://docs.sqlalchemy.org/en/21/core/pooling.html#pooling-multiprocessing)
|
- [SQLAlchemy pooling and multiprocessing](https://docs.sqlalchemy.org/en/21/core/pooling.html#pooling-multiprocessing)
|
||||||
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
- [SQLAlchemy SQLite transaction control](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#enabling-non-legacy-sqlite-transactional-modes-with-the-sqlite3-or-aiosqlite-driver)
|
||||||
|
- [SQLAlchemy SQLite foreign-key support](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#foreign-key-support)
|
||||||
|
- [SQLite PRAGMA reference](https://www.sqlite.org/pragma.html)
|
||||||
|
- [`nicegui-db` engine implementation](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/db/engine.py)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Engine Ownership Model
|
## Engine Ownership Model
|
||||||
|
|
||||||
Create one async engine per process per database URL and keep engine construction independent from FastAPI.
|
Resolve one async engine for each database URL within an application, worker, command, or test lifecycle.
|
||||||
|
|
||||||
- SQLAlchemy guidance: the engine is intended as a long-lived, concurrent registry over pooled DB connections, not a per-request object.
|
- SQLAlchemy guidance: the engine is intended as a long-lived, concurrent registry over pooled DB connections, not a per-operation object.
|
||||||
- A cached function provides stable process-local engine identity without making framework state the only way to obtain it.
|
- `get_engine(database_url)` owns URL-keyed engine construction and caching.
|
||||||
- FastAPI lifespan starts and stops that independently defined resource; it does not contain the construction policy.
|
- The composition root enters `engine_scope(database_url)` once and therefore owns initialization and disposal.
|
||||||
|
- Services and repositories receive a session or session factory; they do not resolve an engine.
|
||||||
|
|
||||||
!!! tip "Practical rule"
|
!!! tip "Practical rule"
|
||||||
- Exactly one `create_async_engine(...)` call in the cached engine factory.
|
- Exactly one cached engine for each database URL during an active application-owned lifecycle.
|
||||||
- Zero `create_async_engine(...)` calls in request handlers.
|
- Exactly one active owning `engine_scope()` for a given URL.
|
||||||
- Zero calls to the cached factory from repository code.
|
- Zero `create_async_engine(...)` calls in feature code.
|
||||||
|
- Zero engine lookup or disposal calls in repository code.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Cached Engine Factory
|
## Cached Engine Resolution
|
||||||
|
|
||||||
Use [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) on a synchronous factory. Creating an `AsyncEngine` configures the dialect and pool; it does not need to await a database connection.
|
[`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) makes the database URL the engine identity:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from functools import cache
|
from functools import cache
|
||||||
|
|
||||||
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
|
|
||||||
|
|
||||||
@cache
|
@cache
|
||||||
def get_engine(database_url: str) -> AsyncEngine:
|
def get_engine(database_url: str) -> AsyncEngine:
|
||||||
return create_async_engine(
|
engine = create_async_engine(database_url, pool_pre_ping=True)
|
||||||
database_url,
|
if engine.dialect.name == "sqlite":
|
||||||
pool_pre_ping=True,
|
configure_aiosqlite_engine(engine)
|
||||||
)
|
return engine
|
||||||
|
```
|
||||||
|
|
||||||
|
Repeated calls with the same exact URL return the same `AsyncEngine`; different URLs produce independent cache entries. Construction configures the dialect and pool but normally does not open a database connection until the first operation. SQLite event listeners are installed only when a new cached engine is constructed, before its first connection.
|
||||||
|
|
||||||
|
Resolve settings into the final URL before calling `get_engine()`. Services and repositories should not call it directly: the cache controls construction identity, not ownership.
|
||||||
|
|
||||||
|
## Owning Engine Scope
|
||||||
|
|
||||||
|
Use one [`asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager) to pair cached resolution and optional schema initialization with disposal:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
from sqlmodel import SQLModel
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def engine_scope(
|
||||||
|
database_url: str,
|
||||||
|
*,
|
||||||
|
initialize: bool = True,
|
||||||
|
) -> AsyncGenerator[AsyncEngine]:
|
||||||
|
engine = get_engine(database_url)
|
||||||
|
if initialize:
|
||||||
|
await initialize_db(database_url)
|
||||||
|
|
||||||
|
try:
|
||||||
|
yield engine
|
||||||
|
finally:
|
||||||
|
await dispose_engine(database_url)
|
||||||
|
|
||||||
|
|
||||||
|
async def initialize_db(database_url: str) -> None:
|
||||||
|
from . import models # noqa: F401
|
||||||
|
|
||||||
|
engine = get_engine(database_url)
|
||||||
|
async with engine.begin() as connection:
|
||||||
|
await connection.run_sync(SQLModel.metadata.create_all)
|
||||||
|
|
||||||
|
|
||||||
async def dispose_engine(database_url: str) -> None:
|
async def dispose_engine(database_url: str) -> None:
|
||||||
@@ -48,52 +95,28 @@ async def dispose_engine(database_url: str) -> None:
|
|||||||
await engine.dispose()
|
await engine.dispose()
|
||||||
finally:
|
finally:
|
||||||
get_engine.cache_clear()
|
get_engine.cache_clear()
|
||||||
|
|
||||||
|
|
||||||
async def refresh_engine(database_url: str) -> AsyncEngine:
|
|
||||||
await dispose_engine(database_url)
|
|
||||||
return get_engine(database_url)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
The database URL is an explicit, hashable cache key. Calls with the same URL return the same engine; a different URL receives a different engine. If engine options vary at runtime, make them explicit hashable arguments too.
|
The code that enters `engine_scope()` owns the engine. It keeps that scope open for the complete application, worker, command, or test lifecycle and passes the yielded engine into session-factory construction. Successful and exceptional exits both dispose the pool and invalidate cached engine resolution.
|
||||||
|
|
||||||
Resolve settings at the composition boundary and call `get_engine(settings.database_url)`. Do not hide settings lookup or engine creation inside feature code.
|
Initialization imports the model package so every table is registered, then runs `SQLModel.metadata.create_all()` in `engine.begin()`. This is suitable for the template and focused tests. Use migrations instead when schema evolution is part of the deployment contract. Pass `initialize=False` only when another owner provisions the schema or a test is directly exercising construction without schema setup.
|
||||||
|
|
||||||
## Thin FastAPI Lifespan Wrapper
|
`dispose_engine()` clears the complete function cache, not only the requested URL. This matches the template and is safe under its intended single-database lifecycle. Applications that own several simultaneously active database URLs need per-key lifecycle management rather than this global invalidation behavior.
|
||||||
|
|
||||||
The lifespan context manager only connects the cached resource to FastAPI ownership:
|
Workers, scripts, and other composition roots enter `database_scope()` directly:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from collections.abc import AsyncIterator
|
async with database_scope(settings.database_url) as session_factory:
|
||||||
from contextlib import asynccontextmanager
|
await run_worker(session_factory)
|
||||||
|
|
||||||
from fastapi import FastAPI
|
|
||||||
|
|
||||||
|
|
||||||
@asynccontextmanager
|
|
||||||
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
|
||||||
database_url = app.state.settings.database_url
|
|
||||||
engine = get_engine(database_url)
|
|
||||||
app.state.engine = engine
|
|
||||||
|
|
||||||
try:
|
|
||||||
yield
|
|
||||||
finally:
|
|
||||||
await dispose_engine(database_url)
|
|
||||||
|
|
||||||
|
|
||||||
app = FastAPI(lifespan=lifespan)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
`dispose()` closes checked-in connections and replaces the pool, but it does not remove the Python object from `functools.cache`. `dispose_engine()` clears the cache even if driver cleanup raises, preventing a later lifespan run or test from retrieving that engine instance.
|
`database_scope()` is defined in [session management](session.md). It enters `engine_scope()` and creates the factory bound to the yielded engine.
|
||||||
|
|
||||||
This simple cleanup assumes one configured database URL per process. If a process intentionally owns several cached engines, use a small registry with per-key removal instead of clearing the whole cache. For a fixed engine, `try/finally` is sufficient; use `AsyncExitStack` when lifespan composes multiple conditional or dynamically acquired resources.
|
Do not overlap two owning scopes for the same URL. Both resolve the same cached engine, and the first scope to exit disposes it and clears the cache while the other still refers to it. For several fixed databases, use one non-overlapping owner per URL and account for global cache invalidation; use [`AsyncExitStack`](https://docs.python.org/3/library/contextlib.html#contextlib.AsyncExitStack) only after adopting lifecycle semantics that support several simultaneous owners.
|
||||||
|
|
||||||
When directly testing engine construction or lifespan behavior:
|
When directly testing engine construction or lifecycle behavior, enter `engine_scope()` in the test or fixture. Exiting the context disposes the engine even when the test fails and clears the cache for the next lifecycle.
|
||||||
|
|
||||||
- Call `get_engine.cache_clear()` before the test to remove process-local state.
|
See [FastAPI database integration](fastapi.md) for adapting `database_scope()` to application lifespan and dependency injection.
|
||||||
- Dispose any engine the test creates.
|
|
||||||
- Clear the cache again during teardown, even when the test fails.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -110,6 +133,62 @@ Use SQLAlchemy async driver URLs:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## SQLite Connection and Transaction Policy
|
||||||
|
|
||||||
|
SQLite settings do not form one indivisible bundle:
|
||||||
|
|
||||||
|
- `PRAGMA foreign_keys=ON` is a correctness requirement when the schema declares foreign keys. SQLite requires it on every connection, including the connection used by `metadata.create_all()`.
|
||||||
|
- Disabling the driver's implicit `BEGIN` and emitting `BEGIN` from SQLAlchemy provides non-legacy transaction behavior for `aiosqlite`. This makes SELECT, DDL, and SAVEPOINT behavior participate in SQLAlchemy's transaction boundary consistently.
|
||||||
|
- `PRAGMA busy_timeout` is a per-connection lock-wait policy. Choose the duration from the application's latency and contention requirements.
|
||||||
|
- `PRAGMA journal_mode=WAL` is an optional file-database concurrency policy. WAL persists in the database file, cannot be enabled for an in-memory database, and is not a substitute for transaction control.
|
||||||
|
|
||||||
|
Install instance-level listeners exactly once, immediately after constructing an `aiosqlite` engine and before its first connection:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from sqlalchemy import event
|
||||||
|
from sqlalchemy.engine import Connection
|
||||||
|
from sqlalchemy.engine.interfaces import DBAPIConnection
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
|
||||||
|
|
||||||
|
def configure_aiosqlite_engine(
|
||||||
|
engine: AsyncEngine,
|
||||||
|
*,
|
||||||
|
busy_timeout_ms: int | None = 30_000,
|
||||||
|
enable_wal: bool = False,
|
||||||
|
) -> None:
|
||||||
|
if engine.dialect.name != "sqlite" or engine.dialect.driver != "aiosqlite":
|
||||||
|
raise ValueError("Expected a sqlite+aiosqlite engine")
|
||||||
|
if busy_timeout_ms is not None and busy_timeout_ms < 0:
|
||||||
|
raise ValueError("busy_timeout_ms must be non-negative")
|
||||||
|
|
||||||
|
@event.listens_for(engine.sync_engine, "connect")
|
||||||
|
def configure_connection(dbapi_connection: DBAPIConnection, _: object) -> None:
|
||||||
|
dbapi_connection.isolation_level = None
|
||||||
|
cursor = dbapi_connection.cursor()
|
||||||
|
try:
|
||||||
|
cursor.execute("PRAGMA foreign_keys=ON")
|
||||||
|
if busy_timeout_ms is not None:
|
||||||
|
cursor.execute(f"PRAGMA busy_timeout={busy_timeout_ms}")
|
||||||
|
if enable_wal:
|
||||||
|
cursor.execute("PRAGMA journal_mode=WAL")
|
||||||
|
journal_mode = cursor.fetchone()
|
||||||
|
if journal_mode is None or journal_mode[0].lower() != "wal":
|
||||||
|
raise RuntimeError("SQLite could not enable WAL mode")
|
||||||
|
finally:
|
||||||
|
cursor.close()
|
||||||
|
|
||||||
|
@event.listens_for(engine.sync_engine, "begin")
|
||||||
|
def begin_transaction(connection: Connection) -> None:
|
||||||
|
connection.exec_driver_sql("BEGIN")
|
||||||
|
```
|
||||||
|
|
||||||
|
The `connect` listener receives the adapted synchronous DBAPI connection exposed by `engine.sync_engine`; event callbacks themselves are synchronous even though application queries use the async engine. Setting `isolation_level=None` and adding the `begin` listener are one transaction-control strategy and must remain paired. Do not combine this pair with SQLAlchemy's driver-level `AUTOCOMMIT` isolation mode.
|
||||||
|
|
||||||
|
The default above enables foreign keys and modern transaction boundaries for file and in-memory databases. Enable WAL only for a file-backed database after deciding that its read/write concurrency model is appropriate. Treat `30_000` as an example policy, not a universal default; `connect_args={"timeout": 30.0}` at engine construction is another way to configure the underlying SQLite lock timeout.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Pooling Defaults and Tuning
|
## Pooling Defaults and Tuning
|
||||||
|
|
||||||
Default behavior is usually correct first:
|
Default behavior is usually correct first:
|
||||||
@@ -123,14 +202,36 @@ When to switch pool strategy:
|
|||||||
- `NullPool` if you explicitly need no pooling (special environments, some tests, or strict cross-loop constraints).
|
- `NullPool` if you explicitly need no pooling (special environments, some tests, or strict cross-loop constraints).
|
||||||
- Keep in mind this increases connect/disconnect churn.
|
- Keep in mind this increases connect/disconnect churn.
|
||||||
|
|
||||||
|
### When `StaticPool` Is Appropriate
|
||||||
|
|
||||||
|
Use [`StaticPool`](https://docs.sqlalchemy.org/en/21/core/pooling.html#sqlalchemy.pool.StaticPool) only when every checkout must reuse one DBAPI connection and all database access is serialized. Typical cases are:
|
||||||
|
|
||||||
|
- A serial test suite using a private in-memory SQLite database. The `sqlite+aiosqlite://` URL already selects `StaticPool` automatically, so specifying `poolclass=StaticPool` is normally redundant.
|
||||||
|
- A narrowly scoped SQLite engine that must preserve connection-local state, such as temporary tables, across SQLAlchemy connection or session checkouts.
|
||||||
|
|
||||||
|
When explicit configuration is required:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
|
from sqlalchemy.pool import StaticPool
|
||||||
|
|
||||||
|
engine = create_async_engine(
|
||||||
|
"sqlite+aiosqlite:///./test.db",
|
||||||
|
poolclass=StaticPool,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
`StaticPool` is not a general performance optimization or a way to make SQLite concurrent. All sessions share one underlying connection and its single transaction state, so one session's `COMMIT` or `ROLLBACK` can interfere with another session. Do not use it when several sessions or tasks may access the engine concurrently. For concurrent in-memory work, use a named shared-cache SQLite URL so pooled connections have independent transaction state, or use a temporary file database. See [SQLite test targets](testing.md#sqlite-targets) for those patterns.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Disposal Semantics
|
## Disposal Semantics
|
||||||
|
|
||||||
`engine.dispose()` replaces/disposes the pool, but only checked-in connections are immediately closed.
|
`dispose_engine(database_url)` resolves the cached engine, awaits `engine.dispose()`, and clears the engine cache in a `finally` block. `engine.dispose()` replaces/disposes the pool, but only checked-in connections are immediately closed.
|
||||||
|
|
||||||
Rules:
|
Rules:
|
||||||
- Dispose when the app is shutting down.
|
- Dispose when the app is shutting down.
|
||||||
|
- Clear cached resolution even when disposal raises, so a later lifecycle cannot receive the failed engine object.
|
||||||
- Dispose before reusing an engine across event loops.
|
- Dispose before reusing an engine across event loops.
|
||||||
- In forked child-process initialization, use `engine.dispose(close=False)` (sync API guidance) so child processes do not touch parent-held connections.
|
- In forked child-process initialization, use `engine.dispose(close=False)` (sync API guidance) so child processes do not touch parent-held connections.
|
||||||
|
|
||||||
@@ -151,21 +252,30 @@ This prevents broken socket state and cross-process connection corruption.
|
|||||||
|
|
||||||
## What Not to Do
|
## What Not to Do
|
||||||
|
|
||||||
- Create an engine inside every request dependency.
|
- Create an engine inside each operation or unit of work.
|
||||||
- Create/dispose engines inside repository methods.
|
- Create/dispose engines inside repository methods.
|
||||||
- Call `get_engine()` from repositories instead of injecting their engine or session dependency.
|
- Resolve an engine from repositories instead of injecting a session dependency.
|
||||||
- Keep engine creation as a hidden side effect of import-time module globals.
|
- Keep engine creation as a hidden side effect of import-time module globals.
|
||||||
- Dispose a cached engine without clearing the cache during final teardown.
|
- Keep a session factory alive after its bound engine scope exits.
|
||||||
- Use deprecated FastAPI startup/shutdown events together with lifespan.
|
- Enter overlapping engine scopes for the same cached URL.
|
||||||
|
- Treat `cache_clear()` as per-URL invalidation when it clears every cached engine.
|
||||||
|
- Use `metadata.create_all()` as a substitute for required production migrations.
|
||||||
|
- Install the same SQLite event listeners more than once on one engine.
|
||||||
|
- Enable WAL blindly for in-memory SQLite or treat a busy timeout as a concurrency guarantee.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Engine Design Checklist
|
## Engine Design Checklist
|
||||||
|
|
||||||
- One engine per process per DB URL.
|
- One cached engine per exact database URL during an active lifecycle.
|
||||||
- Engine created by one cached, framework-independent factory.
|
- One owning engine scope per URL, with no overlapping owners.
|
||||||
- Lifespan only retrieves, exposes, disposes, and uncaches the engine.
|
- Cached resolution, optional initialization, disposal, and cache invalidation follow one framework-independent lifecycle.
|
||||||
|
- The composition root enters the database scope once and keeps it open until shutdown.
|
||||||
|
- Session factory created inside, and never outlives, its engine scope.
|
||||||
|
- Model registration occurs before `metadata.create_all()` when initialization is enabled.
|
||||||
- Async driver URL matches backend (`asyncpg` or `aiosqlite`).
|
- Async driver URL matches backend (`asyncpg` or `aiosqlite`).
|
||||||
|
- `aiosqlite` foreign-key and transaction listeners installed once before first use.
|
||||||
|
- WAL enabled only as an explicit policy for a file-backed SQLite database.
|
||||||
- Pooling strategy is explicit for non-default needs.
|
- Pooling strategy is explicit for non-default needs.
|
||||||
- No request-path engine creation.
|
- No feature-path engine creation.
|
||||||
- Tests dispose engines and clear cached state deterministically.
|
- Tests enter the same scope and receive deterministic disposal plus cache cleanup.
|
||||||
|
|||||||
@@ -0,0 +1,192 @@
|
|||||||
|
# FastAPI Database Integration
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
||||||
|
- [FastAPI dependencies with `yield`](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
|
||||||
|
- [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
||||||
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
|
- [`nicegui-db` application lifespan](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/ui/app.py)
|
||||||
|
- [`nicegui-db` database dependencies](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/ui/dependency.py)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Connect the framework-independent database tools to FastAPI:
|
||||||
|
|
||||||
|
- lifespan enters one application-owned `database_scope()`,
|
||||||
|
- application state holds settings and the resulting session factory,
|
||||||
|
- dependencies create one session per request,
|
||||||
|
- `Annotated` aliases make route ownership concise and explicit.
|
||||||
|
|
||||||
|
The underlying resource and transaction rules remain in [engine lifecycle](engine.md), [session management](session.md), and [transaction boundaries](transactions.md).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Lifespan Ownership
|
||||||
|
|
||||||
|
Enter `database_scope()` once for the complete application lifecycle. Store the session factory, not the engine, because request code needs sessions rather than direct pool access:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from fastapi import FastAPI
|
||||||
|
|
||||||
|
from .config import Settings
|
||||||
|
from .config import get_database_url
|
||||||
|
from .db import database_scope
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(settings: Settings, app: FastAPI) -> AsyncGenerator[None]:
|
||||||
|
app.state.settings = settings
|
||||||
|
db_url = get_database_url(settings)
|
||||||
|
|
||||||
|
try:
|
||||||
|
async with database_scope(db_url) as session_factory:
|
||||||
|
app.state.session_factory = session_factory
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
del app.state.settings
|
||||||
|
del app.state.session_factory
|
||||||
|
```
|
||||||
|
|
||||||
|
The application factory binds `settings` to lifespan, for example with `partial(lifespan, settings)`. Lifespan does not construct resources per request. It enters the same framework-independent scope used by scripts, workers, and tests, keeps that scope open while requests are served, and lets it dispose the engine and clear cached engine resolution during shutdown.
|
||||||
|
|
||||||
|
The template's unconditional `del app.state.session_factory` mirrors an expected successful startup. If `database_scope()` raises before assignment, cleanup can raise `AttributeError` and obscure the startup error. A production hardening option is to assign a sentinel before the `try` or delete conditionally; that changes failure behavior and is not part of the exact template mechanics.
|
||||||
|
|
||||||
|
Only store the engine too when application-level code genuinely needs direct Core operations, pool instrumentation, or engine-specific diagnostics. Routes and repositories should normally receive an `AsyncSession`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Session Factory Dependency
|
||||||
|
|
||||||
|
A synchronous dependency retrieves the already-created factory from application state:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from typing import Annotated
|
||||||
|
|
||||||
|
from fastapi import Depends
|
||||||
|
from fastapi import Request
|
||||||
|
|
||||||
|
from .session import SessionFactory
|
||||||
|
|
||||||
|
|
||||||
|
def _get_session_factory(request: Request) -> SessionFactory:
|
||||||
|
return request.app.state.session_factory
|
||||||
|
|
||||||
|
|
||||||
|
type SessionFactoryDep = Annotated[SessionFactory, Depends(_get_session_factory)]
|
||||||
|
```
|
||||||
|
|
||||||
|
`Depends()` does not create or cache a factory here. It only exposes the lifespan-owned object. This function is also the narrow seam that tests can override when they need a different factory.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Request Session Dependencies
|
||||||
|
|
||||||
|
Use a session-only dependency for reads and other request conversations that must not commit implicitly:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
|
||||||
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
|
|
||||||
|
async def _get_session(session_factory: SessionFactoryDep) -> AsyncGenerator[AsyncSession]:
|
||||||
|
async with session_factory() as owned_session:
|
||||||
|
yield owned_session
|
||||||
|
|
||||||
|
|
||||||
|
type SessionDep = Annotated[AsyncSession, Depends(_get_session)]
|
||||||
|
```
|
||||||
|
|
||||||
|
The dependency creates and closes one session per request. Closing rolls back any unfinished autobegun transaction; it does not commit.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Route Usage
|
||||||
|
|
||||||
|
Read route:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@router.get("/items/{item_id}")
|
||||||
|
async def get_item(item_id: int, session: SessionDep) -> Item | None:
|
||||||
|
return await find_item(session, item_id)
|
||||||
|
```
|
||||||
|
|
||||||
|
Write route:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@router.post("/items")
|
||||||
|
async def create_item(payload: ItemCreate, session: SessionDep) -> Item:
|
||||||
|
async with session.begin():
|
||||||
|
return await insert_item(session, payload)
|
||||||
|
```
|
||||||
|
|
||||||
|
The template exposes only `SessionDep`; it does not hide commit behavior in dependency teardown. Choose one visible write convention per application:
|
||||||
|
|
||||||
|
- place `async with session.begin():` around a complete write unit, which commits on success and rolls back on exception; or
|
||||||
|
- call `await session.commit()` explicitly after all writes when the route is the complete unit, as the template's simple UI action does.
|
||||||
|
|
||||||
|
The context-manager form scales better to several statements and makes exception rollback visible. Direct `commit()` is concise but requires the route to preserve the single-commit invariant and handle any recovery needs. Do not combine both conventions in one route. Lower-level data-access functions receive the existing session and remain unaware of FastAPI.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Background Work
|
||||||
|
|
||||||
|
A request session belongs to that request and must not be retained by a background task. Inject or otherwise provide the application session factory, then create a new session inside the task:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def run_background_job(session_factory: SessionFactory) -> None:
|
||||||
|
async with session_factory.begin() as session:
|
||||||
|
await process_pending_items(session)
|
||||||
|
```
|
||||||
|
|
||||||
|
If work must survive application shutdown, it needs an independently owned worker lifecycle rather than the FastAPI lifespan-owned factory.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing and Overrides
|
||||||
|
|
||||||
|
Override the narrow dependency that matches the test objective:
|
||||||
|
|
||||||
|
- Override `_get_session_factory` to preserve production request-session behavior with a test factory.
|
||||||
|
- Override `_get_session` when a test must inject one transaction-scoped session directly.
|
||||||
|
- Verify each lifespan receives a fresh engine and session factory and removes application state during teardown.
|
||||||
|
- Remove overrides during teardown so mutable application state does not leak between tests.
|
||||||
|
|
||||||
|
```python
|
||||||
|
app.dependency_overrides[_get_session] = get_test_session
|
||||||
|
try:
|
||||||
|
yield app
|
||||||
|
finally:
|
||||||
|
app.dependency_overrides.pop(_get_session, None)
|
||||||
|
```
|
||||||
|
|
||||||
|
See [database testing](testing.md) for outer transactions, SAVEPOINT-backed fixtures, and database target selection.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Anti-Patterns
|
||||||
|
|
||||||
|
- Creating an engine or session factory in a request dependency.
|
||||||
|
- Reading settings and constructing database resources from repositories.
|
||||||
|
- Storing one mutable `AsyncSession` on `app.state`.
|
||||||
|
- Sharing a request session with concurrent or background tasks.
|
||||||
|
- Assuming `SessionDep` commits when dependency cleanup runs.
|
||||||
|
- Keeping `app.state.session_factory` after its `database_scope()` exits.
|
||||||
|
- Using deprecated startup and shutdown event handlers alongside lifespan.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Checklist
|
||||||
|
|
||||||
|
- Lifespan enters exactly one `database_scope()` for each application lifecycle.
|
||||||
|
- Application state stores the yielded session factory.
|
||||||
|
- Session dependencies create and close one session per request.
|
||||||
|
- The session dependency owns request session closure but not commit behavior.
|
||||||
|
- Routes use `Annotated` aliases and receive sessions, not engines.
|
||||||
|
- Background tasks create their own sessions from a still-live factory.
|
||||||
|
- Tests override and restore dependencies deterministically.
|
||||||
@@ -8,13 +8,15 @@ Purpose: concept registry for the principles, mechanics, and implementation guid
|
|||||||
|
|
||||||
| Concept | File | Status | Decision Level | Owner | Last Reviewed |
|
| Concept | File | Status | Decision Level | Owner | Last Reviewed |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| Engine lifecycle and ownership | [engine.md](engine.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
| Engine lifecycle and ownership | [engine.md](engine.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||||
| Session factory and scope | [session.md](session.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
| Session factory and scope | [session.md](session.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||||
|
| FastAPI lifespan and dependency injection | [fastapi.md](fastapi.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||||
| Transaction boundaries | [transactions.md](transactions.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
| Transaction boundaries | [transactions.md](transactions.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
||||||
| Implicit ORM I/O under asyncio | [implicit_io.md](implicit_io.md) | adopted | advisory | platform/backend | 2026-06-17 |
|
| Implicit ORM I/O under asyncio | [implicit_io.md](implicit_io.md) | adopted | advisory | platform/backend | 2026-06-17 |
|
||||||
| Observability and resilience | [observability.md](observability.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
| Observability and resilience | [observability.md](observability.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
||||||
| SQLModel modeling and async boundaries | [sqlmodel.md](sqlmodel.md) | adopted | mandatory | platform/backend | 2026-07-26 |
|
| SQLModel modeling and async boundaries | [sqlmodel.md](sqlmodel.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||||
| Basic CRUD repository and functions | [crud.md](crud.md) | adopted | advisory | platform/backend | 2026-07-26 |
|
| Basic CRUD repository and functions | [crud.md](crud.md) | adopted | advisory | platform/backend | 2026-08-06 |
|
||||||
|
| Test database targets and fixture data | [testing.md](testing.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -1,42 +1,46 @@
|
|||||||
# Async SQLAlchemy Session Management
|
# Async SQLAlchemy Session Management
|
||||||
|
|
||||||
!!! info "Primary sources"
|
!!! info "Primary sources"
|
||||||
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
|
||||||
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
|
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
|
||||||
|
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||||
|
- [Python `inspect.signature`](https://docs.python.org/3/library/inspect.html#inspect.signature)
|
||||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
- [SQLAlchemy session basics](https://docs.sqlalchemy.org/en/21/orm/session_basics.html)
|
- [SQLAlchemy session basics](https://docs.sqlalchemy.org/en/21/orm/session_basics.html)
|
||||||
- [FastAPI dependencies with yield](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
|
- [`nicegui-db` session implementation](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/db/session.py)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
Define one canonical session model for FastAPI + SQLAlchemy asyncio:
|
Define one canonical session model for SQLAlchemy asyncio:
|
||||||
|
|
||||||
- configure one shared session factory,
|
- configure a lifespan-owned factory or resolve a URL-keyed cached factory,
|
||||||
- create one AsyncSession per request or per unit-of-work,
|
- create one AsyncSession per task or unit of work,
|
||||||
|
- let callers supply a session when they already own the scope,
|
||||||
- never share one AsyncSession across concurrent tasks.
|
- never share one AsyncSession across concurrent tasks.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Scope and Non-Goals
|
## Scope and Non-Goals
|
||||||
|
|
||||||
- In scope: session factory creation, FastAPI dependency wiring, request/task scoping, transaction demarcation.
|
- In scope: session factory creation, task scoping, and transaction demarcation.
|
||||||
- Out of scope: ORM model design, query optimization strategy, schema migration tooling.
|
- Out of scope: framework dependency wiring, ORM model design, query optimization strategy, and schema migration tooling.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Rules
|
## Rules
|
||||||
|
|
||||||
- Create one cached `async_sessionmaker` per app-owned AsyncEngine.
|
- Create the application `async_sessionmaker` inside `database_scope()` and store it in application state for request dependencies.
|
||||||
- Let repositories resolve the cached maker by database URL.
|
- Use `get_session_factory(db_url)` and `resolve_session_factory()` for standalone decorated operations that do not receive the application factory.
|
||||||
- Use a fresh AsyncSession for each request or explicit unit-of-work.
|
- Use a fresh AsyncSession for each task or explicit unit of work.
|
||||||
- Pass an `AsyncSession` directly to data-access functions.
|
- Let reusable service functions accept `AsyncSession | None` and apply `@with_session` when standalone invocation is useful.
|
||||||
- Borrow a caller-provided session without closing or committing it.
|
- Pass an `AsyncSession` directly when composing several calls under one caller-owned scope.
|
||||||
|
- Borrow a caller-provided session without beginning, closing, committing, or rolling it back.
|
||||||
- Do not share AsyncSession across `asyncio.gather()` or parallel tasks.
|
- Do not share AsyncSession across `asyncio.gather()` or parallel tasks.
|
||||||
- Prefer direct dependency injection over global scoped-session patterns in new code.
|
- Prefer direct dependency injection over global scoped-session patterns in new code.
|
||||||
- Use explicit transaction boundaries (`async with session.begin():`) for writes.
|
- Use explicit transaction boundaries (`async with session.begin():`) for writes.
|
||||||
- When a use case accepts an optional session, borrow only an active caller-owned transaction or own the complete session-and-transaction scope.
|
- Use `db_transaction_scope()` when a standalone operation must own engine, factory, session, and transaction lifetimes together.
|
||||||
|
- Use `begin_nested()` directly and only when partial rollback through a database SAVEPOINT is required.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -46,7 +50,7 @@ A session and a transaction solve related but different problems:
|
|||||||
|
|
||||||
| Concept | Responsibility | Typical lifetime |
|
| Concept | Responsibility | Typical lifetime |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `AsyncSession` | Provides the ORM workspace: executes queries, tracks loaded and changed objects in its identity map, and flushes pending changes. It also coordinates access to a database connection. | One request, task, or explicit unit of work. |
|
| `AsyncSession` | Provides the ORM workspace: executes queries, tracks loaded and changed objects in its identity map, and flushes pending changes. It also coordinates access to a database connection. | One task or explicit unit of work. |
|
||||||
| Transaction | Defines the atomic database boundary: all work inside it commits together on success or rolls back together on failure. | One complete operation that must have a single outcome. |
|
| Transaction | Defines the atomic database boundary: all work inside it commits together on success or rolls back together on failure. | One complete operation that must have a single outcome. |
|
||||||
|
|
||||||
A transaction belongs to a session; it is not an alternative to one. The session is the interface used by application and data-access code, while the transaction determines when that work becomes permanent. A session may coordinate sequential transactions during its lifetime, although short-lived application scopes commonly use one session for one transaction.
|
A transaction belongs to a session; it is not an alternative to one. The session is the interface used by application and data-access code, while the transaction determines when that work becomes permanent. A session may coordinate sequential transactions during its lifetime, although short-lived application scopes commonly use one session for one transaction.
|
||||||
@@ -76,32 +80,68 @@ For most read-only operations, a session context is sufficient. Use an explicit
|
|||||||
|
|
||||||
An `async_sessionmaker[AsyncSession]` is a reusable configuration object and callable session producer. It stores how sessions should be created, including the engine binding and options such as `expire_on_commit=False`. It is not itself a session, connection, or transaction, and calling it does not make a shared global `AsyncSession`.
|
An `async_sessionmaker[AsyncSession]` is a reusable configuration object and callable session producer. It stores how sessions should be created, including the engine binding and options such as `expire_on_commit=False`. It is not itself a session, connection, or transaction, and calling it does not make a shared global `AsyncSession`.
|
||||||
|
|
||||||
Cache it by the application-owned engine so repeated composition calls return the same maker:
|
The template exposes two construction paths with the same session options.
|
||||||
|
|
||||||
|
The application-owned path creates a factory inside the engine lifecycle:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||||
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
|
from .engine import engine_scope
|
||||||
|
|
||||||
|
type SessionFactory = async_sessionmaker[AsyncSession]
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def database_scope(
|
||||||
|
db_url: str,
|
||||||
|
*,
|
||||||
|
auto_flush: bool = True,
|
||||||
|
) -> AsyncGenerator[SessionFactory]:
|
||||||
|
async with engine_scope(db_url) as engine:
|
||||||
|
yield async_sessionmaker(
|
||||||
|
bind=engine,
|
||||||
|
class_=AsyncSession,
|
||||||
|
expire_on_commit=False,
|
||||||
|
autoflush=auto_flush,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
FastAPI lifespan enters this path once and stores the yielded factory on application state. The factory must not outlive the scope because its bound engine is disposed on exit.
|
||||||
|
|
||||||
|
The standalone path caches a factory by URL and `auto_flush` policy:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from functools import cache
|
from functools import cache
|
||||||
|
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
|
||||||
|
|
||||||
from .engine import dispose_engine
|
|
||||||
from .engine import get_engine
|
from .engine import get_engine
|
||||||
|
|
||||||
|
|
||||||
@cache
|
@cache
|
||||||
def get_session_factory(database_url: str) -> async_sessionmaker[AsyncSession]:
|
def get_session_factory(
|
||||||
|
db_url: str,
|
||||||
|
*,
|
||||||
|
auto_flush: bool = True,
|
||||||
|
) -> SessionFactory:
|
||||||
return async_sessionmaker(
|
return async_sessionmaker(
|
||||||
bind=get_engine(database_url),
|
bind=get_engine(db_url),
|
||||||
class_=AsyncSession,
|
class_=AsyncSession,
|
||||||
expire_on_commit=False,
|
expire_on_commit=False,
|
||||||
|
autoflush=auto_flush,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
async def dispose_session_factory(database_url: str) -> None:
|
def resolve_session_factory(settings: Settings | None = None) -> SessionFactory:
|
||||||
get_session_factory.cache_clear()
|
settings = settings or get_settings()
|
||||||
await dispose_engine(database_url)
|
db_url = get_database_url(settings)
|
||||||
|
return get_session_factory(db_url)
|
||||||
```
|
```
|
||||||
|
|
||||||
`functools.cache` caches by argument equality and requires hashable arguments. The database URL is an explicit string key shared with the cached engine factory. The cache retains the returned maker until `get_session_factory.cache_clear()` runs. Cache the synchronous maker function, never an async function and never a produced `AsyncSession`.
|
This path lets framework-independent helpers resolve one stable factory without receiving it through every call. The tradeoff is hidden configuration resolution and a second lifecycle mechanism. `dispose_engine()` clears `get_engine`'s cache but does not clear `get_session_factory`'s cache in the template. A cached factory remains bound to the disposed engine object; SQLAlchemy can create a new pool when that engine is used again, but a later `database_scope()` for the same URL can own a different engine. Treat cached standalone resolution as process-lifetime convenience, avoid repeated application lifecycles in one process, and clear both caches together if the template evolves to support them.
|
||||||
|
|
||||||
Each call to `session_factory()` creates a distinct `AsyncSession`. The caller that invokes the factory owns that session lifetime and must close it, normally with `async with`:
|
Each call to `session_factory()` creates a distinct `AsyncSession`. The caller that invokes the factory owns that session lifetime and must close it, normally with `async with`:
|
||||||
|
|
||||||
@@ -110,212 +150,189 @@ async with session_factory() as session:
|
|||||||
...
|
...
|
||||||
```
|
```
|
||||||
|
|
||||||
The factory can be shared across requests and tasks. Sessions produced by it cannot be shared across concurrent tasks.
|
The factory can be shared across operations and tasks. Sessions produced by it cannot be shared across concurrent tasks.
|
||||||
|
|
||||||
An `async_sessionmaker` has no connection pool or async `dispose()` method of its own. `dispose_session_factory()` means "invalidate the cached maker, then dispose its engine." Clearing the maker first ensures no subsequent composition call can retrieve a maker bound to the engine being shut down.
|
Passing the application factory directly has three useful consequences:
|
||||||
|
|
||||||
Use the helper when shutting down or replacing the database resources:
|
- Lower layers do not resolve settings or global resources.
|
||||||
|
- Tests can inject a test factory directly through `session_scope(session_factory=...)` or FastAPI state.
|
||||||
```python
|
- Transaction ownership remains independent of engine construction.
|
||||||
await dispose_session_factory(database_url)
|
|
||||||
```
|
|
||||||
|
|
||||||
Otherwise, a later call can return a maker that still references the old engine object. This matters in lifespan tests, application restarts within one process, and test suites that replace engines.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Optional Session Ownership
|
## Database and Convenience Scopes
|
||||||
|
|
||||||
A small [`asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager) can make repository methods composable. It borrows an existing session when supplied; otherwise it creates and closes one from a supplied factory:
|
The template provides three framework-independent context managers:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from collections.abc import AsyncIterator
|
from collections.abc import AsyncGenerator
|
||||||
from contextlib import asynccontextmanager
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def db_session_scope(
|
||||||
|
db_url: str | None = None,
|
||||||
|
) -> AsyncGenerator[AsyncSession]:
|
||||||
|
db_url = db_url or resolve_database_url()
|
||||||
|
async with database_scope(db_url) as session_factory, session_factory() as session:
|
||||||
|
yield session
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def db_transaction_scope(
|
||||||
|
db_url: str | None = None,
|
||||||
|
) -> AsyncGenerator[AsyncSession]:
|
||||||
|
db_url = db_url or resolve_database_url()
|
||||||
|
async with database_scope(db_url) as session_factory, session_factory.begin() as session:
|
||||||
|
yield session
|
||||||
|
|
||||||
|
|
||||||
@asynccontextmanager
|
@asynccontextmanager
|
||||||
async def session_scope(
|
async def session_scope(
|
||||||
*,
|
*,
|
||||||
database_url: str,
|
settings: Settings | None = None,
|
||||||
|
session_factory: SessionFactory | None = None,
|
||||||
session: AsyncSession | None = None,
|
session: AsyncSession | None = None,
|
||||||
) -> AsyncIterator[AsyncSession]:
|
) -> AsyncGenerator[AsyncSession]:
|
||||||
if session is not None:
|
if session is not None:
|
||||||
yield session
|
yield session
|
||||||
return
|
return
|
||||||
|
|
||||||
async with get_session_factory(database_url)() as owned_session:
|
session_factory = session_factory or resolve_session_factory(settings=settings)
|
||||||
|
async with session_factory() as owned_session:
|
||||||
yield owned_session
|
yield owned_session
|
||||||
```
|
```
|
||||||
|
|
||||||
The branch is intentionally explicit. Python's [`nullcontext`](https://docs.python.org/3/library/contextlib.html#contextlib.nullcontext) can express the same borrow-or-own idea, but the branch keeps ownership and typing obvious.
|
`db_session_scope()` owns a complete temporary database lifecycle and a session but does not commit. `db_transaction_scope()` owns the same resources plus a root transaction that commits on successful exit and rolls back on exception. Both initialize the schema by default because `database_scope()` enters `engine_scope()` with its default `initialize=True`. They are appropriate for scripts, commands, and isolated operations, not per-request use inside an already-running application.
|
||||||
|
|
||||||
This helper manages session lifetime only:
|
`session_scope()` is the borrow-or-create helper. Its precedence is supplied session, supplied factory, then settings-based cached factory resolution. A supplied session remains entirely caller-owned; the helper does not require an active transaction and does not begin, commit, roll back, or close it. An owned session is closed on exit, and unfinished autobegun work rolls back.
|
||||||
|
|
||||||
- It does not close, commit, or roll back a supplied session; the caller owns it.
|
Passing `session=None` is the same as omitting the session for `session_scope()` and therefore creates a session. This differs from `with_session`, which tests whether the argument name was bound rather than whether its value is non-null.
|
||||||
- It closes a session that it creates. Closing releases resources and rolls back an unfinished transaction; it does not commit.
|
|
||||||
- It does not start a transaction. Put `session.begin()` at the use-case boundary.
|
|
||||||
- A supplied session wins; the cached factory is not resolved.
|
|
||||||
- Otherwise, `database_url` selects the cached factory returned by `get_session_factory()`.
|
|
||||||
|
|
||||||
Do not turn this into an implicit unit-of-work helper that sometimes commits. Whether work joins an existing transaction or creates a new one must remain visible to the caller.
|
## Signature-Aware Session Injection
|
||||||
|
|
||||||
|
`with_session` allows one async function to support standalone calls and explicit composition:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import Awaitable
|
||||||
|
from collections.abc import Callable
|
||||||
|
from functools import wraps
|
||||||
|
from inspect import signature
|
||||||
|
|
||||||
|
|
||||||
|
def with_session[**P, R](
|
||||||
|
func: Callable[P, Awaitable[R]],
|
||||||
|
) -> Callable[P, Awaitable[R]]:
|
||||||
|
sig = signature(func)
|
||||||
|
|
||||||
|
@wraps(func)
|
||||||
|
async def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
|
||||||
|
bound = sig.bind_partial(*args, **kwargs)
|
||||||
|
|
||||||
|
if "session" in bound.arguments:
|
||||||
|
return await func(*args, **kwargs)
|
||||||
|
|
||||||
|
async with resolve_session_factory()() as session:
|
||||||
|
bound.arguments["session"] = session
|
||||||
|
return await func(*bound.args, **bound.kwargs)
|
||||||
|
|
||||||
|
return wrapper
|
||||||
|
```
|
||||||
|
|
||||||
|
The function must be async and expose a parameter named exactly `session`. The decorator preserves metadata with `wraps()`, binds positional and keyword arguments through the original signature, and injects a fresh session only when the caller omitted that argument.
|
||||||
|
|
||||||
|
The distinction between omitted and explicit `None` is deliberate in the implementation:
|
||||||
|
|
||||||
|
- `await operation()` injects and owns a session.
|
||||||
|
- `await operation(session=existing_session)` borrows the caller's session.
|
||||||
|
- `await operation(None)` or `await operation(session=None)` forwards `None` without injection.
|
||||||
|
|
||||||
|
The decorated function therefore types the parameter as `AsyncSession | None = None` but should assert or guard after decoration. Explicit `None` is not a request for injection. This preserves ordinary Python call binding, but it means wrappers or callers must omit the argument instead of forwarding a nullable value.
|
||||||
|
|
||||||
|
`with_session` owns session lifetime only. It does not begin or commit a transaction, so it is naturally suited to reads. Decorated writes must either manage a visible transaction or be called with a session from `db_transaction_scope()` or another caller-owned transaction. Prefer explicit factory or session injection when lifecycle transparency and test substitution matter more than call-site convenience.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Optional Transaction Ownership
|
## Function and Service Boundaries
|
||||||
|
|
||||||
Use a separate context manager when a service or use-case function must support both a caller-owned transaction and a standalone transaction. A supplied session must already be inside a transaction; otherwise the helper creates a session and transaction together with `async_sessionmaker.begin()`:
|
Template service functions support both standalone and composed use by combining `@with_session` with an optional parameter:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
@asynccontextmanager
|
from sqlmodel import func
|
||||||
async def transaction_scope(
|
from sqlmodel import select
|
||||||
*,
|
|
||||||
database_url: str,
|
|
||||||
session: AsyncSession | None = None,
|
|
||||||
) -> AsyncIterator[AsyncSession]:
|
|
||||||
if session is not None:
|
|
||||||
if not session.in_transaction():
|
|
||||||
raise RuntimeError("A supplied session must have an active transaction")
|
|
||||||
yield session
|
|
||||||
return
|
|
||||||
|
|
||||||
session_factory = get_session_factory(database_url)
|
|
||||||
async with session_factory.begin() as owned_session:
|
@with_session
|
||||||
yield owned_session
|
async def count_items(session: AsyncSession | None = None) -> int:
|
||||||
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
|
result = await session.exec(select(func.count()).select_from(Item))
|
||||||
|
return result.one()
|
||||||
```
|
```
|
||||||
|
|
||||||
Here, `begin()` is intentionally called on the [`async_sessionmaker`](https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin), not on an existing `AsyncSession`. The related APIs have different ownership semantics:
|
The standalone call injects and closes a session:
|
||||||
|
|
||||||
- `session_factory()` creates a session whose lifetime the surrounding code must manage; it does not commit automatically.
|
|
||||||
- `session_factory.begin()` creates a new session and transaction together, commits on successful exit or rolls back on exceptional exit, and then closes the session.
|
|
||||||
- `session.begin()` manages a transaction on an existing session but does not own or close that session.
|
|
||||||
|
|
||||||
The factory form is equivalent in ownership terms to creating a session and then entering that session's transaction:
|
|
||||||
|
|
||||||
```python
|
```python
|
||||||
async with session_factory() as owned_session:
|
count = await count_items()
|
||||||
async with owned_session.begin():
|
|
||||||
yield owned_session
|
|
||||||
```
|
```
|
||||||
|
|
||||||
This helper makes transaction ownership follow the same explicit borrow-or-own mechanics as session ownership:
|
A larger use case passes one caller-owned session through several decorated functions:
|
||||||
|
|
||||||
- A supplied session and its active transaction remain caller-owned. The helper does not commit, roll back, or close them.
|
|
||||||
- Without a supplied session, the helper owns the session and transaction. Successful exit commits; exceptional exit rolls back; either path closes the session.
|
|
||||||
- Use this helper only at a complete operation, service, or use-case boundary. A public CRUD function or repository method may be such a boundary when its optional-session contract explicitly states that omitting the session owns and commits one transaction. Never use it inside a lower-level session-required helper.
|
|
||||||
- Do not silently begin a transaction on a supplied session. That would make commit ownership depend on hidden helper behavior.
|
|
||||||
|
|
||||||
Callers that supply a session make their ownership visible with an outer transaction:
|
|
||||||
|
|
||||||
```python
|
```python
|
||||||
async with session_factory() as session:
|
async with session_factory.begin() as session:
|
||||||
async with session.begin():
|
count = await count_items(session)
|
||||||
await run_use_case(..., session=session)
|
await create_item(payload, session=session)
|
||||||
```
|
```
|
||||||
|
|
||||||
Standalone callers omit the session and let the use case own the complete unit of work:
|
The decorator sees the bound `session` argument and leaves all ownership with the caller. It never creates a SAVEPOINT or nested transaction.
|
||||||
|
|
||||||
|
For low-level helpers that should never resolve settings, require a non-optional session and leave them undecorated. Application service objects may store the immutable session factory, but they must not store a mutable session:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
await run_use_case(...)
|
class ItemService:
|
||||||
|
def __init__(self, session_factory: SessionFactory) -> None:
|
||||||
|
self.session_factory = session_factory
|
||||||
|
|
||||||
|
async def find(self, item_id: int) -> Item | None:
|
||||||
|
async with self.session_factory() as session:
|
||||||
|
return await find_item(session, item_id)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Code that already owns a transaction should call the session-required function directly. Repositories should normally remain in that session-required layer; the service or use-case boundary owns standalone session creation. This avoids optional-session APIs spreading into every data-access function.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Repository and Function Boundaries
|
## SAVEPOINTs and Partial Failure
|
||||||
|
|
||||||
Pass the database URL to repository constructors. The repository stores repeatable database configuration, not mutable session state, and `session_scope()` resolves the cached factory when a standalone operation needs a session:
|
Use [`begin_nested()`](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#using-savepoint) only when failure inside one portion of an operation should roll back that portion while preserving the outer transaction:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from sqlalchemy import select
|
async with db_transaction_scope() as session:
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession
|
order = await insert_order(session, payload)
|
||||||
|
|
||||||
|
try:
|
||||||
|
async with session.begin_nested():
|
||||||
|
await apply_optional_discount(session, order)
|
||||||
|
except DiscountError:
|
||||||
|
pass
|
||||||
|
|
||||||
async def find_item(session: AsyncSession, item_id: int) -> Item | None:
|
await reserve_inventory(session, order)
|
||||||
statement = select(Item).where(Item.id == item_id)
|
|
||||||
return await session.scalar(statement)
|
|
||||||
|
|
||||||
|
|
||||||
class ItemRepository:
|
|
||||||
def __init__(self, database_url: str) -> None:
|
|
||||||
self.database_url = database_url
|
|
||||||
|
|
||||||
async def find(
|
|
||||||
self,
|
|
||||||
item_id: int,
|
|
||||||
*,
|
|
||||||
session: AsyncSession | None = None,
|
|
||||||
) -> Item | None:
|
|
||||||
async with session_scope(
|
|
||||||
database_url=self.database_url,
|
|
||||||
session=session,
|
|
||||||
) as active_session:
|
|
||||||
return await find_item(active_session, item_id)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
This split gives each layer one job:
|
Important SAVEPOINT semantics:
|
||||||
|
|
||||||
- The repository object identifies its database configuration and creates a session only for a standalone call.
|
- `begin_nested()` starts a root transaction if one is not already active, so call it inside a visible outer transaction when that ownership matters.
|
||||||
- Standalone calls reuse the cached factory selected by database URL.
|
- Entering `begin_nested()` unconditionally flushes pending session state, regardless of the `autoflush` setting.
|
||||||
- A caller can pass a session to join an existing unit of work; the repository borrows it.
|
- Successful exit releases the SAVEPOINT; it does not commit the outer transaction.
|
||||||
- The access function owns only the query and requires an existing `AsyncSession`.
|
- Exceptional exit rolls back to the SAVEPOINT and leaves the outer transaction active.
|
||||||
- Application wiring supplies the production factory.
|
- In SQLAlchemy 2.x, `session.commit()` commits the outermost transaction. Never call it to release a SAVEPOINT; let the nested context manager manage its transaction handle.
|
||||||
- Tests can use a test database URL or call `find_item()` with a transaction-scoped test session.
|
|
||||||
|
|
||||||
When several repository operations must share one transaction, pass the same session through each call. Put the transaction at the use-case boundary:
|
Do not create a SAVEPOINT merely because one service calls another. SAVEPOINTs add database work and alter flush and error-recovery behavior. Use them only for explicit partial-failure requirements such as skipping one conflicting row while retaining the rest of a batch.
|
||||||
|
|
||||||
```python
|
|
||||||
async with session_factory() as session:
|
|
||||||
async with session.begin():
|
|
||||||
item = await repository.find(item_id, session=session)
|
|
||||||
await update_item(session, item, changes)
|
|
||||||
```
|
|
||||||
|
|
||||||
This preserves atomicity without making repository objects hold mutable `AsyncSession` instances across calls.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Canonical FastAPI Dependency Pattern
|
## Framework Integration
|
||||||
|
|
||||||
```python
|
Keep framework adapters outside these session primitives. See [FastAPI database integration](fastapi.md) for lifespan ownership, `Annotated` dependency aliases, and read-versus-write request sessions.
|
||||||
from collections.abc import AsyncIterator
|
|
||||||
|
|
||||||
from fastapi import Depends
|
|
||||||
from fastapi import Request
|
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession
|
|
||||||
from sqlalchemy.ext.asyncio import async_sessionmaker
|
|
||||||
|
|
||||||
|
|
||||||
type SessionFactory = async_sessionmaker[AsyncSession]
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_session_factory(request: Request) -> SessionFactory:
|
|
||||||
return get_session_factory(request.app.state.settings.database_url)
|
|
||||||
|
|
||||||
|
|
||||||
async def get_db_session(
|
|
||||||
session_factory: SessionFactory = Depends(resolve_session_factory),
|
|
||||||
) -> AsyncIterator[AsyncSession]:
|
|
||||||
async with session_factory() as session:
|
|
||||||
yield session
|
|
||||||
```
|
|
||||||
|
|
||||||
Route usage:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from fastapi import APIRouter, Depends
|
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession
|
|
||||||
|
|
||||||
from .session import get_db_session
|
|
||||||
|
|
||||||
router = APIRouter()
|
|
||||||
|
|
||||||
|
|
||||||
@router.post("/items")
|
|
||||||
async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
|
|
||||||
async with session.begin():
|
|
||||||
# write operations here
|
|
||||||
...
|
|
||||||
return {"status": "ok"}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -327,9 +344,9 @@ async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
|
|||||||
|
|
||||||
## SQLModel Alignment
|
## SQLModel Alignment
|
||||||
|
|
||||||
- Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per request/unit-of-work.
|
- Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per task or unit of work.
|
||||||
- SQLModel does not replace SQLAlchemy async lifecycle primitives; it provides model declaration, validation, and typing ergonomics on top of them.
|
- SQLModel does not replace SQLAlchemy async lifecycle primitives; it provides model declaration, validation, and typing ergonomics on top of them.
|
||||||
- Do not mix ad hoc session construction with the canonical async dependency.
|
- Do not mix ad hoc session construction with the canonical session factory.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -343,41 +360,45 @@ async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
|
|||||||
|
|
||||||
## Anti-Patterns
|
## Anti-Patterns
|
||||||
|
|
||||||
- A singleton/global AsyncSession reused across requests.
|
- A singleton/global AsyncSession reused across tasks or operations.
|
||||||
- Sharing one AsyncSession across parallel tasks.
|
- Sharing one AsyncSession across parallel tasks.
|
||||||
- Passing an application-global AsyncSession to a repository constructor.
|
- Passing an application-global AsyncSession to a repository constructor.
|
||||||
- Caching an `AsyncSession` instead of caching `async_sessionmaker`.
|
- Creating a new `async_sessionmaker` in each operation.
|
||||||
- Leaving a cached maker pointing at a disposed or replaced engine.
|
- Retaining a session factory after its bound engine scope exits.
|
||||||
- Calling the session factory inside low-level access functions such as `find_item()`.
|
- Using cached standalone factory resolution when the application factory is already available.
|
||||||
- Hidden session creation in lower access functions with no caller control.
|
- Assuming `with_session` starts or commits a transaction.
|
||||||
|
- Forwarding `session=None` to a decorated function when injection was intended.
|
||||||
- Closing or committing a session supplied by the caller.
|
- Closing or committing a session supplied by the caller.
|
||||||
- Starting a new transaction inside a helper that may receive a session already in a transaction.
|
|
||||||
- Silently starting or committing a transaction on a supplied session.
|
- Silently starting or committing a transaction on a supplied session.
|
||||||
|
- Creating a SAVEPOINT for ordinary nested service calls.
|
||||||
|
- Hiding root transaction, joined transaction, and SAVEPOINT behavior behind one mode-driven `atomic_scope()` helper.
|
||||||
|
- Calling `session.commit()` inside a SAVEPOINT scope.
|
||||||
- Mixing commit/rollback ownership across layers without a declared boundary.
|
- Mixing commit/rollback ownership across layers without a declared boundary.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Operational Checks
|
## Operational Checks
|
||||||
|
|
||||||
- Exactly one cached `async_sessionmaker` exists per application engine.
|
- The FastAPI application factory is created inside `database_scope()` and does not outlive its bound engine.
|
||||||
- Session factory caches are cleared before their engines are disposed or replaced.
|
- Cached standalone factories are used only where application-state injection is unavailable.
|
||||||
- Request handlers receive sessions from one canonical dependency.
|
- `session_scope()` precedence is supplied session, supplied factory, then settings-based resolution.
|
||||||
|
- Decorated functions receive injection only when the `session` argument is omitted.
|
||||||
- No code path creates AsyncSession in module import side effects.
|
- No code path creates AsyncSession in module import side effects.
|
||||||
- Background jobs and API handlers each create task-local sessions.
|
- Concurrent jobs and operations each create task-local sessions.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Testing Checks
|
## Testing Checks
|
||||||
|
|
||||||
- Repository constructors accept a test database URL without FastAPI startup.
|
- Service constructors accept a test session factory without framework startup.
|
||||||
- Session-taking access functions accept a transaction-scoped test session directly.
|
- Session-taking access functions accept a transaction-scoped test session directly.
|
||||||
- Optional-session tests verify that borrowed sessions remain open and created sessions close.
|
- `session_scope()` tests cover supplied-session, supplied-factory, and settings-resolution precedence.
|
||||||
- Optional-session tests verify that neither path commits implicitly.
|
- `db_transaction_scope()` tests verify commit on success, rollback on failure, session closure, engine disposal, and cache cleanup.
|
||||||
- Optional-transaction tests verify supplied sessions require an active transaction and remain caller-owned.
|
- `with_session` tests cover omitted, positional, keyword, and explicit-`None` session arguments.
|
||||||
- Optional-transaction tests verify owned transactions commit on success, roll back on failure, and close their sessions.
|
- Composition tests verify decorated service calls borrow one caller-owned session without committing it.
|
||||||
- Cache tests clear `get_session_factory` before and after replacing engines.
|
- SAVEPOINT tests verify local rollback preserves the outer transaction and successful exit does not commit it.
|
||||||
- Dependency override exists for the FastAPI session factory.
|
- Tests that depend on SAVEPOINT timing account for `begin_nested()` flushing pending state on entry.
|
||||||
- Rollback behavior is verified for failed write units.
|
- Rollback behavior is verified for failed write units.
|
||||||
- Parallel-task tests verify no shared AsyncSession instances.
|
- Parallel-task tests verify no shared AsyncSession instances.
|
||||||
- Lifespan tests confirm session factory is initialized and teardown-safe.
|
- Lifecycle tests confirm schema initialization, factory availability, deterministic teardown, and expected cache behavior.
|
||||||
|
|
||||||
|
|||||||
@@ -12,7 +12,7 @@
|
|||||||
- Status: adopted
|
- Status: adopted
|
||||||
- Decision level: mandatory
|
- Decision level: mandatory
|
||||||
- Applies to: api-runtime, workers, tests
|
- Applies to: api-runtime, workers, tests
|
||||||
- Last reviewed: 2026-07-26
|
- Last reviewed: 2026-08-06
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -34,7 +34,7 @@ SQLModel is designed for FastAPI, built on Pydantic and SQLAlchemy, and intended
|
|||||||
## Rules
|
## Rules
|
||||||
|
|
||||||
- Default to SQLModel for new table models and API data models.
|
- Default to SQLModel for new table models and API data models.
|
||||||
- Keep SQLAlchemy async primitives as the runtime base: `create_async_engine`, `async_sessionmaker`, and `AsyncSession`.
|
- Keep SQLAlchemy engine and factory primitives as the runtime base: `create_async_engine` and `async_sessionmaker`. For SQLModel applications, use SQLModel's `AsyncSession` wrapper so its typed `exec()` API remains available.
|
||||||
- Keep transaction and session ownership policies identical whether models are SQLAlchemy Declarative or SQLModel.
|
- Keep transaction and session ownership policies identical whether models are SQLAlchemy Declarative or SQLModel.
|
||||||
- Use SQLModel inheritance to share validated fields while keeping table, create, update, and public contracts distinct where their semantics differ.
|
- Use SQLModel inheritance to share validated fields while keeping table, create, update, and public contracts distinct where their semantics differ.
|
||||||
- Use SQLAlchemy declarative models only for a concrete unsupported mapping or third-party constraint; document the reason.
|
- Use SQLAlchemy declarative models only for a concrete unsupported mapping or third-party constraint; document the reason.
|
||||||
@@ -72,17 +72,14 @@ class UserRead(UserBase):
|
|||||||
### Pattern B: Keep SQLModel models with the async runtime
|
### Pattern B: Keep SQLModel models with the async runtime
|
||||||
|
|
||||||
```python
|
```python
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
|
||||||
from sqlmodel import select
|
from sqlmodel import select
|
||||||
|
|
||||||
engine = create_async_engine(settings.database_url, pool_pre_ping=True)
|
async with database_scope(settings.database_url) as session_factory:
|
||||||
session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
async with session_factory() as session:
|
||||||
|
users = (await session.exec(select(User))).all()
|
||||||
async with session_factory() as session:
|
|
||||||
users = (await session.scalars(select(User))).all()
|
|
||||||
```
|
```
|
||||||
|
|
||||||
`sqlmodel.select()` keeps SQLModel's typing-oriented statement construction, while `AsyncSession.scalars()` and the surrounding lifecycle come from SQLAlchemy.
|
`database_scope()` enters the cached engine lifecycle, initializes registered SQLModel metadata by default, and yields the application session factory while SQLModel supplies the model and statement layer. `sqlmodel.select()` keeps SQLModel's typing-oriented statement construction, and SQLModel's `AsyncSession` adds typed `exec()` results while retaining SQLAlchemy's async lifecycle and transaction behavior. Import `AsyncSession` from `sqlmodel.ext.asyncio.session` when working with SQLModel models; use SQLAlchemy's `AsyncSession` only when the code intentionally has no SQLModel dependency.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,172 @@
|
|||||||
|
# Testing Database Targets and Data
|
||||||
|
|
||||||
|
Use the same engine and session primitives in production and tests. Tests select a different URL and, when transaction isolation is required, bind a test session factory to one test-owned connection and outer transaction. They do not replace repositories, services, or SQLAlchemy mechanics with mocks.
|
||||||
|
|
||||||
|
## Decision Table
|
||||||
|
|
||||||
|
| Test need | Database target | Isolation approach | What it proves |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Fast, serial application tests | `sqlite+aiosqlite://` | Per-test engine or connection-bound session factory over an outer transaction | ORM mappings and ordinary application behavior |
|
||||||
|
| Async code using multiple simultaneous sessions | Named SQLite shared-cache URL or temporary SQLite file | Per-test schema or cleanup strategy | Concurrent-session behavior without a database server |
|
||||||
|
| PostgreSQL-specific behavior | Dedicated PostgreSQL test database | Per-test outer transaction and SAVEPOINT | SQL, constraints, types, locking, and migrations that SQLite cannot represent |
|
||||||
|
|
||||||
|
SQLite is a useful fast target, not a drop-in PostgreSQL substitute. Keep a small PostgreSQL integration suite for PostgreSQL-specific queries, extensions, row locking, JSON semantics, collations, isolation, and migration validation.
|
||||||
|
|
||||||
|
## Shared Construction Primitives
|
||||||
|
|
||||||
|
Make the application factory accept a database URL or settings object. Production, workers, and ordinary integration tests enter the same [`database_scope()`](session.md#database-and-convenience-scopes). Tests enter the lower-level [`engine_scope()`](engine.md#owning-engine-scope) only when they need direct engine or connection ownership for schema setup, an outer transaction, or engine-specific assertions:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
|
||||||
|
import pytest_asyncio
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
|
||||||
|
from .engine import engine_scope
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture(scope="session", loop_scope="session")
|
||||||
|
async def test_engine(database_url: str) -> AsyncGenerator[AsyncEngine]:
|
||||||
|
async with engine_scope(database_url) as engine:
|
||||||
|
yield engine
|
||||||
|
```
|
||||||
|
|
||||||
|
Production passes its `postgresql+asyncpg://...` URL to `database_scope()`. A local SQLite run passes `sqlite+aiosqlite:///./app.db`. Tests pass a dedicated test URL to `database_scope()` or `engine_scope()` and receive schema initialization, deterministic disposal, and engine-cache cleanup when the context exits. Do not create an engine during module import: that makes it easy for tests to retain the production URL before an override is applied.
|
||||||
|
|
||||||
|
Use migrations to provision an integration database when migrations are part of the release contract. `metadata.create_all()` is appropriate for focused ORM tests only when it accurately represents the schema under test. Import all table models before creating metadata; [SQLModel documents that model-registration order matters](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/#import-table-models).
|
||||||
|
|
||||||
|
## Transactional Async Fixture
|
||||||
|
|
||||||
|
For tests that exercise code which commits, start an outer transaction on one test connection. Bind a test `SessionFactory` to that connection with `join_transaction_mode="create_savepoint"`. SQLAlchemy documents this as its test-suite pattern: sessions created by the factory resolve their commits through SAVEPOINTs while fixture teardown rolls back the outer transaction.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
|
||||||
|
import pytest_asyncio
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||||
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
|
from .session import SessionFactory
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture(scope="function", loop_scope="session")
|
||||||
|
async def session_factory(test_engine: AsyncEngine) -> AsyncGenerator[SessionFactory]:
|
||||||
|
async with test_engine.connect() as connection:
|
||||||
|
transaction = await connection.begin()
|
||||||
|
factory = async_sessionmaker(
|
||||||
|
bind=connection,
|
||||||
|
class_=AsyncSession,
|
||||||
|
expire_on_commit=False,
|
||||||
|
join_transaction_mode="create_savepoint",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
yield factory
|
||||||
|
finally:
|
||||||
|
await transaction.rollback()
|
||||||
|
```
|
||||||
|
|
||||||
|
Each factory call still creates a distinct `AsyncSession`, matching [session factory mechanics](session.md#session-factory-mechanics). The factory belongs to the fixture's engine and outer transaction and must not escape either scope.
|
||||||
|
|
||||||
|
For service tests that pass a caller-owned session into decorated or undecorated service functions, derive that session from the same factory:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@pytest_asyncio.fixture(scope="function", loop_scope="session")
|
||||||
|
async def session(session_factory: SessionFactory) -> AsyncGenerator[AsyncSession]:
|
||||||
|
async with session_factory() as test_session:
|
||||||
|
await test_session.begin()
|
||||||
|
yield test_session
|
||||||
|
```
|
||||||
|
|
||||||
|
The explicit `begin()` gives test code one visible transaction from the start. Session closure rolls back unfinished work; the outer connection transaction remains the final isolation boundary even if application code commits its SAVEPOINT.
|
||||||
|
|
||||||
|
For FastAPI request tests, override `_get_session_factory` so the production `SessionDep` retains its session-creation and cleanup behavior while receiving the test-bound factory. Always remove the override after the test because [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/) are stored in a mutable application-level dictionary.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import Generator
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi import FastAPI
|
||||||
|
|
||||||
|
from .fastapi import _get_session_factory
|
||||||
|
from .session import SessionFactory
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def app_with_test_database(app: FastAPI, session_factory: SessionFactory) -> Generator[FastAPI]:
|
||||||
|
def get_test_session_factory() -> SessionFactory:
|
||||||
|
return session_factory
|
||||||
|
|
||||||
|
app.dependency_overrides[_get_session_factory] = get_test_session_factory
|
||||||
|
try:
|
||||||
|
yield app
|
||||||
|
finally:
|
||||||
|
app.dependency_overrides.pop(_get_session_factory, None)
|
||||||
|
```
|
||||||
|
|
||||||
|
Construct `app` with test settings before lifespan starts so startup cannot resolve the production URL. The override changes request session creation; it does not prevent lifespan from entering its configured `database_scope()`.
|
||||||
|
|
||||||
|
The connection-bound factory is deliberately serial even though it creates distinct sessions: those sessions still share one connection and outer transaction. A test that verifies concurrently active sessions must use independent connections and a database target that supports them.
|
||||||
|
|
||||||
|
## SQLite Targets
|
||||||
|
|
||||||
|
### Serial in-memory tests
|
||||||
|
|
||||||
|
Use `sqlite+aiosqlite://` for a fresh in-memory database when the test runs all database work serially. SQLAlchemy's `aiosqlite` dialect uses a single-connection `StaticPool` for this target, so all sessions share one SQLite transaction state. One session's rollback can discard another session's uncommitted work.
|
||||||
|
|
||||||
|
`engine_scope()` imports the model package and creates the schema by default, then disposes the engine and clears cached resolution deterministically:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
|
||||||
|
import pytest_asyncio
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
from .engine import engine_scope
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture(scope="session", loop_scope="session")
|
||||||
|
async def test_engine() -> AsyncGenerator[AsyncEngine]:
|
||||||
|
async with engine_scope("sqlite+aiosqlite://") as engine:
|
||||||
|
yield engine
|
||||||
|
```
|
||||||
|
|
||||||
|
### Concurrent in-memory tests
|
||||||
|
|
||||||
|
Do not use the default `:memory:` target for tests that have multiple active sessions or tasks. Use a named shared-cache database instead, with a name unique to the test process:
|
||||||
|
|
||||||
|
```text
|
||||||
|
sqlite+aiosqlite:///file:test-suite?mode=memory&cache=shared&uri=true
|
||||||
|
```
|
||||||
|
|
||||||
|
This lets connections share the same in-memory database while retaining independent transaction state. A temporary file URL such as `sqlite+aiosqlite:////tmp/test.db` is often simpler when test isolation or cleanup tooling already manages files.
|
||||||
|
|
||||||
|
For both SQLite forms, enable and test the constraints your application depends on. SQLite foreign-key enforcement is disabled by default, and its transaction behavior has driver-specific differences. Keep PostgreSQL integration coverage for behavior that SQLite cannot faithfully model.
|
||||||
|
|
||||||
|
## Test Data Practices
|
||||||
|
|
||||||
|
- Build only the data a test needs, through named factory functions or pytest fixtures rather than a large global seed.
|
||||||
|
- Give each fixture a domain meaning, such as `active_account`, `expired_subscription`, or `admin_user`; avoid opaque rows with unexplained defaults.
|
||||||
|
- Set values relevant to the assertion explicitly, including timestamps, permissions, statuses, and unique identifiers. Use fixed clocks or injected clock values instead of the wall clock.
|
||||||
|
- Construct object graphs through relationships, then `await session.flush()` before reading generated identifiers or passing foreign keys onward. `flush()` exercises database constraints without ending the test transaction.
|
||||||
|
- Seed prerequisite data before creating a client request. Let the endpoint own the mutation being asserted; do not pre-insert the row that the endpoint is supposed to create.
|
||||||
|
- Use `commit()` in fixture setup only when the test specifically needs to prove post-commit behavior. With the transactional fixture, this remains isolated through the outer rollback.
|
||||||
|
- Keep shared reference data immutable and explicit. If it must be reused for performance, load it once into a dedicated test database and reset all mutable tables between tests; never depend on test order.
|
||||||
|
- Include both valid and constraint-breaking graphs where a behavior depends on foreign keys, uniqueness, nullability, or cascading deletes. SQLite-only tests should not be the sole evidence for PostgreSQL constraints.
|
||||||
|
|
||||||
|
## Completion Checks
|
||||||
|
|
||||||
|
- A test run cannot reach the production URL; production credentials are absent from the test environment.
|
||||||
|
- Production PostgreSQL, local SQLite, and in-memory SQLite all use `database_scope()` unless a test explicitly needs lower-level engine or connection ownership.
|
||||||
|
- Every test or fixture scope owns its override, session factory, connection, transaction, and session cleanup; the session-scoped engine fixture owns disposal and cache cleanup.
|
||||||
|
- Request tests override `_get_session_factory`, preserving production request-session creation and cleanup behavior.
|
||||||
|
- Test data is deterministic, minimal, and expresses the scenario under test.
|
||||||
|
- PostgreSQL integration tests cover every PostgreSQL-specific contract and run against migrations where migrations are shipped.
|
||||||
|
|
||||||
|
## Sources
|
||||||
|
|
||||||
|
- [SQLAlchemy: joining a session into an external transaction](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites)
|
||||||
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
|
- [SQLAlchemy SQLite dialect and async in-memory pooling](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#using-a-memory-database-with-multiple-coroutines)
|
||||||
|
- [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
||||||
|
- [SQLModel testing with FastAPI](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/)
|
||||||
|
- [pytest-asyncio fixtures](https://pytest-asyncio.readthedocs.io/en/stable/how-to-guides/index.html)
|
||||||
@@ -1,23 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: copilot-customization
|
name: copilot-customization
|
||||||
description: 'Plan, create, review, and debug GitHub Copilot and VS Code agent customizations, including instructions, prompt files, skills, custom agents, hooks, MCP servers, and repo-specific personal-mcp skill integration.'
|
description: 'Plan, create, review, and debug GitHub Copilot and VS Code agent customizations, including instructions, prompt files, skills, custom agents, hooks, MCP servers, and repo-specific personal-mcp skill integration.'
|
||||||
x-personal-mcp:
|
|
||||||
id: copilot-customization
|
|
||||||
version: 1.0.0
|
|
||||||
tags:
|
|
||||||
- copilot
|
|
||||||
- vscode
|
|
||||||
- customization
|
|
||||||
- instructions
|
|
||||||
- prompts
|
|
||||||
- agent-skills
|
|
||||||
- custom-agents
|
|
||||||
- hooks
|
|
||||||
- mcp
|
|
||||||
- personal-mcp
|
|
||||||
- skills
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/copilot-customization/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Copilot Customization
|
# Copilot Customization
|
||||||
@@ -78,31 +61,30 @@ Use either:
|
|||||||
Choose one of these patterns:
|
Choose one of these patterns:
|
||||||
|
|
||||||
1. Direct URI strategy:
|
1. Direct URI strategy:
|
||||||
- Reference known resources directly, such as:
|
- Read `skill://<skill-name>/SKILL.md` when the required skill is known.
|
||||||
- `resource://catalog/skills_index`
|
- Read `skill://<skill-name>/_manifest` only when supporting material may be useful.
|
||||||
- `resource://catalog/skills/{skill_id}`
|
- Read selected supporting files at `skill://<skill-name>/<supporting-path>`.
|
||||||
- `resource://skills/<skill-id>/document`
|
|
||||||
- `resource://skills/<skill-id>/references/<ref-id>`
|
|
||||||
2. Discovery-first strategy:
|
2. Discovery-first strategy:
|
||||||
- Start at catalog discovery (`resource://catalog/skills_index`), select the best skill match, then load the skill document and only the minimal references needed.
|
- List resources, compare native main-resource names and descriptions, then load the best matching `SKILL.md`.
|
||||||
|
- In tool-only clients, use only `list_resources` and `read_resource` for the same sequence.
|
||||||
|
|
||||||
### Authoring guidance for shims
|
### Authoring guidance for shims
|
||||||
|
|
||||||
1. Keep shim content short and procedural; avoid copying large guidance blocks from Personal MCP.
|
1. Keep shim content short and procedural; avoid copying large guidance blocks from Personal MCP.
|
||||||
2. State trigger conditions clearly (for example: "when creating a new skill" or "when editing docs contracts").
|
2. State trigger conditions clearly (for example: "when creating a new skill" or "when editing docs contracts").
|
||||||
3. Specify whether to use direct URIs or discovery for that repo's common workflows.
|
3. Specify whether to use a direct native URI or resource listing for that repo's common workflows.
|
||||||
4. Prefer loading only the most relevant skill document first; expand to references only when needed.
|
4. Prefer loading only the most relevant main file first; inspect its manifest only when needed.
|
||||||
5. For stable repeated workflows, use explicit URIs. For broader or ambiguous requests, use discovery-first.
|
5. For stable repeated workflows, use explicit URIs. For broader or ambiguous requests, use discovery-first.
|
||||||
|
|
||||||
### Minimal shim examples
|
### Minimal shim examples
|
||||||
|
|
||||||
Instruction-style shim intent:
|
Instruction-style shim intent:
|
||||||
|
|
||||||
1. "For markdown edits (`applyTo: '**/*.md'`), load `resource://skills/zensical-docs/document` and apply Zensical-native documentation conventions unless they conflict with expected MkDocs compatibility."
|
1. "For markdown edits (`applyTo: '**/*.md'`), load `skill://zensical-docs/SKILL.md` and apply Zensical-native documentation conventions unless they conflict with expected MkDocs compatibility."
|
||||||
|
|
||||||
Prompt-style shim intent:
|
Prompt-style shim intent:
|
||||||
|
|
||||||
1. "For docs authoring tasks, consult `resource://skills/zensical-docs/document`, summarize the relevant authoring constraints, then propose the smallest markdown change for this repository."
|
1. "For docs authoring tasks, consult `skill://zensical-docs/SKILL.md`, summarize the relevant authoring constraints, then propose the smallest markdown change for this repository."
|
||||||
|
|
||||||
### Validation for shim implementation
|
### Validation for shim implementation
|
||||||
|
|
||||||
@@ -130,7 +112,7 @@ Before finishing:
|
|||||||
3. Confirm names match directory names where VS Code requires it.
|
3. Confirm names match directory names where VS Code requires it.
|
||||||
4. Confirm descriptions include the phrases users are likely to ask for.
|
4. Confirm descriptions include the phrases users are likely to ask for.
|
||||||
5. Confirm extra skill resources are linked from `SKILL.md`.
|
5. Confirm extra skill resources are linked from `SKILL.md`.
|
||||||
6. Confirm repo skill metadata exposes the correct `resource://skills/<skill-id>/document` capability.
|
6. Confirm native discovery exposes `skill://<skill-name>/SKILL.md`, `_manifest`, and supporting-file reads.
|
||||||
7. State any remaining ambiguity or user choice, such as personal vs workspace scope.
|
7. State any remaining ambiguity or user choice, such as personal vs workspace scope.
|
||||||
|
|
||||||
## Output Contract
|
## Output Contract
|
||||||
|
|||||||
@@ -1,17 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: fastapi-uv-docker
|
name: fastapi-uv-docker
|
||||||
description: 'Audit and migrate an existing Python project to best practices for a cloud-native ASGI FastAPI app managed with uv and run with uvicorn in Docker. Use when: conforming a project to production standards, setting up src layout, configuring pyproject.toml, writing multi-stage Dockerfiles, wiring lifespan and settings, adding health endpoints, enforcing non-root container user, migrating from requirements.txt to uv.'
|
description: 'Audit and migrate an existing Python project to best practices for a cloud-native ASGI FastAPI app managed with uv and run with uvicorn in Docker. Use when: conforming a project to production standards, setting up src layout, configuring pyproject.toml, writing multi-stage Dockerfiles, wiring lifespan and settings, adding health endpoints, enforcing non-root container user, migrating from requirements.txt to uv.'
|
||||||
x-personal-mcp:
|
|
||||||
id: fastapi-uv-docker
|
|
||||||
version: 1.0.0
|
|
||||||
tags:
|
|
||||||
- fastapi
|
|
||||||
- uv
|
|
||||||
- uvicorn
|
|
||||||
- docker
|
|
||||||
- architecture
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/fastapi-uv-docker/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# FastAPI Project Best Practices
|
# FastAPI Project Best Practices
|
||||||
|
|||||||
@@ -1,17 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: mcp-details
|
name: mcp-details
|
||||||
description: "Reference hub for MCP and FastMCP source documentation links. Use when you need authoritative protocol, SDK, transport, and deployment docs without loading broad implementation guidance."
|
description: "Reference hub for MCP and FastMCP source documentation links. Use when you need authoritative protocol, SDK, transport, and deployment docs without loading broad implementation guidance."
|
||||||
x-personal-mcp:
|
|
||||||
id: mcp-details
|
|
||||||
version: 1.0.0
|
|
||||||
tags:
|
|
||||||
- mcp
|
|
||||||
- model-context-protocol
|
|
||||||
- fastmcp
|
|
||||||
- references
|
|
||||||
- source-docs
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/mcp-details/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# MCP Details
|
# MCP Details
|
||||||
|
|||||||
@@ -1,26 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: nicegui
|
name: nicegui
|
||||||
description: 'Reference hub for NiceGUI and FastAPI application structure, typed configuration, ASGI and Uvicorn startup, UI composition, styling, bindable state, interactions, troubleshooting, testing, and source documentation. Use when planning, implementing, reviewing, deploying, or debugging NiceGUI applications; load only the references relevant to the task.'
|
description: 'Reference hub for NiceGUI and FastAPI application structure, typed configuration, ASGI and Uvicorn startup, UI composition, styling, bindable state, interactions, troubleshooting, testing, and source documentation. Use when planning, implementing, reviewing, deploying, or debugging NiceGUI applications; load only the references relevant to the task.'
|
||||||
x-personal-mcp:
|
|
||||||
id: nicegui
|
|
||||||
version: 2.5.0
|
|
||||||
tags:
|
|
||||||
- nicegui
|
|
||||||
- fastapi
|
|
||||||
- asgi
|
|
||||||
- uvicorn
|
|
||||||
- pydantic-settings
|
|
||||||
- configuration
|
|
||||||
- deployment
|
|
||||||
- ui
|
|
||||||
- architecture
|
|
||||||
- scaffolding
|
|
||||||
- customization
|
|
||||||
- frontend
|
|
||||||
- testing
|
|
||||||
- source-docs
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/nicegui/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# NiceGUI Reference
|
# NiceGUI Reference
|
||||||
|
|||||||
@@ -126,7 +126,7 @@ def get_settings() -> Settings:
|
|||||||
The argument-free [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) provider is appropriate here because both the project entry point and Uvicorn's zero-argument factory need process-lifetime access. Each reload or worker process gets its own settings instance. Do not add override arguments to `get_settings()`; inject a `Settings` instance directly into `create_app()` in tests or alternate composition roots. See the [Pydantic settings implementation guide](../../pydantic-settings/SKILL.md) for source precedence, independent settings boundaries, cache clearing, and runtime reload guidance.
|
The argument-free [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) provider is appropriate here because both the project entry point and Uvicorn's zero-argument factory need process-lifetime access. Each reload or worker process gets its own settings instance. Do not add override arguments to `get_settings()`; inject a `Settings` instance directly into `create_app()` in tests or alternate composition roots. See the [Pydantic settings implementation guide](../../pydantic-settings/SKILL.md) for source precedence, independent settings boundaries, cache clearing, and runtime reload guidance.
|
||||||
|
|
||||||
```python title="src/my_app/main.py"
|
```python title="src/my_app/main.py"
|
||||||
from collections.abc import AsyncIterator
|
from collections.abc import AsyncGenerator
|
||||||
from contextlib import asynccontextmanager
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
import uvicorn
|
import uvicorn
|
||||||
@@ -137,7 +137,7 @@ from my_app.config import Settings, get_settings
|
|||||||
|
|
||||||
|
|
||||||
@asynccontextmanager
|
@asynccontextmanager
|
||||||
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
async def lifespan(app: FastAPI) -> AsyncGenerator[None]:
|
||||||
app.state.ready = True
|
app.state.ready = True
|
||||||
try:
|
try:
|
||||||
yield
|
yield
|
||||||
|
|||||||
@@ -1,22 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: pydantic-settings
|
name: pydantic-settings
|
||||||
description: "Practical guide for implementing typed application configuration with pydantic-settings. Use when designing BaseSettings models, choosing nested or independent settings boundaries, managing settings lifecycles, configuring dotenv or secrets, and customizing source priority safely."
|
description: "Practical guide for implementing typed application configuration with pydantic-settings. Use when designing BaseSettings models, choosing nested or independent settings boundaries, managing settings lifecycles, configuring dotenv or secrets, and customizing source priority safely."
|
||||||
x-personal-mcp:
|
|
||||||
id: pydantic-settings
|
|
||||||
version: 1.1.0
|
|
||||||
tags:
|
|
||||||
- python
|
|
||||||
- pydantic
|
|
||||||
- pydantic-settings
|
|
||||||
- configuration
|
|
||||||
- env-vars
|
|
||||||
- secrets
|
|
||||||
- dotenv
|
|
||||||
- source-priority
|
|
||||||
- caching
|
|
||||||
- lifecycle
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/pydantic-settings/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Pydantic Settings Implementation Guide
|
# Pydantic Settings Implementation Guide
|
||||||
@@ -203,11 +187,125 @@ Use independent `BaseSettings` classes when the objects have genuinely independe
|
|||||||
|
|
||||||
Construct independent objects explicitly at the composition root and inject each dependency. Do not nest one `BaseSettings` class inside another merely to reuse its fields. Extract a shared `BaseModel` schema when models need common structure.
|
Construct independent objects explicitly at the composition root and inject each dependency. Do not nest one `BaseSettings` class inside another merely to reuse its fields. Extract a shared `BaseModel` schema when models need common structure.
|
||||||
|
|
||||||
|
### Alternative Database Backends
|
||||||
|
|
||||||
|
When one application can run against one of several database backends, model the selected backend as a [discriminated union](https://docs.pydantic.dev/latest/concepts/unions/#discriminated-unions). Pydantic validates only the variant selected by `driver`, so required PostgreSQL values do not make a SQLite configuration fail, and vice versa.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from typing import Annotated, Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field, SecretStr
|
||||||
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
|
|
||||||
|
|
||||||
|
class SqliteSettings(BaseModel):
|
||||||
|
driver: Literal["sqlite"] = "sqlite"
|
||||||
|
path: str = "app.db"
|
||||||
|
|
||||||
|
|
||||||
|
class PostgresSettings(BaseModel):
|
||||||
|
driver: Literal["postgres"] = "postgres"
|
||||||
|
host: str
|
||||||
|
port: int = 5432
|
||||||
|
database: str
|
||||||
|
user: str
|
||||||
|
password: SecretStr
|
||||||
|
|
||||||
|
|
||||||
|
DatabaseSettings = Annotated[
|
||||||
|
SqliteSettings | PostgresSettings,
|
||||||
|
Field(discriminator="driver"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
model_config = SettingsConfigDict(
|
||||||
|
env_prefix="APP_",
|
||||||
|
env_nested_delimiter="__",
|
||||||
|
env_file=".env",
|
||||||
|
extra="ignore",
|
||||||
|
frozen=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
database: DatabaseSettings
|
||||||
|
```
|
||||||
|
|
||||||
|
Choose one configuration. A SQLite deployment requires no PostgreSQL variables:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
APP_DATABASE__DRIVER=sqlite
|
||||||
|
APP_DATABASE__PATH=./data/app.db
|
||||||
|
```
|
||||||
|
|
||||||
|
A PostgreSQL deployment requires only the PostgreSQL branch:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
APP_DATABASE__DRIVER=postgres
|
||||||
|
APP_DATABASE__HOST=db.internal
|
||||||
|
APP_DATABASE__PORT=5432
|
||||||
|
APP_DATABASE__DATABASE=app
|
||||||
|
APP_DATABASE__USER=app_user
|
||||||
|
APP_DATABASE__PASSWORD=provided-by-the-runtime
|
||||||
|
```
|
||||||
|
|
||||||
|
After settings validation, select an async SQLAlchemy driver URL. This is a pure configuration step; create the engine, session factory, and sessions in their own lifecycle-managed providers:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from functools import cache
|
||||||
|
|
||||||
|
from sqlalchemy import URL
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
|
|
||||||
|
|
||||||
|
def get_database_url(settings: Settings) -> str:
|
||||||
|
match settings.database:
|
||||||
|
case SqliteSettings(path=path):
|
||||||
|
url = URL.create(
|
||||||
|
drivername="sqlite+aiosqlite",
|
||||||
|
database=path,
|
||||||
|
)
|
||||||
|
case PostgresSettings() as database:
|
||||||
|
url = URL.create(
|
||||||
|
drivername="postgresql+asyncpg",
|
||||||
|
host=database.host,
|
||||||
|
port=database.port,
|
||||||
|
database=database.database,
|
||||||
|
username=database.user,
|
||||||
|
password=database.password.get_secret_value(),
|
||||||
|
)
|
||||||
|
return url.render_as_string(hide_password=False)
|
||||||
|
|
||||||
|
|
||||||
|
@cache
|
||||||
|
def get_engine(database_url: str) -> AsyncEngine:
|
||||||
|
return create_async_engine(database_url, pool_pre_ping=True)
|
||||||
|
|
||||||
|
|
||||||
|
async def dispose_engine(database_url: str) -> None:
|
||||||
|
engine = get_engine(database_url)
|
||||||
|
try:
|
||||||
|
await engine.dispose()
|
||||||
|
finally:
|
||||||
|
get_engine.cache_clear()
|
||||||
|
|
||||||
|
|
||||||
|
async def refresh_engine(database_url: str) -> AsyncEngine:
|
||||||
|
await dispose_engine(database_url)
|
||||||
|
return get_engine(database_url)
|
||||||
|
```
|
||||||
|
|
||||||
|
At the composition boundary, resolve the URL once with `get_database_url(settings)` and use it to retrieve the cached engine. In FastAPI, expose that engine through lifespan and build one `async_sessionmaker` from it; each request or unit of work then creates its own `AsyncSession`. Do not call `aiosqlite.connect()` or `asyncpg.create_pool()` directly: `aiosqlite` and `asyncpg` are selected as SQLAlchemy drivers by the URL, while SQLAlchemy owns pooling, disposal, and session integration.
|
||||||
|
|
||||||
|
The nested variants remain `BaseModel` classes. `Settings` is the only `BaseSettings` model and therefore the only object that reads environment variables, dotenv files, or secrets. This keeps one source policy and validated configuration snapshot while keeping the engine, session factory, and sessions in their distinct lifecycles. See the [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html), the [engine lifecycle guidance](../async-fastapi-sqlmodel/references/engine.md), and the [session lifecycle guidance](../async-fastapi-sqlmodel/references/session.md).
|
||||||
|
|
||||||
Quality gate:
|
Quality gate:
|
||||||
|
|
||||||
1. Nested sections share one source policy and lifecycle.
|
1. Nested sections share one source policy and lifecycle.
|
||||||
2. Independent settings have distinct owners, prefixes, or lifecycles.
|
2. Independent settings have distinct owners, prefixes, or lifecycles.
|
||||||
3. The application does not repeatedly scan the same sources through accidental nested `BaseSettings` construction.
|
3. The application does not repeatedly scan the same sources through accidental nested `BaseSettings` construction.
|
||||||
|
4. Each backend configuration validates without values required only by another backend.
|
||||||
|
5. One cached `AsyncEngine` exists per configured driver URL, while each request or unit of work receives a new `AsyncSession`.
|
||||||
|
|
||||||
### 7. Own The Settings Lifecycle
|
### 7. Own The Settings Lifecycle
|
||||||
|
|
||||||
|
|||||||
@@ -1,19 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: pytesting
|
name: pytesting
|
||||||
description: "Reference hub for pytest suite structure, naming, markers, and stack-specific testing patterns. Optimized for progressive discovery so naming and hierarchy guidance are loaded first when shaping or reorganizing tests."
|
description: "Reference hub for pytest suite structure, naming, markers, and stack-specific testing patterns. Optimized for progressive discovery so naming and hierarchy guidance are loaded first when shaping or reorganizing tests."
|
||||||
x-personal-mcp:
|
|
||||||
id: pytesting
|
|
||||||
version: 1.0.0
|
|
||||||
tags:
|
|
||||||
- pytest
|
|
||||||
- testing
|
|
||||||
- python
|
|
||||||
- fastapi
|
|
||||||
- asyncio
|
|
||||||
- anyio
|
|
||||||
- deterministic
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/pytesting/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Pytesting
|
# Pytesting
|
||||||
|
|||||||
@@ -1,15 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: python-logging
|
name: python-logging
|
||||||
description: 'Design, review, or refactor Python logging. Use when choosing logger names, levels, handlers, library/application boundaries, basicConfig, dictConfig, structured logs, or operational logging defaults.'
|
description: 'Design, review, or refactor Python logging. Use when choosing logger names, levels, handlers, library/application boundaries, basicConfig, dictConfig, structured logs, or operational logging defaults.'
|
||||||
x-personal-mcp:
|
|
||||||
id: python-logging
|
|
||||||
version: 1.0.0
|
|
||||||
tags:
|
|
||||||
- logging
|
|
||||||
- python
|
|
||||||
- observability
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/python-logging/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Python Logging
|
# Python Logging
|
||||||
|
|||||||
@@ -1,18 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: python-typing
|
name: python-typing
|
||||||
description: "Reference-first skill for reviewing and modernizing Python typing to the newest supported best practices. Use when auditing annotations, replacing legacy typing syntax, and enforcing latest-syntax-first conventions."
|
description: "Reference-first skill for reviewing and modernizing Python typing to the newest supported best practices. Use when auditing annotations, replacing legacy typing syntax, and enforcing latest-syntax-first conventions."
|
||||||
x-personal-mcp:
|
|
||||||
id: python-typing
|
|
||||||
version: 1.0.0
|
|
||||||
tags:
|
|
||||||
- python
|
|
||||||
- typing
|
|
||||||
- type-hints
|
|
||||||
- pep-695
|
|
||||||
- modernization
|
|
||||||
- static-analysis
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/python-typing/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Modern Python Typing Review Reference
|
# Modern Python Typing Review Reference
|
||||||
|
|||||||
@@ -1,17 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: ruff-linting-formating
|
name: ruff-linting-formating
|
||||||
description: "Reference-first Ruff skill for repository preferences, baseline defaults, and source links. Use to pick consistent Ruff conventions and integration references, not to run migration playbooks."
|
description: "Reference-first Ruff skill for repository preferences, baseline defaults, and source links. Use to pick consistent Ruff conventions and integration references, not to run migration playbooks."
|
||||||
x-personal-mcp:
|
|
||||||
id: ruff-linting-formating
|
|
||||||
version: 1.0.0
|
|
||||||
tags:
|
|
||||||
- ruff
|
|
||||||
- linting
|
|
||||||
- formatting
|
|
||||||
- python
|
|
||||||
- ci
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/ruff-linting-formating/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Ruff Preferences and References
|
# Ruff Preferences and References
|
||||||
|
|||||||
@@ -1,19 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: vscode-configuration
|
name: vscode-configuration
|
||||||
description: 'Create and troubleshoot VS Code workspace configuration for Python projects, with focused patterns for launch.json debugpy/FastAPI debugging and tasks.json task automation.'
|
description: 'Create and troubleshoot VS Code workspace configuration for Python projects, with focused patterns for launch.json debugpy/FastAPI debugging and tasks.json task automation.'
|
||||||
x-personal-mcp:
|
|
||||||
id: vscode-configuration
|
|
||||||
version: 1.0.0
|
|
||||||
tags:
|
|
||||||
- vscode
|
|
||||||
- launch-json
|
|
||||||
- tasks-json
|
|
||||||
- debugpy
|
|
||||||
- fastapi
|
|
||||||
- python
|
|
||||||
- skills
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/vscode-configuration/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# VS Code Configuration
|
# VS Code Configuration
|
||||||
|
|||||||
@@ -1,23 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: zensical-docs
|
name: zensical-docs
|
||||||
description: 'Reference skill for Zensical documentation mechanics. Use for quick lookup of docs structure, feature options, and source links. Prefer inline Markdown links to source docs and avoid bare URLs because this content is rendered as human docs and MCP resources.'
|
description: 'Reference skill for Zensical documentation mechanics. Use for quick lookup of docs structure, feature options, and source links. Prefer inline Markdown links to source docs and avoid bare URLs because this content is rendered as human docs and MCP resources.'
|
||||||
x-personal-mcp:
|
|
||||||
id: zensical-docs
|
|
||||||
version: 1.0.0
|
|
||||||
tags:
|
|
||||||
- zensical
|
|
||||||
- mkdocs
|
|
||||||
- mkdocs-material
|
|
||||||
- mkdocstrings
|
|
||||||
- docs
|
|
||||||
- documentation
|
|
||||||
- information-architecture
|
|
||||||
- skills
|
|
||||||
- bootstrap
|
|
||||||
- discovery
|
|
||||||
- authoring
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/zensical-docs/document
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Zensical Documentation Authoring
|
# Zensical Documentation Authoring
|
||||||
|
|||||||
+6
-3
@@ -27,26 +27,29 @@ tests/
|
|||||||
__init__.py
|
__init__.py
|
||||||
conftest.py
|
conftest.py
|
||||||
registry/
|
registry/
|
||||||
|
test_read.py
|
||||||
ingest/
|
ingest/
|
||||||
conftest.py
|
conftest.py
|
||||||
test_current_docs.py
|
test_current_docs.py
|
||||||
test_document.py
|
test_document.py
|
||||||
test_prompt.py
|
test_prompt.py
|
||||||
test_skill.py
|
|
||||||
models/
|
models/
|
||||||
test_document_validation.py
|
test_document_validation.py
|
||||||
test_prompt_validation.py
|
test_prompt_validation.py
|
||||||
test_registry_payload_models.py
|
test_registry_payload_models.py
|
||||||
test_skill_validation.py
|
skills/
|
||||||
|
test_provider.py
|
||||||
web/
|
web/
|
||||||
conftest.py
|
conftest.py
|
||||||
test_endpoint_connections.py
|
test_endpoint_connections.py
|
||||||
|
test_mcp_prompts.py
|
||||||
test_mcp_skills.py
|
test_mcp_skills.py
|
||||||
```
|
```
|
||||||
|
|
||||||
Source-to-test alignment today:
|
Source-to-test alignment today:
|
||||||
- `src/personal_mcp/registry/ingest/` -> `tests/registry/ingest/`
|
- `src/personal_mcp/registry/ingest/` -> `tests/registry/ingest/`
|
||||||
- `src/personal_mcp/registry/models/` -> `tests/registry/models/`
|
- `src/personal_mcp/registry/models/` -> `tests/registry/models/`
|
||||||
|
- `src/personal_mcp/skills/provider.py` -> `tests/skills/test_provider.py`
|
||||||
- `src/personal_mcp/web/` and MCP HTTP surface -> `tests/web/`
|
- `src/personal_mcp/web/` and MCP HTTP surface -> `tests/web/`
|
||||||
|
|
||||||
## Markers And Strictness
|
## Markers And Strictness
|
||||||
@@ -87,7 +90,7 @@ uv run pytest -m smoke -q
|
|||||||
## Adding New Tests
|
## Adding New Tests
|
||||||
|
|
||||||
When adding coverage:
|
When adding coverage:
|
||||||
1. Place tests under the nearest existing module subtree (`registry/` or `web/`).
|
1. Place tests under the nearest existing module subtree (`registry/`, `skills/`, or `web/`).
|
||||||
2. Mirror the source path where practical.
|
2. Mirror the source path where practical.
|
||||||
3. Reuse existing `conftest.py` files before adding new fixture layers.
|
3. Reuse existing `conftest.py` files before adding new fixture layers.
|
||||||
4. Add markers only when they convey execution intent, and register new markers in `pyproject.toml` first.
|
4. Add markers only when they convey execution intent, and register new markers in `pyproject.toml` first.
|
||||||
|
|||||||
+71
-309
@@ -6,361 +6,123 @@ icon: lucide/workflow
|
|||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
This page explains practical usage mechanics for the GitHub Copilot extension in VS Code when `personal-mcp` is configured as an MCP server:
|
This page describes how clients discover and load `personal-mcp` skills published by the [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills).
|
||||||
|
|
||||||
1. explicit `/` command flows when you want deterministic control
|
Skills are MCP resources. The client remains responsible for selecting guidance, loading only useful supporting material, and applying it to the current workspace.
|
||||||
2. guided skill loading when relevance can be inferred
|
|
||||||
|
|
||||||
The goal is to show how Copilot behaves as a client and how to shape that behavior.
|
## Published Skill Surface
|
||||||
|
|
||||||
## Mental Model
|
Each directory beneath `docs/skills/` publishes:
|
||||||
|
|
||||||
In Copilot Chat, there are two distinct mechanisms:
|
1. `skill://<name>/SKILL.md` for primary instructions
|
||||||
|
2. `skill://<name>/_manifest` for file discovery and integrity metadata
|
||||||
|
3. `skill://<name>/{path*}` for supporting files
|
||||||
|
|
||||||
1. `/` commands are user-invoked orchestration shortcuts.
|
The server uses `supporting_files="template"`. Main files and manifests appear in `resources/list`; supporting files stay behind per-skill wildcard templates so the resource list remains compact.
|
||||||
2. MCP resources are server-published knowledge units that can be attached as read-only context, while MCP tools provide an execution path for discovery and retrieval.
|
|
||||||
|
|
||||||
In this repository, skill guidance is exposed as MCP resources, not as server-owned prompt execution. Copilot remains the orchestrator.
|
The manifest contains every skill-relative path, byte size, and SHA256 hash. References do not have synthetic ids or a separate catalog record.
|
||||||
|
|
||||||
Prompt guidance is now exposed through both prompt resources and MCP prompt objects. Prompt objects are additive; authored markdown remains the canonical source.
|
Prompts remain available through prompt catalog resources, prompt document resources, and MCP prompt objects.
|
||||||
|
|
||||||
## Background Mechanics
|
## Discovery Workflow
|
||||||
|
|
||||||
### What the server publishes
|
Use this bounded sequence:
|
||||||
|
|
||||||
`personal-mcp` registers resources from the validated docs registry and exposes catalog discovery resources:
|
1. List resources or call FastMCP `list_skills()`.
|
||||||
|
2. Compare skill names and descriptions.
|
||||||
|
3. Read one selected `skill://<name>/SKILL.md`.
|
||||||
|
4. Read `skill://<name>/_manifest` only when supporting material may be useful.
|
||||||
|
5. Fetch the minimum supporting paths needed for the task.
|
||||||
|
6. Reconcile the guidance with the actual repository code before making changes.
|
||||||
|
|
||||||
1. `resource://catalog/skills_index`
|
Do not load every skill or every supporting file up front.
|
||||||
2. `resource://catalog/skills_index{?q,tag,capability,cursor,limit}`
|
|
||||||
3. `resource://catalog/skills/{skill_id}`
|
|
||||||
4. `resource://catalog/prompts_index`
|
|
||||||
5. `resource://catalog/prompts_index{?q,tag,cursor,limit}`
|
|
||||||
6. `resource://catalog/prompts/{prompt_id}`
|
|
||||||
|
|
||||||
Each skill publishes a canonical Markdown document resource:
|
## FastMCP Client Utilities
|
||||||
|
|
||||||
1. `resource://skills/<skill-id>/document`
|
FastMCP provides native utilities in `fastmcp.utilities.skills`:
|
||||||
2. `resource://skills/<skill-id>/references/<ref-id>`
|
|
||||||
|
|
||||||
Prompts publish a canonical prompt document resource:
|
1. `list_skills(client)` discovers main skill resources.
|
||||||
|
2. `get_skill_manifest(client, name)` parses a generated manifest.
|
||||||
|
3. `download_skill(client, name, target_dir)` downloads one skill.
|
||||||
|
4. `sync_skills(client, target_dir)` downloads all advertised skills.
|
||||||
|
|
||||||
1. `resource://prompts/<prompt-id>/document`
|
These utilities operate directly on the native `skill://` contract and require no repository-specific adapter.
|
||||||
|
|
||||||
The document payload is loaded from `docs/skills/<skill-id>/SKILL.md` and returned with metadata.
|
## Tool-Only Clients
|
||||||
|
|
||||||
### What Copilot does as the client
|
The server installs [`ResourcesAsTools`](https://gofastmcp.com/servers/transforms/resources-as-tools), which exposes generic tools:
|
||||||
|
|
||||||
When connected to MCP, Copilot can do the following at runtime:
|
1. `list_resources`
|
||||||
|
2. `read_resource`
|
||||||
|
|
||||||
1. interpret the current chat request
|
A tool-only client should list resources, select a `skill://<name>/SKILL.md` URI, and read it. It can then read `_manifest` and selected supporting paths through the same tool.
|
||||||
2. use attached MCP resources that you provide through the chat UI
|
|
||||||
3. invoke MCP tools when the task and tool descriptions make that relevant
|
|
||||||
4. summarize relevant sections into working context
|
|
||||||
5. apply guidance while generating edits or recommendations
|
|
||||||
|
|
||||||
This behavior is shaped by the active chat surface, prompt or instruction guidance, and available MCP tools.
|
There are no skill-specific search, detail, or document tools. This avoids maintaining a second discovery implementation.
|
||||||
|
|
||||||
For reliable progressive discovery, use one of these sequences:
|
## Optional Tool Search
|
||||||
|
|
||||||
1. explicit resource path: attach a catalog resource first, then attach only selected skill documents
|
For large tool inventories, FastMCP search transforms can reduce tool-list noise:
|
||||||
2. tool path: call catalog tools first, then load only selected skill documents
|
|
||||||
|
|
||||||
### What `/` commands do
|
1. `PERSONAL_MCP_TOOL_SEARCH=none|regex|bm25` defaults to `none`.
|
||||||
|
2. `PERSONAL_MCP_TOOL_SEARCH_MAX_RESULTS=<positive int>` defaults to `5`.
|
||||||
|
3. `list_resources` and `read_resource` remain visible in search modes.
|
||||||
|
|
||||||
`/` commands in VS Code are client-side prompt entry points (for example in prompt files). They do not replace MCP resources. In Copilot, they typically:
|
These settings filter tools, not native skill resources.
|
||||||
|
|
||||||
1. enforce a known sequence
|
## Copilot Invocation
|
||||||
2. collect missing inputs
|
|
||||||
3. call discovery/read steps in a predictable order
|
|
||||||
|
|
||||||
Think of `/` commands as orchestration shortcuts on top of MCP resources.
|
In VS Code, skills can arrive through:
|
||||||
|
|
||||||
### What automatic loading means here
|
1. explicit attachment from `Add Context > MCP Resources` or `MCP: Browse Resources`
|
||||||
|
2. generic `list_resources` and `read_resource` tool calls
|
||||||
|
3. a slash-command prompt that names a specific native skill URI
|
||||||
|
|
||||||
In this project, "automatic loading" should be read as a preference you express through instructions and prompts, not as a guaranteed VS Code feature that auto-attaches MCP resources.
|
Instructions can steer retrieval, but they do not guarantee automatic resource attachment in every chat surface.
|
||||||
|
|
||||||
In practice, there are two reliable ways to make skill content available in chat:
|
A reliable prompt for a tool-only session is:
|
||||||
|
|
||||||
1. explicit resource attachment through `Add Context > MCP Resources` or `MCP: Browse Resources`
|
```text
|
||||||
2. MCP tool invocation using `list_resources`/`read_resource` (ResourcesAsTools), with thin catalog tools as parity fallback
|
Use personal-mcp list_resources to find the best matching skill://.../SKILL.md resource. Read one selected skill, inspect its _manifest only if supporting detail is needed, and reconcile the guidance with this workspace.
|
||||||
|
|
||||||
For prompt content, there is a third option when the client supports MCP prompt APIs:
|
|
||||||
|
|
||||||
1. prompt-object discovery and invocation through MCP prompt lists and `get_prompt`
|
|
||||||
|
|
||||||
Instruction quality and metadata quality still matter, because they influence whether Copilot recognizes that the MCP server is relevant and chooses the tool path well.
|
|
||||||
|
|
||||||
## Invocation Mechanics Deep Dive
|
|
||||||
|
|
||||||
This section expands on how invocation works at runtime across chat entry points.
|
|
||||||
|
|
||||||
### Invocation Surfaces
|
|
||||||
|
|
||||||
A user request can arrive through one of these surfaces:
|
|
||||||
|
|
||||||
1. plain chat request in Ask/Edit/Agent mode
|
|
||||||
2. slash command invocation of a prompt or skill
|
|
||||||
3. chat request with manually attached MCP resources
|
|
||||||
|
|
||||||
Each surface changes how much discovery Copilot must do before applying guidance.
|
|
||||||
|
|
||||||
### Resolution Order
|
|
||||||
|
|
||||||
When multiple retrieval paths are possible, use this priority order:
|
|
||||||
|
|
||||||
1. attached MCP resources already in context
|
|
||||||
2. explicit slash-command workflow steps
|
|
||||||
3. catalog-first discovery via MCP resources
|
|
||||||
4. tool fallback (`list_resources` then `read_resource`, then thin catalog parity tools)
|
|
||||||
|
|
||||||
This ordering keeps behavior predictable while minimizing unnecessary context expansion.
|
|
||||||
|
|
||||||
### Prompt Invocation Pipeline
|
|
||||||
|
|
||||||
For prompt-oriented flows, treat invocation as this sequence:
|
|
||||||
|
|
||||||
1. parse prompt frontmatter and argument hints
|
|
||||||
2. validate required inputs and ask one clarifying question if blocked
|
|
||||||
3. run bounded discovery against prompt or skill catalogs
|
|
||||||
4. fetch only selected document resources
|
|
||||||
5. apply instructions to produce edits, recommendations, or commands
|
|
||||||
6. report what was loaded and why
|
|
||||||
|
|
||||||
Prompt objects and prompt document resources are additive mechanisms. The authored Markdown prompt document remains the canonical contract.
|
|
||||||
|
|
||||||
### Argument Syntax Nuance
|
|
||||||
|
|
||||||
Invocation strings such as target_modules=src/personal_mcp/registry/ingest/skill.py, mode=plan-only are a structured authoring convention, not a guaranteed client-level grammar.
|
|
||||||
|
|
||||||
In practice:
|
|
||||||
|
|
||||||
1. Prompt metadata defines expected argument names and intent.
|
|
||||||
2. Prompt body instructions define how those inputs should be interpreted.
|
|
||||||
3. Copilot may receive equivalent intent in freeform phrasing and still resolve it correctly.
|
|
||||||
|
|
||||||
Implication for authors:
|
|
||||||
|
|
||||||
1. Treat key=value examples as clarity aids for users.
|
|
||||||
2. Do not assume strict parser enforcement unless your prompt explicitly validates and rejects malformed input.
|
|
||||||
3. Include accepted invocation examples and one fallback freeform example so behavior is predictable for both humans and the model.
|
|
||||||
|
|
||||||
This distinction is important because argument hints improve discoverability, while robust prompt instructions determine actual runtime reliability.
|
|
||||||
|
|
||||||
### Skill Invocation Pipeline
|
|
||||||
|
|
||||||
For guided skill loading, use this sequence:
|
|
||||||
|
|
||||||
1. start from `resource://catalog/skills_index` or scoped index query
|
|
||||||
2. inspect one or two top candidates for intent and capability fit
|
|
||||||
3. fetch `resource://skills/<skill-id>/document`
|
|
||||||
4. load references only when the task needs deeper detail
|
|
||||||
5. apply only relevant sections and keep context bounded
|
|
||||||
|
|
||||||
This avoids the common failure mode where many skill documents are loaded up front.
|
|
||||||
|
|
||||||
### Determinism vs Flexibility
|
|
||||||
|
|
||||||
Use this decision rule:
|
|
||||||
|
|
||||||
1. choose slash-command invocation when repeatability and step order are critical
|
|
||||||
2. choose guided loading when requests vary and speed matters more than strict orchestration
|
|
||||||
3. escalate from guided loading to slash-command flow when confidence is low or conflicting skills appear
|
|
||||||
|
|
||||||
### Invocation Trace (What to Log in Results)
|
|
||||||
|
|
||||||
For transparent operation, include a concise invocation trace in task outputs:
|
|
||||||
|
|
||||||
1. entry surface used (plain chat, slash command, or attached resource)
|
|
||||||
2. discovery source used (catalog resource or tool path)
|
|
||||||
3. resources fetched (ids only)
|
|
||||||
4. clarifying questions asked (if any)
|
|
||||||
5. reason for fallback or escalation (if used)
|
|
||||||
|
|
||||||
This makes behavior auditable and easier to tune over time.
|
|
||||||
|
|
||||||
## Operating Pattern
|
|
||||||
|
|
||||||
Use both modes intentionally in Copilot Chat.
|
|
||||||
|
|
||||||
### Mode A: Explicit `/` command
|
|
||||||
|
|
||||||
Use when you need predictable, repeatable behavior across teammates.
|
|
||||||
|
|
||||||
Good fits:
|
|
||||||
|
|
||||||
1. onboarding workflows
|
|
||||||
2. compliance-sensitive tasks
|
|
||||||
3. repetitive scaffolding
|
|
||||||
|
|
||||||
### Mode B: Guided skill loading
|
|
||||||
|
|
||||||
Use when requests are varied and you want lower friction during normal chat.
|
|
||||||
|
|
||||||
Good fits:
|
|
||||||
|
|
||||||
1. ad hoc implementation questions
|
|
||||||
2. mixed-topic debugging
|
|
||||||
3. architecture tradeoff discussions
|
|
||||||
|
|
||||||
### Mode C: Fallback flow
|
|
||||||
|
|
||||||
Start with guided loading in chat; escalate to a `/` command when:
|
|
||||||
|
|
||||||
1. confidence is low
|
|
||||||
2. multiple skills conflict
|
|
||||||
3. the user wants strict repeatability
|
|
||||||
|
|
||||||
## Suggested Resolution Flow
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TD
|
|
||||||
A[User request in Copilot Chat] --> B{Deterministic workflow needed?}
|
|
||||||
B -- Yes --> C[/Run slash command/]
|
|
||||||
C --> D[Copilot fetches known catalog and skill resources]
|
|
||||||
B -- No --> E[Copilot uses attached resources or catalog tools]
|
|
||||||
E --> F{Confident skill match?}
|
|
||||||
F -- Yes --> G[Copilot fetches skill documents]
|
|
||||||
F -- No --> H[Ask clarifying question or suggest slash command]
|
|
||||||
D --> I[Apply guidance to task]
|
|
||||||
G --> I
|
|
||||||
H --> I
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Authoring Requirements
|
## Thin Shim Pattern
|
||||||
|
|
||||||
Authoring rules for metadata quality and instruction patterns are maintained in [Authoring Guide](./authoring.md).
|
Consumer repositories can bind file scopes to native skill resources with short `.github/instructions/*.instructions.md` files.
|
||||||
|
|
||||||
## Practical Guidelines
|
| `applyTo` scope | Companion docs | Primary skill resource |
|
||||||
|
|
||||||
1. Keep `/` commands minimal and high-value.
|
|
||||||
2. Do not duplicate full methodology text inside command files.
|
|
||||||
3. Keep canonical guidance in `docs/skills/*/SKILL.md`.
|
|
||||||
4. In Copilot instructions, prefer catalog-first discovery before skill fetch.
|
|
||||||
5. Prefer small, relevant context slices over loading every skill.
|
|
||||||
6. Keep slash commands focused on deterministic orchestration, not content duplication.
|
|
||||||
|
|
||||||
If you skip the catalog/index step, behavior is less predictable and may either miss relevant skills or pull too much context.
|
|
||||||
|
|
||||||
## Optional Tool Search Mode
|
|
||||||
|
|
||||||
When tool catalogs grow, FastMCP search transforms can reduce tool-list noise for tool-only clients.
|
|
||||||
|
|
||||||
Runtime switches:
|
|
||||||
|
|
||||||
1. `PERSONAL_MCP_TOOL_SEARCH=none|regex|bm25` (default `none`)
|
|
||||||
2. `PERSONAL_MCP_TOOL_SEARCH_MAX_RESULTS=<positive int>` (default `5`)
|
|
||||||
|
|
||||||
Behavior:
|
|
||||||
|
|
||||||
1. `regex` uses deterministic regex matching for targeted queries.
|
|
||||||
2. `bm25` uses ranked natural-language matching.
|
|
||||||
3. `list_resources` and `read_resource` stay visible so resource-backed fallback remains primary.
|
|
||||||
|
|
||||||
## Failure Modes and Recovery
|
|
||||||
|
|
||||||
Common failure modes:
|
|
||||||
|
|
||||||
1. No relevant skill selected.
|
|
||||||
2. Too many skills selected (context bloat).
|
|
||||||
3. Stale assumptions from old metadata.
|
|
||||||
4. Slash command bypasses normal discovery and forces the wrong skill.
|
|
||||||
|
|
||||||
Recovery sequence:
|
|
||||||
|
|
||||||
1. re-run catalog lookup
|
|
||||||
2. narrow by tags and intent
|
|
||||||
3. fetch only top candidates
|
|
||||||
4. if still ambiguous, ask one clarifying question
|
|
||||||
5. use explicit `/` workflow for deterministic fallback
|
|
||||||
|
|
||||||
## Checklist
|
|
||||||
|
|
||||||
Use this checklist when configuring GitHub Copilot in VS Code against `personal-mcp`:
|
|
||||||
|
|
||||||
1. confirm server connectivity
|
|
||||||
2. verify catalog resources are readable
|
|
||||||
3. verify at least one `resource://skills/<id>/document` can be fetched
|
|
||||||
4. add one deterministic `/` command for fallback
|
|
||||||
5. confirm your workspace instruction policy exists (see [Authoring Guide](./authoring.md))
|
|
||||||
6. verify context size remains bounded
|
|
||||||
7. validate behavior in Ask/Edit/Agent-style workflows with at least one task each
|
|
||||||
|
|
||||||
## Runtime Discovery Workflow
|
|
||||||
|
|
||||||
Use this runtime sequence in chat sessions:
|
|
||||||
|
|
||||||
1. Start with catalog-first discovery.
|
|
||||||
2. Prefer MCP resources when the chat surface exposes resource attachment.
|
|
||||||
3. Otherwise use tool fallback to load one or two likely skill documents.
|
|
||||||
4. Prefer `list_resources`/`read_resource` first when operating in tool-only clients.
|
|
||||||
5. If confidence is low, ask one clarifying question before loading more.
|
|
||||||
|
|
||||||
## Thin Shim Path Binding Pattern
|
|
||||||
|
|
||||||
For repositories that consume this MCP server, thin shims are a usage pattern for binding path scopes to the right MCP resources. The "thin shims" are just lightweight, repo-specific instructions files that tell Copilot to use certain MCP resources when editing files that match a pattern. That helps with ensuring Copilot uses the intended resources without too much specific goading in the prompt.
|
|
||||||
|
|
||||||
Use thin shims in Copilot instruction files to bind file-path scopes to:
|
|
||||||
|
|
||||||
1. the most relevant docs page for human-readable conventions
|
|
||||||
2. the matching MCP resource URI for machine retrieval
|
|
||||||
|
|
||||||
Keep each shim short: trigger, primary resource, minimal execution pattern, and one fallback rule.
|
|
||||||
|
|
||||||
Recommended binding pattern:
|
|
||||||
|
|
||||||
1. Put shims in `.github/instructions/*.instructions.md`.
|
|
||||||
2. Scope each shim with `applyTo` so it activates only where needed.
|
|
||||||
3. Point to one primary `resource://skills/<skill-id>/document` URI.
|
|
||||||
4. Link one repository docs page as the human-facing companion.
|
|
||||||
5. Expand to references only when the task needs deeper detail.
|
|
||||||
|
|
||||||
Current repository examples:
|
|
||||||
|
|
||||||
| applyTo scope | Primary docs page | Primary MCP resource |
|
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `**/*.md` | [docs/authoring.md](./authoring.md) | `resource://skills/zensical-docs/document` |
|
| `**/*.md` | [Authoring Guide](./authoring.md) | `skill://zensical-docs/SKILL.md` |
|
||||||
| `tests/**` | [docs/testing.md](./testing.md) | `resource://skills/pytesting/document` |
|
| `tests/**` | [Testing](./testing.md) | `skill://pytesting/SKILL.md` |
|
||||||
| `.vscode/**` | [docs/skills/vscode-configuration/SKILL.md](./skills/vscode-configuration/SKILL.md) | `resource://skills/vscode-configuration/document` |
|
| `.vscode/**` | [VS Code Configuration](./skills/vscode-configuration/SKILL.md) | `skill://vscode-configuration/SKILL.md` |
|
||||||
|
|
||||||
Minimal shim shape:
|
Minimal shape:
|
||||||
|
|
||||||
```md
|
```md
|
||||||
---
|
---
|
||||||
name: <short scope name>
|
name: <scope name>
|
||||||
description: Route <path scope> edits to the Personal MCP <skill-id> resource.
|
description: Route <path scope> edits to a personal-mcp skill.
|
||||||
applyTo: '<glob>'
|
applyTo: '<glob>'
|
||||||
---
|
---
|
||||||
|
|
||||||
When editing files matching <glob>, use `resource://skills/<skill-id>/document` as the primary guidance source.
|
Load `skill://<skill-name>/SKILL.md` first. Read `_manifest` and supporting files only when the task needs deeper detail. Apply the guidance to the current repository rather than treating it as generated output.
|
||||||
|
|
||||||
Execution pattern:
|
|
||||||
|
|
||||||
1. Load the primary skill document first.
|
|
||||||
2. Apply only sections relevant to the file being edited.
|
|
||||||
3. Keep edits minimal and aligned with repository conventions.
|
|
||||||
4. If confidence is low, ask one clarifying question before editing.
|
|
||||||
|
|
||||||
Companion docs page: [docs/<page>.md](./<page>.md)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
When to use thin shims:
|
## Failure Recovery
|
||||||
|
|
||||||
1. Repositories that want thin local policy while keeping canonical guidance in MCP resources.
|
When no skill is an obvious match:
|
||||||
2. Stable, repeated workflows with clear path ownership.
|
|
||||||
3. Cases where teams need predictable retrieval behavior.
|
|
||||||
|
|
||||||
When not to use thin shims:
|
1. compare the available main-resource descriptions again
|
||||||
|
2. select at most two candidates
|
||||||
|
3. read their main files, not all supporting files
|
||||||
|
4. ask one clarifying question if the choice remains ambiguous
|
||||||
|
|
||||||
1. Broad, ambiguous tasks with unclear ownership boundaries.
|
When a supporting path fails, refresh `_manifest`; file paths are the public supporting-resource identifiers.
|
||||||
2. Cases where one shim would need many exceptions.
|
|
||||||
3. Situations better handled by catalog-first discovery at runtime.
|
|
||||||
|
|
||||||
## Summary
|
## Runtime Checklist
|
||||||
|
|
||||||
The intended model is:
|
1. Confirm MCP connectivity.
|
||||||
|
2. Confirm at least one `skill://<name>/SKILL.md` resource is listed.
|
||||||
1. skills are canonical MCP resources
|
3. Read its `_manifest` and verify `SKILL.md` appears with a SHA256 hash.
|
||||||
2. `/` commands are explicit Copilot control shortcuts
|
4. Read one supporting file through its manifest path.
|
||||||
3. guided skill loading should be catalog-driven, bounded, and explicit about whether it is using resources or tools
|
5. Confirm `list_resources` and `read_resource` are available for tool-only clients.
|
||||||
|
6. Keep loaded context bounded to the selected skill and relevant files.
|
||||||
Using all three together gives predictable control when needed and low-friction assistance by default in VS Code.
|
|
||||||
|
|||||||
+3
-3
@@ -4,9 +4,9 @@ version = "0.1.0"
|
|||||||
requires-python = ">=3.12"
|
requires-python = ">=3.12"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"fastapi>=0.115.0",
|
"fastapi>=0.115.0",
|
||||||
"fastmcp>=2.10.0",
|
"fastmcp>=3.4.4",
|
||||||
"pydantic-settings>=2.0.0",
|
"pydantic-settings>=2",
|
||||||
"python-json-logger>=4.1.0",
|
"python-json-logger>=4",
|
||||||
"pyyaml>=6.0.2",
|
"pyyaml>=6.0.2",
|
||||||
"uvicorn[standard]>=0.34.0",
|
"uvicorn[standard]>=0.34.0",
|
||||||
"zensical>=0.0.45",
|
"zensical>=0.0.45",
|
||||||
|
|||||||
@@ -1,19 +1,11 @@
|
|||||||
from personal_mcp.catalog.server import build_prompt_detail_payload
|
from personal_mcp.catalog.server import build_prompt_detail_payload
|
||||||
from personal_mcp.catalog.server import build_prompts_index_payload
|
from personal_mcp.catalog.server import build_prompts_index_payload
|
||||||
from personal_mcp.catalog.server import build_skill_detail_payload
|
|
||||||
from personal_mcp.catalog.server import build_skills_index_payload
|
|
||||||
from personal_mcp.catalog.server import get_pattern_by_id_payload
|
|
||||||
from personal_mcp.catalog.server import get_prompt_by_id_payload
|
from personal_mcp.catalog.server import get_prompt_by_id_payload
|
||||||
from personal_mcp.catalog.server import search_patterns_payload
|
|
||||||
from personal_mcp.catalog.server import search_prompts_payload
|
from personal_mcp.catalog.server import search_prompts_payload
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"build_prompt_detail_payload",
|
"build_prompt_detail_payload",
|
||||||
"build_prompts_index_payload",
|
"build_prompts_index_payload",
|
||||||
"build_skill_detail_payload",
|
|
||||||
"build_skills_index_payload",
|
|
||||||
"get_pattern_by_id_payload",
|
|
||||||
"get_prompt_by_id_payload",
|
"get_prompt_by_id_payload",
|
||||||
"search_patterns_payload",
|
|
||||||
"search_prompts_payload",
|
"search_prompts_payload",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -5,42 +5,11 @@ from typing import Any
|
|||||||
from personal_mcp.registry.models.registry import DocsRegistry
|
from personal_mcp.registry.models.registry import DocsRegistry
|
||||||
from personal_mcp.registry.models.registry import PromptRecord
|
from personal_mcp.registry.models.registry import PromptRecord
|
||||||
from personal_mcp.registry.models.registry import PromptSummaryPayload
|
from personal_mcp.registry.models.registry import PromptSummaryPayload
|
||||||
from personal_mcp.registry.models.registry import SkillPatternPayload
|
|
||||||
from personal_mcp.registry.models.registry import SkillRecord
|
|
||||||
from personal_mcp.registry.models.registry import SkillSummaryPayload
|
|
||||||
|
|
||||||
DEFAULT_LIMIT = 20
|
DEFAULT_LIMIT = 20
|
||||||
MAX_LIMIT = 100
|
MAX_LIMIT = 100
|
||||||
|
|
||||||
|
|
||||||
def _skill_matches(
|
|
||||||
skill: SkillRecord,
|
|
||||||
*,
|
|
||||||
query: str | None,
|
|
||||||
tag: str | None,
|
|
||||||
capability: str | None,
|
|
||||||
) -> bool:
|
|
||||||
if query:
|
|
||||||
lowered = query.strip().lower()
|
|
||||||
if lowered:
|
|
||||||
haystack = " ".join(
|
|
||||||
[
|
|
||||||
skill.skill_id,
|
|
||||||
skill.name,
|
|
||||||
skill.description,
|
|
||||||
" ".join(skill.tags),
|
|
||||||
]
|
|
||||||
).lower()
|
|
||||||
terms = [term for term in lowered.replace("-", " ").split() if term]
|
|
||||||
if any(term not in haystack for term in terms):
|
|
||||||
return False
|
|
||||||
|
|
||||||
if tag and tag not in skill.tags:
|
|
||||||
return False
|
|
||||||
|
|
||||||
return not (capability and capability not in skill.capabilities)
|
|
||||||
|
|
||||||
|
|
||||||
def _prompt_matches(
|
def _prompt_matches(
|
||||||
prompt: PromptRecord,
|
prompt: PromptRecord,
|
||||||
*,
|
*,
|
||||||
@@ -66,63 +35,6 @@ def _prompt_matches(
|
|||||||
return not (tag and tag not in prompt.tags)
|
return not (tag and tag not in prompt.tags)
|
||||||
|
|
||||||
|
|
||||||
def build_skills_index_payload(
|
|
||||||
registry: DocsRegistry,
|
|
||||||
*,
|
|
||||||
query: str | None = None,
|
|
||||||
tag: str | None = None,
|
|
||||||
capability: str | None = None,
|
|
||||||
cursor: str | None = None,
|
|
||||||
limit: int | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
normalized_limit = DEFAULT_LIMIT if limit is None else max(1, min(limit, MAX_LIMIT))
|
|
||||||
try:
|
|
||||||
start = 0 if cursor is None else max(0, int(cursor))
|
|
||||||
except ValueError as exc:
|
|
||||||
raise ValueError("cursor must be an integer string") from exc
|
|
||||||
|
|
||||||
ordered = [registry.skills_by_id[skill_id] for skill_id in registry.skills_in_load_order]
|
|
||||||
matches = [skill for skill in ordered if _skill_matches(skill, query=query, tag=tag, capability=capability)]
|
|
||||||
|
|
||||||
page = matches[start : start + normalized_limit]
|
|
||||||
next_cursor = start + normalized_limit
|
|
||||||
|
|
||||||
return {
|
|
||||||
"skills": [SkillSummaryPayload.from_record(skill).model_dump() for skill in page],
|
|
||||||
"total": len(matches),
|
|
||||||
"cursor": str(start),
|
|
||||||
"limit": normalized_limit,
|
|
||||||
"next_cursor": str(next_cursor) if next_cursor < len(matches) else None,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def build_skill_detail_payload(registry: DocsRegistry, skill_id: str) -> dict[str, Any]:
|
|
||||||
if skill_id not in registry.skills_by_id:
|
|
||||||
raise KeyError(skill_id)
|
|
||||||
|
|
||||||
skill = registry.skills_by_id[skill_id]
|
|
||||||
return {
|
|
||||||
"id": skill.skill_id,
|
|
||||||
"name": skill.name,
|
|
||||||
"description": skill.description,
|
|
||||||
"version": skill.version,
|
|
||||||
"tags": list(skill.tags),
|
|
||||||
"capabilities": list(skill.capabilities),
|
|
||||||
"resources": {
|
|
||||||
"document": skill.document_uri,
|
|
||||||
"references": {
|
|
||||||
ref_id: {
|
|
||||||
"uri": ref.uri,
|
|
||||||
"mime_type": ref.mime_type,
|
|
||||||
"title": ref.title,
|
|
||||||
"path": ref.relpath.as_posix(),
|
|
||||||
}
|
|
||||||
for ref_id, ref in sorted(skill.references.items())
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def build_prompts_index_payload(
|
def build_prompts_index_payload(
|
||||||
registry: DocsRegistry,
|
registry: DocsRegistry,
|
||||||
*,
|
*,
|
||||||
@@ -173,43 +85,6 @@ def build_prompt_detail_payload(registry: DocsRegistry, prompt_id: str) -> dict[
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def search_patterns_payload(
|
|
||||||
registry: DocsRegistry,
|
|
||||||
*,
|
|
||||||
query: str = "",
|
|
||||||
tags: list[str] | None = None,
|
|
||||||
skip: int = 0,
|
|
||||||
limit: int = DEFAULT_LIMIT,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
normalized_skip = max(skip, 0)
|
|
||||||
normalized_limit = max(1, min(limit, MAX_LIMIT))
|
|
||||||
|
|
||||||
requested_tags = [tag.strip() for tag in (tags or []) if tag and tag.strip()]
|
|
||||||
|
|
||||||
matches: list[SkillRecord] = []
|
|
||||||
for skill_id in registry.skills_in_load_order:
|
|
||||||
skill = registry.skills_by_id[skill_id]
|
|
||||||
if not _skill_matches(skill, query=query, tag=None, capability=None):
|
|
||||||
continue
|
|
||||||
if requested_tags and any(tag not in skill.tags for tag in requested_tags):
|
|
||||||
continue
|
|
||||||
matches.append(skill)
|
|
||||||
|
|
||||||
page = matches[normalized_skip : normalized_skip + normalized_limit]
|
|
||||||
return {
|
|
||||||
"patterns": [SkillPatternPayload.from_record(skill).model_dump() for skill in page],
|
|
||||||
"total": len(matches),
|
|
||||||
"skip": normalized_skip,
|
|
||||||
"limit": normalized_limit,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def get_pattern_by_id_payload(registry: DocsRegistry, skill_id: str) -> dict[str, Any]:
|
|
||||||
if skill_id not in registry.skills_by_id:
|
|
||||||
return {"found": False, "id": skill_id}
|
|
||||||
return {"found": True, "pattern": SkillPatternPayload.from_record(registry.skills_by_id[skill_id]).model_dump()}
|
|
||||||
|
|
||||||
|
|
||||||
def search_prompts_payload(
|
def search_prompts_payload(
|
||||||
registry: DocsRegistry,
|
registry: DocsRegistry,
|
||||||
*,
|
*,
|
||||||
|
|||||||
+2
-96
@@ -14,18 +14,13 @@ from fastmcp.server.transforms.search import RegexSearchTransform
|
|||||||
|
|
||||||
from personal_mcp.catalog.server import build_prompt_detail_payload
|
from personal_mcp.catalog.server import build_prompt_detail_payload
|
||||||
from personal_mcp.catalog.server import build_prompts_index_payload
|
from personal_mcp.catalog.server import build_prompts_index_payload
|
||||||
from personal_mcp.catalog.server import build_skill_detail_payload
|
|
||||||
from personal_mcp.catalog.server import build_skills_index_payload
|
|
||||||
from personal_mcp.catalog.server import get_pattern_by_id_payload
|
|
||||||
from personal_mcp.catalog.server import get_prompt_by_id_payload
|
from personal_mcp.catalog.server import get_prompt_by_id_payload
|
||||||
from personal_mcp.catalog.server import search_patterns_payload
|
|
||||||
from personal_mcp.catalog.server import search_prompts_payload
|
from personal_mcp.catalog.server import search_prompts_payload
|
||||||
from personal_mcp.registry.load import get_docs_registry
|
from personal_mcp.registry.load import get_docs_registry
|
||||||
from personal_mcp.registry.models.registry import DocsRegistry
|
from personal_mcp.registry.models.registry import DocsRegistry
|
||||||
from personal_mcp.registry.read import read_docs_markdown_path
|
from personal_mcp.registry.read import read_docs_markdown_path
|
||||||
from personal_mcp.registry.read import read_prompt_document
|
from personal_mcp.registry.read import read_prompt_document
|
||||||
from personal_mcp.registry.read import read_skill_document
|
from personal_mcp.skills import create_skills_provider
|
||||||
from personal_mcp.registry.read import read_skill_reference
|
|
||||||
|
|
||||||
TOOL_SEARCH_MODE = os.getenv("PERSONAL_MCP_TOOL_SEARCH", "none").strip().lower()
|
TOOL_SEARCH_MODE = os.getenv("PERSONAL_MCP_TOOL_SEARCH", "none").strip().lower()
|
||||||
TOOL_SEARCH_MAX_RESULTS = os.getenv("PERSONAL_MCP_TOOL_SEARCH_MAX_RESULTS", "5")
|
TOOL_SEARCH_MAX_RESULTS = os.getenv("PERSONAL_MCP_TOOL_SEARCH_MAX_RESULTS", "5")
|
||||||
@@ -123,64 +118,6 @@ def _register_prompt_objects(mcp: FastMCP, registry: DocsRegistry) -> None:
|
|||||||
|
|
||||||
|
|
||||||
def _register_components(mcp: FastMCP, registry: DocsRegistry) -> None:
|
def _register_components(mcp: FastMCP, registry: DocsRegistry) -> None:
|
||||||
@mcp.resource(
|
|
||||||
"resource://catalog/skills_index",
|
|
||||||
mime_type="application/json",
|
|
||||||
tags={"catalog"},
|
|
||||||
annotations=_ro_annotations(),
|
|
||||||
)
|
|
||||||
def skills_index() -> dict[str, Any]:
|
|
||||||
return build_skills_index_payload(registry)
|
|
||||||
|
|
||||||
@mcp.resource(
|
|
||||||
"resource://catalog/skills_index{?q,tag,capability,cursor,limit}",
|
|
||||||
mime_type="application/json",
|
|
||||||
tags={"catalog"},
|
|
||||||
annotations=_ro_annotations(),
|
|
||||||
)
|
|
||||||
def skills_index_query(
|
|
||||||
q: str | None = None,
|
|
||||||
tag: str | None = None,
|
|
||||||
capability: str | None = None,
|
|
||||||
cursor: str | None = None,
|
|
||||||
limit: int | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
return build_skills_index_payload(
|
|
||||||
registry,
|
|
||||||
query=q,
|
|
||||||
tag=tag,
|
|
||||||
capability=capability,
|
|
||||||
cursor=cursor,
|
|
||||||
limit=limit,
|
|
||||||
)
|
|
||||||
|
|
||||||
@mcp.resource(
|
|
||||||
"resource://catalog/skills/{skill_id}",
|
|
||||||
mime_type="application/json",
|
|
||||||
tags={"catalog"},
|
|
||||||
annotations=_ro_annotations(),
|
|
||||||
)
|
|
||||||
def skill_detail(skill_id: str) -> dict[str, Any]:
|
|
||||||
return build_skill_detail_payload(registry, skill_id)
|
|
||||||
|
|
||||||
@mcp.resource(
|
|
||||||
"resource://skills/{skill_id}/document",
|
|
||||||
mime_type="text/markdown",
|
|
||||||
tags={"skill-doc"},
|
|
||||||
annotations=_ro_annotations(),
|
|
||||||
)
|
|
||||||
def skill_document(skill_id: str) -> dict[str, str]:
|
|
||||||
return read_skill_document(registry, skill_id)
|
|
||||||
|
|
||||||
@mcp.resource(
|
|
||||||
"resource://skills/{skill_id}/references/{ref_id}",
|
|
||||||
mime_type="text/markdown",
|
|
||||||
tags={"reference"},
|
|
||||||
annotations=_ro_annotations(),
|
|
||||||
)
|
|
||||||
def skill_reference(skill_id: str, ref_id: str) -> dict[str, str]:
|
|
||||||
return read_skill_reference(registry, skill_id=skill_id, ref_id=ref_id)
|
|
||||||
|
|
||||||
@mcp.resource(
|
@mcp.resource(
|
||||||
"resource://docs/{path*}",
|
"resource://docs/{path*}",
|
||||||
mime_type="text/markdown",
|
mime_type="text/markdown",
|
||||||
@@ -237,38 +174,6 @@ def _register_components(mcp: FastMCP, registry: DocsRegistry) -> None:
|
|||||||
def prompt_document(prompt_id: str) -> dict[str, str]:
|
def prompt_document(prompt_id: str) -> dict[str, str]:
|
||||||
return read_prompt_document(registry, prompt_id)
|
return read_prompt_document(registry, prompt_id)
|
||||||
|
|
||||||
@mcp.tool
|
|
||||||
def search_patterns(
|
|
||||||
query: str = "",
|
|
||||||
tags: list[str] | None = None,
|
|
||||||
skip: int = 0,
|
|
||||||
limit: int = 20,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Search normalized pattern metadata with optional tags and pagination."""
|
|
||||||
return search_patterns_payload(
|
|
||||||
registry,
|
|
||||||
query=query,
|
|
||||||
tags=tags,
|
|
||||||
skip=skip,
|
|
||||||
limit=limit,
|
|
||||||
)
|
|
||||||
|
|
||||||
@mcp.tool
|
|
||||||
def get_pattern_by_id(id: str) -> dict[str, Any]:
|
|
||||||
"""Return one normalized pattern by stable id."""
|
|
||||||
return get_pattern_by_id_payload(registry, id)
|
|
||||||
|
|
||||||
@mcp.tool
|
|
||||||
def get_skill_document_by_id(skill_id: str) -> dict[str, Any]:
|
|
||||||
"""Return the canonical skill document payload for a stable skill id."""
|
|
||||||
if skill_id not in registry.skills_by_id:
|
|
||||||
return {"found": False, "id": skill_id}
|
|
||||||
|
|
||||||
return {
|
|
||||||
"found": True,
|
|
||||||
"document": read_skill_document(registry, skill_id),
|
|
||||||
}
|
|
||||||
|
|
||||||
@mcp.tool
|
@mcp.tool
|
||||||
def search_prompts(
|
def search_prompts(
|
||||||
query: str = "",
|
query: str = "",
|
||||||
@@ -296,5 +201,6 @@ def create_mcp() -> FastMCP:
|
|||||||
mcp = FastMCP("personal-mcp", on_duplicate="error")
|
mcp = FastMCP("personal-mcp", on_duplicate="error")
|
||||||
_register_components(mcp, registry)
|
_register_components(mcp, registry)
|
||||||
_register_prompt_objects(mcp, registry)
|
_register_prompt_objects(mcp, registry)
|
||||||
|
mcp.add_provider(create_skills_provider())
|
||||||
_install_tool_fallback_transforms(mcp)
|
_install_tool_fallback_transforms(mcp)
|
||||||
return mcp
|
return mcp
|
||||||
|
|||||||
@@ -1,15 +1,9 @@
|
|||||||
from .models.registry import DocsRegistry
|
from .models.registry import DocsRegistry
|
||||||
from .models.registry import PromptRecord
|
from .models.registry import PromptRecord
|
||||||
from .models.registry import PromptSummaryRecord
|
from .models.registry import PromptSummaryRecord
|
||||||
from .models.registry import ReferenceRecord
|
|
||||||
from .models.registry import SkillRecord
|
|
||||||
from .models.registry import SkillSummaryRecord
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"DocsRegistry",
|
"DocsRegistry",
|
||||||
"PromptRecord",
|
"PromptRecord",
|
||||||
"PromptSummaryRecord",
|
"PromptSummaryRecord",
|
||||||
"ReferenceRecord",
|
|
||||||
"SkillRecord",
|
|
||||||
"SkillSummaryRecord",
|
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -38,12 +38,6 @@ class MarkdownDocument:
|
|||||||
frontmatter = get_raw_frontmatter(raw)
|
frontmatter = get_raw_frontmatter(raw)
|
||||||
return cls(relpath=relpath, content=raw, frontmatter=frontmatter)
|
return cls(relpath=relpath, content=raw, frontmatter=frontmatter)
|
||||||
|
|
||||||
@property
|
|
||||||
def skill_slug(self) -> str | None:
|
|
||||||
parts = self.relpath.parts
|
|
||||||
if parts[0] == "skills" and len(parts) >= 3:
|
|
||||||
return parts[1]
|
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def prompt_slug(self) -> str | None:
|
def prompt_slug(self) -> str | None:
|
||||||
parts = self.relpath.parts
|
parts = self.relpath.parts
|
||||||
|
|||||||
@@ -1,125 +0,0 @@
|
|||||||
import re
|
|
||||||
from collections.abc import Iterable
|
|
||||||
from collections.abc import Mapping
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from importlib.resources.abc import Traversable
|
|
||||||
from itertools import groupby
|
|
||||||
from itertools import starmap
|
|
||||||
from pathlib import PurePosixPath
|
|
||||||
from typing import Self
|
|
||||||
|
|
||||||
from personal_mcp.registry.models.common import SKILL_ID_RE
|
|
||||||
from personal_mcp.registry.models.common import DocsPath
|
|
||||||
from personal_mcp.registry.models.common import ReferenceEntry
|
|
||||||
from personal_mcp.registry.models.skill import SkillFrontmatter
|
|
||||||
from personal_mcp.registry.models.skill import StoredSkill
|
|
||||||
from personal_mcp.registry.models.skill import StoredSkillReference
|
|
||||||
|
|
||||||
from .document import MarkdownDocument
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True, slots=True)
|
|
||||||
class SkillFilesBundle:
|
|
||||||
"""Represents a skill and all of its associated markdown files."""
|
|
||||||
|
|
||||||
slug: str
|
|
||||||
skill: MarkdownDocument
|
|
||||||
references: tuple[MarkdownDocument, ...]
|
|
||||||
other: tuple[MarkdownDocument, ...]
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_root(cls, root: Traversable) -> list[Self]:
|
|
||||||
# Should only be used for testing
|
|
||||||
return list(cls.from_docs(MarkdownDocument.from_root(root).values()))
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_docs(cls, docs: Iterable[MarkdownDocument]) -> tuple[Self, ...]:
|
|
||||||
return tuple(starmap(cls.from_paths, group_skill_paths(docs).items()))
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_paths(cls, slug: str, paths: set[MarkdownDocument]) -> Self:
|
|
||||||
skill = next(iter(p for p in paths if p.relpath.name == "SKILL.md"))
|
|
||||||
sorted_paths = tuple(sorted(paths, key=lambda p: p.relpath))
|
|
||||||
references_dir = PurePosixPath("skills", slug, "references")
|
|
||||||
references = tuple(p for p in sorted_paths if p.relpath.parent == references_dir)
|
|
||||||
other = tuple(p for p in sorted_paths if p not in references and p != skill)
|
|
||||||
return cls(
|
|
||||||
slug=slug,
|
|
||||||
skill=skill,
|
|
||||||
references=references,
|
|
||||||
other=other,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def group_skill_paths(docs: Iterable[MarkdownDocument]) -> dict[str, set[MarkdownDocument]]:
|
|
||||||
"""Group skills from a list of markdown documents by their skill slug."""
|
|
||||||
s = sorted(
|
|
||||||
filter(lambda d: d.skill_slug is not None, docs),
|
|
||||||
key=lambda d: (d.skill_slug or "", d.relpath.stem),
|
|
||||||
)
|
|
||||||
grouped = groupby(s, key=lambda doc: doc.skill_slug)
|
|
||||||
return {k: set(g) for k, g in grouped if k}
|
|
||||||
|
|
||||||
|
|
||||||
def _title_from_reference_filename(filename: str) -> str:
|
|
||||||
stem = PurePosixPath(filename).stem
|
|
||||||
normalized = stem.replace("-", " ").replace("_", " ").split()
|
|
||||||
if not normalized:
|
|
||||||
return stem
|
|
||||||
return " ".join(token.capitalize() for token in normalized)
|
|
||||||
|
|
||||||
|
|
||||||
def _reference_id_from_filename(filename: str) -> str | None:
|
|
||||||
stem = PurePosixPath(filename).stem.strip().lower().replace("_", "-")
|
|
||||||
normalized = re.sub(r"[^a-z0-9-]+", "-", stem)
|
|
||||||
normalized = re.sub(r"-+", "-", normalized).strip("-")
|
|
||||||
if not normalized or not SKILL_ID_RE.fullmatch(normalized):
|
|
||||||
return None
|
|
||||||
return normalized
|
|
||||||
|
|
||||||
|
|
||||||
def _discover_reference_entries(bundle: SkillFilesBundle) -> dict[str, ReferenceEntry]:
|
|
||||||
discovered: dict[str, ReferenceEntry] = {}
|
|
||||||
for reference_doc in bundle.references:
|
|
||||||
ref_id = _reference_id_from_filename(reference_doc.relpath.name)
|
|
||||||
if ref_id is None:
|
|
||||||
continue
|
|
||||||
discovered[ref_id] = ReferenceEntry(
|
|
||||||
path=PurePosixPath("references", reference_doc.relpath.name),
|
|
||||||
title=_title_from_reference_filename(reference_doc.relpath.name),
|
|
||||||
)
|
|
||||||
return discovered
|
|
||||||
|
|
||||||
|
|
||||||
def build_stored_skill(
|
|
||||||
*,
|
|
||||||
bundle: SkillFilesBundle,
|
|
||||||
docs_by_relpath: Mapping[DocsPath, MarkdownDocument],
|
|
||||||
) -> StoredSkill:
|
|
||||||
frontmatter = SkillFrontmatter.from_raw_yaml(bundle.skill.frontmatter)
|
|
||||||
metadata = frontmatter.x_personal_mcp
|
|
||||||
merged_entries = _discover_reference_entries(bundle)
|
|
||||||
merged_entries.update(dict(metadata.references))
|
|
||||||
|
|
||||||
references: dict[str, StoredSkillReference] = {}
|
|
||||||
for ref_id, entry in sorted(merged_entries.items()):
|
|
||||||
ref_relpath = PurePosixPath("skills", bundle.slug, entry.path)
|
|
||||||
if ref_relpath not in docs_by_relpath:
|
|
||||||
raise KeyError(f"reference document not found for '{metadata.id}:{ref_id}' at {ref_relpath.as_posix()}")
|
|
||||||
ref_doc = docs_by_relpath[ref_relpath]
|
|
||||||
references[ref_id] = StoredSkillReference(
|
|
||||||
ref_id=ref_id,
|
|
||||||
relpath=ref_relpath,
|
|
||||||
content=ref_doc.content,
|
|
||||||
entry=entry,
|
|
||||||
)
|
|
||||||
|
|
||||||
return StoredSkill.model_validate(
|
|
||||||
{
|
|
||||||
"skill_id": metadata.id,
|
|
||||||
"relpath": bundle.skill.relpath,
|
|
||||||
"content": bundle.skill.content,
|
|
||||||
"frontmatter": frontmatter,
|
|
||||||
"references": references,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
@@ -6,44 +6,10 @@ from importlib.resources import files
|
|||||||
|
|
||||||
from personal_mcp.registry.ingest.document import MarkdownDocument
|
from personal_mcp.registry.ingest.document import MarkdownDocument
|
||||||
from personal_mcp.registry.ingest.prompt import PromptFilesBundle
|
from personal_mcp.registry.ingest.prompt import PromptFilesBundle
|
||||||
from personal_mcp.registry.ingest.skill import SkillFilesBundle
|
|
||||||
from personal_mcp.registry.ingest.skill import build_stored_skill
|
|
||||||
from personal_mcp.registry.models.common import DocsPath
|
|
||||||
from personal_mcp.registry.models.prompt import StoredPrompt
|
from personal_mcp.registry.models.prompt import StoredPrompt
|
||||||
from personal_mcp.registry.models.registry import DocsRegistry
|
from personal_mcp.registry.models.registry import DocsRegistry
|
||||||
from personal_mcp.registry.models.registry import PromptRecord
|
from personal_mcp.registry.models.registry import PromptRecord
|
||||||
from personal_mcp.registry.models.registry import PromptSummaryRecord
|
from personal_mcp.registry.models.registry import PromptSummaryRecord
|
||||||
from personal_mcp.registry.models.registry import ReferenceRecord
|
|
||||||
from personal_mcp.registry.models.registry import SkillRecord
|
|
||||||
from personal_mcp.registry.models.registry import SkillSummaryRecord
|
|
||||||
|
|
||||||
|
|
||||||
def _build_skill_record(*, bundle: SkillFilesBundle, docs_by_relpath: dict[DocsPath, MarkdownDocument]) -> SkillRecord:
|
|
||||||
stored = build_stored_skill(bundle=bundle, docs_by_relpath=docs_by_relpath)
|
|
||||||
metadata = stored.frontmatter.x_personal_mcp
|
|
||||||
references: dict[str, ReferenceRecord] = {}
|
|
||||||
for ref_id, ref in sorted(stored.references.items()):
|
|
||||||
references[ref_id] = ReferenceRecord(
|
|
||||||
ref_id=ref_id,
|
|
||||||
uri=f"resource://skills/{metadata.id}/references/{ref_id}",
|
|
||||||
relpath=ref.relpath,
|
|
||||||
mime_type=ref.entry.mime_type,
|
|
||||||
title=ref.entry.title,
|
|
||||||
content=ref.content,
|
|
||||||
)
|
|
||||||
|
|
||||||
return SkillRecord(
|
|
||||||
skill_id=metadata.id,
|
|
||||||
name=stored.frontmatter.name,
|
|
||||||
description=stored.frontmatter.description,
|
|
||||||
version=metadata.version,
|
|
||||||
tags=tuple(metadata.tags),
|
|
||||||
capabilities=tuple(metadata.capabilities),
|
|
||||||
document_uri=f"resource://skills/{metadata.id}/document",
|
|
||||||
document_relpath=stored.relpath,
|
|
||||||
document_content=stored.content,
|
|
||||||
references=references,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _build_prompt_record(*, bundle: PromptFilesBundle) -> PromptRecord:
|
def _build_prompt_record(*, bundle: PromptFilesBundle) -> PromptRecord:
|
||||||
@@ -64,26 +30,6 @@ def _build_prompt_record(*, bundle: PromptFilesBundle) -> PromptRecord:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _build_tag_index_skills(
|
|
||||||
skills_in_order: tuple[str, ...], skills_by_id: dict[str, SkillRecord]
|
|
||||||
) -> dict[str, tuple[str, ...]]:
|
|
||||||
tag_index: defaultdict[str, list[str]] = defaultdict(list)
|
|
||||||
for skill_id in skills_in_order:
|
|
||||||
for tag in skills_by_id[skill_id].tags:
|
|
||||||
tag_index[tag].append(skill_id)
|
|
||||||
return {tag: tuple(ids) for tag, ids in sorted(tag_index.items())}
|
|
||||||
|
|
||||||
|
|
||||||
def _build_capability_index(
|
|
||||||
skills_in_order: tuple[str, ...], skills_by_id: dict[str, SkillRecord]
|
|
||||||
) -> dict[str, tuple[str, ...]]:
|
|
||||||
capability_index: defaultdict[str, list[str]] = defaultdict(list)
|
|
||||||
for skill_id in skills_in_order:
|
|
||||||
for capability in skills_by_id[skill_id].capabilities:
|
|
||||||
capability_index[capability].append(skill_id)
|
|
||||||
return {capability: tuple(ids) for capability, ids in sorted(capability_index.items())}
|
|
||||||
|
|
||||||
|
|
||||||
def _build_tag_index_prompts(
|
def _build_tag_index_prompts(
|
||||||
prompts_in_order: tuple[str, ...],
|
prompts_in_order: tuple[str, ...],
|
||||||
prompts_by_id: dict[str, PromptRecord],
|
prompts_by_id: dict[str, PromptRecord],
|
||||||
@@ -104,44 +50,22 @@ def get_docs_registry() -> DocsRegistry:
|
|||||||
|
|
||||||
docs_markdown_by_path = {relpath: doc.content for relpath, doc in docs.items()}
|
docs_markdown_by_path = {relpath: doc.content for relpath, doc in docs.items()}
|
||||||
|
|
||||||
skill_bundles = SkillFilesBundle.from_docs(docs.values())
|
|
||||||
prompt_bundles = PromptFilesBundle.from_docs(docs.values())
|
prompt_bundles = PromptFilesBundle.from_docs(docs.values())
|
||||||
|
|
||||||
docs_by_relpath = {doc.relpath: doc for doc in docs.values()}
|
|
||||||
|
|
||||||
skills_by_id: dict[str, SkillRecord] = {}
|
|
||||||
skills_in_load_order: list[str] = []
|
|
||||||
for bundle in skill_bundles:
|
|
||||||
record = _build_skill_record(bundle=bundle, docs_by_relpath=docs_by_relpath)
|
|
||||||
if record.skill_id in skills_by_id:
|
|
||||||
raise ValueError(f"duplicate skill_id detected: {record.skill_id}")
|
|
||||||
skills_by_id[record.skill_id] = record
|
|
||||||
skills_in_load_order.append(record.skill_id)
|
|
||||||
|
|
||||||
prompts_by_id: dict[str, PromptRecord] = {}
|
prompts_by_id: dict[str, PromptRecord] = {}
|
||||||
prompts_in_load_order: list[str] = []
|
prompts_in_load_order: list[str] = []
|
||||||
for bundle in prompt_bundles:
|
for bundle in prompt_bundles:
|
||||||
record = _build_prompt_record(bundle=bundle)
|
record = _build_prompt_record(bundle=bundle)
|
||||||
if record.prompt_id in prompts_by_id:
|
if record.prompt_id in prompts_by_id:
|
||||||
raise ValueError(f"duplicate prompt_id detected: {record.prompt_id}")
|
raise ValueError(f"duplicate prompt_id detected: {record.prompt_id}")
|
||||||
if record.prompt_id in skills_by_id:
|
|
||||||
raise ValueError(f"prompt_id collides with existing skill_id: {record.prompt_id}")
|
|
||||||
prompts_by_id[record.prompt_id] = record
|
prompts_by_id[record.prompt_id] = record
|
||||||
prompts_in_load_order.append(record.prompt_id)
|
prompts_in_load_order.append(record.prompt_id)
|
||||||
|
|
||||||
skills_in_order_tuple = tuple(skills_in_load_order)
|
|
||||||
prompts_in_order_tuple = tuple(prompts_in_load_order)
|
prompts_in_order_tuple = tuple(prompts_in_load_order)
|
||||||
|
|
||||||
return DocsRegistry(
|
return DocsRegistry(
|
||||||
skills_by_id=skills_by_id,
|
|
||||||
skills_in_load_order=skills_in_order_tuple,
|
|
||||||
skills_summary_in_load_order=tuple(
|
|
||||||
SkillSummaryRecord.from_record(skills_by_id[skill_id]) for skill_id in skills_in_order_tuple
|
|
||||||
),
|
|
||||||
docs_markdown_by_path=docs_markdown_by_path,
|
docs_markdown_by_path=docs_markdown_by_path,
|
||||||
docs_markdown_path_index=tuple(sorted(docs_markdown_by_path)),
|
docs_markdown_path_index=tuple(sorted(docs_markdown_by_path)),
|
||||||
tag_to_skill_ids=_build_tag_index_skills(skills_in_order_tuple, skills_by_id),
|
|
||||||
capability_to_skill_ids=_build_capability_index(skills_in_order_tuple, skills_by_id),
|
|
||||||
prompts_by_id=prompts_by_id,
|
prompts_by_id=prompts_by_id,
|
||||||
prompts_in_load_order=prompts_in_order_tuple,
|
prompts_in_load_order=prompts_in_order_tuple,
|
||||||
prompts_summary_in_load_order=tuple(
|
prompts_summary_in_load_order=tuple(
|
||||||
|
|||||||
@@ -45,20 +45,4 @@ def parse_docs_path(value: str | PurePosixPath) -> PurePosixPath:
|
|||||||
return path
|
return path
|
||||||
|
|
||||||
|
|
||||||
def parse_reference_path(value: str | PurePosixPath) -> PurePosixPath:
|
|
||||||
path = parse_docs_path(value)
|
|
||||||
if len(path.parts) < 2 or path.parts[0] != "references":
|
|
||||||
raise ValueError("reference path must stay under references/")
|
|
||||||
return path
|
|
||||||
|
|
||||||
|
|
||||||
type DocsPath = Annotated[PurePosixPath, BeforeValidator(parse_docs_path)]
|
type DocsPath = Annotated[PurePosixPath, BeforeValidator(parse_docs_path)]
|
||||||
type ReferencePath = Annotated[PurePosixPath, BeforeValidator(parse_reference_path)]
|
|
||||||
|
|
||||||
|
|
||||||
class ReferenceEntry(StrictFrozenModel):
|
|
||||||
"""Reference metadata for a markdown file within a skill."""
|
|
||||||
|
|
||||||
path: ReferencePath
|
|
||||||
mime_type: str = "text/markdown"
|
|
||||||
title: str | None = None
|
|
||||||
|
|||||||
@@ -13,61 +13,6 @@ def _empty_docs_mapping() -> Mapping[DocsPath, str]:
|
|||||||
return frozen_mapping()
|
return frozen_mapping()
|
||||||
|
|
||||||
|
|
||||||
class ReferenceRecord(StrictFrozenModel):
|
|
||||||
"""Registry record for a resolved skill reference document."""
|
|
||||||
|
|
||||||
ref_id: str
|
|
||||||
uri: str
|
|
||||||
relpath: DocsPath
|
|
||||||
mime_type: str
|
|
||||||
title: str | None
|
|
||||||
content: str
|
|
||||||
|
|
||||||
|
|
||||||
class SkillRecord(StrictFrozenModel):
|
|
||||||
"""Registry record containing a fully resolved skill and references."""
|
|
||||||
|
|
||||||
skill_id: str
|
|
||||||
name: str
|
|
||||||
description: str
|
|
||||||
version: str
|
|
||||||
tags: tuple[str, ...]
|
|
||||||
capabilities: tuple[str, ...]
|
|
||||||
document_uri: str
|
|
||||||
document_relpath: DocsPath
|
|
||||||
document_content: str
|
|
||||||
references: Mapping[str, ReferenceRecord] = Field(default_factory=frozen_mapping)
|
|
||||||
|
|
||||||
@field_validator("references", mode="before")
|
|
||||||
@classmethod
|
|
||||||
def freeze_references(cls, value: Mapping[str, ReferenceRecord] | None) -> Mapping[str, ReferenceRecord]:
|
|
||||||
return frozen_mapping(value)
|
|
||||||
|
|
||||||
|
|
||||||
class SkillSummaryRecord(StrictFrozenModel):
|
|
||||||
"""Compact skill summary exposed by catalog listing APIs."""
|
|
||||||
|
|
||||||
skill_id: str
|
|
||||||
name: str
|
|
||||||
description: str
|
|
||||||
tags: tuple[str, ...]
|
|
||||||
capabilities: tuple[str, ...]
|
|
||||||
document_uri: str
|
|
||||||
version: str
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_record(cls, record: SkillRecord) -> "SkillSummaryRecord":
|
|
||||||
return cls(
|
|
||||||
skill_id=record.skill_id,
|
|
||||||
name=record.name,
|
|
||||||
description=record.description,
|
|
||||||
tags=record.tags,
|
|
||||||
capabilities=record.capabilities,
|
|
||||||
document_uri=record.document_uri,
|
|
||||||
version=record.version,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class PromptRecord(StrictFrozenModel):
|
class PromptRecord(StrictFrozenModel):
|
||||||
"""Registry record containing a fully resolved prompt document."""
|
"""Registry record containing a fully resolved prompt document."""
|
||||||
|
|
||||||
@@ -112,63 +57,6 @@ class PromptSummaryRecord(StrictFrozenModel):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
class SkillPatternPayload(StrictFrozenModel):
|
|
||||||
"""Catalog payload model for skill pattern search results."""
|
|
||||||
|
|
||||||
id: str
|
|
||||||
name: str
|
|
||||||
version: str
|
|
||||||
description: str
|
|
||||||
tags: list[str]
|
|
||||||
capabilities: list[str]
|
|
||||||
resources: list[str]
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_record(cls, record: SkillRecord) -> "SkillPatternPayload":
|
|
||||||
return cls(
|
|
||||||
id=record.skill_id,
|
|
||||||
name=record.name,
|
|
||||||
version=record.version,
|
|
||||||
description=record.description,
|
|
||||||
tags=list(record.tags),
|
|
||||||
capabilities=list(record.capabilities),
|
|
||||||
resources=list(record.capabilities),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class SkillSummaryPayload(StrictFrozenModel):
|
|
||||||
"""Catalog payload model for skill index summaries."""
|
|
||||||
|
|
||||||
id: str
|
|
||||||
name: str
|
|
||||||
description: str
|
|
||||||
tags: list[str]
|
|
||||||
capabilities: list[str]
|
|
||||||
version: str
|
|
||||||
document_uri: str
|
|
||||||
detail_uri: str
|
|
||||||
resources: dict[str, str | list[str]]
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_record(cls, record: SkillRecord) -> "SkillSummaryPayload":
|
|
||||||
return cls(
|
|
||||||
id=record.skill_id,
|
|
||||||
name=record.name,
|
|
||||||
description=record.description,
|
|
||||||
tags=list(record.tags),
|
|
||||||
capabilities=list(record.capabilities),
|
|
||||||
version=record.version,
|
|
||||||
document_uri=record.document_uri,
|
|
||||||
detail_uri=f"resource://catalog/skills/{record.skill_id}",
|
|
||||||
resources={
|
|
||||||
"document": record.document_uri,
|
|
||||||
"references": [
|
|
||||||
f"resource://skills/{record.skill_id}/references/{ref_id}" for ref_id in sorted(record.references)
|
|
||||||
],
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class PromptSummaryPayload(StrictFrozenModel):
|
class PromptSummaryPayload(StrictFrozenModel):
|
||||||
"""Catalog payload model for prompt index summaries."""
|
"""Catalog payload model for prompt index summaries."""
|
||||||
|
|
||||||
@@ -196,22 +84,12 @@ class PromptSummaryPayload(StrictFrozenModel):
|
|||||||
|
|
||||||
|
|
||||||
class DocsRegistry(StrictFrozenModel):
|
class DocsRegistry(StrictFrozenModel):
|
||||||
"""In-memory index of loaded skills, prompts, and docs content."""
|
"""In-memory index of loaded prompts and documentation content."""
|
||||||
|
|
||||||
skills_by_id: Mapping[str, SkillRecord] = Field(default_factory=frozen_mapping)
|
|
||||||
"""Maps each skill identifier to its fully resolved registry record."""
|
|
||||||
skills_in_load_order: tuple[str, ...]
|
|
||||||
"""Preserves skill identifiers in deterministic source loading order."""
|
|
||||||
skills_summary_in_load_order: tuple[SkillSummaryRecord, ...]
|
|
||||||
"""Stores compact skill summaries in the same deterministic loading order."""
|
|
||||||
docs_markdown_by_path: Mapping[DocsPath, str] = Field(default_factory=_empty_docs_mapping)
|
docs_markdown_by_path: Mapping[DocsPath, str] = Field(default_factory=_empty_docs_mapping)
|
||||||
"""Maps each documentation path to its loaded Markdown content."""
|
"""Maps each documentation path to its loaded Markdown content."""
|
||||||
docs_markdown_path_index: tuple[DocsPath, ...]
|
docs_markdown_path_index: tuple[DocsPath, ...]
|
||||||
"""Lists documentation paths in deterministic index order."""
|
"""Lists documentation paths in deterministic index order."""
|
||||||
tag_to_skill_ids: Mapping[str, tuple[str, ...]] = Field(default_factory=frozen_mapping)
|
|
||||||
"""Indexes skill identifiers by tag for catalog filtering and search."""
|
|
||||||
capability_to_skill_ids: Mapping[str, tuple[str, ...]] = Field(default_factory=frozen_mapping)
|
|
||||||
"""Indexes skill identifiers by the capabilities they provide."""
|
|
||||||
prompts_by_id: Mapping[str, PromptRecord] = Field(default_factory=frozen_mapping)
|
prompts_by_id: Mapping[str, PromptRecord] = Field(default_factory=frozen_mapping)
|
||||||
"""Maps each prompt identifier to its fully resolved registry record."""
|
"""Maps each prompt identifier to its fully resolved registry record."""
|
||||||
prompts_in_load_order: tuple[str, ...] = ()
|
prompts_in_load_order: tuple[str, ...] = ()
|
||||||
@@ -222,10 +100,7 @@ class DocsRegistry(StrictFrozenModel):
|
|||||||
"""Indexes prompt identifiers by tag for catalog filtering and search."""
|
"""Indexes prompt identifiers by tag for catalog filtering and search."""
|
||||||
|
|
||||||
@field_validator(
|
@field_validator(
|
||||||
"skills_by_id",
|
|
||||||
"docs_markdown_by_path",
|
"docs_markdown_by_path",
|
||||||
"tag_to_skill_ids",
|
|
||||||
"capability_to_skill_ids",
|
|
||||||
"prompts_by_id",
|
"prompts_by_id",
|
||||||
"tag_to_prompt_ids",
|
"tag_to_prompt_ids",
|
||||||
mode="before",
|
mode="before",
|
||||||
|
|||||||
@@ -1,142 +0,0 @@
|
|||||||
from collections.abc import Mapping
|
|
||||||
|
|
||||||
import yaml
|
|
||||||
from pydantic import Field
|
|
||||||
from pydantic import field_validator
|
|
||||||
from pydantic import model_validator
|
|
||||||
|
|
||||||
from .common import SEMVER_RE
|
|
||||||
from .common import SKILL_ID_RE
|
|
||||||
from .common import DocsPath
|
|
||||||
from .common import ReferenceEntry
|
|
||||||
from .common import StrictFrozenModel
|
|
||||||
from .common import frozen_mapping
|
|
||||||
|
|
||||||
|
|
||||||
class SkillMetadata(StrictFrozenModel):
|
|
||||||
"""Canonical metadata describing a skill."""
|
|
||||||
|
|
||||||
id: str
|
|
||||||
version: str
|
|
||||||
tags: tuple[str, ...] = ()
|
|
||||||
capabilities: tuple[str, ...] = Field(min_length=1)
|
|
||||||
references: Mapping[str, ReferenceEntry] = Field(default_factory=frozen_mapping)
|
|
||||||
|
|
||||||
@field_validator("id")
|
|
||||||
@classmethod
|
|
||||||
def validate_id(cls, value: str) -> str:
|
|
||||||
if not SKILL_ID_RE.fullmatch(value):
|
|
||||||
raise ValueError("id must be lowercase kebab-case and start with a letter")
|
|
||||||
return value
|
|
||||||
|
|
||||||
@field_validator("version")
|
|
||||||
@classmethod
|
|
||||||
def validate_version(cls, value: str) -> str:
|
|
||||||
if not SEMVER_RE.fullmatch(value):
|
|
||||||
raise ValueError("version must be semver")
|
|
||||||
return value
|
|
||||||
|
|
||||||
@field_validator("tags")
|
|
||||||
@classmethod
|
|
||||||
def validate_tags(cls, value: tuple[str, ...]) -> tuple[str, ...]:
|
|
||||||
for tag in value:
|
|
||||||
if not SKILL_ID_RE.fullmatch(tag):
|
|
||||||
raise ValueError(f"invalid tag: {tag}")
|
|
||||||
return value
|
|
||||||
|
|
||||||
@field_validator("references", mode="before")
|
|
||||||
@classmethod
|
|
||||||
def freeze_references(cls, value: Mapping[str, ReferenceEntry] | None) -> Mapping[str, ReferenceEntry]:
|
|
||||||
return frozen_mapping(value)
|
|
||||||
|
|
||||||
@field_validator("references")
|
|
||||||
@classmethod
|
|
||||||
def validate_reference_ids(cls, value: Mapping[str, ReferenceEntry]) -> Mapping[str, ReferenceEntry]:
|
|
||||||
for ref_id in value:
|
|
||||||
if not SKILL_ID_RE.fullmatch(ref_id):
|
|
||||||
raise ValueError(f"invalid reference id: {ref_id}")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
class SkillFrontmatter(StrictFrozenModel):
|
|
||||||
"""Parsed SKILL frontmatter including standard and personal-mcp fields."""
|
|
||||||
|
|
||||||
name: str = Field(min_length=1, max_length=64)
|
|
||||||
description: str = Field(min_length=1, max_length=1024)
|
|
||||||
x_personal_mcp: SkillMetadata = Field(alias="x-personal-mcp")
|
|
||||||
|
|
||||||
@field_validator("name")
|
|
||||||
@classmethod
|
|
||||||
def validate_name(cls, value: str) -> str:
|
|
||||||
if not SKILL_ID_RE.fullmatch(value):
|
|
||||||
raise ValueError("name must be lowercase kebab-case and start with a letter")
|
|
||||||
if "anthropic" in value or "claude" in value:
|
|
||||||
raise ValueError("name must not contain reserved words anthropic or claude")
|
|
||||||
return value
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_raw_yaml(cls, raw: str | None) -> "SkillFrontmatter":
|
|
||||||
if raw is None:
|
|
||||||
raise ValueError("missing YAML frontmatter")
|
|
||||||
try:
|
|
||||||
data = yaml.safe_load(raw)
|
|
||||||
except yaml.YAMLError as e:
|
|
||||||
raise ValueError(f"invalid YAML in frontmatter: {e}") from e
|
|
||||||
if not isinstance(data, dict):
|
|
||||||
raise TypeError("frontmatter must parse to an object")
|
|
||||||
return cls.model_validate(data)
|
|
||||||
|
|
||||||
|
|
||||||
class StoredSkillReference(StrictFrozenModel):
|
|
||||||
"""Structured representation of a skill reference markdown document."""
|
|
||||||
|
|
||||||
ref_id: str
|
|
||||||
relpath: DocsPath
|
|
||||||
content: str
|
|
||||||
entry: ReferenceEntry
|
|
||||||
|
|
||||||
|
|
||||||
class StoredSkill(StrictFrozenModel):
|
|
||||||
"""Structured representation of a skill markdown document."""
|
|
||||||
|
|
||||||
skill_id: str
|
|
||||||
relpath: DocsPath
|
|
||||||
content: str
|
|
||||||
frontmatter: SkillFrontmatter
|
|
||||||
references: Mapping[str, StoredSkillReference] = Field(default_factory=frozen_mapping)
|
|
||||||
|
|
||||||
@field_validator("frontmatter", mode="before")
|
|
||||||
@classmethod
|
|
||||||
def parse_frontmatter_yaml(cls, value: SkillFrontmatter | str | None) -> SkillFrontmatter:
|
|
||||||
if isinstance(value, SkillFrontmatter):
|
|
||||||
return value
|
|
||||||
return SkillFrontmatter.from_raw_yaml(value)
|
|
||||||
|
|
||||||
@field_validator("references", mode="before")
|
|
||||||
@classmethod
|
|
||||||
def freeze_references(cls, value: Mapping[str, StoredSkillReference] | None) -> Mapping[str, StoredSkillReference]:
|
|
||||||
return frozen_mapping(value)
|
|
||||||
|
|
||||||
@model_validator(mode="after")
|
|
||||||
def validate_contract(self) -> "StoredSkill":
|
|
||||||
parts = self.relpath.parts
|
|
||||||
if len(parts) < 3 or parts[0] != "skills":
|
|
||||||
raise ValueError("skill relpath must be under skills/<slug>/")
|
|
||||||
|
|
||||||
skill_dir_name = parts[1]
|
|
||||||
if self.frontmatter.name != skill_dir_name:
|
|
||||||
raise ValueError("frontmatter name must exactly match skill directory name")
|
|
||||||
if self.frontmatter.x_personal_mcp.id != self.frontmatter.name:
|
|
||||||
raise ValueError("x-personal-mcp.id must exactly match name")
|
|
||||||
|
|
||||||
expected_capability = f"resource://skills/{self.frontmatter.name}/document"
|
|
||||||
if expected_capability not in self.frontmatter.x_personal_mcp.capabilities:
|
|
||||||
raise ValueError(f"capabilities must include {expected_capability}")
|
|
||||||
|
|
||||||
if self.skill_id != self.frontmatter.x_personal_mcp.id:
|
|
||||||
raise ValueError("skill_id must exactly match x-personal-mcp.id")
|
|
||||||
|
|
||||||
for ref_id, ref in self.references.items():
|
|
||||||
if ref.ref_id != ref_id:
|
|
||||||
raise ValueError(f"reference key must match ref_id: {ref_id}")
|
|
||||||
return self
|
|
||||||
@@ -2,41 +2,6 @@ from .models.common import parse_docs_path
|
|||||||
from .models.registry import DocsRegistry
|
from .models.registry import DocsRegistry
|
||||||
|
|
||||||
|
|
||||||
def read_skill_document(registry: DocsRegistry, skill_id: str) -> dict[str, str]:
|
|
||||||
if skill_id not in registry.skills_by_id:
|
|
||||||
raise KeyError(f"unknown skill_id: {skill_id}")
|
|
||||||
skill = registry.skills_by_id[skill_id]
|
|
||||||
return {
|
|
||||||
"id": skill.skill_id,
|
|
||||||
"uri": skill.document_uri,
|
|
||||||
"format": "markdown",
|
|
||||||
"source_path": f"docs/{skill.document_relpath.as_posix()}",
|
|
||||||
"content": skill.document_content,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def read_skill_reference(
|
|
||||||
registry: DocsRegistry,
|
|
||||||
*,
|
|
||||||
skill_id: str,
|
|
||||||
ref_id: str,
|
|
||||||
) -> dict[str, str]:
|
|
||||||
if skill_id not in registry.skills_by_id:
|
|
||||||
raise KeyError(f"unknown skill_id: {skill_id}")
|
|
||||||
skill = registry.skills_by_id[skill_id]
|
|
||||||
if ref_id not in skill.references:
|
|
||||||
raise KeyError(f"unknown ref_id '{ref_id}' for skill '{skill_id}'")
|
|
||||||
reference = skill.references[ref_id]
|
|
||||||
return {
|
|
||||||
"id": ref_id,
|
|
||||||
"skill_id": skill_id,
|
|
||||||
"uri": reference.uri,
|
|
||||||
"format": "markdown",
|
|
||||||
"source_path": f"docs/{reference.relpath.as_posix()}",
|
|
||||||
"content": reference.content,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def read_docs_markdown_path(registry: DocsRegistry, path: str) -> dict[str, str]:
|
def read_docs_markdown_path(registry: DocsRegistry, path: str) -> dict[str, str]:
|
||||||
docs_path = parse_docs_path(path)
|
docs_path = parse_docs_path(path)
|
||||||
if docs_path not in registry.docs_markdown_by_path:
|
if docs_path not in registry.docs_markdown_by_path:
|
||||||
|
|||||||
@@ -1 +1,5 @@
|
|||||||
"""Docs registry and markdown loading utilities for personal MCP skills."""
|
"""FastMCP provider composition for personal MCP skills."""
|
||||||
|
|
||||||
|
from .provider import create_skills_provider
|
||||||
|
|
||||||
|
__all__ = ["create_skills_provider"]
|
||||||
|
|||||||
@@ -0,0 +1,20 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from fastmcp.server.providers.skills import SkillsDirectoryProvider
|
||||||
|
|
||||||
|
|
||||||
|
def create_skills_provider() -> SkillsDirectoryProvider:
|
||||||
|
"""Create the provider for skills packaged with personal-mcp."""
|
||||||
|
skills_root = Path(__file__).resolve().parents[1] / "docs" / "skills"
|
||||||
|
if not skills_root.is_dir():
|
||||||
|
raise FileNotFoundError(f"packaged skills directory does not exist: {skills_root}")
|
||||||
|
|
||||||
|
has_skills = any(skill_dir.is_dir() and (skill_dir / "SKILL.md").is_file() for skill_dir in skills_root.iterdir())
|
||||||
|
if not has_skills:
|
||||||
|
raise ValueError(f"packaged skills directory contains no skills: {skills_root}")
|
||||||
|
|
||||||
|
return SkillsDirectoryProvider(
|
||||||
|
roots=skills_root,
|
||||||
|
reload=False,
|
||||||
|
supporting_files="template",
|
||||||
|
)
|
||||||
@@ -6,7 +6,6 @@ import pytest
|
|||||||
|
|
||||||
from personal_mcp.registry.ingest.document import MarkdownDocument
|
from personal_mcp.registry.ingest.document import MarkdownDocument
|
||||||
from personal_mcp.registry.ingest.prompt import PromptFilesBundle
|
from personal_mcp.registry.ingest.prompt import PromptFilesBundle
|
||||||
from personal_mcp.registry.ingest.skill import SkillFilesBundle
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.unit
|
pytestmark = pytest.mark.unit
|
||||||
|
|
||||||
@@ -23,15 +22,6 @@ class TestCurrentDocsIngestion:
|
|||||||
|
|
||||||
assert docs
|
assert docs
|
||||||
|
|
||||||
def test_bundles_current_skills(self) -> None:
|
|
||||||
"""Ensures all current canonical skill documents can be bundled."""
|
|
||||||
docs = MarkdownDocument.from_root(DOCS_ROOT)
|
|
||||||
expected_slugs = {path.parent.name for path in DOCS_ROOT.glob("skills/*/SKILL.md")}
|
|
||||||
|
|
||||||
bundles = SkillFilesBundle.from_docs(docs.values())
|
|
||||||
|
|
||||||
assert {bundle.slug for bundle in bundles} == expected_slugs
|
|
||||||
|
|
||||||
def test_bundles_current_prompts(self) -> None:
|
def test_bundles_current_prompts(self) -> None:
|
||||||
"""Ensures all current canonical prompt documents can be bundled."""
|
"""Ensures all current canonical prompt documents can be bundled."""
|
||||||
docs = MarkdownDocument.from_root(DOCS_ROOT)
|
docs = MarkdownDocument.from_root(DOCS_ROOT)
|
||||||
|
|||||||
@@ -81,27 +81,6 @@ class TestMarkdownDocument:
|
|||||||
|
|
||||||
assert doc.frontmatter is None
|
assert doc.frontmatter is None
|
||||||
|
|
||||||
class TestSkillSlugProperty:
|
|
||||||
"""Covers skill_slug derivation from document relative paths."""
|
|
||||||
|
|
||||||
def test_returns_slug(self) -> None:
|
|
||||||
"""Ensures skill_slug returns the slug for valid skills paths."""
|
|
||||||
doc = MarkdownDocument(relpath=PurePosixPath("skills/demo/SKILL.md"), content="#")
|
|
||||||
|
|
||||||
assert doc.skill_slug == "demo"
|
|
||||||
|
|
||||||
def test_none_for_non_skill(self) -> None:
|
|
||||||
"""Ensures skill_slug is None for non-skills paths."""
|
|
||||||
doc = MarkdownDocument(relpath=PurePosixPath("docs/index.md"), content="#")
|
|
||||||
|
|
||||||
assert doc.skill_slug is None
|
|
||||||
|
|
||||||
def test_none_for_incomplete_skill(self) -> None:
|
|
||||||
"""Ensures skill_slug is None for incomplete skills paths."""
|
|
||||||
doc = MarkdownDocument(relpath=PurePosixPath("skills/demo.md"), content="#")
|
|
||||||
|
|
||||||
assert doc.skill_slug is None
|
|
||||||
|
|
||||||
class TestPromptSlugProperty:
|
class TestPromptSlugProperty:
|
||||||
"""Covers prompt_slug derivation from document relative paths."""
|
"""Covers prompt_slug derivation from document relative paths."""
|
||||||
|
|
||||||
|
|||||||
@@ -1,171 +0,0 @@
|
|||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from collections.abc import Callable
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from personal_mcp.registry.ingest.document import MarkdownDocument
|
|
||||||
from personal_mcp.registry.ingest.skill import SkillFilesBundle
|
|
||||||
from personal_mcp.registry.ingest.skill import group_skill_paths
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.unit
|
|
||||||
|
|
||||||
MakeDoc = Callable[[str], MarkdownDocument]
|
|
||||||
|
|
||||||
|
|
||||||
class TestSkillFilesBundle:
|
|
||||||
"""Covers SkillFilesBundle construction and path-based categorization."""
|
|
||||||
|
|
||||||
class TestFromRoot:
|
|
||||||
"""Covers bundle creation from resource roots."""
|
|
||||||
|
|
||||||
def test_builds_bundles(self, tmp_path: Path) -> None:
|
|
||||||
"""Ensures from_root builds bundles from discovered markdown docs."""
|
|
||||||
alpha = tmp_path / "skills" / "alpha"
|
|
||||||
beta = tmp_path / "skills" / "beta"
|
|
||||||
(alpha / "references").mkdir(parents=True)
|
|
||||||
beta.mkdir(parents=True)
|
|
||||||
(alpha / "SKILL.md").write_text("# alpha\n", encoding="utf-8")
|
|
||||||
(alpha / "references" / "one.md").write_text("ref\n", encoding="utf-8")
|
|
||||||
(beta / "SKILL.md").write_text("# beta\n", encoding="utf-8")
|
|
||||||
|
|
||||||
bundles = SkillFilesBundle.from_root(tmp_path)
|
|
||||||
|
|
||||||
assert {bundle.slug for bundle in bundles} == {"alpha", "beta"}
|
|
||||||
|
|
||||||
def test_delegates_to_from_docs(
|
|
||||||
self,
|
|
||||||
tmp_path: Path,
|
|
||||||
) -> None:
|
|
||||||
"""Ensures from_root delegates bundle assembly to from_docs."""
|
|
||||||
alpha = tmp_path / "skills" / "alpha"
|
|
||||||
beta = tmp_path / "skills" / "beta"
|
|
||||||
(alpha / "references").mkdir(parents=True)
|
|
||||||
beta.mkdir(parents=True)
|
|
||||||
(alpha / "SKILL.md").write_text("# alpha\n", encoding="utf-8")
|
|
||||||
(alpha / "references" / "one.md").write_text("ref\n", encoding="utf-8")
|
|
||||||
(beta / "SKILL.md").write_text("# beta\n", encoding="utf-8")
|
|
||||||
|
|
||||||
from_root = SkillFilesBundle.from_root(tmp_path)
|
|
||||||
from_docs = SkillFilesBundle.from_docs(MarkdownDocument.from_root(tmp_path).values())
|
|
||||||
|
|
||||||
assert tuple(from_root) == from_docs
|
|
||||||
|
|
||||||
class TestFromDocs:
|
|
||||||
"""Covers bundle creation from preloaded markdown documents."""
|
|
||||||
|
|
||||||
def test_groups_by_slug(self, make_doc: MakeDoc) -> None:
|
|
||||||
"""Ensures from_docs groups documents by skill slug."""
|
|
||||||
docs = [
|
|
||||||
make_doc("skills/alpha/SKILL.md"),
|
|
||||||
make_doc("skills/alpha/references/a.md"),
|
|
||||||
make_doc("skills/beta/SKILL.md"),
|
|
||||||
]
|
|
||||||
|
|
||||||
bundles = SkillFilesBundle.from_docs(docs)
|
|
||||||
|
|
||||||
assert {bundle.slug for bundle in bundles} == {"alpha", "beta"}
|
|
||||||
|
|
||||||
def test_one_bundle_per_slug(self, make_doc: MakeDoc) -> None:
|
|
||||||
"""Ensures from_docs produces one SkillFilesBundle per slug."""
|
|
||||||
docs = [
|
|
||||||
make_doc("skills/alpha/SKILL.md"),
|
|
||||||
make_doc("skills/alpha/references/r1.md"),
|
|
||||||
make_doc("skills/alpha/references/r2.md"),
|
|
||||||
]
|
|
||||||
|
|
||||||
bundles = SkillFilesBundle.from_docs(docs)
|
|
||||||
|
|
||||||
assert len(bundles) == 1
|
|
||||||
assert bundles[0].slug == "alpha"
|
|
||||||
|
|
||||||
class TestFromPaths:
|
|
||||||
"""Covers classification of skill, reference, and other documents."""
|
|
||||||
|
|
||||||
def test_selects_skill_md(self, make_doc: MakeDoc) -> None:
|
|
||||||
"""Ensures from_paths selects SKILL.md as the primary document."""
|
|
||||||
docs = {
|
|
||||||
make_doc("skills/alpha/SKILL.md"),
|
|
||||||
make_doc("skills/alpha/notes.md"),
|
|
||||||
}
|
|
||||||
|
|
||||||
bundle = SkillFilesBundle.from_paths("alpha", docs)
|
|
||||||
|
|
||||||
assert bundle.skill.relpath.name == "SKILL.md"
|
|
||||||
|
|
||||||
def test_collects_references(self, make_doc: MakeDoc) -> None:
|
|
||||||
"""Ensures from_paths captures reference docs under references/."""
|
|
||||||
docs = {
|
|
||||||
make_doc("skills/alpha/SKILL.md"),
|
|
||||||
make_doc("skills/alpha/references/r1.md"),
|
|
||||||
make_doc("skills/alpha/references/r2.md"),
|
|
||||||
make_doc("skills/alpha/notes.md"),
|
|
||||||
}
|
|
||||||
|
|
||||||
bundle = SkillFilesBundle.from_paths("alpha", docs)
|
|
||||||
|
|
||||||
assert {doc.relpath.as_posix() for doc in bundle.references} == {
|
|
||||||
"skills/alpha/references/r1.md",
|
|
||||||
"skills/alpha/references/r2.md",
|
|
||||||
}
|
|
||||||
|
|
||||||
def test_collects_other_docs(self, make_doc: MakeDoc) -> None:
|
|
||||||
"""Ensures from_paths classifies non-reference docs as other docs."""
|
|
||||||
docs = {
|
|
||||||
make_doc("skills/alpha/SKILL.md"),
|
|
||||||
make_doc("skills/alpha/references/r1.md"),
|
|
||||||
make_doc("skills/alpha/notes.md"),
|
|
||||||
make_doc("skills/alpha/changelog.md"),
|
|
||||||
}
|
|
||||||
|
|
||||||
bundle = SkillFilesBundle.from_paths("alpha", docs)
|
|
||||||
|
|
||||||
assert {doc.relpath.as_posix() for doc in bundle.other} == {
|
|
||||||
"skills/alpha/changelog.md",
|
|
||||||
"skills/alpha/notes.md",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
class TestGroupSkillPaths:
|
|
||||||
"""Covers grouping markdown documents by derived skill slug."""
|
|
||||||
|
|
||||||
def test_groups_slugged_docs(self, make_doc: MakeDoc) -> None:
|
|
||||||
"""Ensures group_skill_paths groups only documents with a slug."""
|
|
||||||
docs = [
|
|
||||||
make_doc("skills/alpha/SKILL.md"),
|
|
||||||
make_doc("skills/alpha/references/r.md"),
|
|
||||||
make_doc("skills/beta/SKILL.md"),
|
|
||||||
]
|
|
||||||
|
|
||||||
grouped = group_skill_paths(docs)
|
|
||||||
|
|
||||||
assert set(grouped) == {"alpha", "beta"}
|
|
||||||
|
|
||||||
def test_excludes_unslugged_docs(self, make_doc: MakeDoc) -> None:
|
|
||||||
"""Ensures group_skill_paths excludes documents without skill slugs."""
|
|
||||||
docs = [
|
|
||||||
make_doc("docs/index.md"),
|
|
||||||
make_doc("content/usage.md"),
|
|
||||||
]
|
|
||||||
|
|
||||||
assert group_skill_paths(docs) == {}
|
|
||||||
|
|
||||||
def test_returns_sets(self, make_doc: MakeDoc) -> None:
|
|
||||||
"""Ensures group_skill_paths returns sets of docs per slug."""
|
|
||||||
grouped = group_skill_paths([make_doc("skills/alpha/SKILL.md")])
|
|
||||||
|
|
||||||
assert isinstance(grouped["alpha"], set)
|
|
||||||
|
|
||||||
def test_stable_grouping(self, make_doc: MakeDoc) -> None:
|
|
||||||
"""Ensures group_skill_paths behaves consistently after internal sorting."""
|
|
||||||
docs = [
|
|
||||||
make_doc("skills/beta/SKILL.md"),
|
|
||||||
make_doc("skills/alpha/references/a.md"),
|
|
||||||
make_doc("skills/alpha/SKILL.md"),
|
|
||||||
]
|
|
||||||
|
|
||||||
grouped_forward = group_skill_paths(docs)
|
|
||||||
grouped_reverse = group_skill_paths(list(reversed(docs)))
|
|
||||||
|
|
||||||
assert grouped_forward == grouped_reverse
|
|
||||||
@@ -8,9 +8,7 @@ import yaml
|
|||||||
from pydantic import ValidationError
|
from pydantic import ValidationError
|
||||||
|
|
||||||
from personal_mcp.registry.ingest.document import MarkdownDocument
|
from personal_mcp.registry.ingest.document import MarkdownDocument
|
||||||
from personal_mcp.registry.models.common import ReferenceEntry
|
|
||||||
from personal_mcp.registry.models.common import parse_docs_path
|
from personal_mcp.registry.models.common import parse_docs_path
|
||||||
from personal_mcp.registry.models.common import parse_reference_path
|
|
||||||
from personal_mcp.registry.models.registry import DocsRegistry
|
from personal_mcp.registry.models.registry import DocsRegistry
|
||||||
|
|
||||||
pytestmark = pytest.mark.unit
|
pytestmark = pytest.mark.unit
|
||||||
@@ -30,35 +28,6 @@ def make_markdown_document(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def make_skill_frontmatter_payload(
|
|
||||||
*,
|
|
||||||
skill_id: str,
|
|
||||||
name: str | None = None,
|
|
||||||
version: str = "1.0.0",
|
|
||||||
description: str = "demo skill",
|
|
||||||
tags: tuple[str, ...] = ("testing",),
|
|
||||||
capabilities: tuple[str, ...] | None = None,
|
|
||||||
references: dict[str, dict[str, Any]] | None = None,
|
|
||||||
) -> str:
|
|
||||||
"""Builds frontmatter payload YAML for skill conversion tests."""
|
|
||||||
canonical_name = name or skill_id
|
|
||||||
x_personal_mcp: dict[str, Any] = {
|
|
||||||
"id": skill_id,
|
|
||||||
"version": version,
|
|
||||||
"tags": list(tags),
|
|
||||||
"capabilities": list(capabilities or (f"resource://skills/{canonical_name}/document",)),
|
|
||||||
}
|
|
||||||
if references is not None:
|
|
||||||
x_personal_mcp["references"] = references
|
|
||||||
|
|
||||||
payload: dict[str, Any] = {
|
|
||||||
"name": canonical_name,
|
|
||||||
"description": description,
|
|
||||||
"x-personal-mcp": x_personal_mcp,
|
|
||||||
}
|
|
||||||
return yaml.safe_dump(payload, sort_keys=False)
|
|
||||||
|
|
||||||
|
|
||||||
def make_prompt_frontmatter_payload(
|
def make_prompt_frontmatter_payload(
|
||||||
*,
|
*,
|
||||||
prompt_id: str,
|
prompt_id: str,
|
||||||
@@ -121,19 +90,6 @@ class TestGate5ContractValidation:
|
|||||||
with pytest.raises(ValueError):
|
with pytest.raises(ValueError):
|
||||||
parse_docs_path(value)
|
parse_docs_path(value)
|
||||||
|
|
||||||
def test_reference_entry_materializes_reference_path(self) -> None:
|
|
||||||
"""Ensures authored reference strings become constrained path objects."""
|
|
||||||
entry = ReferenceEntry.model_validate({"path": "references/guides/setup.md"})
|
|
||||||
|
|
||||||
assert entry.path == PurePosixPath("references/guides/setup.md")
|
|
||||||
assert parse_reference_path(entry.path) == entry.path
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("value", ("guide.md", "other/guide.md", "references.md"))
|
|
||||||
def test_reference_path_stays_under_references(self, value: str) -> None:
|
|
||||||
"""Ensures in-skill references remain below the references directory."""
|
|
||||||
with pytest.raises(ValueError, match="stay under references"):
|
|
||||||
parse_reference_path(value)
|
|
||||||
|
|
||||||
|
|
||||||
class TestGate6FreezeValidation:
|
class TestGate6FreezeValidation:
|
||||||
"""Gate 6: immutable in-memory registry snapshot semantics."""
|
"""Gate 6: immutable in-memory registry snapshot semantics."""
|
||||||
@@ -143,13 +99,8 @@ class TestGate6FreezeValidation:
|
|||||||
index_path = PurePosixPath("index.md")
|
index_path = PurePosixPath("index.md")
|
||||||
source_docs = {index_path: "# index\n"}
|
source_docs = {index_path: "# index\n"}
|
||||||
registry = DocsRegistry(
|
registry = DocsRegistry(
|
||||||
skills_by_id={},
|
|
||||||
skills_in_load_order=(),
|
|
||||||
skills_summary_in_load_order=(),
|
|
||||||
docs_markdown_by_path=source_docs,
|
docs_markdown_by_path=source_docs,
|
||||||
docs_markdown_path_index=(index_path,),
|
docs_markdown_path_index=(index_path,),
|
||||||
tag_to_skill_ids={},
|
|
||||||
capability_to_skill_ids={},
|
|
||||||
prompts_by_id={},
|
prompts_by_id={},
|
||||||
prompts_in_load_order=(),
|
prompts_in_load_order=(),
|
||||||
prompts_summary_in_load_order=(),
|
prompts_summary_in_load_order=(),
|
||||||
@@ -164,13 +115,8 @@ class TestGate6FreezeValidation:
|
|||||||
def test_docs_registry_instance_is_frozen(self) -> None:
|
def test_docs_registry_instance_is_frozen(self) -> None:
|
||||||
"""Ensures frozen model prevents attribute reassignment."""
|
"""Ensures frozen model prevents attribute reassignment."""
|
||||||
registry = DocsRegistry(
|
registry = DocsRegistry(
|
||||||
skills_by_id={},
|
|
||||||
skills_in_load_order=(),
|
|
||||||
skills_summary_in_load_order=(),
|
|
||||||
docs_markdown_by_path={},
|
docs_markdown_by_path={},
|
||||||
docs_markdown_path_index=(),
|
docs_markdown_path_index=(),
|
||||||
tag_to_skill_ids={},
|
|
||||||
capability_to_skill_ids={},
|
|
||||||
prompts_by_id={},
|
prompts_by_id={},
|
||||||
prompts_in_load_order=(),
|
prompts_in_load_order=(),
|
||||||
prompts_summary_in_load_order=(),
|
prompts_summary_in_load_order=(),
|
||||||
|
|||||||
@@ -5,15 +5,12 @@ from pathlib import PurePosixPath
|
|||||||
import pytest
|
import pytest
|
||||||
from pydantic import ValidationError
|
from pydantic import ValidationError
|
||||||
|
|
||||||
from personal_mcp.registry.ingest.document import MarkdownDocument
|
|
||||||
from personal_mcp.registry.ingest.prompt import PromptFilesBundle
|
from personal_mcp.registry.ingest.prompt import PromptFilesBundle
|
||||||
from personal_mcp.registry.load import _build_prompt_record
|
from personal_mcp.registry.load import _build_prompt_record
|
||||||
from personal_mcp.registry.load import get_docs_registry
|
|
||||||
from personal_mcp.registry.models.registry import PromptSummaryRecord
|
from personal_mcp.registry.models.registry import PromptSummaryRecord
|
||||||
from tests.registry.models.test_document_validation import assert_model_is_frozen
|
from tests.registry.models.test_document_validation import assert_model_is_frozen
|
||||||
from tests.registry.models.test_document_validation import make_markdown_document
|
from tests.registry.models.test_document_validation import make_markdown_document
|
||||||
from tests.registry.models.test_document_validation import make_prompt_frontmatter_payload
|
from tests.registry.models.test_document_validation import make_prompt_frontmatter_payload
|
||||||
from tests.registry.models.test_document_validation import make_skill_frontmatter_payload
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.unit
|
pytestmark = pytest.mark.unit
|
||||||
|
|
||||||
@@ -113,35 +110,6 @@ class TestPromptValidationGates:
|
|||||||
assert record.arguments["topic"].required is True
|
assert record.arguments["topic"].required is True
|
||||||
assert record.arguments["topic"].description == "topic to discuss"
|
assert record.arguments["topic"].description == "topic to discuss"
|
||||||
|
|
||||||
class TestGate4GraphValidation:
|
|
||||||
"""Gate 4: validate cross-entity identifier coherence."""
|
|
||||||
|
|
||||||
def test_prompt_id_collision_with_skill_id_fails(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
"""Ensures prompt and skill ids cannot collide in published registry."""
|
|
||||||
skill_frontmatter = make_skill_frontmatter_payload(skill_id="shared")
|
|
||||||
prompt_frontmatter = make_prompt_frontmatter_payload(prompt_id="shared")
|
|
||||||
documents = {
|
|
||||||
PurePosixPath("skills/shared/SKILL.md"): make_markdown_document(
|
|
||||||
"skills/shared/SKILL.md",
|
|
||||||
frontmatter=skill_frontmatter,
|
|
||||||
),
|
|
||||||
PurePosixPath("prompts/shared/PROMPT.md"): make_markdown_document(
|
|
||||||
"prompts/shared/PROMPT.md",
|
|
||||||
frontmatter=prompt_frontmatter,
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
def fake_from_root(_cls, _root):
|
|
||||||
return documents
|
|
||||||
|
|
||||||
monkeypatch.setattr(MarkdownDocument, "from_root", classmethod(fake_from_root))
|
|
||||||
get_docs_registry.cache_clear()
|
|
||||||
try:
|
|
||||||
with pytest.raises(ValueError, match="prompt_id collides with existing skill_id"):
|
|
||||||
get_docs_registry()
|
|
||||||
finally:
|
|
||||||
get_docs_registry.cache_clear()
|
|
||||||
|
|
||||||
class TestGate5ContractValidation:
|
class TestGate5ContractValidation:
|
||||||
"""Gate 5: validate model_dump contract shape for API surfaces."""
|
"""Gate 5: validate model_dump contract shape for API surfaces."""
|
||||||
|
|
||||||
|
|||||||
@@ -1,56 +1,14 @@
|
|||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
from pathlib import PurePosixPath
|
from pathlib import PurePosixPath
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from personal_mcp.catalog.server import build_skill_detail_payload
|
|
||||||
from personal_mcp.registry.models.prompt import PromptArgumentEntry
|
from personal_mcp.registry.models.prompt import PromptArgumentEntry
|
||||||
from personal_mcp.registry.models.registry import DocsRegistry
|
|
||||||
from personal_mcp.registry.models.registry import PromptRecord
|
from personal_mcp.registry.models.registry import PromptRecord
|
||||||
from personal_mcp.registry.models.registry import PromptSummaryPayload
|
from personal_mcp.registry.models.registry import PromptSummaryPayload
|
||||||
from personal_mcp.registry.models.registry import ReferenceRecord
|
|
||||||
from personal_mcp.registry.models.registry import SkillPatternPayload
|
|
||||||
from personal_mcp.registry.models.registry import SkillRecord
|
|
||||||
from personal_mcp.registry.models.registry import SkillSummaryPayload
|
|
||||||
from personal_mcp.registry.models.registry import SkillSummaryRecord
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.unit
|
pytestmark = pytest.mark.unit
|
||||||
|
|
||||||
|
|
||||||
def _make_skill_record() -> SkillRecord:
|
|
||||||
return SkillRecord(
|
|
||||||
skill_id="demo-skill",
|
|
||||||
name="demo-skill",
|
|
||||||
description="demo skill",
|
|
||||||
version="1.2.3",
|
|
||||||
tags=("testing", "catalog"),
|
|
||||||
capabilities=("resource://skills/demo-skill/document", "resource://catalog/skills_index"),
|
|
||||||
document_uri="resource://skills/demo-skill/document",
|
|
||||||
document_relpath=PurePosixPath("skills/demo-skill/SKILL.md"),
|
|
||||||
document_content="# demo",
|
|
||||||
references={
|
|
||||||
"zeta": ReferenceRecord(
|
|
||||||
ref_id="zeta",
|
|
||||||
uri="resource://skills/demo-skill/references/zeta",
|
|
||||||
relpath=PurePosixPath("skills/demo-skill/references/zeta.md"),
|
|
||||||
mime_type="text/markdown",
|
|
||||||
title="Zeta",
|
|
||||||
content="# zeta",
|
|
||||||
),
|
|
||||||
"alpha": ReferenceRecord(
|
|
||||||
ref_id="alpha",
|
|
||||||
uri="resource://skills/demo-skill/references/alpha",
|
|
||||||
relpath=PurePosixPath("skills/demo-skill/references/alpha.md"),
|
|
||||||
mime_type="text/markdown",
|
|
||||||
title="Alpha",
|
|
||||||
content="# alpha",
|
|
||||||
),
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _make_prompt_record() -> PromptRecord:
|
def _make_prompt_record() -> PromptRecord:
|
||||||
return PromptRecord(
|
return PromptRecord(
|
||||||
prompt_id="demo-prompt",
|
prompt_id="demo-prompt",
|
||||||
@@ -72,46 +30,6 @@ def _make_prompt_record() -> PromptRecord:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_skill_pattern_payload_from_record_shape() -> None:
|
|
||||||
record = _make_skill_record()
|
|
||||||
|
|
||||||
payload = SkillPatternPayload.from_record(record).model_dump()
|
|
||||||
|
|
||||||
assert payload == {
|
|
||||||
"id": "demo-skill",
|
|
||||||
"name": "demo-skill",
|
|
||||||
"version": "1.2.3",
|
|
||||||
"description": "demo skill",
|
|
||||||
"tags": ["testing", "catalog"],
|
|
||||||
"capabilities": ["resource://skills/demo-skill/document", "resource://catalog/skills_index"],
|
|
||||||
"resources": ["resource://skills/demo-skill/document", "resource://catalog/skills_index"],
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def test_skill_summary_payload_from_record_shape() -> None:
|
|
||||||
record = _make_skill_record()
|
|
||||||
|
|
||||||
payload = SkillSummaryPayload.from_record(record).model_dump()
|
|
||||||
|
|
||||||
assert payload == {
|
|
||||||
"id": "demo-skill",
|
|
||||||
"name": "demo-skill",
|
|
||||||
"description": "demo skill",
|
|
||||||
"tags": ["testing", "catalog"],
|
|
||||||
"capabilities": ["resource://skills/demo-skill/document", "resource://catalog/skills_index"],
|
|
||||||
"version": "1.2.3",
|
|
||||||
"document_uri": "resource://skills/demo-skill/document",
|
|
||||||
"detail_uri": "resource://catalog/skills/demo-skill",
|
|
||||||
"resources": {
|
|
||||||
"document": "resource://skills/demo-skill/document",
|
|
||||||
"references": [
|
|
||||||
"resource://skills/demo-skill/references/alpha",
|
|
||||||
"resource://skills/demo-skill/references/zeta",
|
|
||||||
],
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def test_prompt_summary_payload_from_record_shape() -> None:
|
def test_prompt_summary_payload_from_record_shape() -> None:
|
||||||
record = _make_prompt_record()
|
record = _make_prompt_record()
|
||||||
|
|
||||||
@@ -127,25 +45,3 @@ def test_prompt_summary_payload_from_record_shape() -> None:
|
|||||||
"document_uri": "resource://prompts/demo-prompt/document",
|
"document_uri": "resource://prompts/demo-prompt/document",
|
||||||
"detail_uri": "resource://catalog/prompts/demo-prompt",
|
"detail_uri": "resource://catalog/prompts/demo-prompt",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def test_skill_detail_serializes_reference_paths() -> None:
|
|
||||||
record = _make_skill_record()
|
|
||||||
registry = DocsRegistry(
|
|
||||||
skills_by_id={record.skill_id: record},
|
|
||||||
skills_in_load_order=(record.skill_id,),
|
|
||||||
skills_summary_in_load_order=(SkillSummaryRecord.from_record(record),),
|
|
||||||
docs_markdown_by_path={},
|
|
||||||
docs_markdown_path_index=(),
|
|
||||||
tag_to_skill_ids={},
|
|
||||||
capability_to_skill_ids={},
|
|
||||||
prompts_by_id={},
|
|
||||||
prompts_in_load_order=(),
|
|
||||||
prompts_summary_in_load_order=(),
|
|
||||||
tag_to_prompt_ids={},
|
|
||||||
)
|
|
||||||
|
|
||||||
payload = build_skill_detail_payload(registry, record.skill_id)
|
|
||||||
|
|
||||||
assert payload["resources"]["references"]["alpha"]["path"] == ("skills/demo-skill/references/alpha.md")
|
|
||||||
json.dumps(payload)
|
|
||||||
|
|||||||
@@ -1,130 +0,0 @@
|
|||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from personal_mcp.registry.load import get_docs_registry
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.unit
|
|
||||||
|
|
||||||
REGISTRY = get_docs_registry()
|
|
||||||
|
|
||||||
# Convention: every skill should include tags for the core libraries/frameworks
|
|
||||||
# it relies on so search_patterns query terms map to discoverable skills.
|
|
||||||
REQUIRED_LIBRARY_TAGS_BY_SKILL = {
|
|
||||||
"copilot-customization": {"copilot", "vscode", "mcp"},
|
|
||||||
"async-fastapi-sqlmodel": {"fastapi", "sqlalchemy", "asyncio"},
|
|
||||||
"fastapi-uv-docker": {"fastapi", "uv", "uvicorn", "docker"},
|
|
||||||
"mcp-details": {"mcp", "fastmcp"},
|
|
||||||
"nicegui": {"nicegui", "fastapi"},
|
|
||||||
"pytesting": {"pytest", "testing", "fastapi", "asyncio", "anyio"},
|
|
||||||
"python-logging": {"python", "logging"},
|
|
||||||
"python-typing": {"python", "typing"},
|
|
||||||
"ruff-linting-formating": {"ruff", "python"},
|
|
||||||
"vscode-configuration": {"vscode", "debugpy", "fastapi", "python"},
|
|
||||||
"zensical-docs": {"zensical", "mkdocs", "mkdocs-material", "mkdocstrings"},
|
|
||||||
}
|
|
||||||
|
|
||||||
LIBRARY_OR_PLATFORM_TAGS = {
|
|
||||||
"anyio",
|
|
||||||
"asyncio",
|
|
||||||
"copilot",
|
|
||||||
"debugpy",
|
|
||||||
"docker",
|
|
||||||
"fastapi",
|
|
||||||
"fastmcp",
|
|
||||||
"logging",
|
|
||||||
"mcp",
|
|
||||||
"mkdocs",
|
|
||||||
"mkdocs-material",
|
|
||||||
"mkdocstrings",
|
|
||||||
"nicegui",
|
|
||||||
"pytest",
|
|
||||||
"python",
|
|
||||||
"ruff",
|
|
||||||
"sqlalchemy",
|
|
||||||
"typing",
|
|
||||||
"uv",
|
|
||||||
"uvicorn",
|
|
||||||
"vscode",
|
|
||||||
"zensical",
|
|
||||||
}
|
|
||||||
|
|
||||||
DOMAIN_FACET_TAGS = {
|
|
||||||
"agent-skills",
|
|
||||||
"architecture",
|
|
||||||
"authoring",
|
|
||||||
"bootstrap",
|
|
||||||
"ci",
|
|
||||||
"configuration",
|
|
||||||
"custom-agents",
|
|
||||||
"customization",
|
|
||||||
"deterministic",
|
|
||||||
"discovery",
|
|
||||||
"docs",
|
|
||||||
"documentation",
|
|
||||||
"formatting",
|
|
||||||
"frontend",
|
|
||||||
"hooks",
|
|
||||||
"information-architecture",
|
|
||||||
"instructions",
|
|
||||||
"launch-json",
|
|
||||||
"linting",
|
|
||||||
"modernization",
|
|
||||||
"observability",
|
|
||||||
"personal-mcp",
|
|
||||||
"prompts",
|
|
||||||
"references",
|
|
||||||
"scaffolding",
|
|
||||||
"skills",
|
|
||||||
"source-docs",
|
|
||||||
"static-analysis",
|
|
||||||
"tasks-json",
|
|
||||||
"testing",
|
|
||||||
"type-hints",
|
|
||||||
"ui",
|
|
||||||
}
|
|
||||||
|
|
||||||
TAG_CONVENTION_PARAMETERS = tuple(
|
|
||||||
pytest.param(skill_id, required_tags, id=skill_id)
|
|
||||||
for skill_id, required_tags in sorted(REQUIRED_LIBRARY_TAGS_BY_SKILL.items())
|
|
||||||
)
|
|
||||||
|
|
||||||
SKILL_IDS = tuple(pytest.param(skill_id, id=skill_id) for skill_id in sorted(REGISTRY.skills_by_id))
|
|
||||||
|
|
||||||
|
|
||||||
class TestSkillTagConventions:
|
|
||||||
"""Covers tag taxonomy conventions for skill discoverability."""
|
|
||||||
|
|
||||||
class TestRequiredLibraryTags:
|
|
||||||
"""Covers required per-skill library and framework tags."""
|
|
||||||
|
|
||||||
@pytest.mark.parametrize(("skill_id", "required_tags"), TAG_CONVENTION_PARAMETERS)
|
|
||||||
def test_includes_required_library_and_framework_tags(
|
|
||||||
self,
|
|
||||||
skill_id: str,
|
|
||||||
required_tags: set[str],
|
|
||||||
) -> None:
|
|
||||||
"""Ensures each skill includes its required library/framework tags."""
|
|
||||||
skill = REGISTRY.skills_by_id[skill_id]
|
|
||||||
skill_tags = set(skill.tags)
|
|
||||||
|
|
||||||
assert required_tags.issubset(skill_tags)
|
|
||||||
|
|
||||||
class TestTagTaxonomyShape:
|
|
||||||
"""Covers baseline tag-shape guarantees across all skills."""
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("skill_id", SKILL_IDS)
|
|
||||||
def test_includes_library_or_platform_tag(self, skill_id: str) -> None:
|
|
||||||
"""Ensures each skill includes at least one library/platform tag."""
|
|
||||||
skill = REGISTRY.skills_by_id[skill_id]
|
|
||||||
skill_tags = set(skill.tags)
|
|
||||||
|
|
||||||
assert skill_tags & LIBRARY_OR_PLATFORM_TAGS
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("skill_id", SKILL_IDS)
|
|
||||||
def test_includes_domain_facet_tag(self, skill_id: str) -> None:
|
|
||||||
"""Ensures each skill includes at least one domain facet tag."""
|
|
||||||
skill = REGISTRY.skills_by_id[skill_id]
|
|
||||||
skill_tags = set(skill.tags)
|
|
||||||
|
|
||||||
assert skill_tags & DOMAIN_FACET_TAGS
|
|
||||||
@@ -1,172 +0,0 @@
|
|||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import PurePosixPath
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from pydantic import ValidationError
|
|
||||||
|
|
||||||
from personal_mcp.registry.ingest.document import MarkdownDocument
|
|
||||||
from personal_mcp.registry.ingest.skill import SkillFilesBundle
|
|
||||||
from personal_mcp.registry.load import _build_skill_record
|
|
||||||
from personal_mcp.registry.models.registry import SkillSummaryRecord
|
|
||||||
from tests.registry.models.test_document_validation import assert_model_is_frozen
|
|
||||||
from tests.registry.models.test_document_validation import make_markdown_document
|
|
||||||
from tests.registry.models.test_document_validation import make_skill_frontmatter_payload
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.unit
|
|
||||||
|
|
||||||
|
|
||||||
def _make_skill_bundle(
|
|
||||||
*,
|
|
||||||
slug: str,
|
|
||||||
frontmatter: str | None,
|
|
||||||
reference_files: tuple[str, ...] = (),
|
|
||||||
other_files: tuple[str, ...] = (),
|
|
||||||
) -> SkillFilesBundle:
|
|
||||||
skill = make_markdown_document(
|
|
||||||
f"skills/{slug}/SKILL.md",
|
|
||||||
frontmatter=frontmatter,
|
|
||||||
)
|
|
||||||
references = tuple(make_markdown_document(f"skills/{slug}/references/{filename}") for filename in reference_files)
|
|
||||||
other = tuple(make_markdown_document(f"skills/{slug}/{filename}") for filename in other_files)
|
|
||||||
return SkillFilesBundle(slug=slug, skill=skill, references=references, other=other)
|
|
||||||
|
|
||||||
|
|
||||||
def _docs_index(bundle: SkillFilesBundle) -> dict[PurePosixPath, MarkdownDocument]:
|
|
||||||
docs = {bundle.skill.relpath: bundle.skill}
|
|
||||||
for doc in bundle.references:
|
|
||||||
docs[doc.relpath] = doc
|
|
||||||
for doc in bundle.other:
|
|
||||||
docs[doc.relpath] = doc
|
|
||||||
return docs
|
|
||||||
|
|
||||||
|
|
||||||
class TestSkillValidationGates:
|
|
||||||
"""Gate-oriented validation coverage for skill conversion."""
|
|
||||||
|
|
||||||
class TestGate1LayoutValidation:
|
|
||||||
"""Gate 1: enforce required source shape before metadata parsing."""
|
|
||||||
|
|
||||||
def test_missing_frontmatter_fails_fast(self) -> None:
|
|
||||||
"""Ensures conversion rejects missing frontmatter at the layout gate."""
|
|
||||||
bundle = _make_skill_bundle(slug="alpha", frontmatter=None)
|
|
||||||
|
|
||||||
with pytest.raises(ValueError, match="missing YAML frontmatter"):
|
|
||||||
_build_skill_record(bundle=bundle, docs_by_relpath=_docs_index(bundle))
|
|
||||||
|
|
||||||
class TestGate2MetadataValidation:
|
|
||||||
"""Gate 2: validate skill metadata via pydantic models."""
|
|
||||||
|
|
||||||
def test_rejects_non_semver_version(self) -> None:
|
|
||||||
"""Ensures semver violations fail during frontmatter model validation."""
|
|
||||||
frontmatter = make_skill_frontmatter_payload(skill_id="alpha", version="not-semver")
|
|
||||||
bundle = _make_skill_bundle(slug="alpha", frontmatter=frontmatter)
|
|
||||||
|
|
||||||
with pytest.raises(ValidationError, match="version must be semver"):
|
|
||||||
_build_skill_record(bundle=bundle, docs_by_relpath=_docs_index(bundle))
|
|
||||||
|
|
||||||
def test_rejects_reserved_name_tokens(self) -> None:
|
|
||||||
"""Ensures reserved words remain blocked for skill names."""
|
|
||||||
frontmatter = make_skill_frontmatter_payload(skill_id="alpha", name="claude-skill")
|
|
||||||
bundle = _make_skill_bundle(slug="alpha", frontmatter=frontmatter)
|
|
||||||
|
|
||||||
with pytest.raises(ValidationError, match="reserved words anthropic or claude"):
|
|
||||||
_build_skill_record(bundle=bundle, docs_by_relpath=_docs_index(bundle))
|
|
||||||
|
|
||||||
def test_rejects_missing_primary_capability(self) -> None:
|
|
||||||
"""Ensures canonical document capability is required."""
|
|
||||||
frontmatter = make_skill_frontmatter_payload(skill_id="alpha", capabilities=("resource://docs/index.md",))
|
|
||||||
bundle = _make_skill_bundle(slug="alpha", frontmatter=frontmatter)
|
|
||||||
|
|
||||||
with pytest.raises(ValueError, match="capabilities must include"):
|
|
||||||
_build_skill_record(bundle=bundle, docs_by_relpath=_docs_index(bundle))
|
|
||||||
|
|
||||||
class TestGate3ResourceValidation:
|
|
||||||
"""Gate 3: resolve document and reference resources deterministically."""
|
|
||||||
|
|
||||||
def test_discovers_reference_from_filename(self) -> None:
|
|
||||||
"""Ensures top-level reference files are converted into reference records."""
|
|
||||||
frontmatter = make_skill_frontmatter_payload(skill_id="alpha")
|
|
||||||
bundle = _make_skill_bundle(
|
|
||||||
slug="alpha",
|
|
||||||
frontmatter=frontmatter,
|
|
||||||
reference_files=("quick-start.md",),
|
|
||||||
)
|
|
||||||
|
|
||||||
record = _build_skill_record(bundle=bundle, docs_by_relpath=_docs_index(bundle))
|
|
||||||
|
|
||||||
assert set(record.references) == {"quick-start"}
|
|
||||||
assert record.references["quick-start"].uri == "resource://skills/alpha/references/quick-start"
|
|
||||||
assert record.references["quick-start"].title == "Quick Start"
|
|
||||||
|
|
||||||
def test_declared_reference_requires_existing_document(self) -> None:
|
|
||||||
"""Ensures declared reference paths must map to discovered markdown files."""
|
|
||||||
frontmatter = make_skill_frontmatter_payload(
|
|
||||||
skill_id="alpha",
|
|
||||||
references={
|
|
||||||
"guide": {
|
|
||||||
"path": "references/guide.md",
|
|
||||||
"title": "Guide",
|
|
||||||
}
|
|
||||||
},
|
|
||||||
)
|
|
||||||
bundle = _make_skill_bundle(slug="alpha", frontmatter=frontmatter)
|
|
||||||
|
|
||||||
with pytest.raises(KeyError, match="reference document not found"):
|
|
||||||
_build_skill_record(bundle=bundle, docs_by_relpath=_docs_index(bundle))
|
|
||||||
|
|
||||||
class TestGate5ContractValidation:
|
|
||||||
"""Gate 5: validate model_dump contract shape for API surfaces."""
|
|
||||||
|
|
||||||
def test_skill_record_model_dump_contains_contract_fields(self) -> None:
|
|
||||||
"""Ensures skill record serialization contains expected resource contract keys."""
|
|
||||||
frontmatter = make_skill_frontmatter_payload(skill_id="alpha")
|
|
||||||
bundle = _make_skill_bundle(slug="alpha", frontmatter=frontmatter)
|
|
||||||
record = _build_skill_record(bundle=bundle, docs_by_relpath=_docs_index(bundle))
|
|
||||||
|
|
||||||
dumped = record.model_dump()
|
|
||||||
|
|
||||||
assert dumped["skill_id"] == "alpha"
|
|
||||||
assert dumped["document_uri"] == "resource://skills/alpha/document"
|
|
||||||
assert "references" in dumped
|
|
||||||
assert "document_content" in dumped
|
|
||||||
|
|
||||||
def test_skill_summary_projection_stays_stable(self) -> None:
|
|
||||||
"""Ensures summary projection keeps only index-safe fields."""
|
|
||||||
frontmatter = make_skill_frontmatter_payload(skill_id="alpha")
|
|
||||||
bundle = _make_skill_bundle(slug="alpha", frontmatter=frontmatter)
|
|
||||||
record = _build_skill_record(bundle=bundle, docs_by_relpath=_docs_index(bundle))
|
|
||||||
summary = SkillSummaryRecord.from_record(record)
|
|
||||||
|
|
||||||
assert summary.model_dump() == {
|
|
||||||
"skill_id": "alpha",
|
|
||||||
"name": "alpha",
|
|
||||||
"description": "demo skill",
|
|
||||||
"tags": ("testing",),
|
|
||||||
"capabilities": ("resource://skills/alpha/document",),
|
|
||||||
"document_uri": "resource://skills/alpha/document",
|
|
||||||
"version": "1.0.0",
|
|
||||||
}
|
|
||||||
|
|
||||||
class TestGate6FreezeValidation:
|
|
||||||
"""Gate 6: ensure immutable runtime records."""
|
|
||||||
|
|
||||||
def test_skill_record_instance_is_frozen(self) -> None:
|
|
||||||
"""Ensures validated skill records cannot be mutated after creation."""
|
|
||||||
frontmatter = make_skill_frontmatter_payload(skill_id="alpha")
|
|
||||||
bundle = _make_skill_bundle(slug="alpha", frontmatter=frontmatter)
|
|
||||||
record = _build_skill_record(bundle=bundle, docs_by_relpath=_docs_index(bundle))
|
|
||||||
|
|
||||||
assert_model_is_frozen(record, attr="name", value="mutated")
|
|
||||||
|
|
||||||
def test_reference_record_values_are_frozen(self) -> None:
|
|
||||||
"""Ensures nested reference records are immutable after publication."""
|
|
||||||
frontmatter = make_skill_frontmatter_payload(skill_id="alpha")
|
|
||||||
bundle = _make_skill_bundle(
|
|
||||||
slug="alpha",
|
|
||||||
frontmatter=frontmatter,
|
|
||||||
reference_files=("guide.md",),
|
|
||||||
)
|
|
||||||
record = _build_skill_record(bundle=bundle, docs_by_relpath=_docs_index(bundle))
|
|
||||||
|
|
||||||
assert_model_is_frozen(record.references["guide"], attr="title", value="Mutated")
|
|
||||||
@@ -5,42 +5,15 @@ import pytest
|
|||||||
from personal_mcp.registry.models.registry import DocsRegistry
|
from personal_mcp.registry.models.registry import DocsRegistry
|
||||||
from personal_mcp.registry.models.registry import PromptRecord
|
from personal_mcp.registry.models.registry import PromptRecord
|
||||||
from personal_mcp.registry.models.registry import PromptSummaryRecord
|
from personal_mcp.registry.models.registry import PromptSummaryRecord
|
||||||
from personal_mcp.registry.models.registry import ReferenceRecord
|
|
||||||
from personal_mcp.registry.models.registry import SkillRecord
|
|
||||||
from personal_mcp.registry.models.registry import SkillSummaryRecord
|
|
||||||
from personal_mcp.registry.read import read_docs_markdown_path
|
from personal_mcp.registry.read import read_docs_markdown_path
|
||||||
from personal_mcp.registry.read import read_prompt_document
|
from personal_mcp.registry.read import read_prompt_document
|
||||||
from personal_mcp.registry.read import read_skill_document
|
|
||||||
from personal_mcp.registry.read import read_skill_reference
|
|
||||||
|
|
||||||
pytestmark = pytest.mark.unit
|
pytestmark = pytest.mark.unit
|
||||||
|
|
||||||
|
|
||||||
def _make_registry() -> DocsRegistry:
|
def _make_registry() -> DocsRegistry:
|
||||||
skill_path = PurePosixPath("skills/demo/SKILL.md")
|
|
||||||
reference_path = PurePosixPath("skills/demo/references/guide.md")
|
|
||||||
prompt_path = PurePosixPath("prompts/demo-prompt/PROMPT.md")
|
prompt_path = PurePosixPath("prompts/demo-prompt/PROMPT.md")
|
||||||
index_path = PurePosixPath("index.md")
|
index_path = PurePosixPath("index.md")
|
||||||
reference = ReferenceRecord(
|
|
||||||
ref_id="guide",
|
|
||||||
uri="resource://skills/demo/references/guide",
|
|
||||||
relpath=reference_path,
|
|
||||||
mime_type="text/markdown",
|
|
||||||
title="Guide",
|
|
||||||
content="# guide",
|
|
||||||
)
|
|
||||||
skill = SkillRecord(
|
|
||||||
skill_id="demo",
|
|
||||||
name="demo",
|
|
||||||
description="demo skill",
|
|
||||||
version="1.0.0",
|
|
||||||
tags=("testing",),
|
|
||||||
capabilities=("resource://skills/demo/document",),
|
|
||||||
document_uri="resource://skills/demo/document",
|
|
||||||
document_relpath=skill_path,
|
|
||||||
document_content="# demo",
|
|
||||||
references={"guide": reference},
|
|
||||||
)
|
|
||||||
prompt = PromptRecord(
|
prompt = PromptRecord(
|
||||||
prompt_id="demo-prompt",
|
prompt_id="demo-prompt",
|
||||||
name="demo-prompt",
|
name="demo-prompt",
|
||||||
@@ -54,13 +27,8 @@ def _make_registry() -> DocsRegistry:
|
|||||||
document_content="# prompt",
|
document_content="# prompt",
|
||||||
)
|
)
|
||||||
return DocsRegistry(
|
return DocsRegistry(
|
||||||
skills_by_id={skill.skill_id: skill},
|
|
||||||
skills_in_load_order=(skill.skill_id,),
|
|
||||||
skills_summary_in_load_order=(SkillSummaryRecord.from_record(skill),),
|
|
||||||
docs_markdown_by_path={index_path: "# index"},
|
docs_markdown_by_path={index_path: "# index"},
|
||||||
docs_markdown_path_index=(index_path,),
|
docs_markdown_path_index=(index_path,),
|
||||||
tag_to_skill_ids={"testing": (skill.skill_id,)},
|
|
||||||
capability_to_skill_ids={},
|
|
||||||
prompts_by_id={prompt.prompt_id: prompt},
|
prompts_by_id={prompt.prompt_id: prompt},
|
||||||
prompts_in_load_order=(prompt.prompt_id,),
|
prompts_in_load_order=(prompt.prompt_id,),
|
||||||
prompts_summary_in_load_order=(PromptSummaryRecord.from_record(prompt),),
|
prompts_summary_in_load_order=(PromptSummaryRecord.from_record(prompt),),
|
||||||
@@ -87,8 +55,4 @@ def test_rejects_non_posix_docs_path() -> None:
|
|||||||
def test_serializes_record_paths_in_document_payloads() -> None:
|
def test_serializes_record_paths_in_document_payloads() -> None:
|
||||||
registry = _make_registry()
|
registry = _make_registry()
|
||||||
|
|
||||||
assert read_skill_document(registry, "demo")["source_path"] == "docs/skills/demo/SKILL.md"
|
|
||||||
assert read_skill_reference(registry, skill_id="demo", ref_id="guide")["source_path"] == (
|
|
||||||
"docs/skills/demo/references/guide.md"
|
|
||||||
)
|
|
||||||
assert read_prompt_document(registry, "demo-prompt")["source_path"] == ("docs/prompts/demo-prompt/PROMPT.md")
|
assert read_prompt_document(registry, "demo-prompt")["source_path"] == ("docs/prompts/demo-prompt/PROMPT.md")
|
||||||
|
|||||||
@@ -0,0 +1,57 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import yaml
|
||||||
|
from fastmcp import Client
|
||||||
|
from fastmcp import FastMCP
|
||||||
|
from fastmcp.utilities.skills import get_skill_manifest
|
||||||
|
from fastmcp.utilities.skills import list_skills
|
||||||
|
|
||||||
|
from personal_mcp.skills import create_skills_provider
|
||||||
|
|
||||||
|
pytestmark = pytest.mark.unit
|
||||||
|
|
||||||
|
SKILLS_ROOT = Path(__file__).parents[2] / "docs" / "skills"
|
||||||
|
|
||||||
|
|
||||||
|
class TestSkillsProvider:
|
||||||
|
"""Covers native FastMCP skill discovery and retrieval."""
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_discovers_authored_skills(self) -> None:
|
||||||
|
"""Ensures each authored skill is exposed with its description."""
|
||||||
|
expected_names = {path.parent.name for path in SKILLS_ROOT.glob("*/SKILL.md")}
|
||||||
|
mcp = FastMCP("skills-test")
|
||||||
|
mcp.add_provider(create_skills_provider())
|
||||||
|
|
||||||
|
async with Client(mcp) as client:
|
||||||
|
skills = await list_skills(client)
|
||||||
|
|
||||||
|
assert {skill.name for skill in skills} == expected_names
|
||||||
|
assert all(skill.description for skill in skills)
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_reads_manifest_and_supporting_file(self) -> None:
|
||||||
|
"""Ensures manifests disclose hashed files that remain directly readable."""
|
||||||
|
mcp = FastMCP("skills-test")
|
||||||
|
mcp.add_provider(create_skills_provider())
|
||||||
|
|
||||||
|
async with Client(mcp) as client:
|
||||||
|
manifest = await get_skill_manifest(client, "mcp-details")
|
||||||
|
reference = next(file for file in manifest.files if file.path.startswith("references/"))
|
||||||
|
contents = await client.read_resource(f"skill://mcp-details/{reference.path}")
|
||||||
|
|
||||||
|
assert any(file.path == "SKILL.md" for file in manifest.files)
|
||||||
|
assert all(file.hash.startswith("sha256:") for file in manifest.files)
|
||||||
|
assert reference.size > 0
|
||||||
|
assert contents
|
||||||
|
|
||||||
|
def test_frontmatter_names_match_directories(self) -> None:
|
||||||
|
"""Ensures provider identity and authored skill names remain aligned."""
|
||||||
|
for skill_file in SKILLS_ROOT.glob("*/SKILL.md"):
|
||||||
|
raw = skill_file.read_text(encoding="utf-8")
|
||||||
|
frontmatter = yaml.safe_load(raw.split("---", 2)[1])
|
||||||
|
|
||||||
|
assert set(frontmatter) == {"name", "description"}
|
||||||
|
assert frontmatter["name"] == skill_file.parent.name
|
||||||
|
assert frontmatter["description"]
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from collections.abc import AsyncIterator
|
from collections.abc import AsyncGenerator
|
||||||
from contextlib import asynccontextmanager
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -14,7 +14,7 @@ from personal_mcp.web.app import create_app
|
|||||||
|
|
||||||
|
|
||||||
@pytest_asyncio.fixture
|
@pytest_asyncio.fixture
|
||||||
async def client() -> AsyncIterator[AsyncClient]:
|
async def client() -> AsyncGenerator[AsyncClient]:
|
||||||
"""Provides an AsyncClient bound to a fresh application instance."""
|
"""Provides an AsyncClient bound to a fresh application instance."""
|
||||||
app = create_app()
|
app = create_app()
|
||||||
async with AsyncClient(
|
async with AsyncClient(
|
||||||
@@ -30,7 +30,7 @@ def mcp_session_factory():
|
|||||||
"""Provides an in-process context manager factory for MCP SDK sessions."""
|
"""Provides an in-process context manager factory for MCP SDK sessions."""
|
||||||
|
|
||||||
@asynccontextmanager
|
@asynccontextmanager
|
||||||
async def create_session(*, initialize: bool = True) -> AsyncIterator[ClientSession]:
|
async def create_session(*, initialize: bool = True) -> AsyncGenerator[ClientSession]:
|
||||||
app = create_app()
|
app = create_app()
|
||||||
mcp_url = f"http://testserver{app.state.settings.mounts.mcp}"
|
mcp_url = f"http://testserver{app.state.settings.mounts.mcp}"
|
||||||
async with (
|
async with (
|
||||||
|
|||||||
+30
-130
@@ -6,173 +6,73 @@ import pytest
|
|||||||
|
|
||||||
pytestmark = pytest.mark.smoke
|
pytestmark = pytest.mark.smoke
|
||||||
|
|
||||||
|
RETIRED_TOOL_NAMES = {
|
||||||
REQUIRED_TOOL_NAMES = (
|
|
||||||
"search_patterns",
|
"search_patterns",
|
||||||
"get_pattern_by_id",
|
"get_pattern_by_id",
|
||||||
"get_skill_document_by_id",
|
"get_skill_document_by_id",
|
||||||
"search_prompts",
|
}
|
||||||
"get_prompt_by_id",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
REQUIRED_RESOURCE_URIS = (
|
class TestMcpSkillsSurface:
|
||||||
"resource://catalog/skills_index",
|
"""Covers native skill resources over the HTTP MCP surface."""
|
||||||
"resource://catalog/prompts_index",
|
|
||||||
)
|
|
||||||
|
|
||||||
TOOL_NAME_PARAMETERS = tuple(
|
|
||||||
pytest.param(
|
|
||||||
tool_name,
|
|
||||||
id=tool_name.replace("_", "-"),
|
|
||||||
)
|
|
||||||
for tool_name in REQUIRED_TOOL_NAMES
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
SEARCH_QUERY_PARAMETERS = (
|
|
||||||
pytest.param(
|
|
||||||
"pytest",
|
|
||||||
{"pytesting"},
|
|
||||||
id="query-pytest",
|
|
||||||
),
|
|
||||||
pytest.param(
|
|
||||||
"asyncio",
|
|
||||||
{"pytesting", "async-fastapi-sqlmodel"},
|
|
||||||
id="query-asyncio",
|
|
||||||
),
|
|
||||||
pytest.param(
|
|
||||||
"fastapi testing",
|
|
||||||
{"pytesting"},
|
|
||||||
id="query-fastapi-testing",
|
|
||||||
),
|
|
||||||
pytest.param(
|
|
||||||
"asyncio fastapi testing deterministic pytest",
|
|
||||||
{"pytesting"},
|
|
||||||
id="query-composite-async-fastapi-testing-deterministic-pytest",
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
RESOURCE_URI_PARAMETERS = tuple(
|
|
||||||
pytest.param(
|
|
||||||
resource_uri,
|
|
||||||
id=resource_uri.removeprefix("resource://").replace("/", "-"),
|
|
||||||
)
|
|
||||||
for resource_uri in REQUIRED_RESOURCE_URIS
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class TestMcpCatalogSurface:
|
|
||||||
"""Covers smoke-level MCP catalog discovery and tool execution paths."""
|
|
||||||
|
|
||||||
class TestTools:
|
class TestTools:
|
||||||
"""Covers MCP tool-list and tool-call smoke behavior."""
|
"""Covers generic resource fallback tools for native skills."""
|
||||||
|
|
||||||
@pytest.mark.parametrize("tool_name", TOOL_NAME_PARAMETERS)
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_lists_core_catalog_tools(
|
async def test_lists_resource_fallback_tools(self, mcp_session_factory) -> None:
|
||||||
self,
|
"""Ensures generic resource tools replace skill-specific catalog tools."""
|
||||||
mcp_session_factory,
|
|
||||||
tool_name: str,
|
|
||||||
) -> None:
|
|
||||||
"""Ensures tools/list exposes each required core catalog tool name."""
|
|
||||||
async with mcp_session_factory() as mcp_session:
|
async with mcp_session_factory() as mcp_session:
|
||||||
result = await mcp_session.list_tools()
|
result = await mcp_session.list_tools()
|
||||||
tool_names = {tool.name for tool in result.tools}
|
tool_names = {tool.name for tool in result.tools}
|
||||||
|
|
||||||
assert tool_name in tool_names
|
assert {"list_resources", "read_resource"}.issubset(tool_names)
|
||||||
|
assert RETIRED_TOOL_NAMES.isdisjoint(tool_names)
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_calls_search_patterns_tool(self, mcp_session_factory) -> None:
|
async def test_reads_skill_through_fallback_tool(self, mcp_session_factory) -> None:
|
||||||
"""Ensures tools/call succeeds for search_patterns with basic args."""
|
"""Ensures tool-only clients can read a native skill resource."""
|
||||||
async with mcp_session_factory() as mcp_session:
|
async with mcp_session_factory() as mcp_session:
|
||||||
result = await mcp_session.call_tool(
|
result = await mcp_session.call_tool(
|
||||||
"search_patterns",
|
"read_resource",
|
||||||
{
|
{"uri": "skill://mcp-details/SKILL.md"},
|
||||||
"query": "pytest",
|
|
||||||
"limit": 5,
|
|
||||||
},
|
|
||||||
)
|
)
|
||||||
|
|
||||||
assert result.isError is False
|
assert result.isError is False
|
||||||
assert result.content
|
assert result.content
|
||||||
|
|
||||||
@pytest.mark.parametrize(
|
|
||||||
("query", "expected_skill_ids"),
|
|
||||||
SEARCH_QUERY_PARAMETERS,
|
|
||||||
)
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_search_patterns_matches_expected_skills_for_query_terms(
|
|
||||||
self,
|
|
||||||
mcp_session_factory,
|
|
||||||
query: str,
|
|
||||||
expected_skill_ids: set[str],
|
|
||||||
) -> None:
|
|
||||||
"""Ensures query terms return expected skill IDs from search_patterns."""
|
|
||||||
async with mcp_session_factory() as mcp_session:
|
|
||||||
result = await mcp_session.call_tool(
|
|
||||||
"search_patterns",
|
|
||||||
{
|
|
||||||
"query": query,
|
|
||||||
"limit": 20,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
assert result.isError is False
|
|
||||||
assert result.content
|
|
||||||
|
|
||||||
payload = json.loads(result.content[0].text)
|
|
||||||
found_skill_ids = {pattern["id"] for pattern in payload["patterns"]}
|
|
||||||
|
|
||||||
assert expected_skill_ids.issubset(found_skill_ids)
|
|
||||||
|
|
||||||
class TestResources:
|
class TestResources:
|
||||||
"""Covers MCP resource and resource-template discovery."""
|
"""Covers native skill resources, manifests, and file templates."""
|
||||||
|
|
||||||
@pytest.mark.parametrize("resource_uri", RESOURCE_URI_PARAMETERS)
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_lists_catalog_resources(
|
async def test_lists_main_file_and_manifest(self, mcp_session_factory) -> None:
|
||||||
self,
|
"""Ensures resources/list exposes native skill entry points."""
|
||||||
mcp_session_factory,
|
|
||||||
resource_uri: str,
|
|
||||||
) -> None:
|
|
||||||
"""Ensures resources/list exposes each required catalog resource URI."""
|
|
||||||
async with mcp_session_factory() as mcp_session:
|
async with mcp_session_factory() as mcp_session:
|
||||||
result = await mcp_session.list_resources()
|
result = await mcp_session.list_resources()
|
||||||
resource_uris = {str(resource.uri) for resource in result.resources}
|
resource_uris = {str(resource.uri) for resource in result.resources}
|
||||||
|
|
||||||
assert resource_uri in resource_uris
|
assert "skill://mcp-details/SKILL.md" in resource_uris
|
||||||
|
assert "skill://mcp-details/_manifest" in resource_uris
|
||||||
|
assert "resource://catalog/skills_index" not in resource_uris
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_lists_resource_templates(self, mcp_session_factory) -> None:
|
async def test_lists_resource_templates(self, mcp_session_factory) -> None:
|
||||||
"""Ensures resources/templates/list includes skills and prompt templates."""
|
"""Ensures supporting files use per-skill wildcard templates."""
|
||||||
async with mcp_session_factory() as mcp_session:
|
async with mcp_session_factory() as mcp_session:
|
||||||
result = await mcp_session.list_resource_templates()
|
result = await mcp_session.list_resource_templates()
|
||||||
template_uris = {template.uriTemplate for template in result.resourceTemplates}
|
template_uris = {template.uriTemplate for template in result.resourceTemplates}
|
||||||
|
|
||||||
assert "resource://skills/{skill_id}/document" in template_uris
|
assert "skill://mcp-details/{path*}" in template_uris
|
||||||
assert "resource://prompts/{prompt_id}/document" in template_uris
|
assert "resource://skills/{skill_id}/document" not in template_uris
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_reads_mcp_details_skill_document(self, mcp_session_factory) -> None:
|
async def test_reads_manifest_and_supporting_file(self, mcp_session_factory) -> None:
|
||||||
"""Ensures read_resource resolves the mcp-details skill document URI."""
|
"""Ensures manifest paths resolve through the supporting-file template."""
|
||||||
async with mcp_session_factory() as mcp_session:
|
async with mcp_session_factory() as mcp_session:
|
||||||
result = await mcp_session.call_tool(
|
manifest_result = await mcp_session.read_resource("skill://mcp-details/_manifest")
|
||||||
"read_resource",
|
manifest = json.loads(manifest_result.contents[0].text)
|
||||||
{"uri": "resource://skills/mcp-details/document"},
|
reference = next(file["path"] for file in manifest["files"] if file["path"].startswith("references/"))
|
||||||
)
|
reference_result = await mcp_session.read_resource(f"skill://mcp-details/{reference}")
|
||||||
|
|
||||||
assert result.isError is False
|
assert manifest["skill"] == "mcp-details"
|
||||||
assert result.content
|
assert reference_result.contents
|
||||||
|
|
||||||
class TestPrompts:
|
|
||||||
"""Covers MCP prompt discovery surface."""
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_lists_registered_prompts(self, mcp_session_factory) -> None:
|
|
||||||
"""Ensures prompts/list returns at least one registered prompt."""
|
|
||||||
async with mcp_session_factory() as mcp_session:
|
|
||||||
result = await mcp_session.list_prompts()
|
|
||||||
|
|
||||||
assert result.prompts
|
|
||||||
|
|||||||
@@ -65,6 +65,8 @@ nav = [
|
|||||||
] },
|
] },
|
||||||
{ "Prompts" = [
|
{ "Prompts" = [
|
||||||
{ "Authoring" = "prompts/authoring/PROMPT.md" },
|
{ "Authoring" = "prompts/authoring/PROMPT.md" },
|
||||||
|
{ "JSFiddle Page Layout" = "prompts/jsfiddle-page-layout/PROMPT.md" },
|
||||||
|
{ "NiceGUI Component Extraction" = "prompts/nicegui-component-extraction/PROMPT.md" },
|
||||||
{ "Pytest Fill Scaffold" = "prompts/pytest-fill-scaffold/PROMPT.md" },
|
{ "Pytest Fill Scaffold" = "prompts/pytest-fill-scaffold/PROMPT.md" },
|
||||||
{ "Pytest Scaffold" = "prompts/pytest-scaffold/PROMPT.md" },
|
{ "Pytest Scaffold" = "prompts/pytest-scaffold/PROMPT.md" },
|
||||||
{ "Greenfield Architecture" = "prompts/greenfield-architecture/PROMPT.md" },
|
{ "Greenfield Architecture" = "prompts/greenfield-architecture/PROMPT.md" },
|
||||||
@@ -92,9 +94,11 @@ nav = [
|
|||||||
{ "Overview" = "skills/async-fastapi-sqlmodel/SKILL.md" },
|
{ "Overview" = "skills/async-fastapi-sqlmodel/SKILL.md" },
|
||||||
{ "Engine" = "skills/async-fastapi-sqlmodel/references/engine.md" },
|
{ "Engine" = "skills/async-fastapi-sqlmodel/references/engine.md" },
|
||||||
{ "Session" = "skills/async-fastapi-sqlmodel/references/session.md" },
|
{ "Session" = "skills/async-fastapi-sqlmodel/references/session.md" },
|
||||||
|
{ "FastAPI" = "skills/async-fastapi-sqlmodel/references/fastapi.md" },
|
||||||
{ "Tx" = "skills/async-fastapi-sqlmodel/references/transactions.md" },
|
{ "Tx" = "skills/async-fastapi-sqlmodel/references/transactions.md" },
|
||||||
{ "SQLModel" = "skills/async-fastapi-sqlmodel/references/sqlmodel.md" },
|
{ "SQLModel" = "skills/async-fastapi-sqlmodel/references/sqlmodel.md" },
|
||||||
{ "CRUD" = "skills/async-fastapi-sqlmodel/references/crud.md" },
|
{ "CRUD" = "skills/async-fastapi-sqlmodel/references/crud.md" },
|
||||||
|
{ "Testing" = "skills/async-fastapi-sqlmodel/references/testing.md" },
|
||||||
{ "IO" = "skills/async-fastapi-sqlmodel/references/implicit_io.md" },
|
{ "IO" = "skills/async-fastapi-sqlmodel/references/implicit_io.md" },
|
||||||
{ "Obs" = "skills/async-fastapi-sqlmodel/references/observability.md" },
|
{ "Obs" = "skills/async-fastapi-sqlmodel/references/observability.md" },
|
||||||
{ "Template" = "skills/async-fastapi-sqlmodel/references/template.md" },
|
{ "Template" = "skills/async-fastapi-sqlmodel/references/template.md" },
|
||||||
|
|||||||
Reference in New Issue
Block a user