Compare commits
144
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6ec12a100a | ||
|
|
9be7c27410 | ||
|
|
a157489634 | ||
|
|
b5d6e60d45 | ||
|
|
f240486a7e | ||
|
|
5005cd7001 | ||
|
|
88ff4c2c71 | ||
|
|
5b6d5aaec4 | ||
|
|
2a2700b78c | ||
|
|
7162645ff8 | ||
|
|
44edffb8b7 | ||
|
|
c6817a074e | ||
|
|
7ac90d29dd | ||
|
|
0dc06f72ca | ||
|
|
cd11ea8255 | ||
|
|
bc21643e8c | ||
|
|
1ed5856db0 | ||
|
|
70695ff218 | ||
|
|
a238fb4dc3 | ||
|
|
b6f109cf91 | ||
|
|
3abafc4850 | ||
|
|
d4c7952175 | ||
|
|
da58e20b69 | ||
|
|
d79025538b | ||
|
|
7b1e5fcacb | ||
|
|
37461fd880 | ||
|
|
226f19b2c6 | ||
|
|
a18c8456d3 | ||
|
|
34e6d693ab | ||
|
|
efba051cb5 | ||
|
|
c9b6e137f2 | ||
|
|
8f26051a52 | ||
|
|
bc0d6ede49 | ||
|
|
aed2e41ef0 | ||
|
|
d999a04144 | ||
|
|
4818e86a1e | ||
|
|
b6393f1222 | ||
|
|
3897eabfbc | ||
|
|
9e0097708c | ||
|
|
42ea105bee | ||
|
|
5e20f69cfe | ||
|
|
007d823c0a | ||
|
|
27f783fc90 | ||
|
|
7970e76d4f | ||
|
|
70dd0f45d9 | ||
|
|
963805c551 | ||
|
|
a3ca1a65c2 | ||
|
|
94dd47cc19 | ||
|
|
b3d4e55a15 | ||
|
|
7b2b80ecf2 | ||
|
|
d4ca78dbfb | ||
|
|
eeeb6ecdbe | ||
|
|
0177496fab | ||
|
|
00498a2fed | ||
|
|
913ba66d8b | ||
|
|
e2c199c1b7 | ||
|
|
45a1e56d1c | ||
|
|
a6ccc14917 | ||
|
|
a0ae38d0cc | ||
|
|
34d3808bbb | ||
|
|
1cfe9c8e40 | ||
|
|
313c4ecb1e | ||
|
|
ea5450f6cb | ||
|
|
ab53c239bf | ||
|
|
35d7fa1718 | ||
|
|
e93462ec3b | ||
|
|
58a94ad9b6 | ||
|
|
c893173fcc | ||
|
|
123c491413 | ||
|
|
3c7f7e61b7 | ||
|
|
76ea9ebbda | ||
|
|
0aa7ace272 | ||
|
|
4da2b0ac83 | ||
|
|
806bb15bcc | ||
|
|
ff7cd4a07f | ||
|
|
3603471699 | ||
|
|
dcbf570a13 | ||
|
|
d3b336f1e3 | ||
|
|
1f7e63267a | ||
|
|
69cd9037a3 | ||
|
|
b9bb11ac02 | ||
|
|
4958eeb3ef | ||
|
|
5a31ba6390 | ||
|
|
36347ff4a5 | ||
|
|
c189677717 | ||
|
|
37fa9b6c6f | ||
|
|
34923b51d7 | ||
|
|
2d65d83162 | ||
|
|
3a6e2665dd | ||
|
|
b98d8b782a | ||
|
|
36032040ae | ||
|
|
4f05f13e45 | ||
|
|
57347077a9 | ||
|
|
7fec3a4337 | ||
|
|
c5b7733528 | ||
|
|
3c5db37223 | ||
|
|
29130c3a0c | ||
|
|
aec3500370 | ||
|
|
9c8ab70c06 | ||
|
|
9a9432cc55 | ||
|
|
4320a251f5 | ||
|
|
caa4a5079a | ||
|
|
993dc6a879 | ||
|
|
dab539489a | ||
|
|
197fa32f2c | ||
|
|
c653c7024b | ||
|
|
82b50fb63b | ||
|
|
f8e0c14d46 | ||
|
|
098a2418ee | ||
|
|
7f672b9c8f | ||
|
|
3c5efc6018 | ||
|
|
0b2d45d419 | ||
|
|
406fd63a07 | ||
|
|
323f02102d | ||
|
|
906bba427b | ||
|
|
06d5fc18f2 | ||
|
|
38edc4ac36 | ||
|
|
c73771c2f4 | ||
|
|
33144da02f | ||
|
|
0a9dadd5a8 | ||
|
|
660ca88e47 | ||
|
|
e60fc4b27b | ||
|
|
8817d2586f | ||
|
|
bb7508cf65 | ||
|
|
467e1d3c35 | ||
|
|
3885774e5b | ||
|
|
f54cacd6cb | ||
|
|
19f3c1740a | ||
|
|
c273ecfc54 | ||
|
|
fa4498cb78 | ||
|
|
adaa4177fe | ||
|
|
85355a8509 | ||
|
|
127e56692e | ||
|
|
85eb75d188 | ||
|
|
5c4de7b721 | ||
|
|
ed6068f398 | ||
|
|
75b0c8d192 | ||
|
|
45d8beda8a | ||
|
|
7a9e4044f0 | ||
|
|
3347443ca9 | ||
|
|
964cd6f76d | ||
|
|
ef3255544f | ||
|
|
be9551c76e | ||
|
|
9c3fafd2fe |
@@ -0,0 +1,49 @@
|
|||||||
|
# personal-mcp MCP Usage
|
||||||
|
|
||||||
|
This repository is resource-first.
|
||||||
|
|
||||||
|
- Canonical skill guidance lives in `docs/skills/<skill-id>/SKILL.md`.
|
||||||
|
- The machine-facing skill contract is FastMCP's native `skill://` resource family.
|
||||||
|
|
||||||
|
When a task appears to match a documented implementation pattern in `personal-mcp`, use this sequence:
|
||||||
|
|
||||||
|
1. Prefer an already attached native skill resource.
|
||||||
|
2. Otherwise browse native MCP resources and compare `skill://<name>/SKILL.md` descriptions.
|
||||||
|
3. Read the best matching main skill file, or at most 2 candidate main files.
|
||||||
|
4. Read `skill://<name>/_manifest` only when supporting material may be useful.
|
||||||
|
5. Fetch only the relevant supporting paths from that manifest.
|
||||||
|
6. Reconcile skill guidance with the actual repository code before proposing or making changes.
|
||||||
|
|
||||||
|
Preferred MCP resource order:
|
||||||
|
|
||||||
|
1. `skill://<name>/SKILL.md`
|
||||||
|
2. `skill://<name>/_manifest` when needed
|
||||||
|
3. `skill://<name>/<supporting-path>` for selected supporting files
|
||||||
|
|
||||||
|
Selection rules:
|
||||||
|
|
||||||
|
- Prefer the closest `name` and `description` match.
|
||||||
|
- Keep context bounded; do not load many skill documents speculatively.
|
||||||
|
- If confidence is low after reading at most two main files, ask one clarifying question before loading more context.
|
||||||
|
|
||||||
|
Repository-specific guidance:
|
||||||
|
|
||||||
|
- For tasks about adding or modifying a skill, use `skill://copilot-customization/SKILL.md` when relevant.
|
||||||
|
- Keep skills provider-native; do not add custom skill catalogs, per-skill resource modules, or skill-specific discovery tools.
|
||||||
|
|
||||||
|
|
||||||
|
## Python Checks
|
||||||
|
|
||||||
|
After changes run these commands to confirm functionality. Resolve any errors
|
||||||
|
|
||||||
|
```python
|
||||||
|
uv run ruff check
|
||||||
|
```
|
||||||
|
|
||||||
|
```python
|
||||||
|
uv run ty check
|
||||||
|
```
|
||||||
|
|
||||||
|
```python
|
||||||
|
uv run pytest
|
||||||
|
```
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
---
|
||||||
|
name: Authoring Content
|
||||||
|
description: "Use when editing Markdown under docs/. Routes authors to the canonical docs ownership, layout, and symlink guidance."
|
||||||
|
applyTo: 'docs/**/*.md'
|
||||||
|
---
|
||||||
|
|
||||||
|
For edits under `docs/`, use the [Authoring Guide](../../docs/authoring.md) as the entry point for content placement and contracts.
|
||||||
|
|
||||||
|
For source-tree ownership, symlink, packaging, or runtime questions, follow [Source Tree Ownership](../../docs/authoring.md). Treat that section as authoritative instead of restating its guidance here.
|
||||||
|
|
||||||
|
|
||||||
|
Primary references:
|
||||||
|
- [Skill contract](../../docs/contracts/skill_contract.md)
|
||||||
|
- [Prompt contract](../../docs/contracts/prompt.md)
|
||||||
|
- [Frontmatter contract](../../docs/contracts/frontmatter.md)
|
||||||
|
- [URI contract](../../docs/contracts/uris.md)
|
||||||
|
- `skill://zensical-docs/SKILL.md`
|
||||||
|
|
||||||
|
Inspect `skill://zensical-docs/_manifest` only when a supporting documentation reference is needed.
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
---
|
||||||
|
name: FastMCP Python Guidance
|
||||||
|
description: Route FastMCP Python changes to the Personal MCP source-reference skill.
|
||||||
|
applyTo: '**/*.py'
|
||||||
|
---
|
||||||
|
|
||||||
|
A core part of this repository is [FastMCP](https://gofastmcp.com/servers/server).
|
||||||
|
|
||||||
|
For FastMCP implementation or protocol questions, load `skill://mcp-details/SKILL.md` first.
|
||||||
|
|
||||||
|
Inspect `skill://mcp-details/_manifest` only when a source reference is needed, then read the relevant supporting path. For FastMCP Python APIs, prefer the supporting reference that covers SDKs and FastMCP.
|
||||||
|
|
||||||
|
This repository exposes skills through native MCP resources and prompts through native MCP prompt operations.
|
||||||
|
|
||||||
|
Reconcile the skill guidance with the installed FastMCP version and the repository's existing implementation before editing.
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
---
|
||||||
|
name: Pytest Scaffolding Guidance
|
||||||
|
description: Route tests edits to the Personal MCP pytesting resource.
|
||||||
|
applyTo: 'tests/**'
|
||||||
|
---
|
||||||
|
|
||||||
|
When editing files under `tests/`, use `skill://pytesting/SKILL.md` as the primary guidance source for test scaffolding and pytest authoring decisions.
|
||||||
|
|
||||||
|
Execution pattern:
|
||||||
|
|
||||||
|
1. Load `skill://pytesting/SKILL.md` first.
|
||||||
|
2. Inspect `skill://pytesting/_manifest` only when a supporting reference is needed.
|
||||||
|
3. Apply only the portions relevant to the file being edited.
|
||||||
|
4. Keep tests focused, deterministic, and aligned with repository conventions.
|
||||||
|
5. Include source-document links for any feature-level recommendation.
|
||||||
|
|
||||||
|
If task intent is ambiguous, ask one clarifying question before editing.
|
||||||
|
|
||||||
|
Be sure to also refer to the [testing page](../../docs/testing.md) page for design detail
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
---
|
||||||
|
name: VS Code Configuration
|
||||||
|
description: Route .vscode edits to the Personal MCP VS Code configuration skill resource.
|
||||||
|
applyTo: '.vscode/**'
|
||||||
|
---
|
||||||
|
|
||||||
|
When editing files under `.vscode/`, use `skill://vscode-configuration/SKILL.md` as the primary guidance source.
|
||||||
|
|
||||||
|
Execution pattern:
|
||||||
|
|
||||||
|
1. Load `skill://vscode-configuration/SKILL.md` first.
|
||||||
|
2. Inspect `skill://vscode-configuration/_manifest` and select only the matching reference page for the current file type:
|
||||||
|
- `launch.json` -> debug launch configurations.
|
||||||
|
- `tasks.json` -> tasks.json project tasks.
|
||||||
|
- `mcp.json` -> mcp.json MCP server configuration.
|
||||||
|
3. Prefer the smallest safe config change and keep settings explicit.
|
||||||
|
4. Include source-document links for any feature-level recommendation.
|
||||||
|
|
||||||
|
If task intent is ambiguous, ask one clarifying question before editing.
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
---
|
||||||
|
name: Zensical Docs Markdown Guidance
|
||||||
|
description: Use the Zensical docs MCP resource when editing Markdown documentation in this repository.
|
||||||
|
applyTo: '**/*.md'
|
||||||
|
---
|
||||||
|
|
||||||
|
When editing Markdown files in this repository, use `skill://zensical-docs/SKILL.md` for relevant documentation authoring guidance. Inspect `skill://zensical-docs/_manifest` only when a supporting reference is needed.
|
||||||
|
|
||||||
|
Prefer Zensical-native documentation conventions when they cover the need cleanly, while preserving expected MkDocs compatibility unless the Zensical guidance intentionally diverges.
|
||||||
|
|
||||||
|
Always check to make sure the entries in `/home/john/Documents/prompts/zensical.toml` are up to date with any changes.
|
||||||
|
|
||||||
|
Also ensure that top-level pages specify icons in their front matter
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
---
|
||||||
|
name: Pytest Fill Scaffold
|
||||||
|
description: Fill scaffolded pytest test methods with assertions, fixtures, and minimal test data while preserving concise test names and one-line intent docstrings.
|
||||||
|
argument-hint: Target test file(s) under tests plus stack (pure-python, fastapi, sqlalchemy-sync, sqlalchemy-async, or mixed)
|
||||||
|
agent: agent
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pytest Fill Scaffold
|
||||||
|
|
||||||
|
Use this prompt after test scaffolding exists and method names/docstrings are already in place.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
- Target test file(s) under `tests/`.
|
||||||
|
- Stack type:
|
||||||
|
- `pure-python`
|
||||||
|
- `fastapi`
|
||||||
|
- `sqlalchemy-sync`
|
||||||
|
- `sqlalchemy-async`
|
||||||
|
- `mixed`
|
||||||
|
- Optional constraints:
|
||||||
|
- keep implementation minimal vs comprehensive
|
||||||
|
- marker lane target (`unit`, `integration`, `smoke`)
|
||||||
|
|
||||||
|
## Required References
|
||||||
|
|
||||||
|
Load these in order and use only what matches the task:
|
||||||
|
|
||||||
|
1. Core defaults: [pytest scaffolding skill](../../docs/skills/pytesting/SKILL.md)
|
||||||
|
2. Naming/hierarchy preservation: [naming and organization](../../docs/skills/pytesting/references/naming-and-organization.md)
|
||||||
|
3. Baseline pytest fixtures/markers: [pytest docs notes](../../docs/skills/pytesting/references/pytest-docs.md)
|
||||||
|
4. FastAPI-specific behavior (only when needed): [fastapi testing](../../docs/skills/pytesting/references/fastapi-testing.md)
|
||||||
|
5. SQLAlchemy-specific behavior (only when needed): [sqlalchemy testing](../../docs/skills/pytesting/references/sqlalchemy-testing.md)
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. Inspect target files and treat human-reviewed docstring-only scaffolds as invariant.
|
||||||
|
2. Convert each scaffolded method into an executable test with a single behavior focus.
|
||||||
|
3. Keep one-line docstrings for class and method intent.
|
||||||
|
4. Add or refine fixtures at the nearest useful scope:
|
||||||
|
- global in `tests/conftest.py` only when broadly reusable
|
||||||
|
- subtree `conftest.py` for domain-specific fixtures
|
||||||
|
5. Assign markers consistent with cost and dependencies:
|
||||||
|
- `unit` for pure logic
|
||||||
|
- `integration` for framework/DB contracts
|
||||||
|
- `smoke` for thin critical-path checks
|
||||||
|
6. Validate in this order:
|
||||||
|
- `uv run pytest --collect-only -q`
|
||||||
|
- `uv run pytest -m unit -q` when unit tests are touched
|
||||||
|
- `uv run pytest -q` if dependencies are available
|
||||||
|
|
||||||
|
## Authoring Rules
|
||||||
|
|
||||||
|
- Prefer deterministic tests and explicit setup/teardown.
|
||||||
|
- Keep assertions precise and readable.
|
||||||
|
- Do not overfit tests to private implementation details.
|
||||||
|
- If a scaffolded class or method has only a docstring body, treat its name and hierarchy as locked.
|
||||||
|
- Do not rename, move, merge, split, or re-nest docstring-only scaffolded tests unless explicitly requested.
|
||||||
|
- Preserve existing one-line docstrings on scaffolded classes and methods unless they are factually incorrect.
|
||||||
|
- If stack details are missing and would change fixture strategy, ask one concise clarifying question before editing.
|
||||||
|
|
||||||
|
## Output Format
|
||||||
|
|
||||||
|
Return:
|
||||||
|
1. Files updated.
|
||||||
|
2. Fixture and marker decisions.
|
||||||
|
3. Which references were used and why.
|
||||||
|
4. Validation command results.
|
||||||
|
5. Risks or open questions.
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
---
|
||||||
|
name: Pytest Scaffold
|
||||||
|
description: Plan and scaffold pytest test files, class hierarchy, and concise method names for selected Python modules in this repository.
|
||||||
|
argument-hint: Target module path(s) in src plus scope (plan-only or scaffold)
|
||||||
|
agent: agent
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pytest Scaffold
|
||||||
|
|
||||||
|
Use this prompt to do in one run what we have been doing manually in chat:
|
||||||
|
1. Build a naming and hierarchy plan for tests.
|
||||||
|
2. Scaffold test files and class/method skeletons.
|
||||||
|
3. Keep test method names concise because intent is carried by one-line docstrings.
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
- Target module path(s) under `src/`.
|
||||||
|
- Scope mode:
|
||||||
|
- `plan-only`
|
||||||
|
- `scaffold`
|
||||||
|
- Optional constraints:
|
||||||
|
- flattening preferences for path mapping under `tests/`
|
||||||
|
- method naming style preference
|
||||||
|
|
||||||
|
## Repository Rules To Apply
|
||||||
|
|
||||||
|
- Use [pytest scaffolding skill](../../docs/skills/pytesting/SKILL.md) for strategy and defaults.
|
||||||
|
- Use [naming and organization reference](../../docs/skills/pytesting/references/naming-and-organization.md) before finalizing hierarchy.
|
||||||
|
- Use `uv run pytest --collect-only -q` as structural validation.
|
||||||
|
- Default to a source-mirror style adapted to this repository:
|
||||||
|
- map selected modules to `tests/` with concise path segments when requested
|
||||||
|
- keep one test module per source module
|
||||||
|
|
||||||
|
## Execution Steps
|
||||||
|
|
||||||
|
1. Inspect current `tests/` layout and identify existing naming patterns.
|
||||||
|
2. Propose a concise hierarchy plan first:
|
||||||
|
- test file paths
|
||||||
|
- class hierarchy
|
||||||
|
- method naming pattern
|
||||||
|
- fixture placement (`tests/conftest.py` vs subtree `conftest.py`)
|
||||||
|
3. If scope mode is `scaffold`, implement the skeleton:
|
||||||
|
- create missing test modules
|
||||||
|
- create class hierarchy
|
||||||
|
- add one-line docstrings to every class and test method
|
||||||
|
- keep test method names short and behavior-focused
|
||||||
|
- treat resulting docstring-only scaffolds as human-reviewed baseline for future fill-in work
|
||||||
|
4. Validate collection with `uv run pytest --collect-only -q`.
|
||||||
|
5. Report results:
|
||||||
|
- files created or updated
|
||||||
|
- collection outcome
|
||||||
|
- any ambiguities or follow-up choices
|
||||||
|
|
||||||
|
## Class And Method Shape Defaults
|
||||||
|
|
||||||
|
- Class shape:
|
||||||
|
- `Test<PrimarySubject>` as the top-level subject class
|
||||||
|
- nested `Test<MethodOrArea>` classes when it improves context
|
||||||
|
- top-level `Test<FunctionName>` classes for standalone module functions
|
||||||
|
- Method shape:
|
||||||
|
- `test_<short_outcome>` naming
|
||||||
|
- one behavior target per method name
|
||||||
|
- one-line docstring that states the full intent
|
||||||
|
|
||||||
|
## Output Format
|
||||||
|
|
||||||
|
Return:
|
||||||
|
1. Discovery summary and references consulted.
|
||||||
|
2. Proposed or applied test tree.
|
||||||
|
3. Class and method naming map.
|
||||||
|
4. Validation command results.
|
||||||
|
5. Open questions only if they block confident completion.
|
||||||
@@ -0,0 +1,183 @@
|
|||||||
|
## Goal
|
||||||
|
|
||||||
|
Build a local, self-hosted documentation knowledge base that can ingest software docs, generate embeddings, store them in SQLite, and expose high-quality retrieval through MCP tools and resources.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Core Architectural Decisions
|
||||||
|
|
||||||
|
### Storage
|
||||||
|
|
||||||
|
Use:
|
||||||
|
|
||||||
|
* SQLite for metadata and document storage
|
||||||
|
* FTS5 for keyword search
|
||||||
|
* sqlite-vec for vector similarity search
|
||||||
|
|
||||||
|
Avoid a separate vector database unless scale requirements emerge.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Embeddings
|
||||||
|
|
||||||
|
Use local embedding models via:
|
||||||
|
|
||||||
|
* sentence-transformers
|
||||||
|
|
||||||
|
Recommended model:
|
||||||
|
|
||||||
|
```text
|
||||||
|
BAAI/bge-base-en-v1.5
|
||||||
|
```
|
||||||
|
|
||||||
|
Store embeddings alongside document chunks.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Ingestion
|
||||||
|
|
||||||
|
Primary sources:
|
||||||
|
|
||||||
|
1. Git repositories containing Markdown docs
|
||||||
|
2. Documentation websites via Crawl4AI
|
||||||
|
3. Sitemap-driven crawls when available
|
||||||
|
|
||||||
|
Pipeline:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Source
|
||||||
|
↓
|
||||||
|
Extract
|
||||||
|
↓
|
||||||
|
Normalize
|
||||||
|
↓
|
||||||
|
Chunk by headings
|
||||||
|
↓
|
||||||
|
Embed
|
||||||
|
↓
|
||||||
|
Store
|
||||||
|
```
|
||||||
|
|
||||||
|
Track content hashes so unchanged documents are skipped during reindexing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Retrieval
|
||||||
|
|
||||||
|
Implement hybrid retrieval:
|
||||||
|
|
||||||
|
```text
|
||||||
|
FTS5 keyword search
|
||||||
|
+
|
||||||
|
sqlite-vec similarity search
|
||||||
|
↓
|
||||||
|
Candidate set
|
||||||
|
↓
|
||||||
|
Reranker
|
||||||
|
↓
|
||||||
|
Final results
|
||||||
|
```
|
||||||
|
|
||||||
|
Reranker:
|
||||||
|
|
||||||
|
```text
|
||||||
|
BAAI/bge-reranker-v2
|
||||||
|
```
|
||||||
|
|
||||||
|
The retriever owns all ranking logic.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Public Interface
|
||||||
|
|
||||||
|
Do not expose vector search directly.
|
||||||
|
|
||||||
|
Expose a retrieval service through MCP:
|
||||||
|
|
||||||
|
```python
|
||||||
|
search_docs(query)
|
||||||
|
|
||||||
|
get_context(query)
|
||||||
|
|
||||||
|
get_doc(path)
|
||||||
|
```
|
||||||
|
|
||||||
|
The MCP layer becomes the stable API.
|
||||||
|
|
||||||
|
Clients never interact with embeddings or vectors.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Repository Layout
|
||||||
|
|
||||||
|
```text
|
||||||
|
src/
|
||||||
|
├── knowledge/
|
||||||
|
│ ├── models.py
|
||||||
|
│ ├── chunking.py
|
||||||
|
│ ├── embeddings.py
|
||||||
|
│ ├── ingestion.py
|
||||||
|
│ ├── sqlite_store.py
|
||||||
|
│ ├── hybrid_search.py
|
||||||
|
│ ├── reranker.py
|
||||||
|
│ └── retrieval.py
|
||||||
|
│
|
||||||
|
├── sources/
|
||||||
|
│ ├── git_docs.py
|
||||||
|
│ ├── crawl4ai_docs.py
|
||||||
|
│ └── sitemap_docs.py
|
||||||
|
│
|
||||||
|
├── mcp_server/
|
||||||
|
│ ├── tools.py
|
||||||
|
│ └── resources.py
|
||||||
|
│
|
||||||
|
└── cli/
|
||||||
|
├── ingest.py
|
||||||
|
└── reindex.py
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Retrieval Flow
|
||||||
|
|
||||||
|
```text
|
||||||
|
User Query
|
||||||
|
↓
|
||||||
|
Embed Query
|
||||||
|
↓
|
||||||
|
FTS5 Search
|
||||||
|
+
|
||||||
|
Vector Search
|
||||||
|
↓
|
||||||
|
Merge Results
|
||||||
|
↓
|
||||||
|
Rerank
|
||||||
|
↓
|
||||||
|
Return Context Bundle
|
||||||
|
```
|
||||||
|
|
||||||
|
Where a context bundle contains:
|
||||||
|
|
||||||
|
```python
|
||||||
|
ContextBundle(
|
||||||
|
passages=[...],
|
||||||
|
citations=[...],
|
||||||
|
related_docs=[...],
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Future Extensions
|
||||||
|
|
||||||
|
Without changing the architecture:
|
||||||
|
|
||||||
|
* Multiple documentation corpora
|
||||||
|
* Version-aware retrieval
|
||||||
|
* Code snippet indexing
|
||||||
|
* MCP resources for specific topics
|
||||||
|
* LangGraph integration
|
||||||
|
* Docker deployment
|
||||||
|
* Scheduled reindexing
|
||||||
|
|
||||||
|
The key design principle is: **treat the vector store as an internal implementation detail and expose a retrieval-oriented MCP interface instead.**
|
||||||
@@ -2,3 +2,5 @@
|
|||||||
__pycache__
|
__pycache__
|
||||||
.cache*
|
.cache*
|
||||||
site/
|
site/
|
||||||
|
|
||||||
|
*.log*
|
||||||
Vendored
+86
@@ -0,0 +1,86 @@
|
|||||||
|
{
|
||||||
|
"version": "2.0.0",
|
||||||
|
"tasks": [
|
||||||
|
{
|
||||||
|
"label": "Ruff: Check",
|
||||||
|
"type": "shell",
|
||||||
|
"command": "uv",
|
||||||
|
"args": [
|
||||||
|
"run",
|
||||||
|
"ruff",
|
||||||
|
"check",
|
||||||
|
"."
|
||||||
|
],
|
||||||
|
"options": {
|
||||||
|
"cwd": "${workspaceFolder}"
|
||||||
|
},
|
||||||
|
"problemMatcher": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"label": "Ty: Check",
|
||||||
|
"type": "shell",
|
||||||
|
"command": "uv",
|
||||||
|
"args": [
|
||||||
|
"run",
|
||||||
|
"ty",
|
||||||
|
"check"
|
||||||
|
],
|
||||||
|
"options": {
|
||||||
|
"cwd": "${workspaceFolder}"
|
||||||
|
},
|
||||||
|
"problemMatcher": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"label": "Docs: Build",
|
||||||
|
"type": "shell",
|
||||||
|
"command": "uv",
|
||||||
|
"args": [
|
||||||
|
"run",
|
||||||
|
"zensical",
|
||||||
|
"build"
|
||||||
|
],
|
||||||
|
"options": {
|
||||||
|
"cwd": "${workspaceFolder}"
|
||||||
|
},
|
||||||
|
"group": "build",
|
||||||
|
"problemMatcher": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"label": "Server: Run (uvicorn)",
|
||||||
|
"type": "shell",
|
||||||
|
"command": "uv",
|
||||||
|
"args": [
|
||||||
|
"run",
|
||||||
|
"uvicorn",
|
||||||
|
"personal_mcp.main:create_app",
|
||||||
|
"--factory",
|
||||||
|
"--host",
|
||||||
|
"127.0.0.1",
|
||||||
|
"--port",
|
||||||
|
"8000",
|
||||||
|
"--reload"
|
||||||
|
],
|
||||||
|
"options": {
|
||||||
|
"cwd": "${workspaceFolder}"
|
||||||
|
},
|
||||||
|
"isBackground": true,
|
||||||
|
"problemMatcher": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"label": "Docker: Compose Up (Build)",
|
||||||
|
"type": "shell",
|
||||||
|
"command": "docker",
|
||||||
|
"args": [
|
||||||
|
"compose",
|
||||||
|
"up",
|
||||||
|
"--build",
|
||||||
|
"-d"
|
||||||
|
],
|
||||||
|
"options": {
|
||||||
|
"cwd": "${workspaceFolder}"
|
||||||
|
},
|
||||||
|
"isBackground": true,
|
||||||
|
"problemMatcher": []
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
+32
-21
@@ -1,43 +1,54 @@
|
|||||||
# syntax=docker/dockerfile:1
|
FROM python:3.14-slim AS builder
|
||||||
|
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
|
||||||
FROM python:3.12-slim AS builder
|
|
||||||
|
|
||||||
COPY --from=ghcr.io/astral-sh/uv:0.8.4 /uv /uvx /bin/
|
|
||||||
|
|
||||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||||
PYTHONUNBUFFERED=1 \
|
PYTHONUNBUFFERED=1 \
|
||||||
UV_COMPILE_BYTECODE=1 \
|
UV_COMPILE_BYTECODE=1 \
|
||||||
UV_LINK_MODE=copy
|
UV_LINK_MODE=copy \
|
||||||
|
UV_LOCKED=1
|
||||||
|
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
COPY pyproject.toml uv.lock ./
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
||||||
COPY src ./src
|
--mount=type=bind,source=zensical.toml,target=zensical.toml \
|
||||||
|
--mount=type=bind,source=docs/,target=docs/ \
|
||||||
|
uvx zensical build
|
||||||
|
|
||||||
RUN uv sync --frozen --no-dev
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
||||||
|
--mount=type=bind,source=uv.lock,target=uv.lock \
|
||||||
|
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
|
||||||
|
uv sync --no-install-project
|
||||||
|
|
||||||
COPY docs ./docs
|
# COPY --chown=appuser:appuser . /app
|
||||||
COPY zensical.toml ./
|
|
||||||
|
|
||||||
RUN uv run zensical build
|
# RUN --mount=type=cache,target=/root/.cache/uv \
|
||||||
|
# uv sync --no-editable
|
||||||
|
|
||||||
FROM python:3.12-slim AS runtime
|
FROM python:3.14-slim AS runtime
|
||||||
|
|
||||||
ENV PYTHONDONTWRITEBYTECODE=1 \
|
ENV PYTHONDONTWRITEBYTECODE=1 \
|
||||||
PYTHONUNBUFFERED=1 \
|
PYTHONUNBUFFERED=1 \
|
||||||
PATH="/app/.venv/bin:$PATH" \
|
PATH="/app/.venv/bin:$PATH" \
|
||||||
PERSONAL_MCP_HOST=0.0.0.0 \
|
PERSONAL_MCP_SITE_DIR=/app/site
|
||||||
PERSONAL_MCP_PORT=8765
|
|
||||||
|
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
RUN groupadd --system --gid 1001 appuser \
|
|
||||||
&& useradd --system --uid 1001 --gid appuser --create-home --home-dir /home/appuser appuser
|
|
||||||
|
|
||||||
COPY --from=builder --chown=appuser:appuser /app /app
|
|
||||||
|
|
||||||
EXPOSE 8765
|
EXPOSE 8765
|
||||||
|
|
||||||
|
RUN groupadd --system --gid 1001 appuser && \
|
||||||
|
useradd --system --uid 1001 --gid appuser appuser
|
||||||
|
|
||||||
|
COPY --from=ghcr.io/astral-sh/uv:latest --chown=appuser:appuser /uv /uvx /bin/
|
||||||
|
COPY --from=builder --chown=appuser:appuser /app/.venv /app/.venv
|
||||||
|
COPY --from=builder --chown=appuser:appuser /app/site /app/site
|
||||||
|
COPY --chown=appuser:appuser ./docs /app/docs
|
||||||
|
|
||||||
|
RUN --mount=type=cache,target=/root/.cache/uv \
|
||||||
|
--mount=type=bind,source=uv.lock,target=uv.lock \
|
||||||
|
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \
|
||||||
|
--mount=type=bind,source=src/,target=src/ \
|
||||||
|
uv sync --no-editable --refresh-package prompts
|
||||||
|
|
||||||
USER appuser
|
USER appuser
|
||||||
|
|
||||||
CMD ["uvicorn", "personal_mcp.main:app", "--host", "0.0.0.0", "--port", "8765"]
|
CMD ["uvicorn", "personal_mcp.main:create_app", "--factory", "--host", "0.0.0.0", "--port", "8765"]
|
||||||
|
|||||||
@@ -0,0 +1,8 @@
|
|||||||
|
services:
|
||||||
|
personal-mcp:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
restart: unless-stopped
|
||||||
|
ports:
|
||||||
|
- "8765:8765"
|
||||||
@@ -1,166 +0,0 @@
|
|||||||
---
|
|
||||||
icon: lucide/library
|
|
||||||
---
|
|
||||||
|
|
||||||
# Architecture
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The platform is implemented as a resource-first MCP system with an integrated static documentation surface. The same methodology content powers both MCP resources and the published docs site.
|
|
||||||
|
|
||||||
The system is complete in three layers:
|
|
||||||
|
|
||||||
1. Canonical methodology is maintained in Markdown skill documents.
|
|
||||||
2. Catalog resources provide normalized discovery.
|
|
||||||
3. Zensical builds a static site from those same Markdown sources and the FastAPI app serves it in the FastMCP runtime process.
|
|
||||||
|
|
||||||
This architecture keeps authored content human-friendly while preserving machine-stable contracts.
|
|
||||||
|
|
||||||
## Intent
|
|
||||||
|
|
||||||
The architecture is designed to satisfy three long-term requirements:
|
|
||||||
|
|
||||||
1. Methodology must be editable as markdown by humans.
|
|
||||||
2. Agents must consume stable, discoverable resource contracts, with a minimal read-only catalog tool fallback for constrained clients.
|
|
||||||
3. Public documentation must be pre-built static output served from the application runtime without a separate docs service.
|
|
||||||
|
|
||||||
## System Model
|
|
||||||
|
|
||||||
### Pattern Modules
|
|
||||||
|
|
||||||
Each module encapsulates one methodology domain and publishes resource families:
|
|
||||||
|
|
||||||
1. document
|
|
||||||
|
|
||||||
The document resource returns canonical Markdown, while clients can perform any downstream section extraction they need.
|
|
||||||
|
|
||||||
### Catalog Module
|
|
||||||
|
|
||||||
The catalog is the canonical discovery layer and publishes normalized records for all modules. It may also expose a minimal set of read-only discovery tools that resolve back to the same canonical markdown content when a client chat surface does not expose MCP resource attachment.
|
|
||||||
|
|
||||||
Typical catalog resources:
|
|
||||||
|
|
||||||
1. resource://catalog/patterns
|
|
||||||
2. resource://catalog/patterns_by_id
|
|
||||||
3. resource://catalog/skills_index
|
|
||||||
4. resource://catalog/skills_details
|
|
||||||
|
|
||||||
### Content Sources
|
|
||||||
|
|
||||||
Content is authored in markdown under `docs/` and managed as long-form reference material. Skill documents and companion references now live under `docs/skills/`, while project-authored pages remain alongside them in the docs tree. Resource handlers expose the same authored documents through stable resource URIs.
|
|
||||||
|
|
||||||
### Static Docs Surface
|
|
||||||
|
|
||||||
Static docs are built directly from two markdown source streams:
|
|
||||||
|
|
||||||
1. Project-authored docs pages
|
|
||||||
2. Skill and reference markdown pages
|
|
||||||
|
|
||||||
The merged docs tree is built by Zensical into static files and served by the FastAPI app.
|
|
||||||
|
|
||||||
## Data Flow
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TD
|
|
||||||
A[Authored Markdown] --> C[Resource Handlers]
|
|
||||||
B[Pattern Metadata] --> D[Catalog Resources]
|
|
||||||
A --> E[Zensical Static Build]
|
|
||||||
E --> H[FastAPI Static Mount]
|
|
||||||
H --> I[Served Docs Site]
|
|
||||||
D --> I
|
|
||||||
```
|
|
||||||
|
|
||||||
## Contracts
|
|
||||||
|
|
||||||
### Metadata Contract
|
|
||||||
|
|
||||||
Each pattern module declares:
|
|
||||||
|
|
||||||
1. id
|
|
||||||
2. name
|
|
||||||
3. version
|
|
||||||
4. description
|
|
||||||
5. tags
|
|
||||||
6. capabilities
|
|
||||||
7. depends_on
|
|
||||||
|
|
||||||
### URI Contract
|
|
||||||
|
|
||||||
Module resource URIs are stable and follow:
|
|
||||||
|
|
||||||
1. resource://skills/<skill_id>/document
|
|
||||||
|
|
||||||
Catalog resource URIs are stable and discovery-focused.
|
|
||||||
|
|
||||||
### Versioning Rule
|
|
||||||
|
|
||||||
Published URIs are immutable. Behavioral or schema changes are versioned in metadata and documented through additive migration notes.
|
|
||||||
|
|
||||||
## Static Hosting Pattern
|
|
||||||
|
|
||||||
The docs site is pre-built and served by the same FastAPI runtime process used by the MCP app.
|
|
||||||
|
|
||||||
Runtime behavior:
|
|
||||||
|
|
||||||
1. App starts.
|
|
||||||
2. FastAPI mounts the static docs output directory.
|
|
||||||
3. Requests to docs paths are served as static assets.
|
|
||||||
|
|
||||||
This provides a single deployment artifact with no runtime markdown rendering dependency.
|
|
||||||
|
|
||||||
## Advantages
|
|
||||||
|
|
||||||
### Single Source of Truth
|
|
||||||
|
|
||||||
Methodology is authored once and reused in both MCP resources and docs pages.
|
|
||||||
|
|
||||||
### High-Fidelity Agent Context
|
|
||||||
|
|
||||||
Resources expose the same canonical Markdown that humans author and review.
|
|
||||||
|
|
||||||
### Operational Simplicity
|
|
||||||
|
|
||||||
A single app process serves MCP and docs surfaces.
|
|
||||||
|
|
||||||
### Long-Term Maintainability
|
|
||||||
|
|
||||||
Markdown remains easy to review, while contracts remain stable for clients.
|
|
||||||
|
|
||||||
### Client Independence
|
|
||||||
|
|
||||||
Clients can use Ask, Edit, or Agent modes without requiring server-owned prompt orchestration. However, MCP affordances are still chat-surface-dependent: some clients or sessions expose resource attachment directly, while others make tool invocation the more reliable retrieval path.
|
|
||||||
|
|
||||||
## Authoring and Publishing Lifecycle
|
|
||||||
|
|
||||||
1. Update markdown reference content.
|
|
||||||
2. Update metadata if capability surface changes.
|
|
||||||
3. Build static docs with Zensical.
|
|
||||||
4. Serve built output through FastAPI static mount.
|
|
||||||
|
|
||||||
## Scope and Non-Goals
|
|
||||||
|
|
||||||
In-scope:
|
|
||||||
|
|
||||||
1. Resource-first methodology delivery
|
|
||||||
2. Catalog-based discovery
|
|
||||||
3. Pre-built static docs hosting in app runtime
|
|
||||||
|
|
||||||
Out-of-scope:
|
|
||||||
|
|
||||||
1. Prompt-first orchestration as the primary interface
|
|
||||||
2. Large tool inventories duplicating static guidance across skill modules
|
|
||||||
3. Separate dynamic docs service at runtime
|
|
||||||
|
|
||||||
Allowed exception:
|
|
||||||
|
|
||||||
1. A small catalog-level tool layer is acceptable when it improves client interoperability without creating a second source of truth for skill content.
|
|
||||||
|
|
||||||
## Example Content Inputs
|
|
||||||
|
|
||||||
Existing markdown reference sets are valid examples of authored source material for this architecture:
|
|
||||||
|
|
||||||
1. ../docs/skills/pytest-scaffolding/references/pytest-docs.md
|
|
||||||
2. ../docs/skills/python-logging-dictconfig/references/python-logging-docs.md
|
|
||||||
3. ../docs/skills/fastapi-uv-docker/references/fastapi-best-practices.md
|
|
||||||
|
|
||||||
These inputs are treated as content sources, while resource URIs and catalog payloads remain the machine-facing contracts.
|
|
||||||
-177
@@ -1,177 +0,0 @@
|
|||||||
---
|
|
||||||
icon: lucide/bot
|
|
||||||
---
|
|
||||||
|
|
||||||
# Copilot MCP Mechanics
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
This page explains how the GitHub Copilot extension in VS Code behaves as an MCP client when connected to `personal-mcp`, including why tools can appear while resource attachment appears unavailable.
|
|
||||||
|
|
||||||
## Core Model
|
|
||||||
|
|
||||||
Copilot interacts with MCP servers through separate capability lanes:
|
|
||||||
|
|
||||||
1. tools (invoked by the model during execution)
|
|
||||||
2. resources (attached as read-only context)
|
|
||||||
3. prompts (server-provided prompt templates)
|
|
||||||
|
|
||||||
These lanes are related but independently gated in the client.
|
|
||||||
|
|
||||||
The documented reliable paths are:
|
|
||||||
|
|
||||||
1. attach MCP resources explicitly through `Add Context > MCP Resources` or `MCP: Browse Resources`
|
|
||||||
2. let Copilot invoke MCP tools when the task and tool descriptions make that relevant
|
|
||||||
3. invoke MCP prompts explicitly with `/server.prompt` when your server exposes them
|
|
||||||
|
|
||||||
## What Actually Happens In VS Code
|
|
||||||
|
|
||||||
### MCP server side
|
|
||||||
|
|
||||||
Your server can advertise resources and serve them correctly. In this project that includes catalog resources and skill document resources.
|
|
||||||
|
|
||||||
### Copilot session side
|
|
||||||
|
|
||||||
The chat surface exposes tools, resources, and prompts through different UI paths. In practice, you can encounter sessions where tool use is available but MCP resource attachment is not exposed in `Add Context`.
|
|
||||||
|
|
||||||
That is why you can sometimes see MCP tools before you see `Add Context > MCP Resources`.
|
|
||||||
|
|
||||||
## Why The Picker Sometimes Shows Only Tools
|
|
||||||
|
|
||||||
`MCP Resources...` in Add Context requires at least:
|
|
||||||
|
|
||||||
1. at least one connected MCP server advertises resource capability
|
|
||||||
2. the current chat surface exposes MCP resource attachment
|
|
||||||
|
|
||||||
If the second condition is not met, resources can be available on the server while still being absent from the picker.
|
|
||||||
|
|
||||||
## Practical Workflow
|
|
||||||
|
|
||||||
Use this sequence to confirm behavior:
|
|
||||||
|
|
||||||
1. run `MCP: Browse Resources` and verify resources exist
|
|
||||||
2. use `MCP: List Servers` to verify the server is enabled and running
|
|
||||||
3. open Copilot Chat
|
|
||||||
4. check `Add Context` for `MCP Resources...`
|
|
||||||
5. if still missing, restart the server and reload VS Code window
|
|
||||||
|
|
||||||
## Recommended Usage Pattern
|
|
||||||
|
|
||||||
1. rely on catalog resources for discovery (`skills_index`, `patterns`, etc.)
|
|
||||||
2. fetch only selected skill documents for context
|
|
||||||
3. keep slash commands for deterministic fallback flows
|
|
||||||
|
|
||||||
When resource attachment is unavailable in the active session, use thin catalog discovery tools as operational fallback:
|
|
||||||
|
|
||||||
1. `search_patterns`
|
|
||||||
2. `get_pattern_by_id`
|
|
||||||
3. `get_skill_document_by_id`
|
|
||||||
|
|
||||||
These should stay read-only, minimal, and schema-aligned with catalog resources.
|
|
||||||
|
|
||||||
## What To Type In Copilot Chat
|
|
||||||
|
|
||||||
Use prompts that tell Copilot which MCP feature path to take.
|
|
||||||
|
|
||||||
### If `MCP Resources...` is available
|
|
||||||
|
|
||||||
Use the resource attachment UI first, then ask Copilot to work from the attached material.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
```text
|
|
||||||
I attached the catalog resources and the FastAPI async SQLAlchemy modernization skill document. Use that context to propose a migration plan for this repo.
|
|
||||||
```
|
|
||||||
|
|
||||||
If you want to keep the attachment sequence explicit, use:
|
|
||||||
|
|
||||||
```text
|
|
||||||
I attached personal-mcp catalog resources first. Use them to identify the best matching skill, then work only from the selected skill document.
|
|
||||||
```
|
|
||||||
|
|
||||||
### If only tools are available
|
|
||||||
|
|
||||||
Ask Copilot to explicitly use the catalog tools.
|
|
||||||
|
|
||||||
Example discovery prompt:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Use the personal-mcp catalog tools to search for the most relevant skill for FastAPI async SQLAlchemy modernization. Then load the selected skill document and use it as context for your answer.
|
|
||||||
```
|
|
||||||
|
|
||||||
Example direct-load prompt:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Call get_skill_document_by_id for fastapi-async-sqlalchemy-modernization and use that document as the main context for this task.
|
|
||||||
```
|
|
||||||
|
|
||||||
Example bounded-selection prompt:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Search personal-mcp skills for NiceGUI UI customization, select at most 2 strong matches, load the best skill document, and answer using only that material plus the workspace code.
|
|
||||||
```
|
|
||||||
|
|
||||||
## Repo Instructions Example
|
|
||||||
|
|
||||||
Repo instructions are the best place to teach Copilot when MCP content is relevant and which path to prefer.
|
|
||||||
|
|
||||||
If you add a repo-level `copilot-instructions.md`, keep the rule simple: prefer catalog-first discovery, keep loaded skill context small, and fall back to tools when resource attachment is unavailable.
|
|
||||||
|
|
||||||
Instructions can strongly steer behavior, but they do not guarantee that VS Code will auto-attach MCP resources for a request. For reliable resource use, either attach resources explicitly or prompt Copilot to use the fallback tools.
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
```md
|
|
||||||
# MCP Usage
|
|
||||||
|
|
||||||
When a task may benefit from personal-mcp skills, use this sequence:
|
|
||||||
|
|
||||||
1. Start with personal-mcp catalog discovery when the task appears to match documented implementation patterns.
|
|
||||||
2. Prefer MCP resources when the chat surface exposes resource attachment.
|
|
||||||
3. If MCP resource attachment is unavailable, use catalog tools instead.
|
|
||||||
4. Load only the most relevant skill document or at most 2 skill documents.
|
|
||||||
5. Treat skill documents as guidance, then reconcile them with the actual repository code before making changes.
|
|
||||||
|
|
||||||
Preferred discovery order:
|
|
||||||
|
|
||||||
1. `resource://catalog/skills_index` or `resource://catalog/patterns`
|
|
||||||
2. `resource://skills/<skill-id>/document`
|
|
||||||
|
|
||||||
Tool fallback order:
|
|
||||||
|
|
||||||
1. `search_patterns`
|
|
||||||
2. `get_pattern_by_id`
|
|
||||||
3. `get_skill_document_by_id`
|
|
||||||
|
|
||||||
If confidence is low after catalog discovery, ask one clarifying question before loading more skill documents.
|
|
||||||
```
|
|
||||||
|
|
||||||
That instruction style does two useful things:
|
|
||||||
|
|
||||||
1. it tells Copilot to prefer the MCP server when relevant without forcing it on every prompt
|
|
||||||
2. it keeps context size bounded so skill loading does not become noisy or expensive
|
|
||||||
|
|
||||||
If you want stronger behavior, add one more line that names the MCP server directly:
|
|
||||||
|
|
||||||
```md
|
|
||||||
Use the `personal-mcp` server for skill discovery whenever the task involves documented implementation patterns available from the catalog.
|
|
||||||
```
|
|
||||||
|
|
||||||
## Known Gotcha
|
|
||||||
|
|
||||||
A successful `resources/list` response from the server does not guarantee the resource picker appears in every Copilot session type. UI availability is session-capability-dependent.
|
|
||||||
|
|
||||||
## Further Reading
|
|
||||||
|
|
||||||
### VS Code docs
|
|
||||||
|
|
||||||
1. [Add and manage MCP servers](https://code.visualstudio.com/docs/agent-customization/mcp-servers)
|
|
||||||
2. [MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration)
|
|
||||||
3. [Manage context for AI](https://code.visualstudio.com/docs/chat/copilot-chat-context)
|
|
||||||
4. [AI features cheat sheet](https://code.visualstudio.com/docs/agents/reference/ai-features-cheat-sheet)
|
|
||||||
|
|
||||||
### Project docs
|
|
||||||
|
|
||||||
1. [Resource-First Pattern Module Architecture](./architecture.md)
|
|
||||||
2. [Static Docs Hosting Pattern](./mcp_layout.md)
|
|
||||||
3. [Skill Usage Mechanics](./usage.md)
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
---
|
|
||||||
icon: lucide/rocket
|
|
||||||
---
|
|
||||||
|
|
||||||
# Get started
|
|
||||||
|
|
||||||
## Quick start
|
|
||||||
|
|
||||||
Install dependencies first:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
uv sync
|
|
||||||
```
|
|
||||||
|
|
||||||
Run the app locally with the static docs rebuilt first:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
uv run zensical build && uv run uvicorn personal_mcp.main:app --host 127.0.0.1 --port 8765
|
|
||||||
```
|
|
||||||
|
|
||||||
Build and run the Docker image with the same exposed port:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker build -t personal-mcp . && docker run --rm -p 8765:8765 personal-mcp
|
|
||||||
```
|
|
||||||
|
|
||||||
When the server is running, the health check is available at `/healthz` and the generated docs are available at `/docs/`.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
- [Resource-First Pattern Module Architecture](./architecture.md)
|
|
||||||
- [Static Docs Hosting Pattern](./mcp_layout.md)
|
|
||||||
- [Skill Usage Mechanics](./usage.md)
|
|
||||||
- [Copilot MCP Mechanics](./copilot.md)
|
|
||||||
@@ -1,162 +0,0 @@
|
|||||||
# Static Docs Hosting Pattern
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
This document describes the completed layout and runtime pattern used to host a pre-built static documentation site from the same FastAPI app process that runs the FastMCP server.
|
|
||||||
|
|
||||||
This design intentionally avoids runtime docs rendering and avoids a separate docs hosting service.
|
|
||||||
|
|
||||||
It also treats Markdown as the single source of truth for both MCP resources and published docs.
|
|
||||||
|
|
||||||
## Completed-State Layout
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
---
|
|
||||||
config:
|
|
||||||
treeView:
|
|
||||||
rowIndent: 40
|
|
||||||
lineThickness: 2
|
|
||||||
themeVariables:
|
|
||||||
treeView:
|
|
||||||
labelColor: '#FFFFFF'
|
|
||||||
lineColor: '#FFFFFF'
|
|
||||||
---
|
|
||||||
treeView-beta
|
|
||||||
"project-root"
|
|
||||||
"pyproject.toml"
|
|
||||||
"uv.lock"
|
|
||||||
"zensical.toml"
|
|
||||||
"docs"
|
|
||||||
"index.md"
|
|
||||||
"architecture.md"
|
|
||||||
"mcp_layout.md"
|
|
||||||
"skills"
|
|
||||||
"pytest-scaffolding"
|
|
||||||
"SKILL.md"
|
|
||||||
"references"
|
|
||||||
"python-logging-dictconfig"
|
|
||||||
"SKILL.md"
|
|
||||||
"references"
|
|
||||||
"fastapi-uv-docker"
|
|
||||||
"SKILL.md"
|
|
||||||
"references"
|
|
||||||
"site"
|
|
||||||
"static build output"
|
|
||||||
"src"
|
|
||||||
"personal_mcp"
|
|
||||||
"main.py"
|
|
||||||
"web"
|
|
||||||
"app.py"
|
|
||||||
"docs_mount.py"
|
|
||||||
"catalog"
|
|
||||||
"server.py"
|
|
||||||
"skills"
|
|
||||||
"pytest_scaffolding"
|
|
||||||
"python_logging_dictconfig"
|
|
||||||
"fastapi_uv_docker"
|
|
||||||
```
|
|
||||||
|
|
||||||
Notes:
|
|
||||||
|
|
||||||
1. docs contains both project-authored pages and the canonical skill Markdown tree.
|
|
||||||
2. site contains static build output only.
|
|
||||||
3. docs/skills contains canonical skill Markdown and reference Markdown.
|
|
||||||
4. MCP resources and docs site read from the same Markdown sources.
|
|
||||||
|
|
||||||
## Runtime Composition
|
|
||||||
|
|
||||||
The runtime process serves two surfaces:
|
|
||||||
|
|
||||||
1. MCP protocol surface from FastMCP
|
|
||||||
2. Static docs surface from FastAPI static mount
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TD
|
|
||||||
A[FastMCP Root Server] --> B[MCP Transport]
|
|
||||||
A --> C[FastAPI Application]
|
|
||||||
C --> D[Static Mount /docs]
|
|
||||||
D --> E[Zensical site output directory]
|
|
||||||
```
|
|
||||||
|
|
||||||
## Build and Publish Flow
|
|
||||||
|
|
||||||
The docs flow is pre-build only.
|
|
||||||
|
|
||||||
1. Read authored docs pages and skill markdown sources.
|
|
||||||
2. Build static site with Zensical into site.
|
|
||||||
3. Start app and serve site directory as static files.
|
|
||||||
|
|
||||||
No runtime markdown conversion is required.
|
|
||||||
|
|
||||||
## Content Merge Pattern
|
|
||||||
|
|
||||||
The published docs site always contains both:
|
|
||||||
|
|
||||||
1. Project-authored docs pages
|
|
||||||
2. Skill Markdown content from docs/skills/*/SKILL.md and references
|
|
||||||
|
|
||||||
This ensures the public docs reflect architectural guidance and the exact Markdown served by MCP.
|
|
||||||
|
|
||||||
## Markdown-to-Resource Mapping
|
|
||||||
|
|
||||||
MCP resources map directly to canonical Markdown documents.
|
|
||||||
|
|
||||||
Example mapping model:
|
|
||||||
|
|
||||||
1. docs/skills/<slug>/SKILL.md -> resource://skills/<id>/document
|
|
||||||
2. docs/skills/<slug>/references/*.md -> referenced sections or linked companion documents
|
|
||||||
|
|
||||||
Catalog resources provide discovery metadata and stable identifiers.
|
|
||||||
|
|
||||||
When clients cannot attach MCP resources directly, catalog-level tools may retrieve the same underlying skill documents indirectly. This does not create a second content source; it is only an alternate access path to the same markdown-backed contract.
|
|
||||||
|
|
||||||
## Why This Pattern
|
|
||||||
|
|
||||||
### Operational Simplicity
|
|
||||||
|
|
||||||
One application process serves both protocol and static docs surfaces.
|
|
||||||
|
|
||||||
### Deterministic Docs
|
|
||||||
|
|
||||||
Published docs are immutable static assets for a given build.
|
|
||||||
|
|
||||||
### Documentation Fidelity
|
|
||||||
|
|
||||||
The docs site and MCP resources resolve from the same Markdown sources.
|
|
||||||
|
|
||||||
### Maintainer Experience
|
|
||||||
|
|
||||||
Authors continue to work in markdown while resource contracts remain machine-consumable.
|
|
||||||
|
|
||||||
## FastAPI Static Mount Expectations
|
|
||||||
|
|
||||||
The FastAPI app is expected to:
|
|
||||||
|
|
||||||
1. Mount static directory containing Zensical output.
|
|
||||||
2. Serve index and asset files from that directory.
|
|
||||||
3. Keep docs route stable across releases.
|
|
||||||
|
|
||||||
Recommended route conventions:
|
|
||||||
|
|
||||||
1. /docs for static site root
|
|
||||||
2. /docs/* for static assets and page routes
|
|
||||||
|
|
||||||
## Update Lifecycle
|
|
||||||
|
|
||||||
For each documentation update:
|
|
||||||
|
|
||||||
1. Edit authored docs and skill markdown content.
|
|
||||||
2. Rebuild static site.
|
|
||||||
3. Restart runtime if needed.
|
|
||||||
|
|
||||||
This keeps docs publication explicit and predictable.
|
|
||||||
|
|
||||||
## Example Source Material
|
|
||||||
|
|
||||||
Existing reference docs remain valid content inputs in this pattern:
|
|
||||||
|
|
||||||
1. ../docs/skills/pytest-scaffolding/references/pytest-docs.md
|
|
||||||
2. ../docs/skills/python-logging-dictconfig/references/python-logging-docs.md
|
|
||||||
3. ../docs/skills/fastapi-uv-docker/references/fastapi-best-practices.md
|
|
||||||
|
|
||||||
These are source documents, not deployment artifacts.
|
|
||||||
@@ -1,178 +0,0 @@
|
|||||||
# Hooking Up a New Skill
|
|
||||||
|
|
||||||
Use this checklist after generating a new skill under `docs/skills/<slug>/`.
|
|
||||||
|
|
||||||
## Checklist
|
|
||||||
|
|
||||||
1. Create the authored docs content.
|
|
||||||
Add `docs/skills/<slug>/SKILL.md` and any companion files under `docs/skills/<slug>/references/`.
|
|
||||||
|
|
||||||
2. Choose the three names up front.
|
|
||||||
Use a docs slug like `fastapi-uv-docker`, a resource id like `fastapi-uv-docker`, and a Python package name like `fastapi_uv_docker`.
|
|
||||||
|
|
||||||
3. Add the runtime package.
|
|
||||||
Create `src/personal_mcp/skills/<python_namespace>/` with `__init__.py`, `server.py`, and `metadata.yaml`.
|
|
||||||
|
|
||||||
4. Expose the document resource in `server.py`.
|
|
||||||
Follow the existing pattern: create a `FastMCP` instance, register `resource://skills/<skill-id>/document`, and return the shared loader result.
|
|
||||||
For the default layout, load `docs/skills/<slug>/SKILL.md`.
|
|
||||||
For special cases, set `document_path` in `metadata.yaml` to a repo-relative Markdown file and load from metadata instead of hardcoding a path in the server.
|
|
||||||
|
|
||||||
5. Register the catalog metadata.
|
|
||||||
In `metadata.yaml`, add the skill `id`, `name`, `version`, `description`, `tags`, `capabilities`, and `depends_on`. The `capabilities` list should include `resource://skills/<skill-id>/document`.
|
|
||||||
|
|
||||||
6. Mount the skill in the root server.
|
|
||||||
Import the new server in `src/personal_mcp/mcp.py` and add an `mcp.mount(...)` call with the Python namespace.
|
|
||||||
|
|
||||||
7. Let the loader and catalog do the rest.
|
|
||||||
The document loader reads canonical Markdown from `docs/skills/<slug>/SKILL.md` by default, or from `metadata.yaml`'s optional `document_path` override when present. The catalog discovers metadata from `src/personal_mcp/skills/*/metadata.yaml` automatically.
|
|
||||||
|
|
||||||
8. Rebuild and smoke-test.
|
|
||||||
Run `uv run zensical build` to publish the docs site, then run a quick Python check or start the app to confirm the new resource loads.
|
|
||||||
|
|
||||||
## Discovery Tool Policy
|
|
||||||
|
|
||||||
To keep behavior consistent across MCP clients and Copilot session types, follow this boundary:
|
|
||||||
|
|
||||||
1. Keep per-skill servers resource-only.
|
|
||||||
2. Keep discovery/query tools centralized in the catalog server.
|
|
||||||
3. Keep canonical content in `docs/skills/<slug>/SKILL.md` and expose it through `resource://skills/<skill-id>/document`.
|
|
||||||
|
|
||||||
### Do
|
|
||||||
|
|
||||||
1. Add or update `metadata.yaml` fields (`id`, `description`, `tags`, `capabilities`) so catalog discovery quality stays high.
|
|
||||||
2. Use `document_path` when a skill should expose a Markdown file outside `docs/skills/<slug>/SKILL.md`.
|
|
||||||
3. Use catalog resources as the primary discovery surface.
|
|
||||||
4. Add thin, read-only catalog tools only when client behavior needs a fallback path.
|
|
||||||
|
|
||||||
### Don't
|
|
||||||
|
|
||||||
1. Do not add duplicate discovery tools to each skill package.
|
|
||||||
2. Do not duplicate canonical skill guidance in tool descriptions.
|
|
||||||
3. Do not create mutating catalog tools for skill discovery.
|
|
||||||
|
|
||||||
## Minimal Shape
|
|
||||||
|
|
||||||
- Docs content: `docs/skills/<slug>/SKILL.md`
|
|
||||||
- Optional references: `docs/skills/<slug>/references/*.md`
|
|
||||||
- Runtime package: `src/personal_mcp/skills/<python_namespace>/`
|
|
||||||
- Resource URI: `resource://skills/<skill-id>/document`
|
|
||||||
|
|
||||||
## Quick Validation
|
|
||||||
|
|
||||||
1. Confirm the Markdown document resolves through the loader.
|
|
||||||
`uv run python -c "from personal_mcp.skills.document_loader import load_skill_document; print(load_skill_document(skill_id='<skill-id>', skill_slug='<slug>')['source_path'])"`
|
|
||||||
|
|
||||||
2. Confirm the docs build still works.
|
|
||||||
`uv run zensical build`
|
|
||||||
|
|
||||||
## server.py Template
|
|
||||||
|
|
||||||
```python
|
|
||||||
from fastmcp import FastMCP
|
|
||||||
|
|
||||||
from personal_mcp.skills.document_loader import load_skill_document
|
|
||||||
|
|
||||||
<python_namespace>_server = FastMCP("<skill-id>")
|
|
||||||
|
|
||||||
|
|
||||||
@<python_namespace>_server.resource("resource://skills/<skill-id>/document")
|
|
||||||
def skill_document() -> dict[str, str]:
|
|
||||||
"""Return the canonical Markdown document for this skill."""
|
|
||||||
return load_skill_document(
|
|
||||||
skill_id="<skill-id>",
|
|
||||||
skill_slug="<slug>",
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
## metadata.yaml Template
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
id: <skill-id>
|
|
||||||
name: <Human Readable Name>
|
|
||||||
version: 1.0.0
|
|
||||||
description: <One sentence describing what the skill provides.>
|
|
||||||
tags:
|
|
||||||
- <tag-one>
|
|
||||||
- <tag-two>
|
|
||||||
document_path: <optional repo-relative path to a markdown file>
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/<skill-id>/document
|
|
||||||
depends_on: []
|
|
||||||
```
|
|
||||||
|
|
||||||
Omit `document_path` when the canonical document is `docs/skills/<slug>/SKILL.md`.
|
|
||||||
|
|
||||||
## Root Mount Template
|
|
||||||
|
|
||||||
Add an import in `src/personal_mcp/mcp.py`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from personal_mcp.skills.<python_namespace>.server import <python_namespace>_server
|
|
||||||
```
|
|
||||||
|
|
||||||
Add a mount call:
|
|
||||||
|
|
||||||
```python
|
|
||||||
mcp.mount(<python_namespace>_server, namespace="<python_namespace>")
|
|
||||||
```
|
|
||||||
|
|
||||||
## Example Scaffold
|
|
||||||
|
|
||||||
For a new skill called `sqlmodel-patterns`:
|
|
||||||
|
|
||||||
1. Docs content lives in `docs/skills/sqlmodel-patterns/SKILL.md`.
|
|
||||||
2. The Python package lives in `src/personal_mcp/skills/sqlmodel_patterns/`.
|
|
||||||
3. The resource id is `sqlmodel-patterns`.
|
|
||||||
|
|
||||||
Example `server.py`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from fastmcp import FastMCP
|
|
||||||
|
|
||||||
from personal_mcp.skills.document_loader import load_skill_document
|
|
||||||
|
|
||||||
sqlmodel_patterns_server = FastMCP("sqlmodel-patterns")
|
|
||||||
|
|
||||||
|
|
||||||
@sqlmodel_patterns_server.resource("resource://skills/sqlmodel-patterns/document")
|
|
||||||
def skill_document() -> dict[str, str]:
|
|
||||||
"""Return the canonical Markdown document for this skill."""
|
|
||||||
return load_skill_document(
|
|
||||||
skill_id="sqlmodel-patterns",
|
|
||||||
skill_slug="sqlmodel-patterns",
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
Example `metadata.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
id: sqlmodel-patterns
|
|
||||||
name: SQLModel Patterns
|
|
||||||
version: 1.0.0
|
|
||||||
description: Provide reusable patterns for building apps with SQLModel.
|
|
||||||
tags:
|
|
||||||
- sqlmodel
|
|
||||||
- python
|
|
||||||
- patterns
|
|
||||||
capabilities:
|
|
||||||
- resource://skills/sqlmodel-patterns/document
|
|
||||||
depends_on: []
|
|
||||||
```
|
|
||||||
|
|
||||||
Example `mcp.py` additions:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from personal_mcp.skills.sqlmodel_patterns.server import sqlmodel_patterns_server
|
|
||||||
|
|
||||||
mcp.mount(sqlmodel_patterns_server, namespace="sqlmodel_patterns")
|
|
||||||
```
|
|
||||||
|
|
||||||
## Bootstrap Sequence
|
|
||||||
|
|
||||||
1. Create `docs/skills/<slug>/SKILL.md`.
|
|
||||||
2. Copy the `server.py` template into `src/personal_mcp/skills/<python_namespace>/server.py`.
|
|
||||||
3. Copy the `metadata.yaml` template into `src/personal_mcp/skills/<python_namespace>/metadata.yaml`.
|
|
||||||
4. Add `__init__.py` in the new package directory.
|
|
||||||
5. Import and mount the server in `src/personal_mcp/mcp.py`.
|
|
||||||
6. Run the validation commands above.
|
|
||||||
@@ -1,240 +0,0 @@
|
|||||||
---
|
|
||||||
name: fastapi-async-sqlalchemy-modernization
|
|
||||||
description: 'Create a step-by-step modernization plan for an existing FastAPI app using SQLAlchemy async patterns, context managers, and AsyncExitStack. Use when: planning migration from legacy DB setup, standardizing async engine/session lifecycles, defining transaction boundaries, and aligning with SQLAlchemy 2.x best practices.'
|
|
||||||
argument-hint: 'What is your current FastAPI + SQLAlchemy setup (sync/async driver, session pattern, lifespan usage, and deployment model)?'
|
|
||||||
---
|
|
||||||
|
|
||||||
# FastAPI Async SQLAlchemy Modernization Plan
|
|
||||||
|
|
||||||
Create an implementation-ready plan that brings an existing FastAPI application in line with modern async SQLAlchemy practices, with explicit resource lifecycles and deterministic cleanup using async context managers and AsyncExitStack.
|
|
||||||
|
|
||||||
Primary targets: PostgreSQL with asyncpg and SQLite with aiosqlite.
|
|
||||||
|
|
||||||
## When to Use
|
|
||||||
|
|
||||||
- Existing FastAPI app has ad hoc database setup or mixed sync/async access.
|
|
||||||
- Session management is inconsistent across routes/services.
|
|
||||||
- Lifespan startup and shutdown work is spread across globals and side effects.
|
|
||||||
- Team needs a migration plan first, not immediate large-scale rewrites.
|
|
||||||
|
|
||||||
## Outcome
|
|
||||||
|
|
||||||
Produce a practical modernization plan with:
|
|
||||||
|
|
||||||
- Current-state gap assessment.
|
|
||||||
- Target architecture for engine/session/transaction lifecycle.
|
|
||||||
- Branch-based migration path (low-risk staged rollout).
|
|
||||||
- Quality gates and completion checks.
|
|
||||||
- Risks, rollback strategy, and test plan.
|
|
||||||
|
|
||||||
## Top-Level Concepts
|
|
||||||
|
|
||||||
Use these concepts as the planning backbone:
|
|
||||||
|
|
||||||
1. Engine lifecycle and ownership:
|
|
||||||
One AsyncEngine per process for each DB URL, created once and disposed explicitly when the app lifecycle ends.
|
|
||||||
See: [references/engine.md](references/engine.md)
|
|
||||||
2. Session factory and scope:
|
|
||||||
Use async_sessionmaker for configuration; create one AsyncSession per request or unit-of-work, never shared across concurrent tasks.
|
|
||||||
See: [references/session.md](references/session.md)
|
|
||||||
3. Transaction boundaries:
|
|
||||||
Prefer context-managed begin blocks for write units and explicit read-only sessions for queries.
|
|
||||||
See: [references/transactions.md](references/transactions.md)
|
|
||||||
4. Lifespan composition:
|
|
||||||
Compose startup/shutdown resources with AsyncExitStack so cleanup is deterministic and ordered.
|
|
||||||
See: [references/engine.md](references/engine.md)
|
|
||||||
5. Dependency injection:
|
|
||||||
Provide sessions via FastAPI dependencies with async generators/context managers, not globals.
|
|
||||||
See: [references/session.md](references/session.md)
|
|
||||||
6. Implicit I/O control in ORM:
|
|
||||||
Avoid accidental lazy loads; use explicit eager-loading/refresh strategies for asyncio safety.
|
|
||||||
See: [references/implicit_io.md](references/implicit_io.md)
|
|
||||||
7. Observability and resilience:
|
|
||||||
Add pool/connection settings, logging, timeout, and health checks as first-class plan items.
|
|
||||||
See: [references/observability.md](references/observability.md)
|
|
||||||
|
|
||||||
### Concept Reference Map
|
|
||||||
|
|
||||||
| Concept | Reference |
|
|
||||||
|---|---|
|
|
||||||
| Engine lifecycle and ownership | [references/engine.md](references/engine.md) |
|
|
||||||
| Session factory and scope | [references/session.md](references/session.md) |
|
|
||||||
| Transaction boundaries | [references/transactions.md](references/transactions.md) |
|
|
||||||
| Lifespan composition | [references/engine.md](references/engine.md) |
|
|
||||||
| Dependency injection | [references/session.md](references/session.md) |
|
|
||||||
| Implicit I/O control in ORM | [references/implicit_io.md](references/implicit_io.md) |
|
|
||||||
| Observability and resilience | [references/observability.md](references/observability.md) |
|
|
||||||
|
|
||||||
## Decision Points
|
|
||||||
|
|
||||||
Use these branching decisions before proposing migration steps.
|
|
||||||
|
|
||||||
| Decision | Branch A | Branch B |
|
|
||||||
|---|---|---|
|
|
||||||
| DB driver | Already async driver (e.g. asyncpg, aiosqlite): modernize in place | Sync driver: plan driver migration first |
|
|
||||||
| ORM usage | Already ORM 2.x style (`select`, `session.execute`) | Legacy Query API: add compatibility stage and refactor incrementally |
|
|
||||||
| Session scope | Request-scoped already | Global/shared sessions found: prioritize session-scope fix first |
|
|
||||||
| Lifespan | Existing FastAPI lifespan hook | No lifespan hook: introduce lifespan before broader DB changes |
|
|
||||||
| Concurrency | Background jobs/tasks use DB | No background DB use |
|
|
||||||
| Transaction style | Explicit context-managed transactions | Implicit/autobegin side effects |
|
|
||||||
|
|
||||||
## Procedure
|
|
||||||
|
|
||||||
### Step 0: Audit Current State
|
|
||||||
|
|
||||||
Inventory the app and write a concise gap list.
|
|
||||||
|
|
||||||
- Engine creation location(s) and count.
|
|
||||||
- Driver URL(s) and async compatibility.
|
|
||||||
- Session creation patterns in routes/services/background tasks.
|
|
||||||
- Transaction handling style (explicit begin/commit/rollback vs implicit).
|
|
||||||
- Lifespan startup/shutdown and cleanup behavior.
|
|
||||||
- ORM loading patterns that may trigger implicit I/O.
|
|
||||||
|
|
||||||
Completion check: every DB touchpoint is mapped to its engine, session, and transaction source.
|
|
||||||
|
|
||||||
### Step 1: Define the Target Runtime Model
|
|
||||||
|
|
||||||
Define one canonical model to migrate toward.
|
|
||||||
|
|
||||||
- Create AsyncEngine once per process.
|
|
||||||
- Configure async_sessionmaker once.
|
|
||||||
- Use per-request AsyncSession dependency.
|
|
||||||
- Keep one AsyncSession per concurrent task.
|
|
||||||
- Use context-managed transactions for writes.
|
|
||||||
|
|
||||||
Completion check: architecture diagram can explain where engine/session are created, used, and closed.
|
|
||||||
|
|
||||||
### Step 2: Plan Engine Modernization
|
|
||||||
|
|
||||||
Plan engine creation and pool behavior.
|
|
||||||
|
|
||||||
- Use `create_async_engine()` with async dialect URL.
|
|
||||||
- Standardize pool settings and pre-ping strategy where relevant.
|
|
||||||
- Decide isolation level strategy at engine level (avoid ad hoc per-operation switching unless justified).
|
|
||||||
- Define explicit disposal policy for short-lived scopes and tests.
|
|
||||||
|
|
||||||
Completion check: engine configuration is centralized and no per-request engine creation remains.
|
|
||||||
|
|
||||||
### Step 3: Plan Session Lifecycle Modernization
|
|
||||||
|
|
||||||
Define session factory and request dependency pattern.
|
|
||||||
|
|
||||||
- Build `async_sessionmaker(engine, expire_on_commit=False)` unless a strict reason says otherwise.
|
|
||||||
- Provide session via dependency that yields exactly one AsyncSession.
|
|
||||||
- Explicitly prohibit sharing a single AsyncSession across concurrent tasks.
|
|
||||||
- Prefer direct dependency passing over async_scoped_session for new designs.
|
|
||||||
|
|
||||||
Completion check: all route/service entry points receive a session from one canonical dependency.
|
|
||||||
|
|
||||||
### Step 4: Plan Transaction Demarcation
|
|
||||||
|
|
||||||
Establish consistent write and read behavior.
|
|
||||||
|
|
||||||
- Writes: `async with session.begin(): ...` for atomic units.
|
|
||||||
- Reads: execute in managed session context with explicit loader options.
|
|
||||||
- Nested/SAVEPOINT use only where required; call out backend caveats.
|
|
||||||
- Define rollback behavior for service-layer exceptions.
|
|
||||||
|
|
||||||
Completion check: every mutating use case has a declared transaction boundary.
|
|
||||||
|
|
||||||
### Step 5: Compose Lifespan with AsyncExitStack
|
|
||||||
|
|
||||||
Use async context composition as the preferred orchestration pattern.
|
|
||||||
|
|
||||||
```python
|
|
||||||
from contextlib import AsyncExitStack, asynccontextmanager
|
|
||||||
from fastapi import FastAPI
|
|
||||||
|
|
||||||
@asynccontextmanager
|
|
||||||
async def lifespan(app: FastAPI):
|
|
||||||
async with AsyncExitStack() as stack:
|
|
||||||
# Compose resources in acquisition order; cleanup is automatic in reverse order.
|
|
||||||
engine = create_async_engine(settings.database_url)
|
|
||||||
stack.push_async_callback(engine.dispose)
|
|
||||||
|
|
||||||
session_factory = async_sessionmaker(engine, expire_on_commit=False)
|
|
||||||
app.state.session_factory = session_factory
|
|
||||||
|
|
||||||
# Add other async resources with stack.enter_async_context(...) as needed.
|
|
||||||
yield
|
|
||||||
```
|
|
||||||
|
|
||||||
Planning rules:
|
|
||||||
|
|
||||||
- Register every acquired resource with AsyncExitStack at acquisition time.
|
|
||||||
- Prefer `enter_async_context()` for resources that already expose async context managers.
|
|
||||||
- Prefer `push_async_callback()` for async cleanup callables.
|
|
||||||
- Keep resource ownership in lifespan, not in route handlers.
|
|
||||||
|
|
||||||
Completion check: startup/shutdown ordering is explicit and deterministic.
|
|
||||||
|
|
||||||
### Step 6: Prevent Implicit ORM I/O Under Asyncio (Advisory Mode)
|
|
||||||
|
|
||||||
Plan for explicit loading behavior, but treat this as progressive guidance rather than a hard gate.
|
|
||||||
|
|
||||||
- Recommend eager-loading strategies (for example selectin-style loading) where relationship access is required.
|
|
||||||
- For lazy/deferred attributes, define explicit awaitable or refresh paths on high-risk and high-traffic paths first.
|
|
||||||
- Document model-level defaults and known exceptions so teams can migrate incrementally.
|
|
||||||
|
|
||||||
Completion check: critical request paths have explicit loading plans; non-critical paths have tracked follow-up items.
|
|
||||||
|
|
||||||
### Step 7: Testing and Verification Plan
|
|
||||||
|
|
||||||
Create modernization quality gates.
|
|
||||||
|
|
||||||
- Unit tests for session dependency and transaction behavior.
|
|
||||||
- Integration tests for commit/rollback semantics.
|
|
||||||
- Concurrency tests confirming one-session-per-task behavior.
|
|
||||||
- Lifespan tests verifying cleanup calls and ordering.
|
|
||||||
- Health/readiness tests including DB connectivity checks.
|
|
||||||
|
|
||||||
Completion check: all quality gates pass under the target async configuration.
|
|
||||||
|
|
||||||
### Step 8: Rollout Strategy
|
|
||||||
|
|
||||||
Plan low-risk migration phases.
|
|
||||||
|
|
||||||
1. Introduce centralized engine/session factory and lifespan orchestration.
|
|
||||||
2. Migrate read paths to new session dependency.
|
|
||||||
3. Migrate write paths to explicit transaction blocks.
|
|
||||||
4. Remove legacy globals/helpers and dead code.
|
|
||||||
5. Enable stricter linting/review checks for forbidden patterns.
|
|
||||||
|
|
||||||
Completion check: no legacy session/engine creation path remains in production code.
|
|
||||||
|
|
||||||
## Quality Criteria
|
|
||||||
|
|
||||||
A plan is complete only when it includes:
|
|
||||||
|
|
||||||
- Clear current vs target architecture.
|
|
||||||
- Branch decisions with rationale.
|
|
||||||
- Explicit context-manager patterns for resource ownership.
|
|
||||||
- AsyncExitStack composition strategy.
|
|
||||||
- Transaction policy and exception behavior.
|
|
||||||
- Concrete tests and rollout checkpoints.
|
|
||||||
- A documented advisory backlog for non-critical implicit I/O improvements.
|
|
||||||
|
|
||||||
## Anti-Patterns to Flag
|
|
||||||
|
|
||||||
- Creating engines inside request handlers.
|
|
||||||
- Sharing one AsyncSession across concurrent tasks.
|
|
||||||
- Implicit commit/rollback behavior with unclear ownership.
|
|
||||||
- Global mutable session state.
|
|
||||||
- Lifespan cleanup that depends on implicit garbage collection.
|
|
||||||
|
|
||||||
## Output Contract
|
|
||||||
|
|
||||||
Return the plan as:
|
|
||||||
|
|
||||||
1. Current-state gap summary.
|
|
||||||
2. Target architecture summary.
|
|
||||||
3. Phased migration checklist with branch notes.
|
|
||||||
4. Risk register and rollback approach.
|
|
||||||
5. Verification matrix (tests + operational checks).
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- SQLAlchemy engine/connections: https://docs.sqlalchemy.org/en/21/core/connections.html
|
|
||||||
- SQLAlchemy asyncio extension: https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html
|
|
||||||
- Python async context managers and AsyncExitStack: https://docs.python.org/3/library/contextlib.html
|
|
||||||
@@ -1,133 +0,0 @@
|
|||||||
# Async SQLAlchemy Engine
|
|
||||||
|
|
||||||
Source:
|
|
||||||
- https://docs.sqlalchemy.org/en/21/core/connections.html
|
|
||||||
- https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html
|
|
||||||
- https://docs.sqlalchemy.org/en/21/core/pooling.html#pooling-multiprocessing
|
|
||||||
- https://fastapi.tiangolo.com/advanced/events/
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Engine Ownership Model
|
|
||||||
|
|
||||||
Create one async engine per process per database URL and keep it for the app lifetime.
|
|
||||||
|
|
||||||
- SQLAlchemy guidance: the engine is intended as a long-lived, concurrent registry over pooled DB connections, not a per-request object.
|
|
||||||
- In FastAPI, app startup and shutdown ownership belongs in lifespan.
|
|
||||||
- Use `FastAPI(lifespan=...)` (not startup/shutdown events) for modern lifecycle wiring.
|
|
||||||
|
|
||||||
Practical rule:
|
|
||||||
- Exactly one `create_async_engine(...)` call in app bootstrap code.
|
|
||||||
- Zero `create_async_engine(...)` calls in request handlers.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Canonical Lifespan Pattern (AsyncExitStack)
|
|
||||||
|
|
||||||
Use `@asynccontextmanager` + `AsyncExitStack` to make teardown deterministic and composable.
|
|
||||||
|
|
||||||
```python
|
|
||||||
from contextlib import AsyncExitStack, asynccontextmanager
|
|
||||||
|
|
||||||
from fastapi import FastAPI
|
|
||||||
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
|
|
||||||
|
|
||||||
|
|
||||||
@asynccontextmanager
|
|
||||||
async def lifespan(app: FastAPI):
|
|
||||||
async with AsyncExitStack() as stack:
|
|
||||||
engine: AsyncEngine = create_async_engine(
|
|
||||||
app.state.settings.database_url,
|
|
||||||
pool_pre_ping=True,
|
|
||||||
# Optional examples:
|
|
||||||
# echo=app.state.settings.sql_echo,
|
|
||||||
# pool_size=10,
|
|
||||||
# max_overflow=20,
|
|
||||||
)
|
|
||||||
app.state.engine = engine
|
|
||||||
|
|
||||||
# Ensure engine disposal always runs at shutdown.
|
|
||||||
stack.push_async_callback(engine.dispose)
|
|
||||||
|
|
||||||
yield
|
|
||||||
|
|
||||||
|
|
||||||
app = FastAPI(lifespan=lifespan)
|
|
||||||
```
|
|
||||||
|
|
||||||
Why this pattern:
|
|
||||||
- FastAPI executes code before `yield` at startup and after `yield` at shutdown.
|
|
||||||
- `AsyncExitStack` lets you register multiple async cleanups in one place while preserving order.
|
|
||||||
- Explicit disposal (directly awaited or via `AsyncExitStack` callback) avoids event-loop-closed warnings when objects fall out of scope.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Driver URLs (Project Requirement: asyncpg + aiosqlite)
|
|
||||||
|
|
||||||
Use SQLAlchemy async driver URLs:
|
|
||||||
|
|
||||||
- PostgreSQL: `postgresql+asyncpg://user:pass@host:5432/dbname`
|
|
||||||
- SQLite: `sqlite+aiosqlite:///./app.db`
|
|
||||||
|
|
||||||
Notes:
|
|
||||||
- Do not mix sync drivers (for example `psycopg2`) with `create_async_engine()`.
|
|
||||||
- Keep URL construction centralized in settings/config, not in feature modules.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Pooling Defaults and Tuning
|
|
||||||
|
|
||||||
Default behavior is usually correct first:
|
|
||||||
|
|
||||||
- Async engines use async-compatible pooling (`AsyncAdaptedQueuePool`) by default.
|
|
||||||
- Start with defaults, then tune from observed load (`pool_size`, `max_overflow`, `pool_timeout`, `pool_recycle`).
|
|
||||||
- Enable `pool_pre_ping=True` for safer stale-connection handling in long-running services.
|
|
||||||
|
|
||||||
When to switch pool strategy:
|
|
||||||
|
|
||||||
- `NullPool` if you explicitly need no pooling (special environments, some tests, or strict cross-loop constraints).
|
|
||||||
- Keep in mind this increases connect/disconnect churn.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Disposal Semantics
|
|
||||||
|
|
||||||
`engine.dispose()` replaces/disposes the pool, but only checked-in connections are immediately closed.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
- Dispose when the app is shutting down.
|
|
||||||
- Dispose before reusing an engine across event loops.
|
|
||||||
- In forked child-process initialization, use `engine.dispose(close=False)` (sync API guidance) so child processes do not touch parent-held connections.
|
|
||||||
|
|
||||||
Avoid relying on garbage collection for engine cleanup in async code.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Event Loop and Process Boundaries
|
|
||||||
|
|
||||||
Do not share pooled connections across boundaries:
|
|
||||||
|
|
||||||
- Multiple event loops: do not reuse the same pooled async engine across loops unless you intentionally disable pooling (`NullPool`) or dispose before handoff.
|
|
||||||
- Multiprocessing/fork: pooled connections must not be inherited for active use across process boundaries.
|
|
||||||
|
|
||||||
This prevents broken socket state and cross-process connection corruption.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## What Not to Do
|
|
||||||
|
|
||||||
- Create an engine inside every request dependency.
|
|
||||||
- Create/dispose engines inside repository methods.
|
|
||||||
- Keep engine creation as a hidden side effect of import-time module globals.
|
|
||||||
- Use deprecated FastAPI startup/shutdown events together with lifespan.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Engine Design Checklist
|
|
||||||
|
|
||||||
- One engine per process per DB URL.
|
|
||||||
- Engine created in lifespan startup.
|
|
||||||
- Engine disposed in lifespan shutdown.
|
|
||||||
- Async driver URL matches backend (`asyncpg` or `aiosqlite`).
|
|
||||||
- Pooling strategy is explicit for non-default needs.
|
|
||||||
- No request-path engine creation.
|
|
||||||
@@ -1,140 +0,0 @@
|
|||||||
# Async SQLAlchemy Session Management
|
|
||||||
|
|
||||||
Source:
|
|
||||||
- https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html
|
|
||||||
- https://docs.sqlalchemy.org/en/21/orm/session_basics.html
|
|
||||||
- https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/
|
|
||||||
|
|
||||||
Status: adopted
|
|
||||||
Decision level: mandatory
|
|
||||||
Applies to: api-runtime, workers, tests
|
|
||||||
Last reviewed: 2026-06-17
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Define one canonical session model for FastAPI + SQLAlchemy asyncio:
|
|
||||||
|
|
||||||
- configure one shared session factory,
|
|
||||||
- create one AsyncSession per request or per unit-of-work,
|
|
||||||
- never share one AsyncSession across concurrent tasks.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Scope and Non-Goals
|
|
||||||
|
|
||||||
- In scope: session factory creation, FastAPI dependency wiring, request/task scoping, transaction demarcation.
|
|
||||||
- Out of scope: ORM model design, query optimization strategy, schema migration tooling.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Rules
|
|
||||||
|
|
||||||
- Create `async_sessionmaker` once from app-owned AsyncEngine.
|
|
||||||
- Use a fresh AsyncSession for each request or explicit unit-of-work.
|
|
||||||
- Do not share AsyncSession across `asyncio.gather()` or parallel tasks.
|
|
||||||
- Prefer direct dependency injection over global scoped-session patterns in new code.
|
|
||||||
- Use explicit transaction boundaries (`async with session.begin():`) for writes.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Canonical FastAPI Dependency Pattern
|
|
||||||
|
|
||||||
```python
|
|
||||||
from collections.abc import AsyncIterator
|
|
||||||
|
|
||||||
from fastapi import Depends, Request
|
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
|
||||||
|
|
||||||
|
|
||||||
def get_session_factory(request: Request) -> async_sessionmaker[AsyncSession]:
|
|
||||||
return request.app.state.session_factory
|
|
||||||
|
|
||||||
|
|
||||||
async def get_db_session(
|
|
||||||
session_factory: async_sessionmaker[AsyncSession] = Depends(get_session_factory),
|
|
||||||
) -> AsyncIterator[AsyncSession]:
|
|
||||||
async with session_factory() as session:
|
|
||||||
yield session
|
|
||||||
```
|
|
||||||
|
|
||||||
Route usage:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from fastapi import APIRouter, Depends
|
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession
|
|
||||||
|
|
||||||
router = APIRouter()
|
|
||||||
|
|
||||||
|
|
||||||
@router.post("/items")
|
|
||||||
async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
|
|
||||||
async with session.begin():
|
|
||||||
# write operations here
|
|
||||||
...
|
|
||||||
return {"status": "ok"}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Configuration Guidance
|
|
||||||
|
|
||||||
Typical session factory setup:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker
|
|
||||||
|
|
||||||
session_factory = async_sessionmaker(
|
|
||||||
engine,
|
|
||||||
class_=AsyncSession,
|
|
||||||
expire_on_commit=False,
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
Notes:
|
|
||||||
|
|
||||||
- `expire_on_commit=False` is commonly preferred in asyncio applications to reduce accidental post-commit reload behavior.
|
|
||||||
- `AsyncSession.refresh()` is preferred over broad expiration patterns when state refresh is needed.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Concurrency Rules
|
|
||||||
|
|
||||||
- One session per concurrent task.
|
|
||||||
- If work fans out into parallel tasks, each task receives its own AsyncSession.
|
|
||||||
- Pass sessions explicitly to service functions; avoid mutable global session state.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Anti-Patterns
|
|
||||||
|
|
||||||
- A singleton/global AsyncSession reused across requests.
|
|
||||||
- Sharing one AsyncSession across parallel tasks.
|
|
||||||
- Hidden session creation in lower repository helpers with no caller control.
|
|
||||||
- Mixing commit/rollback ownership across layers without a declared boundary.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Operational Checks
|
|
||||||
|
|
||||||
- Exactly one `async_sessionmaker` is registered in app lifecycle.
|
|
||||||
- Request handlers receive sessions from one canonical dependency.
|
|
||||||
- No code path creates AsyncSession in module import side effects.
|
|
||||||
- Background jobs and API handlers each create task-local sessions.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Testing Checks
|
|
||||||
|
|
||||||
- Dependency override exists for test session factory.
|
|
||||||
- Rollback behavior is verified for failed write units.
|
|
||||||
- Parallel-task tests verify no shared AsyncSession instances.
|
|
||||||
- Lifespan tests confirm session factory is initialized and teardown-safe.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Migration Notes
|
|
||||||
|
|
||||||
- If current code uses global/shared sessions, fix scope first before refactoring query style.
|
|
||||||
- If legacy sync patterns are present, keep session boundary rules stable while migrating incrementally.
|
|
||||||
@@ -1,134 +0,0 @@
|
|||||||
---
|
|
||||||
name: nicegui-ui-customization
|
|
||||||
description: 'Design and implement production NiceGUI UIs with reusable components, Tailwind-first styling, event-driven interactions, and troubleshooting for uploads, state, and static assets. Use when building or refactoring NiceGUI pages and interaction flows.'
|
|
||||||
argument-hint: 'What UI outcome should this workflow produce?'
|
|
||||||
---
|
|
||||||
|
|
||||||
# NiceGUI UI Customization Workflow
|
|
||||||
|
|
||||||
Create, style, and ship production NiceGUI UI flows with a repeatable process. The workflow keeps structure in Python, favors Tailwind and Quasar APIs for styling, and uses event-driven interaction patterns over ad-hoc polling.
|
|
||||||
|
|
||||||
## When To Use
|
|
||||||
|
|
||||||
- Building a new NiceGUI page or dashboard
|
|
||||||
- Refactoring a page into reusable components
|
|
||||||
- Adding file upload, form submission, live status, or background-job UX
|
|
||||||
- Troubleshooting race conditions, stale assets, or inconsistent state updates
|
|
||||||
|
|
||||||
## Target Outcome
|
|
||||||
|
|
||||||
Deliver a responsive, accessible UI flow that:
|
|
||||||
|
|
||||||
- keeps clear boundaries between page adapters, reusable components, and services
|
|
||||||
- uses Tailwind-first styling with minimal custom CSS
|
|
||||||
- updates UI through events and bindings
|
|
||||||
- has validation, user feedback, and failure handling
|
|
||||||
- passes a production-readiness check at the end
|
|
||||||
|
|
||||||
## Progressive Loading References
|
|
||||||
|
|
||||||
Load these references only when needed:
|
|
||||||
|
|
||||||
- Architecture and styling rules: [./references/architecture-and-styling.md](./references/architecture-and-styling.md)
|
|
||||||
- Event and state interaction patterns: [./references/interaction-patterns.md](./references/interaction-patterns.md)
|
|
||||||
- Troubleshooting and release gates: [./references/troubleshooting-and-quality-gates.md](./references/troubleshooting-and-quality-gates.md)
|
|
||||||
|
|
||||||
## Procedure
|
|
||||||
|
|
||||||
### 1. Define the UI Slice
|
|
||||||
|
|
||||||
- Capture the user-visible outcome for this task in one sentence.
|
|
||||||
- Identify route-level page modules to touch.
|
|
||||||
- Identify service operations needed by the UI.
|
|
||||||
|
|
||||||
Completion check:
|
|
||||||
|
|
||||||
- You can name the target page, component candidates, and service calls before coding.
|
|
||||||
|
|
||||||
### 2. Choose Component Extraction Strategy
|
|
||||||
|
|
||||||
Decision point:
|
|
||||||
|
|
||||||
- If a layout pattern appears in 2 or more pages, extract it to `ui/components/`.
|
|
||||||
- If a pattern is page-specific, keep it in the page module.
|
|
||||||
|
|
||||||
Completion check:
|
|
||||||
|
|
||||||
- Reused UI patterns are encapsulated as callable components.
|
|
||||||
|
|
||||||
### 3. Build Responsive Layout First
|
|
||||||
|
|
||||||
- Use Tailwind utility classes for structure and spacing.
|
|
||||||
- Use responsive breakpoints (`sm:`, `md:`, `lg:`).
|
|
||||||
- Reserve `.style()` for dynamic values that cannot be expressed with classes.
|
|
||||||
|
|
||||||
Completion check:
|
|
||||||
|
|
||||||
- Layout works at mobile and desktop widths without custom CSS overrides.
|
|
||||||
|
|
||||||
### 4. Add Reactive State And Events
|
|
||||||
|
|
||||||
- Use bindable dataclasses for local page state.
|
|
||||||
- Prefer event handlers (`on_click`, `on_upload`, etc.) over periodic polling.
|
|
||||||
- Trigger explicit refreshes with `@ui.refreshable` where needed.
|
|
||||||
|
|
||||||
Decision point by interaction type:
|
|
||||||
|
|
||||||
- File upload: validate size/type, delegate storage to a service, notify success/failure.
|
|
||||||
- Form submit: bind inputs to dataclass fields, validate in service layer, clear state on success.
|
|
||||||
- Real-time status: use SSE or WebSocket for push updates.
|
|
||||||
- Long jobs: run in background task, update status endpoint or stream.
|
|
||||||
|
|
||||||
Completion check:
|
|
||||||
|
|
||||||
- Every user action has explicit positive and negative feedback via `ui.notify()`.
|
|
||||||
|
|
||||||
### 5. Apply Styling Strategy
|
|
||||||
|
|
||||||
Preferred order:
|
|
||||||
|
|
||||||
1. Tailwind utility classes
|
|
||||||
2. Quasar props
|
|
||||||
3. Reusable styled component functions
|
|
||||||
|
|
||||||
Only if absolutely necessary:
|
|
||||||
|
|
||||||
- Load minimal custom CSS once at startup in `bootstrap.py`.
|
|
||||||
- Keep custom CSS tokenized (variables) and documented.
|
|
||||||
|
|
||||||
Completion check:
|
|
||||||
|
|
||||||
- Styling is mostly class/props-driven and not dependent on scattered ad-hoc CSS.
|
|
||||||
|
|
||||||
### 6. Harden Against Common Failures
|
|
||||||
|
|
||||||
- Prevent duplicate submissions by disabling controls during in-flight operations.
|
|
||||||
- Avoid overlapping timers for the same state target.
|
|
||||||
- Serialize dependent updates (`await` service call before mutation/render).
|
|
||||||
- Verify static mount paths and cache behavior for changed assets.
|
|
||||||
|
|
||||||
Completion check:
|
|
||||||
|
|
||||||
- Race conditions and stale asset symptoms are addressed with explicit safeguards.
|
|
||||||
|
|
||||||
### 7. Final Production Readiness Review
|
|
||||||
|
|
||||||
Pass all checks:
|
|
||||||
|
|
||||||
- Structure: pages, components, services follow one-way dependency flow.
|
|
||||||
- Responsiveness: tested at small and large viewport widths.
|
|
||||||
- Accessibility: labels, button text, and action visibility are clear.
|
|
||||||
- Reliability: validation and exception paths produce user-facing notifications.
|
|
||||||
- Maintainability: repeated UI patterns are extracted; business logic stays in services.
|
|
||||||
|
|
||||||
If any check fails, return to the relevant step and iterate.
|
|
||||||
|
|
||||||
## Completion Contract
|
|
||||||
|
|
||||||
This workflow is complete when:
|
|
||||||
|
|
||||||
- the page flow meets the target outcome
|
|
||||||
- architecture boundaries are preserved
|
|
||||||
- chosen interaction pattern is implemented with explicit success and failure feedback
|
|
||||||
- troubleshooting checks pass
|
|
||||||
- production-readiness gate passes
|
|
||||||
@@ -1,76 +0,0 @@
|
|||||||
# Architecture and Styling Reference
|
|
||||||
|
|
||||||
## Project Boundaries
|
|
||||||
|
|
||||||
Use this dependency direction:
|
|
||||||
|
|
||||||
- pages import components and services
|
|
||||||
- components contain presentation logic only
|
|
||||||
- services contain business logic and do not import UI
|
|
||||||
- static assets are mounted and loaded once at bootstrap
|
|
||||||
|
|
||||||
Suggested module split:
|
|
||||||
|
|
||||||
```text
|
|
||||||
src/app/
|
|
||||||
ui/pages/
|
|
||||||
ui/components/
|
|
||||||
ui/static/
|
|
||||||
services/
|
|
||||||
api/
|
|
||||||
bootstrap.py
|
|
||||||
```
|
|
||||||
|
|
||||||
## Component Extraction Rules
|
|
||||||
|
|
||||||
Extract to ui/components when a pattern appears in two or more pages.
|
|
||||||
|
|
||||||
Keep in-page if the layout is specific to a single route.
|
|
||||||
|
|
||||||
```python
|
|
||||||
def card_section(title: str, content: str) -> ui.card:
|
|
||||||
with ui.card().classes("w-full max-w-md") as card:
|
|
||||||
ui.label(title).classes("text-lg font-bold")
|
|
||||||
ui.label(content).classes("text-gray-600")
|
|
||||||
return card
|
|
||||||
```
|
|
||||||
|
|
||||||
## Tailwind-First Layout Pattern
|
|
||||||
|
|
||||||
Use Tailwind utility classes for structure and spacing.
|
|
||||||
Use breakpoint classes for responsive behavior.
|
|
||||||
Use .style() only for values that must be computed dynamically.
|
|
||||||
|
|
||||||
```python
|
|
||||||
with ui.column().classes("w-full"):
|
|
||||||
with ui.row().classes("w-full gap-4 flex-wrap sm:flex-nowrap"):
|
|
||||||
ui.card().classes("flex-1 min-w-64")
|
|
||||||
ui.card().classes("flex-1 min-w-64")
|
|
||||||
```
|
|
||||||
|
|
||||||
## Styling Decision Order
|
|
||||||
|
|
||||||
1. Tailwind utility classes
|
|
||||||
2. Quasar props
|
|
||||||
3. Reusable styled component functions
|
|
||||||
4. Minimal custom CSS loaded once at bootstrap (only when needed)
|
|
||||||
|
|
||||||
```python
|
|
||||||
from fastapi.staticfiles import StaticFiles
|
|
||||||
|
|
||||||
app.mount("/static", StaticFiles(directory="src/app/static"), name="static")
|
|
||||||
ui.add_css(open("src/app/static/css/base.css").read())
|
|
||||||
```
|
|
||||||
|
|
||||||
## Static Asset Rules
|
|
||||||
|
|
||||||
- Keep custom CSS small and tokenized with variables.
|
|
||||||
- Avoid per-page CSS injection.
|
|
||||||
- Verify static mount paths and reverse proxy rewrites.
|
|
||||||
|
|
||||||
## Links
|
|
||||||
|
|
||||||
- NiceGUI elements: https://nicegui.io/documentation/element
|
|
||||||
- NiceGUI binding: https://nicegui.io/documentation/section_binding_properties
|
|
||||||
- Tailwind: https://tailwindcss.com/docs/utility-first
|
|
||||||
- Quasar components: https://quasar.dev/vue-components
|
|
||||||
@@ -1,196 +0,0 @@
|
|||||||
---
|
|
||||||
name: nicegui
|
|
||||||
description: 'Design and scaffold a production-ready NiceGUI + FastAPI application architecture. Use for multi-page app planning, package boundaries, optional DB/LangGraph/docs integration, and implementation checklists.'
|
|
||||||
argument-hint: 'What should this app include (pages, DB, AI, docs, constraints)?'
|
|
||||||
---
|
|
||||||
|
|
||||||
# NiceGUI
|
|
||||||
|
|
||||||
Design a production-minded NiceGUI + FastAPI architecture with clear boundaries, optional extensions, and a concrete implementation checklist.
|
|
||||||
|
|
||||||
## When to Use
|
|
||||||
|
|
||||||
- You need a reusable architecture plan before implementing a NiceGUI app.
|
|
||||||
- You want FastAPI app-factory structure and lifespan wiring.
|
|
||||||
- You need optional guidance for database, LangGraph workflows, or mounted static docs.
|
|
||||||
- You want output that is concise, structured, and implementation-ready.
|
|
||||||
|
|
||||||
## Inputs to Collect
|
|
||||||
|
|
||||||
Collect these inputs up front. If not provided, make safe defaults and state assumptions.
|
|
||||||
|
|
||||||
- Product scope and primary user journeys.
|
|
||||||
- Required pages and route map.
|
|
||||||
- Whether persistent data is required.
|
|
||||||
- Whether AI orchestration (multi-step, streaming, approvals) is required.
|
|
||||||
- Whether generated docs should be mounted in-app.
|
|
||||||
- Runtime/deployment constraints (single service vs split services, environment requirements).
|
|
||||||
|
|
||||||
## Outcome
|
|
||||||
|
|
||||||
Produce:
|
|
||||||
|
|
||||||
- A concise architecture explanation.
|
|
||||||
- How core services, UI pages, and UI components fit together.
|
|
||||||
- Explicit decision on DB ownership or involvement.
|
|
||||||
- Explicit decision on AI workflow (or no AI).
|
|
||||||
- A checklist implementation plan organized by package and domain.
|
|
||||||
|
|
||||||
## Procedure
|
|
||||||
|
|
||||||
1. Frame the baseline architecture.
|
|
||||||
2. Choose optional extensions (DB, AI, docs) using decision points below.
|
|
||||||
3. Map modules, dependencies, and key boundaries.
|
|
||||||
4. Define async behavior and UI responsiveness expectations.
|
|
||||||
5. Define key functions/classes and configuration surfaces.
|
|
||||||
6. Produce phased checklist with rollout or migration notes when relevant.
|
|
||||||
7. Run completion checks before returning.
|
|
||||||
|
|
||||||
### 1) Baseline architecture
|
|
||||||
|
|
||||||
Use a src-layout with FastAPI as the ASGI app and NiceGUI registered via composition.
|
|
||||||
|
|
||||||
- App factory pattern: `create_app()`.
|
|
||||||
- Lifespan for startup and shutdown resource management.
|
|
||||||
- `api/` for HTTP handlers, `services/` for business logic.
|
|
||||||
- `ui/pages/` for page modules, `ui/components/` for shared UI.
|
|
||||||
- Health endpoint on FastAPI side: `/healthz`.
|
|
||||||
|
|
||||||
Recommended base shape:
|
|
||||||
|
|
||||||
```text
|
|
||||||
.
|
|
||||||
├─ pyproject.toml
|
|
||||||
├─ .env.example
|
|
||||||
├─ README.md
|
|
||||||
├─ src/
|
|
||||||
│ └─ app/
|
|
||||||
│ ├─ __init__.py
|
|
||||||
│ ├─ main.py
|
|
||||||
│ ├─ bootstrap.py
|
|
||||||
│ ├─ config.py
|
|
||||||
│ ├─ logging.py
|
|
||||||
│ ├─ api/
|
|
||||||
│ │ ├─ __init__.py
|
|
||||||
│ │ └─ health.py
|
|
||||||
│ ├─ services/
|
|
||||||
│ │ ├─ __init__.py
|
|
||||||
│ │ └─ example_service.py
|
|
||||||
│ └─ ui/
|
|
||||||
│ ├─ __init__.py
|
|
||||||
│ ├─ components/
|
|
||||||
│ │ ├─ __init__.py
|
|
||||||
│ │ └─ nav.py
|
|
||||||
│ └─ pages/
|
|
||||||
│ ├─ __init__.py
|
|
||||||
│ ├─ home.py
|
|
||||||
│ ├─ dashboard.py
|
|
||||||
│ └─ about.py
|
|
||||||
└─ tests/
|
|
||||||
├─ test_health.py
|
|
||||||
└─ test_pages_registration.py
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2) Decision points
|
|
||||||
|
|
||||||
#### Database needed?
|
|
||||||
|
|
||||||
- If no: keep `services/` pure and skip persistence layers.
|
|
||||||
- If yes: add `db/` package with engine/session/model/repository layering.
|
|
||||||
- Prefer one process-level engine and request-scoped sessions via `yield`.
|
|
||||||
- Prefer Alembic migrations for schema changes.
|
|
||||||
|
|
||||||
#### AI workflow needed?
|
|
||||||
|
|
||||||
- If no: keep `services/` focused on app logic only.
|
|
||||||
- If yes: add `ai/` package (state, nodes, graph, runtime, contracts).
|
|
||||||
- Keep graph internals out of `ui/pages/` and API handlers.
|
|
||||||
- Use stable thread/session IDs for resumable flows.
|
|
||||||
|
|
||||||
#### Mounted docs needed?
|
|
||||||
|
|
||||||
- If no: skip docs mounting.
|
|
||||||
- If yes: mount generated static site under configurable route (default `/docs`).
|
|
||||||
- Keep docs mounting in composition layer, not page modules.
|
|
||||||
|
|
||||||
### 3) Page and component registration
|
|
||||||
|
|
||||||
- Require at minimum page modules for `/`, `/dashboard`, `/about`.
|
|
||||||
- Prefer explicit registration pattern:
|
|
||||||
- `ui/pages/__init__.py` exports `register_pages()`.
|
|
||||||
- Each page module exports `register_page()`.
|
|
||||||
- Shared shell components (header/nav/drawer) live in `ui/components/`.
|
|
||||||
|
|
||||||
### 4) Dependency direction rules
|
|
||||||
|
|
||||||
Prefer:
|
|
||||||
|
|
||||||
- `main/bootstrap` -> `config/logging` + `api` + `ui/pages` + `services`
|
|
||||||
- `api` -> `services`
|
|
||||||
- `ui/pages` -> `ui/components` + `services`
|
|
||||||
- `services` -> helpers/clients (and `db/` when enabled)
|
|
||||||
|
|
||||||
Avoid reverse imports from services into API or UI modules.
|
|
||||||
|
|
||||||
### 5) Async and UI responsiveness rules
|
|
||||||
|
|
||||||
- Prefer `async def` for page handlers, service methods, and integrations when the call path includes I/O.
|
|
||||||
- Use non-blocking clients/libraries where possible so long-running I/O does not freeze UI updates.
|
|
||||||
- Do not run blocking calls (`time.sleep`, blocking HTTP/database clients) in UI event handlers.
|
|
||||||
- For heavy CPU work, offload to worker/background execution and keep the UI loop free.
|
|
||||||
- Show progress states for long actions (disable action button, show spinner/progress text, re-enable on completion).
|
|
||||||
- Stream or chunk incremental results to the UI when workflows are multi-step or long-running.
|
|
||||||
- Keep cancellation and timeout behavior explicit for user-triggered long tasks.
|
|
||||||
- Ensure exceptions from async tasks are surfaced with user-friendly feedback and logged for diagnostics.
|
|
||||||
|
|
||||||
### 6) Testing minimums
|
|
||||||
|
|
||||||
- Test FastAPI health route behavior.
|
|
||||||
- Test page registration wiring.
|
|
||||||
- If DB enabled: session lifecycle and rollback behavior tests.
|
|
||||||
- If AI enabled: graph happy path and interrupt/resume coverage.
|
|
||||||
- If docs enabled: mounted docs route returns index page.
|
|
||||||
- For async flows: test long-running actions preserve UI responsiveness (loading state, completion state, and error state).
|
|
||||||
|
|
||||||
### 7) Styling architecture
|
|
||||||
|
|
||||||
- Keep structure and layout in Python modules using NiceGUI class composition.
|
|
||||||
- Keep visual polish in shared CSS files, loaded once at startup.
|
|
||||||
- Prefer semantic reusable classes over ad hoc per-page styling.
|
|
||||||
|
|
||||||
## Completion Checks
|
|
||||||
|
|
||||||
- Uses app factory and FastAPI lifespan.
|
|
||||||
- Pages are modularized (not single-file UI).
|
|
||||||
- Health endpoint exists on FastAPI side.
|
|
||||||
- Dependency direction is clean and one-way.
|
|
||||||
- Async-first guidance is applied where I/O exists, with explicit non-blocking UX states.
|
|
||||||
- Optional DB/AI/docs decisions are explicit and reflected in structure.
|
|
||||||
- Output includes architecture summary and package-organized checklist.
|
|
||||||
|
|
||||||
## Output Contract
|
|
||||||
|
|
||||||
Return:
|
|
||||||
|
|
||||||
- Concise high-level architecture.
|
|
||||||
- How core services, pages, and shared components fit.
|
|
||||||
- DB involvement and ownership stance.
|
|
||||||
- AI workflow stance and runtime flow.
|
|
||||||
- Checklist plan by package and domain:
|
|
||||||
- key functions/classes
|
|
||||||
- settings/config surfaces
|
|
||||||
- rollout/migration notes (when relevant)
|
|
||||||
|
|
||||||
## Guardrails
|
|
||||||
|
|
||||||
- Do not collapse all pages into one file.
|
|
||||||
- Do not use globals or implicit global side effects.
|
|
||||||
- Do not block UI event handlers with synchronous I/O or long CPU tasks.
|
|
||||||
- Always define loading/progress/error states for long user-triggered actions.
|
|
||||||
- Keep code minimal but production-minded.
|
|
||||||
- Prefer clarity and maintainability over clever abstractions.
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Architecture and integration details: [NiceGUI architecture reference](./references/architecture.md)
|
|
||||||
- Source documentation links: [NiceGUI source documentation](./references/source-documentation.md)
|
|
||||||
@@ -1,92 +0,0 @@
|
|||||||
# NiceGUI Architecture Reference
|
|
||||||
|
|
||||||
This reference expands the workflow in the main skill file and is loaded only when needed.
|
|
||||||
|
|
||||||
## Baseline package boundaries
|
|
||||||
|
|
||||||
- `main.py`: process entrypoint only.
|
|
||||||
- `bootstrap.py`: app composition, router wiring, page registration, lifespan orchestration.
|
|
||||||
- `config.py`: typed settings and env parsing.
|
|
||||||
- `logging.py`: centralized logging setup.
|
|
||||||
- `api/`: HTTP transport layer; delegates to services.
|
|
||||||
- `services/`: business/use-case logic.
|
|
||||||
- `ui/pages/`: route-level NiceGUI pages.
|
|
||||||
- `ui/components/`: shared UI building blocks.
|
|
||||||
|
|
||||||
## Required baseline behavior
|
|
||||||
|
|
||||||
- FastAPI is the base ASGI app.
|
|
||||||
- NiceGUI pages are modular and registered from page modules.
|
|
||||||
- Minimum pages: `/`, `/dashboard`, `/about`.
|
|
||||||
- FastAPI health route: `/healthz`.
|
|
||||||
- Lifespan handles startup/shutdown resources.
|
|
||||||
- No global side effects at import time.
|
|
||||||
|
|
||||||
## Optional extension: Database
|
|
||||||
|
|
||||||
Use only if persistence is required.
|
|
||||||
|
|
||||||
Suggested additions:
|
|
||||||
|
|
||||||
```text
|
|
||||||
src/app/db/
|
|
||||||
├─ __init__.py
|
|
||||||
├─ base.py
|
|
||||||
├─ session.py
|
|
||||||
├─ models/
|
|
||||||
└─ repositories/
|
|
||||||
```
|
|
||||||
|
|
||||||
Guidelines:
|
|
||||||
|
|
||||||
- One engine and one sessionmaker per process.
|
|
||||||
- Request-scoped session dependency using `yield`.
|
|
||||||
- Explicit transaction boundaries in service/repository flows.
|
|
||||||
- Avoid shared sessions across concurrent tasks.
|
|
||||||
- Use Alembic as schema source of truth.
|
|
||||||
|
|
||||||
## Optional extension: LangGraph AI
|
|
||||||
|
|
||||||
Use only for multi-step AI orchestration or human-in-the-loop workflows.
|
|
||||||
|
|
||||||
Suggested additions:
|
|
||||||
|
|
||||||
```text
|
|
||||||
src/app/ai/
|
|
||||||
├─ state.py
|
|
||||||
├─ nodes/
|
|
||||||
├─ graphs/
|
|
||||||
├─ runtime.py
|
|
||||||
└─ contracts.py
|
|
||||||
```
|
|
||||||
|
|
||||||
Guidelines:
|
|
||||||
|
|
||||||
- Keep graph internals outside API/UI modules.
|
|
||||||
- Invoke graph through `services/ai_service.py`.
|
|
||||||
- Use stable thread/session IDs for resumable sessions.
|
|
||||||
- Keep interrupt payloads JSON-serializable.
|
|
||||||
|
|
||||||
## Optional extension: Mounted static docs
|
|
||||||
|
|
||||||
Use only when generated docs should be served in-app.
|
|
||||||
|
|
||||||
Suggested settings:
|
|
||||||
|
|
||||||
- `docs_enabled`
|
|
||||||
- `docs_mount_path`
|
|
||||||
- `docs_site_dir`
|
|
||||||
- `docs_require_build` (optional)
|
|
||||||
|
|
||||||
Guidelines:
|
|
||||||
|
|
||||||
- Mount docs in composition layer (`bootstrap.py`).
|
|
||||||
- Normalize mount path and avoid route conflicts.
|
|
||||||
- Warn on missing build artifacts unless strict mode is enabled.
|
|
||||||
|
|
||||||
## Suggested output quality criteria
|
|
||||||
|
|
||||||
- Clear architecture summary with assumptions.
|
|
||||||
- Explicit decisions for DB, AI, and docs.
|
|
||||||
- Package-scoped implementation checklist.
|
|
||||||
- Minimal test plan aligned to enabled features.
|
|
||||||
@@ -1,35 +0,0 @@
|
|||||||
# Source Documentation
|
|
||||||
|
|
||||||
Use these links for framework-specific details.
|
|
||||||
|
|
||||||
## FastAPI
|
|
||||||
|
|
||||||
- FastAPI lifespan events: https://fastapi.tiangolo.com/advanced/events/
|
|
||||||
- FastAPI settings and environment variables: https://fastapi.tiangolo.com/advanced/settings/
|
|
||||||
- FastAPI dependencies with yield: https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/
|
|
||||||
- FastAPI SQL databases tutorial: https://fastapi.tiangolo.com/tutorial/sql-databases/
|
|
||||||
|
|
||||||
## SQLAlchemy and Alembic
|
|
||||||
|
|
||||||
- SQLAlchemy engine configuration and pooling: https://docs.sqlalchemy.org/en/20/core/engines.html
|
|
||||||
- SQLAlchemy session lifecycle basics: https://docs.sqlalchemy.org/en/20/orm/session_basics.html
|
|
||||||
- Alembic tutorial: https://alembic.sqlalchemy.org/en/latest/tutorial.html
|
|
||||||
|
|
||||||
## Pydantic
|
|
||||||
|
|
||||||
- Pydantic settings management: https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/
|
|
||||||
|
|
||||||
## NiceGUI
|
|
||||||
|
|
||||||
- NiceGUI pages/routing and FastAPI integration: https://www.nicegui.io/documentation/section_pages_routing
|
|
||||||
- NiceGUI security best practices: https://www.nicegui.io/documentation/section_security
|
|
||||||
|
|
||||||
## LangGraph
|
|
||||||
|
|
||||||
- LangGraph overview: https://docs.langchain.com/oss/python/langgraph/overview
|
|
||||||
- LangGraph quickstart: https://docs.langchain.com/oss/python/langgraph/quickstart
|
|
||||||
- LangGraph workflows and agents: https://docs.langchain.com/oss/python/langgraph/workflows-agents
|
|
||||||
- LangGraph persistence: https://docs.langchain.com/oss/python/langgraph/persistence
|
|
||||||
- LangGraph memory concepts: https://docs.langchain.com/oss/python/concepts/memory
|
|
||||||
- LangGraph streaming: https://docs.langchain.com/oss/python/langgraph/streaming
|
|
||||||
- LangGraph interrupts and human-in-the-loop: https://docs.langchain.com/oss/python/langgraph/interrupts
|
|
||||||
@@ -1,136 +0,0 @@
|
|||||||
---
|
|
||||||
name: pytest-scaffolding
|
|
||||||
description: "Scaffold a maintainable, hierarchical pytest suite for core functionality first, then extend safely. Use when setting up tests, organizing fixtures by dependency, mirroring src structure in tests, or enforcing fast-by-default test runs."
|
|
||||||
argument-hint: "Target scope (for example: app/services/job, app/ai, or full repo)"
|
|
||||||
---
|
|
||||||
|
|
||||||
# Pytest Scaffolding
|
|
||||||
|
|
||||||
Create test scaffolding that is:
|
|
||||||
- Hierarchical: test layout roughly mirrors source layout.
|
|
||||||
- Fast by default: most tests run in under a second total for core units.
|
|
||||||
- Dependency-aware: slow/external dependencies are isolated behind markers and fixture scope.
|
|
||||||
- Extensible: minimal initial skeleton supports adding detailed tests later without refactors.
|
|
||||||
|
|
||||||
This repository currently uses:
|
|
||||||
- `uv run pytest` as the canonical test invocation.
|
|
||||||
- `pyproject.toml` pytest config under `[tool.pytest.ini_options]`.
|
|
||||||
- strict marker checking (`--strict-markers`).
|
|
||||||
|
|
||||||
Load [pytest references](./references/pytest-docs.md) when you need detailed rules.
|
|
||||||
|
|
||||||
## When To Use
|
|
||||||
- Bootstrapping tests for a new or existing Python repo.
|
|
||||||
- Reorganizing tests that have become flat, slow, or difficult to extend.
|
|
||||||
- Defining fixture boundaries before writing many assertions.
|
|
||||||
- Creating only the first-layer scaffold for core behavior (not exhaustive coverage yet).
|
|
||||||
|
|
||||||
## Inputs To Collect
|
|
||||||
1. Target test scope: full repo, package, or module.
|
|
||||||
2. Dependency profile: pure Python, DB, network/API, filesystem, UI/browser.
|
|
||||||
3. Runtime expectation: what must be instant vs allowed to be slower.
|
|
||||||
4. CI policy: which marker groups must block merges.
|
|
||||||
|
|
||||||
If these are missing, ask concise clarifying questions before editing.
|
|
||||||
|
|
||||||
## Workflow
|
|
||||||
1. Map source tree to test tree.
|
|
||||||
2. Classify tests by dependency cost.
|
|
||||||
3. Create minimal directories and placeholder test modules.
|
|
||||||
4. Create fixture layers (`tests/conftest.py` plus local `conftest.py` in subtrees only when needed).
|
|
||||||
5. Register markers and default selection behavior.
|
|
||||||
6. Run collection and fast path tests.
|
|
||||||
7. Report gaps and next extension points.
|
|
||||||
|
|
||||||
## Step 1: Map Source To Tests
|
|
||||||
Create a mirrored structure rooted at `tests/` that follows major source concepts.
|
|
||||||
|
|
||||||
Example mapping pattern:
|
|
||||||
- `src/app/services/job.py` -> `tests/app/services/test_job.py`
|
|
||||||
- `src/app/ai/graphs/transcription.py` -> `tests/app/ai/graphs/test_transcription.py`
|
|
||||||
- `src/app/api/routes.py` -> `tests/app/api/test_routes.py`
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
- One initial test module per core source module.
|
|
||||||
- Prefer `test_<module>.py` naming.
|
|
||||||
- Keep directory mirrors shallow first; add deeper modules only where behavior is complex.
|
|
||||||
|
|
||||||
## Step 2: Classify By Dependency Cost
|
|
||||||
Assign each test module to one initial class:
|
|
||||||
- `unit`: no DB/network/filesystem side effects; instant execution.
|
|
||||||
- `integration`: touches DB, HTTP stack, workflow runtime, or external services.
|
|
||||||
- `smoke`: thin end-to-end confidence checks.
|
|
||||||
|
|
||||||
Decision logic:
|
|
||||||
- If logic can run with fakes/stubs, make it `unit`.
|
|
||||||
- If contract with framework/DB is essential, make it `integration`.
|
|
||||||
- If validating a user-critical path across layers, make it `smoke`.
|
|
||||||
|
|
||||||
## Step 3: Scaffold Minimal Test Modules
|
|
||||||
For each target module, scaffold:
|
|
||||||
- import section
|
|
||||||
- one happy-path test function
|
|
||||||
- one error/edge test function
|
|
||||||
- TODO comments indicating detail expansion points
|
|
||||||
|
|
||||||
Keep assertions minimal but behavior-focused. Avoid large fixtures in module files.
|
|
||||||
|
|
||||||
## Step 4: Fixture Layering Strategy
|
|
||||||
Use fixture scopes based on cost:
|
|
||||||
- `function` scope by default.
|
|
||||||
- broader scopes (`module`/`session`) only for expensive setup with clear teardown.
|
|
||||||
|
|
||||||
Layer fixtures by directory:
|
|
||||||
- `tests/conftest.py`: global, lightweight fixtures only (factories, deterministic defaults).
|
|
||||||
- subtree `conftest.py`: domain-specific fixtures (API client, DB session, AI runtime stubs).
|
|
||||||
|
|
||||||
Guidelines:
|
|
||||||
- Prefer yield fixtures for setup/teardown.
|
|
||||||
- Keep fixtures atomic (one state-changing responsibility per fixture).
|
|
||||||
- Avoid autouse except for truly universal behavior.
|
|
||||||
|
|
||||||
## Step 5: Marker Taxonomy And Config
|
|
||||||
Ensure marker names are explicit and registered in `pyproject.toml` because strict markers are enabled.
|
|
||||||
|
|
||||||
Recommended baseline markers:
|
|
||||||
- `unit`
|
|
||||||
- `integration`
|
|
||||||
- `smoke`
|
|
||||||
- `slow`
|
|
||||||
- `external` (requires network/service credentials)
|
|
||||||
|
|
||||||
Default run strategy:
|
|
||||||
- Fast local path: run only `unit` by default in day-to-day iteration.
|
|
||||||
- Full validation path: run all markers in CI or pre-release checks.
|
|
||||||
|
|
||||||
## Step 6: Execution And Verification
|
|
||||||
Run commands in this order:
|
|
||||||
1. `uv run pytest --collect-only -q`
|
|
||||||
2. `uv run pytest -m unit -q`
|
|
||||||
3. `uv run pytest -q` (if dependencies are available)
|
|
||||||
|
|
||||||
Optional targeted runs:
|
|
||||||
- by node id for one test
|
|
||||||
- by `-k` expression for focused iteration
|
|
||||||
|
|
||||||
## Step 7: Completion Checks
|
|
||||||
A scaffold pass is complete when all are true:
|
|
||||||
1. Every core source area has at least one corresponding test module.
|
|
||||||
2. Unit tests run quickly and deterministically.
|
|
||||||
3. Integration/external tests are isolated by marker and fixture boundaries.
|
|
||||||
4. No unregistered marker warnings/errors.
|
|
||||||
5. `tests/` structure is understandable without extra documentation.
|
|
||||||
6. A clear TODO path exists for deepening assertions later.
|
|
||||||
|
|
||||||
## Branching Scenarios
|
|
||||||
- If external APIs are required: provide stubs/mocks for unit tests; guard real calls behind `external` marker.
|
|
||||||
- If DB is required: build a dedicated integration fixture layer and keep unit tests DB-free.
|
|
||||||
- If tests become slow: split slow tests via marker and widen fixture scope only where safe.
|
|
||||||
- If naming conflicts appear: keep unique test module names or package test directories explicitly.
|
|
||||||
|
|
||||||
## Output Format
|
|
||||||
When applying this skill, provide:
|
|
||||||
1. Proposed test tree diff.
|
|
||||||
2. Marker and fixture plan.
|
|
||||||
3. Exact commands for fast path and full path.
|
|
||||||
4. Risks/open questions before writing detailed assertions.
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
# Pytest Documentation Notes
|
|
||||||
|
|
||||||
Primary references used:
|
|
||||||
- https://docs.pytest.org/en/stable/explanation/goodpractices.html
|
|
||||||
- https://docs.pytest.org/en/stable/how-to/fixtures.html
|
|
||||||
- https://docs.pytest.org/en/stable/example/markers.html
|
|
||||||
- https://docs.pytest.org/en/stable/reference/customize.html
|
|
||||||
- https://docs.pytest.org/en/stable/explanation/flaky.html
|
|
||||||
|
|
||||||
## Practical Guidance For This Skill
|
|
||||||
- Use src-aligned test layout and keep test discovery conventional.
|
|
||||||
- Keep fixtures small, composable, and explicit; use `yield` for teardown.
|
|
||||||
- Register custom markers and keep strict marker validation on.
|
|
||||||
- Separate quick unit runs from slower integration/external runs.
|
|
||||||
- Minimize flakiness by controlling shared state and avoiding hidden dependencies.
|
|
||||||
- Use `--collect-only` and marker-filtered runs to validate scaffold quality early.
|
|
||||||
|
|
||||||
## Commands Worth Remembering
|
|
||||||
- `uv run pytest --collect-only -q`
|
|
||||||
- `uv run pytest -m unit -q`
|
|
||||||
- `uv run pytest -m "not external" -q`
|
|
||||||
- `uv run pytest -q`
|
|
||||||
@@ -1,97 +0,0 @@
|
|||||||
---
|
|
||||||
name: python-logging-dictconfig
|
|
||||||
description: 'Set up idiomatic Python logging with logging.config.dictConfig. Use when creating or refactoring logging setup, standardizing handlers/formatters, and enforcing centralized config.'
|
|
||||||
argument-hint: 'Target context (single script, package, FastAPI app, or CLI) and desired log destinations'
|
|
||||||
---
|
|
||||||
|
|
||||||
# Idiomatic Python Logging with dictConfig
|
|
||||||
|
|
||||||
Use this skill to produce a minimal, centralized logging setup using `logging.config.dictConfig`.
|
|
||||||
|
|
||||||
Load references only when needed:
|
|
||||||
- Python logging overview and hierarchy: [./references/python-logging-docs.md](./references/python-logging-docs.md)
|
|
||||||
|
|
||||||
## When to Use
|
|
||||||
- A project configures logging ad hoc with `basicConfig` across multiple modules.
|
|
||||||
- You need one canonical logging configuration for app startup.
|
|
||||||
- You need consistent formatting and levels across console/file handlers.
|
|
||||||
- You want library modules to use named loggers without configuring logging themselves.
|
|
||||||
|
|
||||||
## Inputs To Collect
|
|
||||||
1. Runtime type: script, library, web app, worker, CLI.
|
|
||||||
2. Destinations: stdout only, file only, or both.
|
|
||||||
3. Desired default level: `INFO`, `DEBUG`, etc.
|
|
||||||
4. Whether third-party loggers should be tuned (for example `uvicorn`, `sqlalchemy`).
|
|
||||||
|
|
||||||
If missing, assume:
|
|
||||||
- stdout handler
|
|
||||||
- human-readable formatter
|
|
||||||
- root level `INFO`
|
|
||||||
- `disable_existing_loggers: False`
|
|
||||||
|
|
||||||
## Procedure
|
|
||||||
1. Define a single `LOGGING` dictionary in one startup-oriented module (for example `logging_config.py`).
|
|
||||||
2. Include `version: 1` and set `disable_existing_loggers: False` unless there is a specific reason to silence existing loggers.
|
|
||||||
3. Define formatters first, then handlers, then logger routing (`root` and optional named `loggers`).
|
|
||||||
4. Use `logging.config.dictConfig(LOGGING)` exactly once during application startup.
|
|
||||||
5. In all modules, get loggers via `logger = logging.getLogger(__name__)` and never call `basicConfig`.
|
|
||||||
6. Keep libraries configuration-free: libraries should emit logs, applications decide routing.
|
|
||||||
7. Verify behavior with a quick smoke check at multiple levels (`DEBUG`, `INFO`, `WARNING`, `ERROR`).
|
|
||||||
|
|
||||||
## Minimal Baseline Template
|
|
||||||
```python
|
|
||||||
# logging_config.py
|
|
||||||
from logging.config import dictConfig
|
|
||||||
|
|
||||||
LOGGING = {
|
|
||||||
"version": 1,
|
|
||||||
"disable_existing_loggers": False,
|
|
||||||
"formatters": {
|
|
||||||
"standard": {
|
|
||||||
"format": "%(asctime)s %(levelname)s %(name)s: %(message)s"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"handlers": {
|
|
||||||
"console": {
|
|
||||||
"class": "logging.StreamHandler",
|
|
||||||
"level": "INFO",
|
|
||||||
"formatter": "standard",
|
|
||||||
"stream": "ext://sys.stdout",
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"root": {
|
|
||||||
"level": "INFO",
|
|
||||||
"handlers": ["console"],
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
def configure_logging() -> None:
|
|
||||||
dictConfig(LOGGING)
|
|
||||||
```
|
|
||||||
|
|
||||||
```python
|
|
||||||
# app startup
|
|
||||||
from .logging_config import configure_logging
|
|
||||||
|
|
||||||
configure_logging()
|
|
||||||
```
|
|
||||||
|
|
||||||
```python
|
|
||||||
# any module
|
|
||||||
import logging
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
logger.info("module initialized")
|
|
||||||
```
|
|
||||||
|
|
||||||
## Completion Checks
|
|
||||||
1. `dictConfig` is called once at startup, not per module.
|
|
||||||
2. No `basicConfig` calls remain.
|
|
||||||
3. Modules use `getLogger(__name__)`.
|
|
||||||
4. Logs appear at expected level and destination.
|
|
||||||
5. Third-party logger noise is intentionally configured or left at defaults.
|
|
||||||
|
|
||||||
## Branching Guidance
|
|
||||||
- If structured logs are required: switch formatter output to JSON while keeping `dictConfig` topology unchanged.
|
|
||||||
- If both console and file output are needed: add a file handler and attach it to `root`.
|
|
||||||
- If a specific framework logger is too noisy: add a named logger override under `loggers`.
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
# Python Logging References
|
|
||||||
|
|
||||||
Use these official Python docs when applying this skill.
|
|
||||||
|
|
||||||
## Core Documentation
|
|
||||||
- Logging HOWTO: https://docs.python.org/3/howto/logging.html
|
|
||||||
- Logging Cookbook: https://docs.python.org/3/howto/logging-cookbook.html
|
|
||||||
- logging API reference: https://docs.python.org/3/library/logging.html
|
|
||||||
- logging.config reference: https://docs.python.org/3/library/logging.config.html
|
|
||||||
|
|
||||||
## dictConfig-Specific
|
|
||||||
- Dictionary schema details (`version`, formatters, handlers, loggers, root): https://docs.python.org/3/library/logging.config.html#logging-config-dictschema
|
|
||||||
- `logging.config.dictConfig` function: https://docs.python.org/3/library/logging.config.html#logging.config.dictConfig
|
|
||||||
|
|
||||||
## Practical Notes
|
|
||||||
- Prefer app-level centralized config with one startup call to `dictConfig`.
|
|
||||||
- In modules, use `logging.getLogger(__name__)`.
|
|
||||||
- Avoid calling `basicConfig` in libraries or scattered modules.
|
|
||||||
@@ -1,175 +0,0 @@
|
|||||||
---
|
|
||||||
name: zensical-docs
|
|
||||||
description: 'Plan, write, and improve high-quality documentation with Zensical. Use for information architecture, progressive discoverability, writing standards, navigation/search tuning, and feature-driven docs site configuration.'
|
|
||||||
argument-hint: 'What are you documenting, who is the audience, and what Zensical features are in scope?'
|
|
||||||
---
|
|
||||||
|
|
||||||
# Zensical Documentation Authoring
|
|
||||||
|
|
||||||
Create documentation that is easy to discover, easy to scan, and easy to trust, then configure Zensical so the site reinforces those outcomes.
|
|
||||||
|
|
||||||
## When to Use
|
|
||||||
|
|
||||||
- You are creating or restructuring docs in a Zensical project.
|
|
||||||
- You need a repeatable workflow for docs quality and discoverability.
|
|
||||||
- You want guidance that combines writing best practices with Zensical feature choices.
|
|
||||||
- You need a checklist-driven review before publishing.
|
|
||||||
|
|
||||||
## Progressive Loading References
|
|
||||||
|
|
||||||
Load only the references required for the current task:
|
|
||||||
|
|
||||||
- Source map for official docs and APIs: [./references/index.md](./references/index.md)
|
|
||||||
- Zensical features and configuration links: [./references/zensical-features.md](./references/zensical-features.md)
|
|
||||||
- Theme customization, colors, icons, and extension patterns: [./references/theme-customization-and-icons.md](./references/theme-customization-and-icons.md)
|
|
||||||
- Docs writing quality and style guidance: [./references/documentation-quality.md](./references/documentation-quality.md)
|
|
||||||
- Information architecture and discoverability patterns: [./references/discoverability-and-ia.md](./references/discoverability-and-ia.md)
|
|
||||||
- Code-heavy API docs with mkdocstrings: [./references/code-heavy-docs-and-mkdocstrings.md](./references/code-heavy-docs-and-mkdocstrings.md)
|
|
||||||
|
|
||||||
## Project Bootstrap Rule
|
|
||||||
|
|
||||||
Always start a new documentation project with `uv run zensical new`.
|
|
||||||
|
|
||||||
- This command provides the baseline scaffolding you need for structure, configuration, and theme setup.
|
|
||||||
- Do not hand-build a new project skeleton when this command is available.
|
|
||||||
|
|
||||||
## Related Skill Discovery
|
|
||||||
|
|
||||||
When the task is not just writing docs but creating or wiring a new MCP skill in this repository, use catalog discovery to load the bootstrap skill before drafting implementation steps.
|
|
||||||
|
|
||||||
1. Search the catalog with terms such as `new skill`, `skill bootstrap`, or `scaffold skill`.
|
|
||||||
2. Fetch the `new-skill` document through the catalog tool path.
|
|
||||||
3. Use that skill for runtime package, metadata, and mount wiring, then return here for documentation architecture and Zensical-specific authoring guidance.
|
|
||||||
|
|
||||||
This keeps implementation guidance and documentation guidance separated while still making both discoverable from one request.
|
|
||||||
|
|
||||||
## Inputs To Collect
|
|
||||||
|
|
||||||
Collect these before writing. If missing, make explicit assumptions.
|
|
||||||
|
|
||||||
1. Audience: beginner, intermediate, advanced, or mixed.
|
|
||||||
2. User goals: top tasks users come to complete.
|
|
||||||
3. Product scope: features, versions, and deployment context.
|
|
||||||
4. Content constraints: release deadlines, localization, legal/compliance needs.
|
|
||||||
5. Navigation constraints: explicit `nav` vs implicit structure.
|
|
||||||
6. Site-level expectations: search quality, code example depth, diagrams, API docs.
|
|
||||||
|
|
||||||
## Procedure
|
|
||||||
|
|
||||||
### Step 0: Audit Current Documentation
|
|
||||||
|
|
||||||
Build a quick baseline of what exists now.
|
|
||||||
|
|
||||||
1. List docs sections and current nav structure.
|
|
||||||
2. Identify orphan pages (not in nav, weak internal linking, or no inbound links).
|
|
||||||
3. Identify stale sections (version drift, missing prerequisites, broken commands).
|
|
||||||
4. Capture recurring support questions and map each to a missing or weak doc page.
|
|
||||||
|
|
||||||
Completion check: you can name the top 5 current documentation gaps.
|
|
||||||
|
|
||||||
### Step 1: Define Outcomes and Page Taxonomy
|
|
||||||
|
|
||||||
Organize content by user intent, not by internal team boundaries.
|
|
||||||
|
|
||||||
1. Split content into at least four buckets:
|
|
||||||
- Learn (concepts and mental models)
|
|
||||||
- Do (task/how-to guides)
|
|
||||||
- Reference (API, config, CLI, schemas)
|
|
||||||
- Troubleshoot (symptoms, diagnostics, fixes)
|
|
||||||
2. For each bucket, define success criteria and expected time-to-answer.
|
|
||||||
3. Add clear page entry criteria (what belongs here, what does not).
|
|
||||||
|
|
||||||
Completion check: every planned page maps to one primary user intent.
|
|
||||||
|
|
||||||
### Step 2: Design Progressive Discoverability
|
|
||||||
|
|
||||||
Make docs progressively discoverable from overview to detail.
|
|
||||||
|
|
||||||
1. Start each section with an index/overview page that answers:
|
|
||||||
- who this section is for
|
|
||||||
- what problems it solves
|
|
||||||
- where to go next
|
|
||||||
2. Keep page openings front-loaded:
|
|
||||||
- first paragraph states purpose and outcome
|
|
||||||
- first heading after intro is usually prerequisites or quickstart path
|
|
||||||
3. Add consistent wayfinding on every page:
|
|
||||||
- links to prerequisite concepts
|
|
||||||
- links to next steps
|
|
||||||
- links to relevant reference sections
|
|
||||||
4. Prefer short sections, descriptive headings, and stable anchor names.
|
|
||||||
|
|
||||||
Completion check: users can navigate from high-level overview to exact procedure in 3 clicks or fewer.
|
|
||||||
|
|
||||||
### Step 3: Author High-Trust Content
|
|
||||||
|
|
||||||
Use writing patterns that reduce ambiguity and failure.
|
|
||||||
|
|
||||||
1. Use imperative, task-oriented titles (for example: "Configure SSO with OIDC").
|
|
||||||
2. State assumptions and prerequisites before commands.
|
|
||||||
3. Provide copy-paste-safe examples and expected outputs.
|
|
||||||
4. Include rollback or recovery steps for risky operations.
|
|
||||||
5. Separate normative guidance (must/should) from optional patterns.
|
|
||||||
6. Call out version-specific behavior explicitly.
|
|
||||||
|
|
||||||
Completion check: each task page includes prerequisites, steps, expected result, and failure recovery.
|
|
||||||
|
|
||||||
### Step 4: Apply Zensical Features Intentionally
|
|
||||||
|
|
||||||
Load [./references/zensical-features.md](./references/zensical-features.md) and choose features based on content needs.
|
|
||||||
Load [./references/code-heavy-docs-and-mkdocstrings.md](./references/code-heavy-docs-and-mkdocstrings.md) when the docs include Python APIs or other generated references.
|
|
||||||
|
|
||||||
Decision matrix:
|
|
||||||
|
|
||||||
- If docs are deep and section-heavy: enable `navigation.indexes`, `navigation.path`, and `navigation.sections`.
|
|
||||||
- If speed and perceived responsiveness matter: enable `navigation.instant` and `navigation.instant.prefetch`.
|
|
||||||
- If code-heavy content exists: enable `content.code.copy`, `content.code.select`, and `content.code.annotate`.
|
|
||||||
- If code-heavy content includes API references: standardize on mkdocstrings for generated API docs and keep hand-written task guides alongside generated reference pages.
|
|
||||||
- If users rely on search often: enable `search.highlight` and improve heading quality for better snippets.
|
|
||||||
- If many pages share tab labels (for example Python/JS): enable `content.tabs.link`.
|
|
||||||
|
|
||||||
Completion check: each enabled feature has a documented rationale tied to a user outcome.
|
|
||||||
|
|
||||||
### Step 5: Standardize Navigation and Search Quality
|
|
||||||
|
|
||||||
1. Use explicit nav for critical docs journeys and release-stable ordering.
|
|
||||||
2. Keep page titles and H1 aligned with search intent language users actually type.
|
|
||||||
3. Avoid duplicate page titles across sections.
|
|
||||||
4. Use concise first paragraphs because search snippets often rely on early content.
|
|
||||||
5. Add cross-links between concept, task, and reference pages.
|
|
||||||
|
|
||||||
Completion check: top support queries return a correct page within first search results.
|
|
||||||
|
|
||||||
### Step 6: Publish and Validate
|
|
||||||
|
|
||||||
1. Build the site: `uv run zensical build`
|
|
||||||
2. Validate links, code blocks, and navigation paths.
|
|
||||||
3. Review on desktop and mobile for scanability and heading rhythm.
|
|
||||||
4. Check that key journeys (new user setup, common task, troubleshooting flow) are uninterrupted.
|
|
||||||
|
|
||||||
Completion check: no broken links, no dead-end pages, and all critical journeys are complete.
|
|
||||||
|
|
||||||
## Completion Checks
|
|
||||||
|
|
||||||
- Page taxonomy is intent-based: Learn, Do, Reference, Troubleshoot.
|
|
||||||
- Every major section has an overview page and next-step links.
|
|
||||||
- Task docs contain prerequisites, steps, expected results, and recovery guidance.
|
|
||||||
- Enabled Zensical features are justified and aligned to user needs.
|
|
||||||
- Code-heavy documentation strategy is explicit, including when and how mkdocstrings is used.
|
|
||||||
- Navigation and search behavior are verified with real user tasks.
|
|
||||||
- Site builds cleanly with `uv run zensical build`.
|
|
||||||
|
|
||||||
## Output Contract
|
|
||||||
|
|
||||||
Return:
|
|
||||||
|
|
||||||
1. Proposed docs architecture and navigation map.
|
|
||||||
2. Zensical feature configuration recommendations with rationale.
|
|
||||||
3. A prioritized writing backlog (must-have, should-have, nice-to-have).
|
|
||||||
4. A quality-gate checklist for pre-publish review.
|
|
||||||
|
|
||||||
## Guardrails
|
|
||||||
|
|
||||||
- Do not produce architecture-only docs without concrete task pages.
|
|
||||||
- Do not bury prerequisites or version constraints below the fold.
|
|
||||||
- Do not rely only on search; preserve strong navigational paths.
|
|
||||||
- Do not enable features without documenting the expected user benefit.
|
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
# Zensical Docs Skill References
|
|
||||||
|
|
||||||
Use this index to load only the source references needed for the current task.
|
|
||||||
|
|
||||||
## Zensical Official Docs
|
|
||||||
|
|
||||||
- New project scaffolding (`uv run zensical new`): https://zensical.org/docs/
|
|
||||||
- Home: https://zensical.org/docs/
|
|
||||||
- Setup basics: https://zensical.org/docs/setup/basics/
|
|
||||||
- Navigation setup: https://zensical.org/docs/setup/navigation/
|
|
||||||
- Header setup and announcement bar: https://zensical.org/docs/setup/header/
|
|
||||||
- Footer setup: https://zensical.org/docs/setup/footer/
|
|
||||||
- Repository and content actions: https://zensical.org/docs/setup/repository/
|
|
||||||
- Search setup: https://zensical.org/docs/setup/search/
|
|
||||||
- Customization overview: https://zensical.org/docs/customization/
|
|
||||||
- Additional CSS: https://zensical.org/docs/customization/#additional-css
|
|
||||||
- Additional JavaScript: https://zensical.org/docs/customization/#additional-javascript
|
|
||||||
- Theme extension and overrides: https://zensical.org/docs/customization/#extending-the-theme
|
|
||||||
- Language setup: https://zensical.org/docs/setup/language/
|
|
||||||
- Logo and icons: https://zensical.org/docs/setup/logo-and-icons/
|
|
||||||
- Code blocks and annotations: https://zensical.org/docs/authoring/code-blocks/
|
|
||||||
- Content tabs: https://zensical.org/docs/authoring/content-tabs/
|
|
||||||
- Footnotes: https://zensical.org/docs/authoring/footnotes/
|
|
||||||
- Tooltips: https://zensical.org/docs/authoring/tooltips/
|
|
||||||
|
|
||||||
## Adjacent Documentation Quality Sources
|
|
||||||
|
|
||||||
- Divio documentation system: https://documentation.divio.com/
|
|
||||||
- Write the Docs guide: https://www.writethedocs.org/guide/
|
|
||||||
- Google developer documentation style guide: https://developers.google.com/style
|
|
||||||
- Microsoft Writing Style Guide: https://learn.microsoft.com/style-guide/welcome/
|
|
||||||
- MDN writing guidelines: https://developer.mozilla.org/en-US/docs/MDN/Writing_guidelines
|
|
||||||
- Diataxis framework: https://diataxis.fr/
|
|
||||||
|
|
||||||
## Related Tooling References
|
|
||||||
|
|
||||||
- Markdown guide: https://www.markdownguide.org/
|
|
||||||
- MkDocs configuration reference: https://www.mkdocs.org/user-guide/configuration/
|
|
||||||
- Material for MkDocs setup reference: https://squidfunk.github.io/mkdocs-material/setup/
|
|
||||||
- mkdocstrings: https://mkdocstrings.github.io/
|
|
||||||
|
|
||||||
## Skill-Specific Deep Dives
|
|
||||||
|
|
||||||
- Theme customization, colors, icons: [./theme-customization-and-icons.md](./theme-customization-and-icons.md)
|
|
||||||
- Code-heavy docs with mkdocstrings: [./code-heavy-docs-and-mkdocstrings.md](./code-heavy-docs-and-mkdocstrings.md)
|
|
||||||
-238
@@ -1,238 +0,0 @@
|
|||||||
---
|
|
||||||
icon: lucide/workflow
|
|
||||||
---
|
|
||||||
|
|
||||||
# Skill Usage Mechanics
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
This page explains practical usage mechanics for the GitHub Copilot extension in VS Code when `personal-mcp` is configured as an MCP server:
|
|
||||||
|
|
||||||
1. explicit `/` command flows when you want deterministic control
|
|
||||||
2. automatic skill context loading when relevance can be inferred
|
|
||||||
|
|
||||||
The goal is to show how Copilot behaves as a client and how to shape that behavior.
|
|
||||||
|
|
||||||
## Mental Model
|
|
||||||
|
|
||||||
In Copilot Chat, there are two distinct mechanisms:
|
|
||||||
|
|
||||||
1. `/` commands are user-invoked orchestration shortcuts.
|
|
||||||
2. MCP resources are server-published knowledge units that can be attached as read-only context, while MCP tools provide an execution path for discovery and retrieval.
|
|
||||||
|
|
||||||
In this repository, skill guidance is exposed as MCP resources, not as server-owned prompt execution. Copilot remains the orchestrator.
|
|
||||||
|
|
||||||
## Background Mechanics
|
|
||||||
|
|
||||||
### What the server publishes
|
|
||||||
|
|
||||||
`personal-mcp` mounts skill modules and a catalog module. The catalog exposes discovery resources:
|
|
||||||
|
|
||||||
1. `resource://catalog/skills_index`
|
|
||||||
2. `resource://catalog/skills_details`
|
|
||||||
3. `resource://catalog/patterns`
|
|
||||||
4. `resource://catalog/patterns_by_id`
|
|
||||||
|
|
||||||
Each skill publishes a canonical Markdown document resource:
|
|
||||||
|
|
||||||
1. `resource://skills/<skill-id>/document`
|
|
||||||
|
|
||||||
The document payload is loaded from `docs/skills/<slug>/SKILL.md` and returned with metadata.
|
|
||||||
|
|
||||||
### What Copilot does as the client
|
|
||||||
|
|
||||||
When connected to MCP, Copilot can do the following at runtime:
|
|
||||||
|
|
||||||
1. interpret the current chat request
|
|
||||||
2. use attached MCP resources that you provide through the chat UI
|
|
||||||
3. invoke MCP tools when the task and tool descriptions make that relevant
|
|
||||||
4. summarize relevant sections into working context
|
|
||||||
5. apply guidance while generating edits or recommendations
|
|
||||||
|
|
||||||
This behavior is shaped by the active chat surface, prompt or instruction guidance, and available MCP tools.
|
|
||||||
|
|
||||||
For reliable progressive discovery, use one of these sequences:
|
|
||||||
|
|
||||||
1. explicit resource path: attach a catalog resource first, then attach only selected skill documents
|
|
||||||
2. tool path: call catalog tools first, then load only selected skill documents
|
|
||||||
|
|
||||||
### What `/` commands do
|
|
||||||
|
|
||||||
`/` commands in VS Code are client-side prompt entry points (for example in prompt files). They do not replace MCP resources. In Copilot, they typically:
|
|
||||||
|
|
||||||
1. enforce a known sequence
|
|
||||||
2. collect missing inputs
|
|
||||||
3. call discovery/read steps in a predictable order
|
|
||||||
|
|
||||||
Think of `/` commands as orchestration shortcuts on top of MCP resources.
|
|
||||||
|
|
||||||
### What automatic loading means here
|
|
||||||
|
|
||||||
In this project, "automatic loading" should be read as a preference you express through instructions and prompts, not as a guaranteed VS Code feature that auto-attaches MCP resources.
|
|
||||||
|
|
||||||
In practice, there are two reliable ways to make skill content available in chat:
|
|
||||||
|
|
||||||
1. explicit resource attachment through `Add Context > MCP Resources` or `MCP: Browse Resources`
|
|
||||||
2. MCP tool invocation such as `search_patterns` followed by `get_skill_document_by_id`
|
|
||||||
|
|
||||||
Instruction quality and metadata quality still matter, because they influence whether Copilot recognizes that the MCP server is relevant and chooses the tool path well.
|
|
||||||
|
|
||||||
## Operating Pattern
|
|
||||||
|
|
||||||
Use both modes intentionally in Copilot Chat.
|
|
||||||
|
|
||||||
### Mode A: Explicit `/` command
|
|
||||||
|
|
||||||
Use when you need predictable, repeatable behavior across teammates.
|
|
||||||
|
|
||||||
Good fits:
|
|
||||||
|
|
||||||
1. onboarding workflows
|
|
||||||
2. compliance-sensitive tasks
|
|
||||||
3. repetitive scaffolding
|
|
||||||
|
|
||||||
### Mode B: Guided skill loading
|
|
||||||
|
|
||||||
Use when requests are varied and you want lower friction during normal chat.
|
|
||||||
|
|
||||||
Good fits:
|
|
||||||
|
|
||||||
1. ad hoc implementation questions
|
|
||||||
2. mixed-topic debugging
|
|
||||||
3. architecture tradeoff discussions
|
|
||||||
|
|
||||||
### Mode C: Fallback flow
|
|
||||||
|
|
||||||
Start with guided loading in chat; escalate to a `/` command when:
|
|
||||||
|
|
||||||
1. confidence is low
|
|
||||||
2. multiple skills conflict
|
|
||||||
3. the user wants strict repeatability
|
|
||||||
|
|
||||||
## Suggested Resolution Flow
|
|
||||||
|
|
||||||
```mermaid
|
|
||||||
flowchart TD
|
|
||||||
A[User request in Copilot Chat] --> B{Deterministic workflow needed?}
|
|
||||||
B -- Yes --> C[/Run slash command/]
|
|
||||||
C --> D[Copilot fetches known catalog and skill resources]
|
|
||||||
B -- No --> E[Copilot uses attached resources or catalog tools]
|
|
||||||
E --> F{Confident skill match?}
|
|
||||||
F -- Yes --> G[Copilot fetches skill documents]
|
|
||||||
F -- No --> H[Ask clarifying question or suggest slash command]
|
|
||||||
D --> I[Apply guidance to task]
|
|
||||||
G --> I
|
|
||||||
H --> I
|
|
||||||
```
|
|
||||||
|
|
||||||
## Authoring Requirements For Reliable Matching
|
|
||||||
|
|
||||||
For resource selection or tool-based matching to work well, each skill should have:
|
|
||||||
|
|
||||||
1. precise `description`
|
|
||||||
2. focused `tags`
|
|
||||||
3. explicit `capabilities`
|
|
||||||
4. stable `id` and slug naming
|
|
||||||
|
|
||||||
Weak metadata reduces Copilot match quality and increases wrong context injection.
|
|
||||||
|
|
||||||
## Practical Guidelines
|
|
||||||
|
|
||||||
1. Keep `/` commands minimal and high-value.
|
|
||||||
2. Do not duplicate full methodology text inside command files.
|
|
||||||
3. Keep canonical guidance in `docs/skills/*/SKILL.md`.
|
|
||||||
4. In Copilot instructions, prefer catalog-first discovery before skill fetch.
|
|
||||||
5. Prefer small, relevant context slices over loading every skill.
|
|
||||||
6. Keep slash commands focused on deterministic orchestration, not content duplication.
|
|
||||||
|
|
||||||
If you skip the catalog/index step, behavior is less predictable and may either miss relevant skills or pull too much context.
|
|
||||||
|
|
||||||
## Copilot Instruction Pattern
|
|
||||||
|
|
||||||
If you want Copilot to use `personal-mcp` skill content more reliably, the instruction file should describe three things clearly:
|
|
||||||
|
|
||||||
1. when MCP-backed skill guidance is relevant
|
|
||||||
2. which retrieval path Copilot should prefer first
|
|
||||||
3. how much skill context it should load before answering
|
|
||||||
|
|
||||||
That matters because instructions can strongly steer discovery behavior, but they do not force VS Code to auto-attach MCP resources. A good instruction tells Copilot to prefer the canonical MCP content path while remaining accurate about the fallback path.
|
|
||||||
|
|
||||||
In this repository, the right policy is:
|
|
||||||
|
|
||||||
1. start from catalog discovery
|
|
||||||
2. prefer MCP resources when the current chat surface exposes resource attachment
|
|
||||||
3. fall back to catalog tools when resource attachment is unavailable
|
|
||||||
4. keep loaded skill context bounded
|
|
||||||
|
|
||||||
Suggested instruction text:
|
|
||||||
|
|
||||||
```md
|
|
||||||
When a task may match a documented implementation pattern from `personal-mcp`:
|
|
||||||
|
|
||||||
1. Start with catalog-first discovery.
|
|
||||||
2. Prefer MCP resources when the chat surface exposes resource attachment.
|
|
||||||
3. If MCP resource attachment is unavailable, use catalog tools instead.
|
|
||||||
4. Load only the most relevant skill document, or at most 2 skill documents.
|
|
||||||
5. Reconcile loaded skill guidance with the actual repository code before making changes.
|
|
||||||
|
|
||||||
Preferred resource order:
|
|
||||||
|
|
||||||
1. `resource://catalog/skills_index` or `resource://catalog/patterns`
|
|
||||||
2. `resource://skills/<skill-id>/document`
|
|
||||||
|
|
||||||
Preferred tool fallback order:
|
|
||||||
|
|
||||||
1. `search_patterns`
|
|
||||||
2. `get_pattern_by_id`
|
|
||||||
3. `get_skill_document_by_id`
|
|
||||||
|
|
||||||
If confidence is low after discovery, ask one clarifying question before loading more context.
|
|
||||||
```
|
|
||||||
|
|
||||||
This is intentionally guidance, not a guarantee. It gives Copilot a strong policy for when to use resources and when to fall back to discovery tools, while preserving the resource-first architecture.
|
|
||||||
|
|
||||||
## Failure Modes and Recovery
|
|
||||||
|
|
||||||
Common failure modes:
|
|
||||||
|
|
||||||
1. No relevant skill selected.
|
|
||||||
2. Too many skills selected (context bloat).
|
|
||||||
3. Stale assumptions from old metadata.
|
|
||||||
4. Slash command bypasses normal discovery and forces the wrong skill.
|
|
||||||
|
|
||||||
Recovery sequence:
|
|
||||||
|
|
||||||
1. re-run catalog lookup
|
|
||||||
2. narrow by tags and intent
|
|
||||||
3. fetch only top candidates
|
|
||||||
4. if still ambiguous, ask one clarifying question
|
|
||||||
5. use explicit `/` workflow for deterministic fallback
|
|
||||||
|
|
||||||
## Checklist
|
|
||||||
|
|
||||||
Use this checklist when configuring GitHub Copilot in VS Code against `personal-mcp`:
|
|
||||||
|
|
||||||
1. confirm server connectivity
|
|
||||||
2. verify catalog resources are readable
|
|
||||||
3. verify at least one `resource://skills/<id>/document` can be fetched
|
|
||||||
4. add one deterministic `/` command for fallback
|
|
||||||
5. add Copilot instruction: prefer catalog-first discovery, then targeted skill fetch
|
|
||||||
6. verify context size remains bounded
|
|
||||||
7. validate behavior in Ask/Edit/Agent-style workflows with at least one task each
|
|
||||||
|
|
||||||
Suggested instruction policy text:
|
|
||||||
|
|
||||||
1. Start with catalog-first discovery.
|
|
||||||
2. Prefer MCP resources when the chat surface exposes resource attachment.
|
|
||||||
3. Otherwise use catalog tools to search and load one or two likely skill documents.
|
|
||||||
4. If confidence is low, ask one clarifying question before loading more.
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
The intended model is:
|
|
||||||
|
|
||||||
1. skills are canonical MCP resources
|
|
||||||
2. `/` commands are explicit Copilot control shortcuts
|
|
||||||
3. guided skill loading should be catalog-driven, bounded, and explicit about whether it is using resources or tools
|
|
||||||
|
|
||||||
Using all three together gives predictable control when needed and low-friction assistance by default in VS Code.
|
|
||||||
+37
-5
@@ -1,18 +1,22 @@
|
|||||||
[project]
|
[project]
|
||||||
name = "prompts"
|
name = "prompts"
|
||||||
version = "0.1.0"
|
version = "2.0.0"
|
||||||
requires-python = ">=3.12"
|
requires-python = ">=3.12"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"fastapi>=0.115.0",
|
"fastapi>=0.133.0",
|
||||||
"fastmcp>=2.10.0",
|
"fastmcp==4.0.0b1",
|
||||||
"pydantic-settings>=2.0.0",
|
"pydantic-settings>=2",
|
||||||
"pyyaml>=6.0.2",
|
"pyyaml>=6.0.2",
|
||||||
|
"python-json-logger>=4",
|
||||||
"uvicorn[standard]>=0.34.0",
|
"uvicorn[standard]>=0.34.0",
|
||||||
"zensical>=0.0.45",
|
"zensical>=0.0.45",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
[tool.uv]
|
||||||
|
constraint-dependencies = ["fastmcp-slim==4.0.0b1"]
|
||||||
|
|
||||||
[project.scripts]
|
[project.scripts]
|
||||||
personal-mcp = "personal_mcp.main:main"
|
personal-mcp = "personal_mcp.__main__:main"
|
||||||
|
|
||||||
[build-system]
|
[build-system]
|
||||||
requires = ["hatchling"]
|
requires = ["hatchling"]
|
||||||
@@ -20,3 +24,31 @@ build-backend = "hatchling.build"
|
|||||||
|
|
||||||
[tool.hatch.build.targets.wheel]
|
[tool.hatch.build.targets.wheel]
|
||||||
packages = ["src/personal_mcp"]
|
packages = ["src/personal_mcp"]
|
||||||
|
|
||||||
|
[dependency-groups]
|
||||||
|
dev = [
|
||||||
|
"ipywidgets>=8.1.8",
|
||||||
|
"pre-commit>=4.6.0",
|
||||||
|
"ruff>=0.15.18",
|
||||||
|
"ty>=0.0.51",
|
||||||
|
]
|
||||||
|
test = [
|
||||||
|
"httpx2>=2.9.1",
|
||||||
|
"pytest>=9.1.1",
|
||||||
|
"pytest-asyncio>=1.4.0",
|
||||||
|
"pytest-cov>=7.1.0",
|
||||||
|
"pyyaml>=6.0.2",
|
||||||
|
]
|
||||||
|
|
||||||
|
[tool.pytest.ini_options]
|
||||||
|
addopts = ["--strict-markers"]
|
||||||
|
asyncio_mode = "auto"
|
||||||
|
asyncio_default_fixture_loop_scope = "function"
|
||||||
|
markers = [
|
||||||
|
"unit: fast deterministic tests with no external dependencies",
|
||||||
|
"integration: framework or component integration tests",
|
||||||
|
"smoke: thin critical-path checks",
|
||||||
|
]
|
||||||
|
|
||||||
|
[tool.ty.src]
|
||||||
|
include = ["src", "tests"]
|
||||||
|
|||||||
@@ -0,0 +1,63 @@
|
|||||||
|
line-length = 120
|
||||||
|
indent-width = 4
|
||||||
|
target-version = "py313"
|
||||||
|
|
||||||
|
exclude = [
|
||||||
|
".venv",
|
||||||
|
".devenv",
|
||||||
|
".git",
|
||||||
|
".vscode",
|
||||||
|
"build",
|
||||||
|
"site",
|
||||||
|
"__pycache__",
|
||||||
|
]
|
||||||
|
|
||||||
|
[lint]
|
||||||
|
preview = true
|
||||||
|
|
||||||
|
extend-select = [
|
||||||
|
"ARG", # https://docs.astral.sh/ruff/rules/#flake8-unused-arguments-arg
|
||||||
|
"B", # https://docs.astral.sh/ruff/rules/#flake8-bugbear-b
|
||||||
|
"C4", # https://docs.astral.sh/ruff/rules/#flake8-comprehensions-c4
|
||||||
|
"DOC102", # https://docs.astral.sh/ruff/rules/docstring-extraneous-parameter/
|
||||||
|
"DOC202", # https://docs.astral.sh/ruff/rules/docstring-extraneous-returns/
|
||||||
|
"DOC403", # https://docs.astral.sh/ruff/rules/docstring-extraneous-yields/
|
||||||
|
"DOC502", # https://docs.astral.sh/ruff/rules/docstring-extraneous-exception/
|
||||||
|
"E", "W", # https://docs.astral.sh/ruff/rules/#pycodestyle-e-w
|
||||||
|
"F", # https://docs.astral.sh/ruff/rules/#pyflakes-f
|
||||||
|
"FURB", # https://docs.astral.sh/ruff/rules/#refurb-furb
|
||||||
|
"I", # https://docs.astral.sh/ruff/rules/#isort-i
|
||||||
|
"N", # https://docs.astral.sh/ruff/rules/#pep8-naming-n
|
||||||
|
"PD", # https://docs.astral.sh/ruff/rules/#pandas-vet-pd
|
||||||
|
"PTH", # https://docs.astral.sh/ruff/rules/#flake8-use-pathlib-pth
|
||||||
|
"UP", # https://docs.astral.sh/ruff/rules/#pyupgrade-up
|
||||||
|
"SIM", # https://docs.astral.sh/ruff/rules/#flake8-simplify-sim
|
||||||
|
"PLR0202", # https://docs.astral.sh/ruff/rules/no-classmethod-decorator/
|
||||||
|
"PLR0203", # https://docs.astral.sh/ruff/rules/no-staticmethod-decorator/
|
||||||
|
"PLR0206", # https://docs.astral.sh/ruff/rules/property-with-parameters/
|
||||||
|
"PLR0915", # https://docs.astral.sh/ruff/rules/too-many-statements/
|
||||||
|
"PLR1702", # https://docs.astral.sh/ruff/rules/too-many-nested-blocks/
|
||||||
|
"TRY002",
|
||||||
|
]
|
||||||
|
extend-fixable = ["ALL"]
|
||||||
|
ignore = [
|
||||||
|
"UP046",
|
||||||
|
"UP047",
|
||||||
|
]
|
||||||
|
|
||||||
|
[lint.extend-per-file-ignores]
|
||||||
|
"*.ipynb" = [
|
||||||
|
"F401", # unused imports
|
||||||
|
"F841", # unused local variable
|
||||||
|
"F821", # undefined name in exploratory notebook cells,
|
||||||
|
"LOG015", # root logger calls
|
||||||
|
]
|
||||||
|
|
||||||
|
[lint.isort]
|
||||||
|
force-single-line = true
|
||||||
|
|
||||||
|
[format]
|
||||||
|
quote-style = "double"
|
||||||
|
indent-style = "space"
|
||||||
|
skip-magic-trailing-comma = false
|
||||||
|
line-ending = "auto"
|
||||||
Executable
+80
@@ -0,0 +1,80 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Create directory and symlink for Copilot skills."""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def add_skill(markdown_path: str) -> None:
|
||||||
|
"""Add a skill by creating symlink in ~/.copilot/skills/.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
markdown_path: Path to the markdown file (relative or absolute)
|
||||||
|
|
||||||
|
Raises:
|
||||||
|
FileNotFoundError: If markdown file doesn't exist
|
||||||
|
RuntimeError: If symlink creation fails
|
||||||
|
"""
|
||||||
|
md_file = Path(markdown_path)
|
||||||
|
|
||||||
|
if not md_file.exists():
|
||||||
|
raise FileNotFoundError(f"Markdown file not found: {markdown_path}")
|
||||||
|
|
||||||
|
# Get absolute path
|
||||||
|
abs_md_path = md_file.resolve()
|
||||||
|
|
||||||
|
# Extract skill name from filename (without .md extension)
|
||||||
|
skill_name = md_file.stem
|
||||||
|
|
||||||
|
# Create skill directory
|
||||||
|
skills_dir = Path.home() / ".copilot" / "skills" / skill_name
|
||||||
|
skills_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
# Create symlink
|
||||||
|
symlink_path = skills_dir / "SKILL.md"
|
||||||
|
|
||||||
|
# Remove existing symlink if it exists
|
||||||
|
if symlink_path.exists() or symlink_path.is_symlink():
|
||||||
|
symlink_path.unlink()
|
||||||
|
|
||||||
|
# Create the symlink using ln -s for compatibility
|
||||||
|
result = subprocess.run(
|
||||||
|
["ln", "-s", str(abs_md_path), str(symlink_path)],
|
||||||
|
capture_output=True,
|
||||||
|
check=False,
|
||||||
|
text=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
if result.returncode != 0:
|
||||||
|
raise RuntimeError(f"Failed to create symlink: {result.stderr}")
|
||||||
|
|
||||||
|
print(f"✓ Created skill link: {symlink_path} -> {abs_md_path}")
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
"""Main entry point."""
|
||||||
|
parser = argparse.ArgumentParser(description="Create directory and symlink for Copilot skills")
|
||||||
|
subparsers = parser.add_subparsers(dest="command", help="Command to run")
|
||||||
|
|
||||||
|
# Add subcommand
|
||||||
|
add_parser = subparsers.add_parser("add", help="Add a skill")
|
||||||
|
add_parser.add_argument("markdown", help="Path to markdown skill file")
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if not args.command:
|
||||||
|
parser.print_help()
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
if args.command == "add":
|
||||||
|
try:
|
||||||
|
add_skill(args.markdown)
|
||||||
|
except (FileNotFoundError, RuntimeError, OSError) as e:
|
||||||
|
print(f"✗ Error: {e}", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
import uvicorn
|
||||||
|
|
||||||
|
from .config import get_settings
|
||||||
|
|
||||||
|
|
||||||
|
def main(cli: bool = True) -> None:
|
||||||
|
"""Run the root MCP server."""
|
||||||
|
settings = get_settings(cli=cli)
|
||||||
|
uvicorn.run(
|
||||||
|
"personal_mcp.web.app:create_app",
|
||||||
|
factory=True,
|
||||||
|
host=settings.host,
|
||||||
|
port=settings.port,
|
||||||
|
reload=settings.reload,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
from personal_mcp.catalog.server import catalog_server
|
|
||||||
|
|
||||||
__all__ = ["catalog_server"]
|
|
||||||
@@ -1,172 +0,0 @@
|
|||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import yaml
|
|
||||||
from fastmcp import FastMCP
|
|
||||||
|
|
||||||
from personal_mcp.skills.document_loader import load_skill_document_from_metadata
|
|
||||||
|
|
||||||
catalog_server = FastMCP("catalog")
|
|
||||||
|
|
||||||
|
|
||||||
def _skills_dir() -> Path:
|
|
||||||
return Path(__file__).resolve().parents[1] / "skills"
|
|
||||||
|
|
||||||
|
|
||||||
def _load_skill_registry() -> dict[str, Any]:
|
|
||||||
registry: dict[str, Any] = {}
|
|
||||||
for metadata_path in sorted(_skills_dir().glob("*/metadata.yaml")):
|
|
||||||
with metadata_path.open("r", encoding="utf-8") as handle:
|
|
||||||
metadata = yaml.safe_load(handle) or {}
|
|
||||||
skill_key = metadata_path.parent.name
|
|
||||||
registry[skill_key] = {
|
|
||||||
"namespace": skill_key,
|
|
||||||
"metadata": metadata,
|
|
||||||
}
|
|
||||||
return registry
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_pattern(namespace: str, metadata: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
pattern_id = metadata.get("id", namespace)
|
|
||||||
capabilities = metadata.get("capabilities", [])
|
|
||||||
return {
|
|
||||||
"id": pattern_id,
|
|
||||||
"namespace": namespace,
|
|
||||||
"name": metadata.get("name", pattern_id),
|
|
||||||
"version": metadata.get("version", "0.1.0"),
|
|
||||||
"description": metadata.get("description", ""),
|
|
||||||
"tags": metadata.get("tags", []),
|
|
||||||
"depends_on": metadata.get("depends_on", []),
|
|
||||||
"capabilities": capabilities,
|
|
||||||
# Expose resources explicitly for clients that treat resources as the primary interface.
|
|
||||||
"resources": capabilities,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _normalized_patterns() -> list[dict[str, Any]]:
|
|
||||||
registry = _load_skill_registry()
|
|
||||||
return [
|
|
||||||
_normalize_pattern(namespace, entry["metadata"])
|
|
||||||
for namespace, entry in registry.items()
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def _matches_query(pattern: dict[str, Any], query: str) -> bool:
|
|
||||||
if not query:
|
|
||||||
return True
|
|
||||||
|
|
||||||
lowered = query.strip().lower()
|
|
||||||
if not lowered:
|
|
||||||
return True
|
|
||||||
|
|
||||||
query_terms = [term for term in lowered.replace("-", " ").split() if term]
|
|
||||||
if not query_terms:
|
|
||||||
return True
|
|
||||||
|
|
||||||
haystack = " ".join(
|
|
||||||
[
|
|
||||||
str(pattern.get("id", "")),
|
|
||||||
str(pattern.get("namespace", "")),
|
|
||||||
str(pattern.get("name", "")),
|
|
||||||
str(pattern.get("description", "")),
|
|
||||||
" ".join(str(tag) for tag in pattern.get("tags", [])),
|
|
||||||
]
|
|
||||||
).lower()
|
|
||||||
return all(term in haystack for term in query_terms)
|
|
||||||
|
|
||||||
|
|
||||||
def _matches_tags(pattern: dict[str, Any], tags: list[str] | None) -> bool:
|
|
||||||
if not tags:
|
|
||||||
return True
|
|
||||||
|
|
||||||
requested = [tag.strip().lower() for tag in tags if tag and tag.strip()]
|
|
||||||
if not requested:
|
|
||||||
return True
|
|
||||||
|
|
||||||
pattern_tags = {str(tag).lower() for tag in pattern.get("tags", [])}
|
|
||||||
return all(tag in pattern_tags for tag in requested)
|
|
||||||
|
|
||||||
|
|
||||||
@catalog_server.resource("resource://catalog/skills_index")
|
|
||||||
def skills_index() -> dict[str, Any]:
|
|
||||||
"""Return a compact discovery index for all available pattern modules."""
|
|
||||||
return {"patterns": _normalized_patterns()}
|
|
||||||
|
|
||||||
|
|
||||||
@catalog_server.resource("resource://catalog/skills_details")
|
|
||||||
def skills_details() -> dict[str, Any]:
|
|
||||||
"""Return full metadata for all mounted pattern modules."""
|
|
||||||
return {"patterns": _load_skill_registry()}
|
|
||||||
|
|
||||||
|
|
||||||
@catalog_server.resource("resource://catalog/patterns")
|
|
||||||
def patterns() -> dict[str, Any]:
|
|
||||||
"""Return normalized pattern records for resource-first clients."""
|
|
||||||
return {"patterns": _normalized_patterns()}
|
|
||||||
|
|
||||||
|
|
||||||
@catalog_server.resource("resource://catalog/patterns_by_id")
|
|
||||||
def patterns_by_id() -> dict[str, Any]:
|
|
||||||
"""Return normalized pattern records indexed by stable pattern id."""
|
|
||||||
indexed: dict[str, Any] = {}
|
|
||||||
for pattern in _normalized_patterns():
|
|
||||||
indexed[pattern["id"]] = pattern
|
|
||||||
return {"patterns_by_id": indexed}
|
|
||||||
|
|
||||||
|
|
||||||
@catalog_server.tool
|
|
||||||
def search_patterns(
|
|
||||||
query: str = "",
|
|
||||||
tags: list[str] | None = None,
|
|
||||||
skip: int = 0,
|
|
||||||
limit: int = 20,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Search normalized pattern metadata with optional tags and pagination."""
|
|
||||||
normalized_skip = max(skip, 0)
|
|
||||||
normalized_limit = min(max(limit, 1), 100)
|
|
||||||
|
|
||||||
matches = [
|
|
||||||
pattern
|
|
||||||
for pattern in _normalized_patterns()
|
|
||||||
if _matches_query(pattern, query) and _matches_tags(pattern, tags)
|
|
||||||
]
|
|
||||||
|
|
||||||
page = matches[normalized_skip : normalized_skip + normalized_limit]
|
|
||||||
return {
|
|
||||||
"patterns": page,
|
|
||||||
"total": len(matches),
|
|
||||||
"skip": normalized_skip,
|
|
||||||
"limit": normalized_limit,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@catalog_server.tool
|
|
||||||
def get_pattern_by_id(id: str) -> dict[str, Any]:
|
|
||||||
"""Return one normalized pattern by stable id."""
|
|
||||||
for pattern in _normalized_patterns():
|
|
||||||
if pattern["id"] == id:
|
|
||||||
return {"found": True, "pattern": pattern}
|
|
||||||
|
|
||||||
return {"found": False, "id": id}
|
|
||||||
|
|
||||||
|
|
||||||
@catalog_server.tool
|
|
||||||
def get_skill_document_by_id(skill_id: str) -> dict[str, Any]:
|
|
||||||
"""Return the canonical skill document payload for a stable skill id."""
|
|
||||||
registry = _load_skill_registry()
|
|
||||||
for namespace, entry in registry.items():
|
|
||||||
metadata = entry.get("metadata", {})
|
|
||||||
pattern_id = metadata.get("id", namespace)
|
|
||||||
if pattern_id != skill_id:
|
|
||||||
continue
|
|
||||||
|
|
||||||
return {
|
|
||||||
"found": True,
|
|
||||||
"document": load_skill_document_from_metadata(
|
|
||||||
skill_id=skill_id,
|
|
||||||
namespace=namespace,
|
|
||||||
metadata=metadata,
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
return {"found": False, "id": skill_id}
|
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
from functools import cache
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from pydantic import BaseModel
|
||||||
|
from pydantic import DirectoryPath
|
||||||
|
from pydantic import Field
|
||||||
|
from pydantic_settings import BaseSettings
|
||||||
|
from pydantic_settings import SettingsConfigDict
|
||||||
|
|
||||||
|
DEFAULT_ENV_FILE = Path(".env").resolve()
|
||||||
|
DEFAULT_SITE_DIR = Path("site").resolve()
|
||||||
|
|
||||||
|
|
||||||
|
class Mounts(BaseModel):
|
||||||
|
docs: str = "/docs"
|
||||||
|
mcp: str = "/mcp"
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
"""Runtime settings for the HTTP MCP and docs server."""
|
||||||
|
|
||||||
|
model_config = SettingsConfigDict(
|
||||||
|
env_file=DEFAULT_ENV_FILE,
|
||||||
|
env_prefix="PERSONAL_MCP_",
|
||||||
|
extra="ignore",
|
||||||
|
cli_implicit_flags=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
debug: bool = False
|
||||||
|
log_level: str = "info"
|
||||||
|
mounts: Mounts = Field(default_factory=Mounts)
|
||||||
|
site_dir: DirectoryPath = Field(default=DEFAULT_SITE_DIR)
|
||||||
|
host: str = "localhost"
|
||||||
|
port: int = 8080
|
||||||
|
reload: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
@cache
|
||||||
|
def get_settings(*, cli: bool = False, **overrides) -> Settings:
|
||||||
|
return Settings(**overrides, _cli_parse_args=cli) # pyright: ignore[reportCallIssue]
|
||||||
|
|
||||||
|
|
||||||
|
def refresh_settings(**overrides):
|
||||||
|
get_settings.cache_clear()
|
||||||
|
return get_settings(**overrides)
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/library
|
||||||
|
---
|
||||||
|
|
||||||
|
# Architecture
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
The application combines a FastMCP server with a pre-built Zensical documentation site. Markdown under `docs/` is the single authored content tree, while native FastMCP providers own skill and prompt discovery.
|
||||||
|
|
||||||
|
The runtime has four content paths:
|
||||||
|
|
||||||
|
1. `SkillsDirectoryProvider` publishes native `skill://` resources from packaged skill directories.
|
||||||
|
2. A custom prompt provider loads declarative prompt definitions from packaged Markdown.
|
||||||
|
3. The general docs registry publishes non-skill Markdown through `resource://docs/{path*}`.
|
||||||
|
4. FastAPI serves the pre-built `site/` directory.
|
||||||
|
|
||||||
|
There is no custom skill catalog, prompt catalog, or per-prompt Python module.
|
||||||
|
|
||||||
|
## Source Ownership
|
||||||
|
|
||||||
|
### Skills
|
||||||
|
|
||||||
|
Each skill owns one directory:
|
||||||
|
|
||||||
|
1. `docs/skills/<skill-id>/SKILL.md`
|
||||||
|
2. `docs/skills/<skill-id>/<supporting-path>`
|
||||||
|
|
||||||
|
`SkillsDirectoryProvider` publishes:
|
||||||
|
|
||||||
|
1. `skill://<name>/SKILL.md`
|
||||||
|
2. `skill://<name>/_manifest`
|
||||||
|
3. `skill://<name>/{path*}`
|
||||||
|
|
||||||
|
The provider parses standard skill frontmatter and generates the manifest. The general docs registry excludes `skills/**`, so only the native provider owns this namespace.
|
||||||
|
|
||||||
|
### Prompts
|
||||||
|
|
||||||
|
Each prompt has one source: `docs/prompts/<prompt-id>/PROMPT.md`. Its nested `prompt` frontmatter owns runtime metadata and argument declarations, while its body owns canonical prose.
|
||||||
|
|
||||||
|
The custom provider reads packaged Markdown with `importlib.resources`, validates metadata and exact placeholder-to-argument equality, and creates native FastMCP prompt objects. It rescans on each list and get request, so an editable deployment observes file additions, edits, and deletions without a restart.
|
||||||
|
|
||||||
|
FastMCP exposes prompts through native `prompts/list` and `prompts/get` operations.
|
||||||
|
|
||||||
|
### General Docs
|
||||||
|
|
||||||
|
The docs registry indexes packaged Markdown for `resource://docs/{path*}`. It rejects `skills/**` because skills are provider-owned. Prompt Markdown can remain visible as general documentation, but prompt invocation is owned by the native prompt provider.
|
||||||
|
|
||||||
|
## Runtime Composition
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
A[Packaged Skill Directories] --> B[SkillsDirectoryProvider]
|
||||||
|
C[Packaged Prompt Markdown] --> D[Markdown Prompt Provider]
|
||||||
|
F[General Markdown] --> G[Docs Registry]
|
||||||
|
B --> H[FastMCP Server]
|
||||||
|
D --> H
|
||||||
|
G --> H
|
||||||
|
H --> K[MCP Transport]
|
||||||
|
L[Zensical Site Output] --> M[FastAPI Static Mount]
|
||||||
|
K --> M
|
||||||
|
```
|
||||||
|
|
||||||
|
Server construction is lazy with respect to package import. Each application process creates its providers and docs snapshot when the server factory runs. Skills use startup discovery, while prompts are reloaded when a client lists or gets prompts.
|
||||||
|
|
||||||
|
## Packaging
|
||||||
|
|
||||||
|
The repository root `docs/` directory is the only authored Markdown source. `src/personal_mcp/docs` is a relative symlink used by source checkouts and editable installs. Hatchling follows it and stores regular files beneath `personal_mcp/docs/` in the wheel.
|
||||||
|
|
||||||
|
Runtime reads are package-relative:
|
||||||
|
|
||||||
|
1. Prompt content and general docs use `importlib.resources` and `Traversable` APIs.
|
||||||
|
2. `SkillsDirectoryProvider` receives the packaged `personal_mcp/docs/skills` filesystem path.
|
||||||
|
3. No runtime content lookup depends on the current working directory.
|
||||||
|
|
||||||
|
## Public Contracts
|
||||||
|
|
||||||
|
The machine-facing surfaces are:
|
||||||
|
|
||||||
|
1. Native skill resources under `skill://<name>/...`.
|
||||||
|
2. Native MCP prompt list and get operations.
|
||||||
|
3. `resource://docs/{path*}` for general Markdown.
|
||||||
|
|
||||||
|
Canonical contracts are documented in:
|
||||||
|
|
||||||
|
1. [Prompt Contract](./contracts/prompt.md)
|
||||||
|
2. [Skill Contract](./contracts/skill_contract.md)
|
||||||
|
3. [Frontmatter Contract](./contracts/frontmatter.md)
|
||||||
|
4. [URI Contract](./contracts/uris.md)
|
||||||
|
|
||||||
|
Only these canonical provider and protocol surfaces are registered.
|
||||||
|
|
||||||
|
## Static Documentation
|
||||||
|
|
||||||
|
Zensical builds `docs/` into `site/` before deployment. FastAPI mounts that immutable output in the same process that hosts FastMCP. Generated `site/` files are deployment assets and are never an authored source.
|
||||||
|
|
||||||
|
## Validation
|
||||||
|
|
||||||
|
Changes are accepted only after:
|
||||||
|
|
||||||
|
1. focused provider and protocol tests
|
||||||
|
2. Ruff and ty checks
|
||||||
|
3. a Zensical build
|
||||||
|
4. the full pytest suite
|
||||||
|
5. an installed-wheel smoke test when packaging or provider paths change
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/pencil
|
||||||
|
---
|
||||||
|
|
||||||
|
# Authoring Guide
|
||||||
|
|
||||||
|
This page defines the practical workflow for maintaining skills, prompts, and project documentation while keeping root `docs/` as the only authored source.
|
||||||
|
|
||||||
|
Primary references:
|
||||||
|
|
||||||
|
1. [Skill Contract](./contracts/skill_contract.md)
|
||||||
|
2. [Prompt Contract](./contracts/prompt.md)
|
||||||
|
3. [Frontmatter Contract](./contracts/frontmatter.md)
|
||||||
|
4. [URI Contract](./contracts/uris.md)
|
||||||
|
5. [Zensical documentation skill](./skills/zensical-docs/SKILL.md)
|
||||||
|
|
||||||
|
## Source Tree Ownership
|
||||||
|
|
||||||
|
Edit content only under root `docs/`. The `src/personal_mcp/docs` path is a relative symlink for editable installs; do not author through a copied package tree.
|
||||||
|
|
||||||
|
Hatchling's normal package traversal follows `src/personal_mcp/docs` during wheel builds and archives the linked targets as regular files under `personal_mcp/docs/`. Do not add a `force-include` entry for root `docs/`; it duplicates those wheel paths. The installed package therefore gives `SkillsDirectoryProvider` a regular filesystem directory while Zensical builds the human site directly from root `docs/`.
|
||||||
|
|
||||||
|
Generated `site/` content is a build artifact and must not be edited by hand.
|
||||||
|
|
||||||
|
## Content Layout
|
||||||
|
|
||||||
|
```text
|
||||||
|
docs/
|
||||||
|
*.md
|
||||||
|
contracts/
|
||||||
|
prompts/<prompt-id>/
|
||||||
|
PROMPT.md
|
||||||
|
references/
|
||||||
|
skills/<skill-name>/
|
||||||
|
SKILL.md
|
||||||
|
references/
|
||||||
|
```
|
||||||
|
|
||||||
|
Keep skill and prompt files inside their owning directories. Relative links may cross sections, but content ownership should remain clear.
|
||||||
|
|
||||||
|
## Skill Authoring
|
||||||
|
|
||||||
|
A skill is discovered when a direct child of `docs/skills/` contains `SKILL.md`.
|
||||||
|
|
||||||
|
Required frontmatter:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
name: <skill-name>
|
||||||
|
description: <what the skill does and when to use it>
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
1. Use lowercase kebab-case for the directory and `name`.
|
||||||
|
2. Keep `name` exactly equal to the directory name.
|
||||||
|
3. Write a specific description because clients use it for discovery.
|
||||||
|
4. Do not add `x-personal-mcp`, versions, tags, capabilities, or reference mappings.
|
||||||
|
5. Put supporting material anywhere beneath the skill directory, normally under `references/`.
|
||||||
|
6. Link supporting files from `SKILL.md` so humans and agents understand when to load them.
|
||||||
|
|
||||||
|
FastMCP recursively scans every skill file and generates `skill://<name>/_manifest`. Supporting-resource identity is the real relative path, not a synthetic reference id.
|
||||||
|
|
||||||
|
Recommended sequence:
|
||||||
|
|
||||||
|
1. Draft or revise `SKILL.md` routing guidance.
|
||||||
|
2. Add focused supporting files.
|
||||||
|
3. Verify relative links.
|
||||||
|
4. Run the provider tests and docs build.
|
||||||
|
5. Restart running servers because production uses `reload=False`.
|
||||||
|
|
||||||
|
## Prompt Authoring
|
||||||
|
|
||||||
|
A prompt is one self-describing `docs/prompts/<prompt-id>/PROMPT.md` file:
|
||||||
|
|
||||||
|
1. Create a lowercase kebab-case directory beneath `docs/prompts/`.
|
||||||
|
2. Add a nested `prompt` frontmatter mapping with version, description, tags, and ordered arguments.
|
||||||
|
3. Give every argument a description and explicit required flag.
|
||||||
|
4. Add `choices` only when a string argument accepts a fixed set of values.
|
||||||
|
5. Use each argument exactly once or more as a `{{argument_name}}` placeholder in the body.
|
||||||
|
6. Do not add a Python component, name field, metadata sidecar, or central catalog entry.
|
||||||
|
|
||||||
|
The custom provider rescans prompt documents during every native list and get request. Changes in an editable checkout are therefore visible on the next request without a process restart. Invalid metadata or placeholder drift fails that request with a configuration error.
|
||||||
|
|
||||||
|
## Frontmatter Safety
|
||||||
|
|
||||||
|
1. Quote scalar values containing `:`.
|
||||||
|
2. Quote values with reserved YAML characters such as `#`, `{}`, `[]`, or leading `*`.
|
||||||
|
3. Use block scalars for punctuation-heavy multiline text.
|
||||||
|
4. Keep fields within the applicable skill or documentation contract.
|
||||||
|
|
||||||
|
## Writing Quality
|
||||||
|
|
||||||
|
1. Prefer focused sections and descriptive headings.
|
||||||
|
2. Link feature-level claims to authoritative sources.
|
||||||
|
3. Use relative links for internal pages.
|
||||||
|
4. Keep code examples minimal and actionable.
|
||||||
|
5. Avoid bare URLs in prose.
|
||||||
|
6. Load only supporting material relevant to the immediate task.
|
||||||
|
|
||||||
|
## Copilot Routing
|
||||||
|
|
||||||
|
Active instructions should point directly to native main resources:
|
||||||
|
|
||||||
|
1. `skill://zensical-docs/SKILL.md`
|
||||||
|
2. `skill://pytesting/SKILL.md`
|
||||||
|
3. `skill://vscode-configuration/SKILL.md`
|
||||||
|
|
||||||
|
When deeper guidance is needed, read the selected skill's `_manifest` and fetch supporting files by their listed path.
|
||||||
|
|
||||||
|
## Validation Checklist
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run pytest tests/skills/test_provider.py tests/web/test_mcp_skills.py -q
|
||||||
|
uv run zensical build
|
||||||
|
uv run ruff check .
|
||||||
|
uv run ty check
|
||||||
|
uv run pytest
|
||||||
|
```
|
||||||
|
|
||||||
|
For packaging changes, also build and inspect an installed wheel so provider path resolution is verified outside the editable checkout.
|
||||||
|
|
||||||
|
## Navigation
|
||||||
|
|
||||||
|
When adding or moving pages:
|
||||||
|
|
||||||
|
1. update `zensical.toml`
|
||||||
|
2. keep top-level page icons in frontmatter
|
||||||
|
3. rebuild the site
|
||||||
|
4. verify internal links and navigation labels
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/braces
|
||||||
|
---
|
||||||
|
|
||||||
|
# Frontmatter Contract
|
||||||
|
|
||||||
|
This page defines frontmatter ownership for native skills and prompt documentation.
|
||||||
|
|
||||||
|
## Skill Frontmatter
|
||||||
|
|
||||||
|
Skills use the standard Agent Skills fields consumed by the [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
name: <skill-id>
|
||||||
|
description: <what the skill does and when to use it>
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
1. `name` and `description` are required.
|
||||||
|
2. `name` must equal the skill directory name.
|
||||||
|
3. The repository uses lowercase kebab-case directory names.
|
||||||
|
4. Skill frontmatter contains no `x-personal-mcp` catalog metadata.
|
||||||
|
5. Supporting files require no frontmatter manifest. The provider discovers files recursively and generates `_manifest` with relative paths, byte sizes, and SHA256 hashes.
|
||||||
|
|
||||||
|
The provider uses the directory name as the URI identity and the frontmatter `description` as the main resource description. Repository tests enforce directory/name parity and reject extra skill frontmatter fields.
|
||||||
|
|
||||||
|
## Prompt Documentation Frontmatter
|
||||||
|
|
||||||
|
Each prompt stores runtime metadata in a nested `prompt` mapping beside fields consumed by the static documentation site. The runtime mapping uses this shape:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
icon: lucide/messages-square
|
||||||
|
prompt:
|
||||||
|
version: "1.0.0"
|
||||||
|
description: Describe when to use the prompt.
|
||||||
|
tags: [example, prompts]
|
||||||
|
arguments: {topic: {description: "Topic to process.", required: true, choices: [first, second]}, notes: {description: "Optional constraints.", required: false}}
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
Prompt rules:
|
||||||
|
|
||||||
|
1. `version`, `description`, `tags`, and `arguments` are required; unknown fields inside `prompt` or an argument are rejected.
|
||||||
|
2. The directory name supplies the prompt id. Do not add a duplicate `name` field.
|
||||||
|
3. Argument names must be valid identifiers and preserve their authored mapping order.
|
||||||
|
4. Every argument requires a non-empty `description` and explicit `required` boolean.
|
||||||
|
5. Optional `choices` must be a non-empty list of unique, non-empty strings.
|
||||||
|
6. Markdown placeholders must exactly match the declared argument names.
|
||||||
|
7. Top-level fields such as `icon` remain owned by the documentation site and are not runtime prompt metadata.
|
||||||
|
|
||||||
|
See the MCP [prompts concept documentation](https://modelcontextprotocol.io/docs/learn/server-concepts#prompts) and [schema reference](https://modelcontextprotocol.io/specification/latest/schema) for the protocol-level prompt shape.
|
||||||
|
|
||||||
|
## Validation Timing
|
||||||
|
|
||||||
|
Skill validation is file- and provider-oriented:
|
||||||
|
|
||||||
|
1. `SkillsDirectoryProvider` discovers each directory containing `SKILL.md`.
|
||||||
|
2. FastMCP parses the description and scans all files when the provider is created.
|
||||||
|
3. Repository tests enforce the stricter standard-only frontmatter and directory/name rules.
|
||||||
|
|
||||||
|
Prompt validation is provider- and renderer-oriented. Every list or get request reloads and validates the authored files. A malformed definition fails the request instead of publishing a partial prompt set.
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
|
||||||
|
1. Skills remain directly portable to tools that understand standard Agent Skills directories.
|
||||||
|
2. Native skill discovery has no parallel catalog metadata source.
|
||||||
|
3. Prompts use FastMCP's native component metadata and protocol surface without a parallel catalog or Python component file.
|
||||||
|
4. All authored content remains under `docs/`.
|
||||||
@@ -0,0 +1,88 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/file-check-2
|
||||||
|
---
|
||||||
|
|
||||||
|
# Contracts
|
||||||
|
|
||||||
|
This section groups the core data and contract documents for the repository.
|
||||||
|
|
||||||
|
## Pages
|
||||||
|
|
||||||
|
1. [Prompt Contract](./prompt.md)
|
||||||
|
2. [Skill Contract](./skill_contract.md)
|
||||||
|
3. [Frontmatter Contract](./frontmatter.md)
|
||||||
|
4. [URI Contract](./uris.md)
|
||||||
|
|
||||||
|
Use these pages as the normative source for authored content layout, frontmatter schema, and canonical MCP URI semantics.
|
||||||
|
|
||||||
|
## Content Contract
|
||||||
|
|
||||||
|
This page defines the authored content contract for the docs-first MCP architecture.
|
||||||
|
|
||||||
|
## Canonical Source Of Truth
|
||||||
|
|
||||||
|
1. All authored Markdown lives under `docs/`.
|
||||||
|
2. MCP resources and static docs are two distribution surfaces of the same authored files.
|
||||||
|
3. No parallel authored markdown is allowed in `src/` or other package-only paths.
|
||||||
|
|
||||||
|
## Canonical Content Shape
|
||||||
|
|
||||||
|
Authored content is organized under `docs/`:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
---
|
||||||
|
config:
|
||||||
|
treeView:
|
||||||
|
rowIndent: 20
|
||||||
|
lineThickness: 2
|
||||||
|
themeVariables:
|
||||||
|
treeView:
|
||||||
|
labelColor: '#FFFFFF'
|
||||||
|
lineColor: '#FFFFFF'
|
||||||
|
---
|
||||||
|
treeView-beta
|
||||||
|
"docs/"
|
||||||
|
"*.md (top-level docs pages)"
|
||||||
|
"contracts/"
|
||||||
|
"prompt.md"
|
||||||
|
"skill_contract.md"
|
||||||
|
"frontmatter.md"
|
||||||
|
"uris.md"
|
||||||
|
"prompts/"
|
||||||
|
"<prompt-id>/"
|
||||||
|
"PROMPT.md"
|
||||||
|
"skills/"
|
||||||
|
"<skill-id>/"
|
||||||
|
"SKILL.md"
|
||||||
|
"references/..."
|
||||||
|
```
|
||||||
|
|
||||||
|
## File Placement And Ownership Boundaries
|
||||||
|
|
||||||
|
1. Top-level project docs stay in `docs/*.md`.
|
||||||
|
2. Skill docs stay in `docs/skills/<skill-id>/...`.
|
||||||
|
3. Prompt docs stay in `docs/prompts/<prompt-id>/...`.
|
||||||
|
4. A skill or prompt may link across sections, but must not store content in another artifact's directory.
|
||||||
|
5. Server and runtime code may index and serve docs, but must not be the source of authored markdown.
|
||||||
|
|
||||||
|
## Delegated Contracts
|
||||||
|
|
||||||
|
1. Skill-specific directory, metadata, and id rules are defined in [Skill Contract](./skill_contract.md).
|
||||||
|
2. Prompt-specific directory, metadata, and id rules are defined in [Prompt Contract](./prompt.md).
|
||||||
|
|
||||||
|
## Invariants
|
||||||
|
|
||||||
|
This contract guarantees:
|
||||||
|
|
||||||
|
1. One authored source tree in `docs/` for both website and MCP.
|
||||||
|
2. Skill and prompt artifacts remain path-stable within their own sections.
|
||||||
|
3. Cross-surface publishing remains deterministic because authored content paths are canonical.
|
||||||
|
|
||||||
|
## Non-Goals
|
||||||
|
|
||||||
|
This contract does not define:
|
||||||
|
|
||||||
|
1. URI versioning policy details.
|
||||||
|
2. The full frontmatter schema.
|
||||||
|
3. Detailed skill rules (see [Skill Contract](./skill_contract.md)).
|
||||||
|
4. Detailed prompt rules (see [Prompt Contract](./prompt.md)).
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/messages-square
|
||||||
|
---
|
||||||
|
|
||||||
|
# Prompt Contract
|
||||||
|
|
||||||
|
This page defines the canonical contract for declarative prompts published through a custom [FastMCP provider](https://gofastmcp.com/servers/providers/custom).
|
||||||
|
|
||||||
|
## Canonical Prompt Shape
|
||||||
|
|
||||||
|
Each prompt is one self-describing Markdown document:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
---
|
||||||
|
config:
|
||||||
|
treeView:
|
||||||
|
rowIndent: 20
|
||||||
|
lineThickness: 2
|
||||||
|
themeVariables:
|
||||||
|
treeView:
|
||||||
|
labelColor: '#FFFFFF'
|
||||||
|
lineColor: '#FFFFFF'
|
||||||
|
---
|
||||||
|
treeView-beta
|
||||||
|
"docs/prompts/"
|
||||||
|
"<prompt-id>/"
|
||||||
|
"PROMPT.md"
|
||||||
|
"src/personal_mcp/prompts/"
|
||||||
|
"content.py"
|
||||||
|
"models.py"
|
||||||
|
"provider.py"
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
1. The parent directory name defines the public prompt id.
|
||||||
|
2. The nested `prompt` frontmatter block defines version, description, tags, and arguments.
|
||||||
|
3. Argument declarations define names, descriptions, requiredness, and optional string choices.
|
||||||
|
4. The Markdown body owns the rendered prompt prose and uses `{{argument_name}}` placeholders.
|
||||||
|
5. Declared arguments and body placeholders must match exactly.
|
||||||
|
6. No Python file is added when authoring a prompt.
|
||||||
|
|
||||||
|
## Ownership Boundary
|
||||||
|
|
||||||
|
1. Each `PROMPT.md` owns both its runtime metadata and prose.
|
||||||
|
2. Python owns only generic parsing, validation, rendering, and provider behavior.
|
||||||
|
3. There is no central prompt catalog, generated signature, or metadata sidecar.
|
||||||
|
4. The provider scans direct children of packaged `docs/prompts/` on each list or get request.
|
||||||
|
5. Additions, edits, and deletions become visible on the next request without restarting the server.
|
||||||
|
6. Reload is pull-based; the provider does not watch files or emit proactive change notifications.
|
||||||
|
|
||||||
|
## Prompt Id Contract
|
||||||
|
|
||||||
|
`prompt-id` is the public identifier and should satisfy all rules below:
|
||||||
|
|
||||||
|
1. Format: lowercase kebab-case only.
|
||||||
|
2. Character set: `a-z`, `0-9`, and `-`.
|
||||||
|
3. Must start with a letter.
|
||||||
|
4. No underscores, spaces, dots, or uppercase characters.
|
||||||
|
5. Directory name should equal `prompt-id` in each committed revision.
|
||||||
|
6. The provider derives the prompt name from the directory; frontmatter must not duplicate it.
|
||||||
|
7. Treat `prompt-id` as immutable after release; a rename is a breaking replacement.
|
||||||
|
|
||||||
|
Valid examples:
|
||||||
|
|
||||||
|
1. `pytest-fill-scaffold`
|
||||||
|
2. `review-pr-comments`
|
||||||
|
3. `scaffold-fastapi-service`
|
||||||
|
|
||||||
|
Invalid examples:
|
||||||
|
|
||||||
|
1. `fill_pytest_scaffold`
|
||||||
|
2. `Prompt-Template`
|
||||||
|
3. `docs.prompt`
|
||||||
|
|
||||||
|
## Rendering Contract
|
||||||
|
|
||||||
|
1. The loader requires one leading YAML frontmatter block and validates its nested `prompt` mapping strictly.
|
||||||
|
2. All MCP arguments are strings; `choices` optionally restricts accepted values.
|
||||||
|
3. Missing required arguments, unknown arguments, and invalid choices fail before rendering.
|
||||||
|
4. An omitted optional value renders as `Not provided`.
|
||||||
|
5. Unknown prompt ids, malformed metadata, and mismatched placeholders fail immediately.
|
||||||
|
6. Prompt content is read through [importlib resources](https://docs.python.org/3/library/importlib.resources.html) and does not depend on the working directory.
|
||||||
|
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/brain-circuit
|
||||||
|
---
|
||||||
|
|
||||||
|
# Skill Contract
|
||||||
|
|
||||||
|
This page defines the canonical contract for skills in the docs-first MCP architecture.
|
||||||
|
|
||||||
|
## Canonical Skill Shape
|
||||||
|
|
||||||
|
Each skill is one directory under `docs/skills/`:
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
---
|
||||||
|
config:
|
||||||
|
treeView:
|
||||||
|
rowIndent: 20
|
||||||
|
lineThickness: 2
|
||||||
|
themeVariables:
|
||||||
|
treeView:
|
||||||
|
labelColor: '#FFFFFF'
|
||||||
|
lineColor: '#FFFFFF'
|
||||||
|
---
|
||||||
|
treeView-beta
|
||||||
|
"docs/"
|
||||||
|
"... (other docs)"
|
||||||
|
"skills/"
|
||||||
|
"<skill-id>/"
|
||||||
|
"SKILL.md"
|
||||||
|
"references/"
|
||||||
|
"... (one or more markdown files, optional nested folders)"
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
|
||||||
|
1. `SKILL.md` is required for every skill.
|
||||||
|
2. `references/` is the only place for skill-specific supporting docs.
|
||||||
|
3. Nested folders inside `references/` are allowed so a skill can reorganize internals without changing global architecture.
|
||||||
|
4. Skill directories are independent ownership boundaries; no cross-skill file writes.
|
||||||
|
|
||||||
|
## Metadata Location Constraint
|
||||||
|
|
||||||
|
1. `SKILL.md` frontmatter contains only standard `name` and `description` fields.
|
||||||
|
2. No `metadata.yaml` sidecar or repository-specific skill metadata block exists.
|
||||||
|
3. The provider discovers supporting files recursively; their real relative paths are published in the generated `_manifest`.
|
||||||
|
|
||||||
|
## Skill Id Contract
|
||||||
|
|
||||||
|
`skill-id` is the public identifier and should satisfy all rules below:
|
||||||
|
|
||||||
|
1. Format: lowercase kebab-case only.
|
||||||
|
2. Character set: `a-z`, `0-9`, and `-`.
|
||||||
|
3. Must start with a letter.
|
||||||
|
4. No underscores, spaces, dots, or uppercase characters.
|
||||||
|
5. Directory name equals `skill-id` in each committed revision.
|
||||||
|
6. Frontmatter `name` equals the directory name.
|
||||||
|
7. Treat `skill-id` as immutable after release; any rename is a breaking replacement and clients must move to the new id.
|
||||||
|
|
||||||
|
Valid examples:
|
||||||
|
|
||||||
|
1. `fastapi-uv-docker`
|
||||||
|
2. `zensical-docs`
|
||||||
|
3. `pytesting`
|
||||||
|
|
||||||
|
Invalid examples:
|
||||||
|
|
||||||
|
1. `fastapi_uv_docker`
|
||||||
|
2. `Zensical-Docs`
|
||||||
|
3. `docs.zensical`
|
||||||
|
|
||||||
|
## Provider Publication
|
||||||
|
|
||||||
|
[`SkillsDirectoryProvider`](https://gofastmcp.com/servers/providers/skills) scans `docs/skills/` with `supporting_files="template"` and publishes:
|
||||||
|
|
||||||
|
1. `skill://<skill-id>/SKILL.md`
|
||||||
|
2. `skill://<skill-id>/_manifest`
|
||||||
|
3. `skill://<skill-id>/{path*}` for supporting files
|
||||||
|
|
||||||
|
Only the main file and manifest appear in `resources/list`. Clients inspect the manifest before reading supporting paths.
|
||||||
|
|
||||||
|
## Direct Documentation Inclusion
|
||||||
|
|
||||||
|
1. For direct API documentation, use mkdocstrings directives rather than pasting large code blocks.
|
||||||
|
2. Keep manually-authored code examples short and task-focused; large implementation excerpts are out of scope for this contract.
|
||||||
|
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/link
|
||||||
|
---
|
||||||
|
|
||||||
|
# URI Contract
|
||||||
|
|
||||||
|
This page defines the public resource URI contract for native skills and general authored documentation.
|
||||||
|
|
||||||
|
## Native Skill URIs
|
||||||
|
|
||||||
|
The [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills) publishes each skill through the `skill://` scheme:
|
||||||
|
|
||||||
|
1. `skill://<skill-name>/SKILL.md`
|
||||||
|
2. `skill://<skill-name>/_manifest`
|
||||||
|
3. `skill://<skill-name>/<supporting-path>`
|
||||||
|
|
||||||
|
The first two are concrete resources returned by `resources/list`. Supporting files use a per-skill wildcard resource template when the provider is configured with `supporting_files="template"`:
|
||||||
|
|
||||||
|
```text
|
||||||
|
skill://<skill-name>/{path*}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Main File
|
||||||
|
|
||||||
|
`skill://<skill-name>/SKILL.md` returns the canonical authored skill document. The skill directory name supplies `<skill-name>`, and the resource description comes from `SKILL.md` frontmatter.
|
||||||
|
|
||||||
|
### Manifest
|
||||||
|
|
||||||
|
`skill://<skill-name>/_manifest` returns JSON containing the skill name and every file beneath its directory. Each file entry includes:
|
||||||
|
|
||||||
|
1. relative POSIX path
|
||||||
|
2. byte size
|
||||||
|
3. SHA256 hash
|
||||||
|
|
||||||
|
Clients read the manifest before requesting supporting files. FastMCP client utilities such as `list_skills()` and `get_skill_manifest()` understand this contract directly.
|
||||||
|
|
||||||
|
### Supporting Files
|
||||||
|
|
||||||
|
Supporting files retain their real skill-relative paths. For example:
|
||||||
|
|
||||||
|
```text
|
||||||
|
skill://pytesting/references/pytest-docs.md
|
||||||
|
```
|
||||||
|
|
||||||
|
FastMCP confines reads to the selected skill directory. Absolute paths, traversal outside the directory, missing files, directories, and symlinks that resolve outside the skill root are rejected.
|
||||||
|
|
||||||
|
## General Docs URI
|
||||||
|
|
||||||
|
General authored documentation is exposed through `resource://docs/{path*}`. The wildcard accepts normalized relative POSIX Markdown paths beneath `docs/`, excludes the provider-owned `skills/` subtree, and rejects absolute paths, traversal segments, backslashes, and non-Markdown targets.
|
||||||
|
|
||||||
|
Prompts are MCP prompt components rather than resources. Clients discover them with the protocol `prompts/list` operation and render them with `prompts/get`.
|
||||||
|
|
||||||
|
## Discovery Order
|
||||||
|
|
||||||
|
For skills:
|
||||||
|
|
||||||
|
1. list resources or call FastMCP `list_skills()`
|
||||||
|
2. select a skill by name and description
|
||||||
|
3. read `skill://<skill-name>/SKILL.md`
|
||||||
|
4. read `_manifest` when supporting material may be needed
|
||||||
|
5. fetch only the supporting paths relevant to the task
|
||||||
|
|
||||||
|
Tool-only agents may call `search_skills` instead of retrieving the complete resource list. Search results contain only provider-derived names, descriptions, and canonical main-resource URIs; skill content remains available exclusively through the native resource contract.
|
||||||
|
|
||||||
|
For prompts, use the native MCP prompt APIs or their generic tool projection.
|
||||||
|
|
||||||
|
## Stability Policy
|
||||||
|
|
||||||
|
The provider and protocol surfaces documented here are the complete public contract. Contract changes replace the affected surface directly.
|
||||||
|
|
||||||
|
Skill renames are breaking because the directory name is part of every native skill URI. Supporting-file renames change the corresponding manifest path and URI.
|
||||||
|
|
||||||
|
## Sources
|
||||||
|
|
||||||
|
1. [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills)
|
||||||
|
2. [MCP resources](https://modelcontextprotocol.io/specification/latest/server/resources)
|
||||||
|
3. [RFC 3986 URI syntax](https://www.rfc-editor.org/rfc/rfc3986)
|
||||||
|
4. [RFC 6570 URI templates](https://www.rfc-editor.org/rfc/rfc6570)
|
||||||
|
5. [FastMCP prompts](https://gofastmcp.com/servers/prompts)
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/bot
|
||||||
|
---
|
||||||
|
|
||||||
|
# Copilot MCP Mechanics
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
This page explains how GitHub Copilot in VS Code consumes native skill resources and prompts from `personal-mcp`.
|
||||||
|
|
||||||
|
## Capability Lanes
|
||||||
|
|
||||||
|
Copilot interacts with MCP servers through independently exposed lanes:
|
||||||
|
|
||||||
|
1. tools invoked during execution
|
||||||
|
2. resources attached as read-only context
|
||||||
|
3. server-provided prompts
|
||||||
|
|
||||||
|
This server publishes skills as native `skill://` resources and prompts as native MCP prompt objects. It also exposes `search_skills`, `list_resources`, and `read_resource` tools for agents whose tool catalog does not include direct MCP resource operations.
|
||||||
|
|
||||||
|
## VS Code Feature Coverage
|
||||||
|
|
||||||
|
The server uses every FastMCP feature that applies to its read-only guidance workload:
|
||||||
|
|
||||||
|
| Feature | Usage |
|
||||||
|
| --- | --- |
|
||||||
|
| Server identity | The initialize response includes a stable name, usage instructions, and a self-contained icon for VS Code's MCP server UI. |
|
||||||
|
| Tools | Compatibility tools have display titles, structured output schemas, and read-only, idempotent, closed-world annotations. FastMCP's default schema dereferencing remains enabled for clients such as VS Code that require flat schemas. |
|
||||||
|
| Resources | Documentation and skills use native resources and wildcard resource templates with explicit Markdown MIME types. |
|
||||||
|
| Prompts | Declarative workflows use native prompt objects with descriptions, display titles, typed arguments, and slash-command access. |
|
||||||
|
| Argument completion | Prompt arguments with authored `choices` are returned through `completion/complete` as the user types. |
|
||||||
|
|
||||||
|
[FastMCP server identity](https://gofastmcp.com/servers/server), [component icons](https://gofastmcp.com/servers/icons), [tool metadata](https://gofastmcp.com/servers/tools), and [argument completion](https://gofastmcp.com/servers/completions) define the implementation details. [VS Code's MCP documentation](https://code.visualstudio.com/docs/agent-customization/mcp-servers) describes how tools, resources, prompts, and MCP Apps appear in the client.
|
||||||
|
|
||||||
|
The following capabilities are conditional rather than useful by default:
|
||||||
|
|
||||||
|
1. MCP Apps require an interactive tool result such as a form or visualization; this server returns guidance and structured resource data only.
|
||||||
|
2. Sampling is appropriate only when server-side work must ask VS Code to run an LLM. The current server retrieves authored content and does not generate it.
|
||||||
|
3. Elicitation is appropriate only when a running operation needs additional user input. Prompt arguments already collect all required input before execution.
|
||||||
|
4. Progress, client logging, and background tasks require long-running operations. Current reads and prompt rendering are bounded local operations.
|
||||||
|
5. Client roots matter only when server behavior depends on client filesystem roots. This server reads packaged content and never traverses a client workspace.
|
||||||
|
6. `website_url` requires a canonical public deployment URL. None is configured, so the server does not advertise a guessed address.
|
||||||
|
|
||||||
|
Add one of these capabilities when a concrete workflow needs it, then cover its negotiated capability and protocol response in the HTTP MCP smoke tests. See the [FastMCP Apps overview](https://gofastmcp.com/apps/overview), [sampling](https://gofastmcp.com/servers/sampling), [elicitation](https://gofastmcp.com/servers/elicitation), [progress reporting](https://gofastmcp.com/servers/progress), and [MCP context](https://gofastmcp.com/servers/context) for the activation criteria.
|
||||||
|
|
||||||
|
## Native Skill Resources
|
||||||
|
|
||||||
|
For every skill, Copilot can discover:
|
||||||
|
|
||||||
|
1. `skill://<name>/SKILL.md`
|
||||||
|
2. `skill://<name>/_manifest`
|
||||||
|
3. `skill://<name>/{path*}` supporting-file template
|
||||||
|
|
||||||
|
The main resource description comes from `SKILL.md`. The manifest discloses supporting paths, sizes, and SHA256 hashes. Native resources remain the only skill content and discovery contract; the tools search or delegate to that same resource surface rather than maintaining a parallel catalog.
|
||||||
|
|
||||||
|
## Resource Picker Availability
|
||||||
|
|
||||||
|
`MCP Resources...` in Add Context requires both:
|
||||||
|
|
||||||
|
1. a connected server advertising resource capability
|
||||||
|
2. a chat surface that exposes MCP resource attachment
|
||||||
|
|
||||||
|
A successful `resources/list` response does not guarantee the picker appears in every session type. Use `MCP: Browse Resources` to distinguish server availability from chat UI availability.
|
||||||
|
|
||||||
|
## Recommended Workflow
|
||||||
|
|
||||||
|
For autonomous agents:
|
||||||
|
|
||||||
|
1. call `search_skills` with the task, capability, or technology
|
||||||
|
2. compare the bounded main-skill matches
|
||||||
|
3. call `read_resource` for one relevant `skill://<name>/SKILL.md`
|
||||||
|
4. read `_manifest` only if supporting detail may be needed
|
||||||
|
5. read only selected supporting files
|
||||||
|
|
||||||
|
For manual context attachment, browse the server's resources and attach the same bounded set of files.
|
||||||
|
|
||||||
|
## Prompt Examples
|
||||||
|
|
||||||
|
Resource attachment:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Use the attached personal-mcp skill as guidance, then reconcile it with the repository before proposing changes.
|
||||||
|
```
|
||||||
|
|
||||||
|
Direct loading:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Read skill://async-fastapi-sqlmodel/SKILL.md and apply only the sections relevant to this repository.
|
||||||
|
```
|
||||||
|
|
||||||
|
Supporting material:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Read skill://pytesting/_manifest, select the one reference relevant to async test lifecycle, and use that file with the main skill instructions.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Repository Instruction Pattern
|
||||||
|
|
||||||
|
A repo-level instruction should name the native retrieval order and context budget:
|
||||||
|
|
||||||
|
```md
|
||||||
|
When a task matches a personal-mcp skill:
|
||||||
|
|
||||||
|
1. Prefer an already attached native skill resource.
|
||||||
|
2. Otherwise call `search_skills` and select one `skill://<name>/SKILL.md` result by description.
|
||||||
|
3. Call `read_resource` for the selected skill and read `_manifest` only when supporting material is needed.
|
||||||
|
4. Load at most two candidate main files and only the relevant supporting paths.
|
||||||
|
5. Reconcile guidance with the current repository before editing.
|
||||||
|
```
|
||||||
|
|
||||||
|
Instructions steer behavior but do not force VS Code to attach resources automatically. The generic tools provide an agent-callable fallback when direct resource operations are absent from the deferred-tool catalog.
|
||||||
|
|
||||||
|
## Prompt Objects
|
||||||
|
|
||||||
|
Prompts remain separate from skills. When the client supports MCP prompt APIs, use prompt listing and `get_prompt` for parameterized workflows. Each authored `PROMPT.md` is the complete source of truth for its metadata, arguments, and prose; changes are loaded on the next prompt request.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
1. Use `MCP: List Servers` to confirm the server is enabled.
|
||||||
|
2. Use `MCP: Browse Resources` to confirm native skill resources exist.
|
||||||
|
3. Confirm `search_skills` and `read_resource` appear in the chat tool picker when autonomous retrieval is required.
|
||||||
|
4. Restart the MCP server after changing skill files because production uses `reload=False`.
|
||||||
|
5. Reload the VS Code window if the server is healthy but the resource or tool picker remains stale.
|
||||||
|
|
||||||
|
## Further Reading
|
||||||
|
|
||||||
|
1. [FastMCP Skills Provider](https://gofastmcp.com/servers/providers/skills)
|
||||||
|
2. [Add and manage MCP servers](https://code.visualstudio.com/docs/agent-customization/mcp-servers)
|
||||||
|
3. [MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration)
|
||||||
|
4. [Manage context for AI](https://code.visualstudio.com/docs/chat/copilot-chat-context)
|
||||||
|
5. [Skill Usage Mechanics](./usage.md)
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/rocket
|
||||||
|
---
|
||||||
|
|
||||||
|
# Personal MCP
|
||||||
|
|
||||||
|
This project is a document library of software patterns, best practices, and structured references to external documentation. The same markdown files are published through two equivalent surfaces, so human-readable docs and MCP resources stay aligned.
|
||||||
|
|
||||||
|
## MCP Server
|
||||||
|
|
||||||
|
An [MCP server](https://modelcontextprotocol.io/docs/getting-started/intro) at `/mcp` provides context for AI systems. The markdown files are exposed as [resources](https://modelcontextprotocol.io/docs/learn/server-concepts#resources) and are structured to be easily consumed by [MCP clients](https://modelcontextprotocol.io/docs/learn/client-concepts), such as VS Code.
|
||||||
|
|
||||||
|
## Docs
|
||||||
|
|
||||||
|
A website at `/docs` for humans to read and review.
|
||||||
|
|
||||||
|
## Quick start
|
||||||
|
|
||||||
|
Install dependencies first:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv sync
|
||||||
|
```
|
||||||
|
|
||||||
|
Run the app locally with the static docs rebuilt first, using [Uvicorn factory mode](https://www.uvicorn.org/settings/#application):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run zensical build && uv run uvicorn personal_mcp.web.app:create_app --factory --host 127.0.0.1 --port 8765
|
||||||
|
```
|
||||||
|
|
||||||
|
Build and run the Docker image with the same exposed port:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker build -t personal-mcp . && docker run --rm -p 8765:8765 personal-mcp
|
||||||
|
```
|
||||||
|
|
||||||
|
When the server is running, the health check is available at `/healthz` and the generated docs are available at `/docs/`.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
- [Resource-First Pattern Module Architecture](./architecture.md)
|
||||||
|
- [Contracts](./contracts/index.md)
|
||||||
|
- [Content Contract](./contracts/index.md#content-contract)
|
||||||
|
- [Frontmatter Contract](./contracts/frontmatter.md)
|
||||||
|
- [URI Contract](./contracts/uris.md)
|
||||||
|
- [Static Docs Hosting Pattern](./mcp_layout.md)
|
||||||
|
- [Skill Usage Mechanics](./usage.md)
|
||||||
|
- [Copilot MCP Mechanics](./copilot.md)
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
window.MathJax = {
|
||||||
|
tex: {
|
||||||
|
inlineMath: [['\\(', '\\)']],
|
||||||
|
displayMath: [['\\[', '\\]']],
|
||||||
|
processEscapes: true,
|
||||||
|
processEnvironments: true
|
||||||
|
},
|
||||||
|
options: {
|
||||||
|
ignoreHtmlClass: '.*|',
|
||||||
|
processHtmlClass: 'arithmatex'
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
document$.subscribe(() => {
|
||||||
|
MathJax.startup.output.clearCache();
|
||||||
|
MathJax.typesetClear();
|
||||||
|
MathJax.texReset();
|
||||||
|
MathJax.typesetPromise();
|
||||||
|
});
|
||||||
|
|
||||||
|
component$.subscribe(({ ref }) => {
|
||||||
|
if (ref.classList.contains('md-annotation')) {
|
||||||
|
MathJax.typesetPromise([ref]);
|
||||||
|
}
|
||||||
|
});
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/server
|
||||||
|
---
|
||||||
|
|
||||||
|
# Runtime And Static Docs Layout
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
The project serves native MCP content and a pre-built documentation site from one FastAPI process. Markdown is authored once under `docs/`; runtime providers and Zensical consume that same packaged tree for different purposes.
|
||||||
|
|
||||||
|
## Repository Layout
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
---
|
||||||
|
config:
|
||||||
|
treeView:
|
||||||
|
rowIndent: 32
|
||||||
|
lineThickness: 2
|
||||||
|
---
|
||||||
|
treeView-beta
|
||||||
|
"project-root"
|
||||||
|
"docs"
|
||||||
|
"prompts/<prompt-id>/PROMPT.md"
|
||||||
|
"skills/<skill-id>/SKILL.md"
|
||||||
|
"skills/<skill-id>/<supporting-files>"
|
||||||
|
"<general-pages>.md"
|
||||||
|
"site"
|
||||||
|
"static build output"
|
||||||
|
"src/personal_mcp"
|
||||||
|
"mcp.py"
|
||||||
|
"prompts/content.py"
|
||||||
|
"prompts/models.py"
|
||||||
|
"prompts/provider.py"
|
||||||
|
"registry/"
|
||||||
|
"skills/provider.py"
|
||||||
|
"web/"
|
||||||
|
```
|
||||||
|
|
||||||
|
Ownership rules:
|
||||||
|
|
||||||
|
1. `docs/skills/` is owned exclusively by `SkillsDirectoryProvider` at runtime.
|
||||||
|
2. Each file under `docs/prompts/` owns its prompt metadata, argument schema, and prose.
|
||||||
|
3. The docs registry owns only general Markdown resources and explicitly excludes skills.
|
||||||
|
4. `site/` is generated output.
|
||||||
|
5. The deleted custom `catalog/` package is not part of the runtime.
|
||||||
|
|
||||||
|
## Runtime Composition
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
A[Packaged Skills] --> B[SkillsDirectoryProvider]
|
||||||
|
C[Packaged Prompt Markdown] --> D[Markdown Prompt Provider]
|
||||||
|
E[Packaged Markdown] --> F[Docs Registry]
|
||||||
|
B --> G[FastMCP]
|
||||||
|
D --> G
|
||||||
|
F --> G
|
||||||
|
G --> H[MCP Transport]
|
||||||
|
H --> K[FastAPI Application]
|
||||||
|
L[Pre-built site] --> M[Static /docs Mount]
|
||||||
|
K --> M
|
||||||
|
```
|
||||||
|
|
||||||
|
Runtime guarantees:
|
||||||
|
|
||||||
|
1. Providers are installed before serving requests.
|
||||||
|
2. Prompt discovery rescans authored files on each list and get request.
|
||||||
|
3. Duplicate components fail according to FastMCP's configured duplicate policy.
|
||||||
|
4. Skills and prompts use native FastMCP component surfaces.
|
||||||
|
5. General docs path parsing rejects traversal, backslashes, non-Markdown paths, and the skill namespace.
|
||||||
|
|
||||||
|
## Build And Publish Flow
|
||||||
|
|
||||||
|
1. Author prompt definitions and prose under `docs/prompts/`.
|
||||||
|
2. Run `uv run zensical build` to produce `site/`.
|
||||||
|
3. Build the wheel, which packages the authored docs under `personal_mcp/docs/`.
|
||||||
|
4. Start the app and serve MCP plus the static site.
|
||||||
|
|
||||||
|
No runtime Markdown-to-HTML conversion occurs.
|
||||||
|
|
||||||
|
## Machine-Facing Mapping
|
||||||
|
|
||||||
|
1. `docs/skills/<skill-id>/SKILL.md` maps to `skill://<skill-id>/SKILL.md`.
|
||||||
|
2. Skill supporting files map to `skill://<skill-id>/<path>`.
|
||||||
|
3. Declarative prompt documents map to native MCP prompt names.
|
||||||
|
4. General `docs/<path>.md` maps to `resource://docs/{path*}`.
|
||||||
|
|
||||||
|
The server publishes no tool projections of resources or prompts.
|
||||||
|
|
||||||
|
## Public Surface Policy
|
||||||
|
|
||||||
|
Canonical provider and protocol surfaces are the only public interfaces.
|
||||||
|
|
||||||
|
## Static Mount Expectations
|
||||||
|
|
||||||
|
The FastAPI app mounts the Zensical output, serves index and asset files, and returns a clear unavailable response when the static output is absent. The site directory is immutable for a given build and remains separate from packaged authored Markdown.
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/messages-square
|
||||||
|
prompt:
|
||||||
|
version: "1.0.0"
|
||||||
|
description: Provide a practical checklist and baseline template for authoring docs-first MCP modules and repository-specific Copilot instruction shims.
|
||||||
|
tags: [authoring, mcp, fastmcp, copilot, prompts, scaffolding]
|
||||||
|
arguments:
|
||||||
|
artifact_type:
|
||||||
|
description: Artifact type to create.
|
||||||
|
required: true
|
||||||
|
choices: [skill, prompt, shim]
|
||||||
|
artifact_id:
|
||||||
|
description: Lowercase kebab-case id for the module or shim.
|
||||||
|
required: true
|
||||||
|
goal:
|
||||||
|
description: One-sentence capability statement.
|
||||||
|
required: true
|
||||||
|
scope_glob:
|
||||||
|
description: Optional applyTo glob for shim outputs.
|
||||||
|
required: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# Authoring Bootstrap
|
||||||
|
|
||||||
|
Use this prompt to author or update docs-first MCP modules in this repository, including repository-specific Copilot thin shims.
|
||||||
|
|
||||||
|
## Supplied Inputs
|
||||||
|
|
||||||
|
- `artifact_type`: {{artifact_type}}
|
||||||
|
- `artifact_id`: {{artifact_id}}
|
||||||
|
- `goal`: {{goal}}
|
||||||
|
- `scope_glob`: {{scope_glob}}
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
1. artifact_type: one of skill, prompt, shim
|
||||||
|
2. artifact_id: lowercase kebab-case id
|
||||||
|
3. goal: one-sentence capability statement
|
||||||
|
4. optional scope_glob for shim outputs
|
||||||
|
|
||||||
|
## Required References
|
||||||
|
|
||||||
|
Load only what matches the requested artifact:
|
||||||
|
|
||||||
|
1. Authoring workflow and validation policy: [Authoring Guide](../../authoring.md)
|
||||||
|
2. Prompt metadata and structure: [Prompt Contract](../../contracts/prompt.md)
|
||||||
|
3. Skill metadata and structure (only for skill outputs): [Skill Contract](../../contracts/skill_contract.md)
|
||||||
|
4. Thin shim mechanics and path binding: [Skill Usage Mechanics](../../usage.md)
|
||||||
|
5. Copilot resource attachment behavior: [Copilot MCP Mechanics](../../copilot.md)
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. Validate required inputs and ask one clarifying question if any required input is missing.
|
||||||
|
2. Keep ids and slugs aligned with folder names and frontmatter ids.
|
||||||
|
3. Enforce artifact_type enum values exactly: skill, prompt, shim.
|
||||||
|
4. If artifact_type is outside the enum, ask one correction question and stop before generating output.
|
||||||
|
5. Apply YAML safety rules for frontmatter values:
|
||||||
|
- quote values containing `:`
|
||||||
|
- prefer quotes for punctuation-heavy scalars
|
||||||
|
- use block scalars for multiline descriptions
|
||||||
|
6. Run immediate validation after frontmatter edits:
|
||||||
|
- `uv run zensical build`
|
||||||
|
- `uv run pytest -q`
|
||||||
|
7. Produce only the requested artifact type.
|
||||||
|
8. Keep guidance deterministic and minimal, with explicit references to source docs.
|
||||||
|
9. If artifact_type is shim:
|
||||||
|
- bind one applyTo scope to one `skill://<name>/SKILL.md` resource URI
|
||||||
|
- use MCP resource attachment
|
||||||
|
- inspect the selected skill's `_manifest` only when supporting material is needed
|
||||||
|
10. Return created or updated file paths and any validation commands that should be run.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
Return:
|
||||||
|
|
||||||
|
1. Files created or updated.
|
||||||
|
2. Which references were used.
|
||||||
|
3. Validation commands and outcomes (or commands to run if execution is not requested).
|
||||||
@@ -0,0 +1,106 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/messages-square
|
||||||
|
prompt:
|
||||||
|
version: "1.0.0"
|
||||||
|
description: Research established patterns and design a high-level architecture for a new app or library with explicit tradeoffs and test strategy.
|
||||||
|
tags: [architecture, planning, greenfield, design, testing, prompts]
|
||||||
|
arguments:
|
||||||
|
scope_type:
|
||||||
|
description: Scope type to design.
|
||||||
|
required: true
|
||||||
|
choices: [app, library]
|
||||||
|
intent_document:
|
||||||
|
description: Optional full document describing goals and context.
|
||||||
|
required: false
|
||||||
|
problem_domain:
|
||||||
|
description: Problem domain and business goal.
|
||||||
|
required: false
|
||||||
|
constraints:
|
||||||
|
description: Runtime, deployment, and non-functional constraints.
|
||||||
|
required: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# Greenfield Architecture Planner
|
||||||
|
|
||||||
|
Use this prompt to design a new software app or library architecture in generic terms.
|
||||||
|
|
||||||
|
## Supplied Inputs
|
||||||
|
|
||||||
|
- `scope_type`: {{scope_type}}
|
||||||
|
- `intent_document`: {{intent_document}}
|
||||||
|
- `problem_domain`: {{problem_domain}}
|
||||||
|
- `constraints`: {{constraints}}
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
1. intent_document: optional full document that explains goals, context, constraints, and desired outcomes
|
||||||
|
2. problem_domain: concise domain and one-sentence business goal when no full intent document is provided
|
||||||
|
3. scope_type: app or library
|
||||||
|
4. optional constraints: runtime, deployment, scale, non-functional priorities
|
||||||
|
|
||||||
|
If both intent_document and problem_domain are provided, treat intent_document as the primary source and use problem_domain as a summary cross-check.
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. Validate required inputs.
|
||||||
|
- scope_type is required
|
||||||
|
- at least one of intent_document or problem_domain must be provided
|
||||||
|
- ask one concise clarification question if inputs are incomplete or contradictory
|
||||||
|
2. Start with research before proposing architecture:
|
||||||
|
- identify at least three established patterns or methodologies used for similar systems
|
||||||
|
- summarize what each pattern optimizes for
|
||||||
|
- compare strengths, risks, and implementation complexity
|
||||||
|
3. Ask which aspects of those patterns matter most for the user context.
|
||||||
|
4. Identify major libraries or frameworks commonly used for this problem space and explain tradeoffs for each:
|
||||||
|
- strengths and weaknesses
|
||||||
|
- ecosystem maturity
|
||||||
|
- performance profile
|
||||||
|
- operational complexity
|
||||||
|
- learning curve
|
||||||
|
5. Recommend one primary stack and one fallback stack, with rationale tied to stated priorities.
|
||||||
|
6. Produce the architecture deliverables:
|
||||||
|
- high-level concepts, features, and requirements
|
||||||
|
- intended use cases and key workflows
|
||||||
|
- high-level package/module structure
|
||||||
|
- conceptual boundaries for each module (what belongs there and what does not)
|
||||||
|
- dependency and data-flow direction between modules
|
||||||
|
7. Plan incremental delivery with explicit growth paths:
|
||||||
|
- define the initial prototype slice with the smallest valuable feature set
|
||||||
|
- identify which features are intentionally deferred from the prototype
|
||||||
|
- describe extension paths that add complexity in controlled stages
|
||||||
|
- ensure each stage preserves clean module boundaries and low migration risk
|
||||||
|
8. Design a test strategy aligned to the proposed structure and staged delivery plan:
|
||||||
|
- unit, integration, contract, and end-to-end layers
|
||||||
|
- what each layer should cover in prototype stage vs extension stages
|
||||||
|
- fixture and environment setup for fast, deterministic tests
|
||||||
|
- boundary seams for mocks/fakes and minimization of nondeterministic external I/O
|
||||||
|
- CI execution approach for fast feedback and confidence
|
||||||
|
9. Call out key risks, assumptions, and open questions.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
Return these sections in order:
|
||||||
|
|
||||||
|
1. Research Summary
|
||||||
|
2. Pattern Comparison
|
||||||
|
3. Library and Framework Tradeoffs
|
||||||
|
4. Recommended Stack
|
||||||
|
5. Architecture Overview
|
||||||
|
6. Concepts, Features, and Requirements
|
||||||
|
7. Intended Use Cases
|
||||||
|
8. Package and Module Layout
|
||||||
|
9. Conceptual Boundary Map
|
||||||
|
10. Initial Prototype Scope
|
||||||
|
11. Extension Roadmap
|
||||||
|
12. Test Strategy
|
||||||
|
13. Risks and Open Questions
|
||||||
|
14. Next Implementation Steps
|
||||||
|
|
||||||
|
## Quality Rules
|
||||||
|
|
||||||
|
1. Keep language generic and project-agnostic.
|
||||||
|
2. Prefer established patterns over novelty unless there is a strong reason to diverge.
|
||||||
|
3. Tie each recommendation to an explicit requirement or tradeoff.
|
||||||
|
4. Make assumptions explicit and concise.
|
||||||
|
5. Ask one focused clarifying question when confidence is low instead of over-speculating.
|
||||||
|
6. Prefer architecture decisions that support starting simple and growing complexity without major rewrites.
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/messages-square
|
||||||
|
prompt:
|
||||||
|
version: "1.1.0"
|
||||||
|
description: Create a responsive sample page layout for a user-supplied domain and return paste-ready HTML and CSS for JSFiddle.
|
||||||
|
tags: [frontend, html, css, jsfiddle, layout, prototyping, prompts]
|
||||||
|
arguments:
|
||||||
|
domain:
|
||||||
|
description: Product, service, organization, or subject represented by the page.
|
||||||
|
required: true
|
||||||
|
layout_brief:
|
||||||
|
description: Optional page type, sections, priorities, or visual constraints.
|
||||||
|
required: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# JSFiddle Page Layout
|
||||||
|
|
||||||
|
Create a polished sample page layout for the supplied domain. The result must run by pasting the markup and styles into the [JSFiddle](https://jsfiddle.net/) HTML and CSS panes.
|
||||||
|
|
||||||
|
## Supplied Inputs
|
||||||
|
|
||||||
|
- `domain`: {{domain}}
|
||||||
|
- `layout_brief`: {{layout_brief}}
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
1. `domain`: the product, service, organization, or subject represented by the page, including its intended audience when known
|
||||||
|
2. `layout_brief`: optional page type, required sections, content priorities, visual direction, or constraints
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. Infer the page's primary purpose, audience, content hierarchy, and most important user action from the inputs.
|
||||||
|
2. If the domain does not provide enough information to choose a useful page type or primary action, ask one concise clarification question before generating code.
|
||||||
|
3. Choose a visual direction and information density appropriate to the domain. Build the usable page itself, not a marketing explanation of the page.
|
||||||
|
4. Write semantic HTML with realistic domain-specific sample content. Do not use placeholder text such as lorem ipsum.
|
||||||
|
5. Build the layout with modern CSS, using [CSS Grid](https://css-tricks.com/complete-guide-css-grid-layout/) for two-dimensional page structure and [Flexbox](https://css-tricks.com/snippets/css/a-guide-to-flexbox/) for one-dimensional alignment where each fits naturally.
|
||||||
|
6. Make the page responsive at narrow mobile and desktop widths without horizontal overflow, overlapping content, or clipped text.
|
||||||
|
7. Keep the example self-contained. Use no JavaScript, build tools, external stylesheets, images, or icon libraries unless the layout brief explicitly requires them.
|
||||||
|
8. Include accessible landmarks, heading order, labels, focus styles, color contrast, and reduced-motion handling when animation is present.
|
||||||
|
9. Use CSS custom properties for the color, typography, spacing, border, and shadow system. Avoid generic framework styling and tailor the visual language to the domain.
|
||||||
|
|
||||||
|
## Design References
|
||||||
|
|
||||||
|
Use these references as comparative guidance, not as templates to copy. Select principles that fit the domain and layout brief, and do not reproduce a vendor's visual language unless the user requests it.
|
||||||
|
|
||||||
|
1. [Material Design 3 foundations](https://m3.material.io/foundations) for current approaches to layout, interaction states, design tokens, and adaptable UI systems.
|
||||||
|
2. [Apple Human Interface Guidelines](https://developer.apple.com/design/human-interface-guidelines/) for contemporary principles covering hierarchy, typography, controls, and platform-aware interaction.
|
||||||
|
3. [web.dev responsive web design basics](https://web.dev/articles/responsive-web-design-basics) for content-led breakpoints, flexible layouts, and input-aware responsiveness.
|
||||||
|
4. [Web Content Accessibility Guidelines (WCAG) 2.2](https://www.w3.org/TR/WCAG22/) as the accessibility baseline for structure, contrast, focus, reflow, and target sizing.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
Return exactly two fenced code blocks in this order:
|
||||||
|
|
||||||
|
1. An `html` block containing only the content for JSFiddle's HTML pane.
|
||||||
|
2. A `css` block containing only the content for JSFiddle's CSS pane.
|
||||||
|
|
||||||
|
Do not include setup instructions, design commentary, JavaScript, or prose outside the two code blocks.
|
||||||
|
|
||||||
|
## Quality Rules
|
||||||
|
|
||||||
|
1. Prefer semantic elements such as `header`, `nav`, `main`, `section`, `article`, `aside`, and `footer` when they match the content.
|
||||||
|
2. Reserve large display type for a true hero or primary page title; keep operational interfaces compact and easy to scan.
|
||||||
|
3. Use cards only for repeated items or genuinely framed tools. Do not place cards inside cards.
|
||||||
|
4. Use stable responsive constraints for grids, controls, media, and navigation so dynamic content does not shift the layout unexpectedly.
|
||||||
|
5. Avoid decorative gradients, floating color blobs, excessive rounding, and one-note palettes unless they are explicitly appropriate to the domain.
|
||||||
|
6. Ensure controls look and behave like their purpose, with visible hover and keyboard-focus states.
|
||||||
|
7. Keep all visible copy relevant to the fictional domain rather than describing the mockup or its implementation.
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/messages-square
|
||||||
|
prompt:
|
||||||
|
version: "1.0.0"
|
||||||
|
description: Create one repository-specific thin shim instruction file that binds a file scope to a user-selected Personal MCP skill resource.
|
||||||
|
tags: [copilot, mcp, instructions, shims, prompts]
|
||||||
|
arguments:
|
||||||
|
apply_to_glob:
|
||||||
|
description: File glob scope for the shim applyTo field.
|
||||||
|
required: true
|
||||||
|
primary_skill_resource:
|
||||||
|
description: Primary native skill:// resource URI.
|
||||||
|
required: true
|
||||||
|
shim_title:
|
||||||
|
description: Optional human-readable instruction shim name.
|
||||||
|
required: false
|
||||||
|
companion_docs_page:
|
||||||
|
description: Optional relative companion documentation link.
|
||||||
|
required: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# MCP Consumer Repository Shim
|
||||||
|
|
||||||
|
Use this prompt to generate exactly one repository-scoped Copilot instruction shim for an MCP consumer repository.
|
||||||
|
|
||||||
|
## Supplied Inputs
|
||||||
|
|
||||||
|
- `apply_to_glob`: {{apply_to_glob}}
|
||||||
|
- `primary_skill_resource`: {{primary_skill_resource}}
|
||||||
|
- `shim_title`: {{shim_title}}
|
||||||
|
- `companion_docs_page`: {{companion_docs_page}}
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
- Required:
|
||||||
|
- apply_to_glob
|
||||||
|
- primary_skill_resource
|
||||||
|
- Optional:
|
||||||
|
- shim_title
|
||||||
|
- companion_docs_page
|
||||||
|
|
||||||
|
## Required References
|
||||||
|
|
||||||
|
Load only sections relevant to the requested shim:
|
||||||
|
|
||||||
|
1. Thin shim pattern and scope guidance: [Skill Usage Mechanics](../../usage.md)
|
||||||
|
2. VS Code Copilot MCP resource behavior: [Copilot MCP Mechanics](../../copilot.md)
|
||||||
|
3. Authoring workflow and validation checklist: [Authoring Guide](../../authoring.md)
|
||||||
|
4. Instruction metadata expectations and examples: [Copilot customization skill](../../skills/copilot-customization/SKILL.md)
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. Validate that apply_to_glob and primary_skill_resource are present.
|
||||||
|
2. Validate that primary_skill_resource uses the `skill://<skill-name>/SKILL.md` form.
|
||||||
|
3. If either value is missing or ambiguous, ask exactly one clarifying question before generating output.
|
||||||
|
4. Generate one .instructions.md file content block only.
|
||||||
|
5. Keep the shim concise and deterministic:
|
||||||
|
- include YAML frontmatter with name, description, and applyTo
|
||||||
|
- include a primary rule that uses the selected primary_skill_resource first
|
||||||
|
- include a bounded execution pattern (load primary doc, apply only relevant sections, keep edits minimal)
|
||||||
|
6. Include VS Code/Copilot integration mechanics in the shim body:
|
||||||
|
- use MCP resource attachment
|
||||||
|
- inspect `_manifest` only when the task needs supporting material
|
||||||
|
- ask one clarifying question when confidence is low
|
||||||
|
7. If companion_docs_page is provided, include it as a companion docs link line.
|
||||||
|
8. Do not generate additional files, code changes, or batch shim packs.
|
||||||
|
|
||||||
|
## Output Format
|
||||||
|
|
||||||
|
Return exactly:
|
||||||
|
|
||||||
|
1. Suggested file path line under .github/instructions/.
|
||||||
|
2. One fenced markdown block containing the full .instructions.md content.
|
||||||
|
3. A brief note (max 3 lines) describing what the shim routes and why.
|
||||||
|
|
||||||
|
## Output Template
|
||||||
|
|
||||||
|
````md
|
||||||
|
Path: .github/instructions/<slug>.instructions.md
|
||||||
|
|
||||||
|
```md
|
||||||
|
---
|
||||||
|
name: <shim-title>
|
||||||
|
description: Route <scope> edits to the Personal MCP <skill-id> resource.
|
||||||
|
applyTo: '<apply_to_glob>'
|
||||||
|
---
|
||||||
|
|
||||||
|
When editing files matching <apply_to_glob>, use <primary_skill_resource> as the primary guidance source.
|
||||||
|
|
||||||
|
Execution pattern:
|
||||||
|
|
||||||
|
1. Load the primary skill document first.
|
||||||
|
2. Apply only sections relevant to the file being edited.
|
||||||
|
3. Keep edits minimal and aligned with repository conventions.
|
||||||
|
4. Prefer MCP resource attachment when available in the current chat surface.
|
||||||
|
5. Read the selected skill's `_manifest` only when supporting material is needed.
|
||||||
|
6. If confidence is low, ask one clarifying question before editing.
|
||||||
|
|
||||||
|
Companion docs page: <optional-relative-doc-link>
|
||||||
|
```
|
||||||
|
````
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/messages-square
|
||||||
|
prompt:
|
||||||
|
version: "1.0.0"
|
||||||
|
description: Extract a user-selected component from a JSFiddle page layout and implement it as a reusable NiceGUI render function.
|
||||||
|
tags: [nicegui, components, frontend, refactoring, jsfiddle, prompts]
|
||||||
|
arguments:
|
||||||
|
component:
|
||||||
|
description: Visible label, semantic role, or selector identifying the component.
|
||||||
|
required: true
|
||||||
|
source_layout:
|
||||||
|
description: Optional source HTML and CSS.
|
||||||
|
required: false
|
||||||
|
target_location:
|
||||||
|
description: Optional target NiceGUI page, module, or package.
|
||||||
|
required: false
|
||||||
|
behavior_requirements:
|
||||||
|
description: Optional interactions, state, callbacks, or variations.
|
||||||
|
required: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# NiceGUI Component Extraction
|
||||||
|
|
||||||
|
Extract one user-selected component from the output of the [JSFiddle Page Layout](../jsfiddle-page-layout/PROMPT.md) prompt and implement it as a reusable NiceGUI component in the target repository.
|
||||||
|
|
||||||
|
## Supplied Inputs
|
||||||
|
|
||||||
|
- `component`: {{component}}
|
||||||
|
- `source_layout`: {{source_layout}}
|
||||||
|
- `target_location`: {{target_location}}
|
||||||
|
- `behavior_requirements`: {{behavior_requirements}}
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
1. `component`: required visible label, semantic role, or selector identifying the component to extract
|
||||||
|
2. `source_layout`: optional HTML and CSS; when omitted, use the latest applicable JSFiddle page layout output in the conversation
|
||||||
|
3. `target_location`: optional target page, module, or package; infer it from the repository when omitted
|
||||||
|
4. `behavior_requirements`: optional interactions, state, callbacks, or content variations
|
||||||
|
|
||||||
|
If the selected component or source layout cannot be identified unambiguously, ask one concise clarification question before editing.
|
||||||
|
|
||||||
|
## Required References
|
||||||
|
|
||||||
|
Apply these references before implementation:
|
||||||
|
|
||||||
|
1. Package boundaries, dependency direction, and page or component ownership: [NiceGUI Application Architecture](../../skills/nicegui/references/architecture.md)
|
||||||
|
2. Responsive layout, Quasar props, Tailwind utilities, and shared CSS: [NiceGUI Styling and Customization](../../skills/nicegui/references/styling-and-customization.md)
|
||||||
|
3. Typed UI state, propagation, mutable defaults, binding strictness, and version checks: [Binding Dataclasses Deep Dive](../../skills/nicegui/references/binding-dataclasses.md)
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. Locate the selected region in the source HTML and CSS, including its responsive rules, states, and dependencies on surrounding layout.
|
||||||
|
2. Inspect the target repository's NiceGUI version, package structure, component conventions, shared CSS loading, and nearest page call site.
|
||||||
|
3. Define the smallest reusable API for the component:
|
||||||
|
- name the public function `render_<component_name>` using snake_case
|
||||||
|
- accept content, typed state, and event callbacks as explicit parameters
|
||||||
|
- keep business rules, persistence, and service access outside the component
|
||||||
|
- preserve an established return-value convention; otherwise return the component's root NiceGUI element
|
||||||
|
4. Translate semantic HTML into native NiceGUI and Quasar elements. Do not embed the original page wholesale with `ui.html` when standard components express the structure.
|
||||||
|
5. Recreate only the CSS needed by the extracted component:
|
||||||
|
- use Quasar props for component appearance and behavior
|
||||||
|
- use NiceGUI classes and Tailwind utilities for spacing, sizing, alignment, and responsive layout
|
||||||
|
- use scoped shared CSS only where props and utilities are insufficient
|
||||||
|
- do not override Quasar field internals or duplicate globally loaded styles
|
||||||
|
6. Model editable or shared component state with a typed `@binding.bindable_dataclass` only when binding improves the interaction:
|
||||||
|
- use `field(default_factory=...)` for mutable defaults
|
||||||
|
- scope state to the appropriate page, client, or user
|
||||||
|
- keep binding transforms pure and inexpensive
|
||||||
|
- assign updated collections back to bound fields instead of relying on in-place mutation
|
||||||
|
7. Integrate the render function at the nearest target page or call site without moving unrelated page composition or domain logic into the component.
|
||||||
|
8. Preserve accessibility, focus behavior, text wrapping, stable dimensions, and the source layout's visual hierarchy.
|
||||||
|
9. Run the narrowest available tests, lint, and type checks for the changed files. For visual components, verify representative mobile, landscape desktop, and portrait desktop viewports when browser tooling is available.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
Complete the implementation in the target repository, then report:
|
||||||
|
|
||||||
|
1. Files created or updated.
|
||||||
|
2. The `render_*` function signature and its state or callback contract.
|
||||||
|
3. Any deliberate visual or interaction differences from the JSFiddle source.
|
||||||
|
4. Validation commands and outcomes, including viewport checks when performed.
|
||||||
|
|
||||||
|
## Quality Rules
|
||||||
|
|
||||||
|
1. Extract exactly the requested component and its necessary local dependencies.
|
||||||
|
2. Prefer the target repository's established patterns over introducing a new abstraction style.
|
||||||
|
3. Keep the component presentation-focused and reusable across pages with compatible data.
|
||||||
|
4. Do not add a bindable dataclass for static content or event-local state that is clearer as ordinary parameters.
|
||||||
|
5. Do not create a second component tree for mobile; use responsive classes and stable layout constraints.
|
||||||
|
6. Keep custom CSS tokenized, scoped to the component, and loaded once by the application's composition layer.
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/messages-square
|
||||||
|
prompt:
|
||||||
|
version: "1.0.0"
|
||||||
|
description: Fill scaffolded pytest methods with assertions, fixtures, and minimal test data while preserving reviewed structure.
|
||||||
|
tags: [pytest, testing, scaffolding, prompts]
|
||||||
|
arguments:
|
||||||
|
target_files:
|
||||||
|
description: Target test file paths under tests/.
|
||||||
|
required: true
|
||||||
|
stack:
|
||||||
|
description: Runtime stack type for fixture and marker choices.
|
||||||
|
required: true
|
||||||
|
choices: [pure-python, fastapi, sqlalchemy-sync, sqlalchemy-async, mixed]
|
||||||
|
strategy:
|
||||||
|
description: Optional minimal or comprehensive implementation preference.
|
||||||
|
required: false
|
||||||
|
marker_lane:
|
||||||
|
description: Optional pytest marker lane.
|
||||||
|
required: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pytest Fill Scaffold
|
||||||
|
|
||||||
|
Use this prompt after test scaffolding exists and method names/docstrings are already in place.
|
||||||
|
|
||||||
|
## Supplied Inputs
|
||||||
|
|
||||||
|
- `target_files`: {{target_files}}
|
||||||
|
- `stack`: {{stack}}
|
||||||
|
- `strategy`: {{strategy}}
|
||||||
|
- `marker_lane`: {{marker_lane}}
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
- Target test file(s) under tests/.
|
||||||
|
- Stack type:
|
||||||
|
- pure-python
|
||||||
|
- fastapi
|
||||||
|
- sqlalchemy-sync
|
||||||
|
- sqlalchemy-async
|
||||||
|
- mixed
|
||||||
|
- Optional constraints:
|
||||||
|
- keep implementation minimal vs comprehensive
|
||||||
|
- marker lane target (unit, integration, smoke)
|
||||||
|
|
||||||
|
## Required References
|
||||||
|
|
||||||
|
Load these in order and use only what matches the task:
|
||||||
|
|
||||||
|
1. Core defaults: [pytest scaffolding skill](../../skills/pytesting/SKILL.md)
|
||||||
|
2. Naming/hierarchy preservation: [naming and organization](../../skills/pytesting/references/naming-and-organization.md)
|
||||||
|
3. Baseline pytest fixtures/markers: [pytest docs notes](../../skills/pytesting/references/pytest-docs.md)
|
||||||
|
4. FastAPI-specific behavior (only when needed): [fastapi testing](../../skills/pytesting/references/fastapi-testing.md)
|
||||||
|
5. SQLAlchemy-specific behavior (only when needed): [sqlalchemy testing](../../skills/pytesting/references/sqlalchemy-testing.md)
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. Inspect target files and treat human-reviewed docstring-only scaffolds as invariant.
|
||||||
|
2. Convert each scaffolded method into an executable test with a single behavior focus.
|
||||||
|
3. Keep one-line docstrings for class and method intent.
|
||||||
|
4. Add or refine fixtures at the nearest useful scope:
|
||||||
|
- global in tests/conftest.py only when broadly reusable
|
||||||
|
- subtree conftest.py for domain-specific fixtures
|
||||||
|
5. Assign markers consistent with cost and dependencies:
|
||||||
|
- unit for pure logic
|
||||||
|
- integration for framework/DB contracts
|
||||||
|
- smoke for thin critical-path checks
|
||||||
|
6. Validate in this order:
|
||||||
|
- uv run pytest --collect-only -q
|
||||||
|
- uv run pytest -m unit -q when unit tests are touched
|
||||||
|
- uv run pytest -q if dependencies are available
|
||||||
|
|
||||||
|
## Authoring Rules
|
||||||
|
|
||||||
|
- Prefer deterministic tests and explicit setup/teardown.
|
||||||
|
- Keep assertions precise and readable.
|
||||||
|
- Do not overfit tests to private implementation details.
|
||||||
|
- If a scaffolded class or method has only a docstring body, treat its name and hierarchy as locked.
|
||||||
|
- Do not rename, move, merge, split, or re-nest docstring-only scaffolded tests unless explicitly requested.
|
||||||
|
- Preserve existing one-line docstrings on scaffolded classes and methods unless they are factually incorrect.
|
||||||
|
- If stack details are missing and would change fixture strategy, ask one concise clarifying question before editing.
|
||||||
|
|
||||||
|
## Output Format
|
||||||
|
|
||||||
|
Return:
|
||||||
|
1. Files updated.
|
||||||
|
2. Fixture and marker decisions.
|
||||||
|
3. Which references were used and why.
|
||||||
|
4. Validation command results.
|
||||||
|
5. Risks or open questions.
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/messages-square
|
||||||
|
prompt:
|
||||||
|
version: "1.0.0"
|
||||||
|
description: Plan and optionally scaffold pytest file and class structure for selected Python modules.
|
||||||
|
tags: [pytest, testing, scaffolding, prompts]
|
||||||
|
arguments:
|
||||||
|
target_modules:
|
||||||
|
description: Target module paths under src/.
|
||||||
|
required: true
|
||||||
|
mode:
|
||||||
|
description: Whether to plan only or create scaffold files.
|
||||||
|
required: true
|
||||||
|
choices: [plan-only, scaffold]
|
||||||
|
path_strategy:
|
||||||
|
description: Optional src-to-tests path mapping preference.
|
||||||
|
required: false
|
||||||
|
naming_style:
|
||||||
|
description: Optional concise test naming preference.
|
||||||
|
required: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pytest Scaffold
|
||||||
|
|
||||||
|
Use this prompt to consistently plan and scaffold pytest test modules for selected Python source modules.
|
||||||
|
|
||||||
|
## Supplied Inputs
|
||||||
|
|
||||||
|
- `target_modules`: {{target_modules}}
|
||||||
|
- `mode`: {{mode}}
|
||||||
|
- `path_strategy`: {{path_strategy}}
|
||||||
|
- `naming_style`: {{naming_style}}
|
||||||
|
|
||||||
|
## Inputs
|
||||||
|
|
||||||
|
- Required:
|
||||||
|
- target_modules: one or more module paths under src/
|
||||||
|
- mode: one of plan-only or scaffold
|
||||||
|
- Optional:
|
||||||
|
- path_strategy: preference for how source paths map into tests/
|
||||||
|
- naming_style: preference for concise method naming style
|
||||||
|
|
||||||
|
## Required References
|
||||||
|
|
||||||
|
Load these in order and apply only the relevant sections:
|
||||||
|
|
||||||
|
1. Primary conventions: [Pytesting Skill](../../skills/pytesting/SKILL.md)
|
||||||
|
2. Hierarchy and naming: [Naming and Organization](../../skills/pytesting/references/naming-and-organization.md)
|
||||||
|
3. Marker and fixture defaults: [Pytest Docs Notes](../../skills/pytesting/references/pytest-docs.md)
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. Inspect the current tests/ layout and infer existing naming and grouping conventions.
|
||||||
|
2. Propose a concise hierarchy plan first:
|
||||||
|
- test file paths
|
||||||
|
- class hierarchy
|
||||||
|
- method naming pattern
|
||||||
|
- fixture placement choices (tests/conftest.py or subtree conftest.py)
|
||||||
|
3. If mode is scaffold, implement only the scaffold structure:
|
||||||
|
- create missing test modules
|
||||||
|
- create class hierarchy
|
||||||
|
- add one-line docstrings to each class and test method
|
||||||
|
- keep test method names short and behavior-focused
|
||||||
|
4. Treat docstring-only scaffolds as an intentionally stable baseline for later fill-in work.
|
||||||
|
5. Validate collection with:
|
||||||
|
- uv run pytest --collect-only -q
|
||||||
|
6. Report outcomes:
|
||||||
|
- files created or updated
|
||||||
|
- collection result
|
||||||
|
- ambiguities and follow-up choices
|
||||||
|
|
||||||
|
## Naming Defaults
|
||||||
|
|
||||||
|
- Class naming:
|
||||||
|
- Test<PrimarySubject> as a top-level subject class
|
||||||
|
- nested Test<MethodOrArea> classes where extra context improves readability
|
||||||
|
- Test<FunctionName> top-level classes for standalone module functions
|
||||||
|
- Method naming:
|
||||||
|
- test_<short_outcome>
|
||||||
|
- one behavior target per method
|
||||||
|
- one-line docstring for full intent
|
||||||
|
|
||||||
|
## Authoring Rules
|
||||||
|
|
||||||
|
1. Keep scope focused on structure and naming in this prompt.
|
||||||
|
2. Do not fill test implementation details unless explicitly requested.
|
||||||
|
3. Preserve established repository conventions when they are already present.
|
||||||
|
4. If input constraints conflict, ask one concise clarifying question before editing.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
Return:
|
||||||
|
|
||||||
|
1. Discovery summary and references used.
|
||||||
|
2. Proposed or applied test tree.
|
||||||
|
3. Class and method naming map.
|
||||||
|
4. Validation command result.
|
||||||
|
5. Open questions only when they block completion.
|
||||||
@@ -0,0 +1,139 @@
|
|||||||
|
---
|
||||||
|
icon: lucide/shield-check
|
||||||
|
---
|
||||||
|
|
||||||
|
# Securing Remote Access
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
This project exposes two related surfaces from the same runtime:
|
||||||
|
|
||||||
|
1. a static documentation site under `/docs`
|
||||||
|
2. a Streamable HTTP MCP endpoint under `/mcp`
|
||||||
|
|
||||||
|
The same Markdown content backs both surfaces. For the current project shape, the MCP server is resource-first and primarily exposes public skill and documentation text. It is not intended to expose secrets, private data, shell access, filesystem access, or tools with side effects.
|
||||||
|
|
||||||
|
The expected deployment path is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Public internet
|
||||||
|
-> Cloudflare Tunnel
|
||||||
|
-> Caddy
|
||||||
|
-> personal-mcp container
|
||||||
|
```
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
For the current use case, heavy application-level authentication is not required.
|
||||||
|
|
||||||
|
The recommended posture is:
|
||||||
|
|
||||||
|
1. Keep the service behind Cloudflare Tunnel and Caddy.
|
||||||
|
2. Do not expose the container port directly to the public internet.
|
||||||
|
3. Treat everything exposed through MCP as publishable public documentation.
|
||||||
|
4. Add stronger authentication only if the MCP surface later includes sensitive content or tools with meaningful side effects.
|
||||||
|
|
||||||
|
This keeps the deployment simple while preserving a clear upgrade path.
|
||||||
|
|
||||||
|
## Tradeoffs
|
||||||
|
|
||||||
|
### Leaving `/mcp` Public
|
||||||
|
|
||||||
|
This is acceptable if `/mcp` exposes only the same public Markdown already available through `/docs`.
|
||||||
|
|
||||||
|
Benefits:
|
||||||
|
|
||||||
|
1. lowest operational friction
|
||||||
|
2. fewer compatibility issues with MCP clients
|
||||||
|
3. no need to implement OAuth, mTLS, JWT validation, or custom auth middleware
|
||||||
|
4. consistent with the project assumption that documentation content is public
|
||||||
|
|
||||||
|
Risks:
|
||||||
|
|
||||||
|
1. random scraping, probing, or fuzzing of a machine endpoint
|
||||||
|
2. possible bandwidth or CPU nuisance traffic
|
||||||
|
3. accidental future exposure if new tools or private resources are added
|
||||||
|
4. less control over who can use the MCP endpoint
|
||||||
|
|
||||||
|
### Protecting `/mcp` With Cloudflare Access
|
||||||
|
|
||||||
|
Cloudflare Access can add a lightweight gate using GitHub, Google, one-time PIN, or service tokens.
|
||||||
|
|
||||||
|
Benefits:
|
||||||
|
|
||||||
|
1. reduces random internet traffic
|
||||||
|
2. requires little app code
|
||||||
|
3. works well for a small trusted team
|
||||||
|
4. provides logs and centralized access control
|
||||||
|
|
||||||
|
Costs:
|
||||||
|
|
||||||
|
1. browser-based login may not work with all MCP clients
|
||||||
|
2. non-browser MCP clients may need Cloudflare Access service tokens
|
||||||
|
3. adds operational configuration for a low-sensitivity endpoint
|
||||||
|
|
||||||
|
### Using mTLS
|
||||||
|
|
||||||
|
mTLS is useful when both client and server environments are tightly controlled.
|
||||||
|
|
||||||
|
Benefits:
|
||||||
|
|
||||||
|
1. strong client identity
|
||||||
|
2. good fit for service-to-service or private infrastructure
|
||||||
|
3. can be used between Cloudflare, Caddy, and the backend if desired
|
||||||
|
|
||||||
|
Costs:
|
||||||
|
|
||||||
|
1. harder certificate provisioning and rotation
|
||||||
|
2. weaker compatibility with normal MCP clients
|
||||||
|
3. unnecessary for public documentation-only content
|
||||||
|
|
||||||
|
For this project, mTLS is not the primary recommendation.
|
||||||
|
|
||||||
|
## Practical Recommendation
|
||||||
|
|
||||||
|
Use a simple public-docs posture unless the endpoint changes.
|
||||||
|
|
||||||
|
Recommended current setup:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/docs public
|
||||||
|
/mcp public or lightly protected
|
||||||
|
```
|
||||||
|
|
||||||
|
If `/mcp` remains public, add only basic operational safeguards:
|
||||||
|
|
||||||
|
1. keep Cloudflare Tunnel and Caddy in front
|
||||||
|
2. avoid publishing `8765` directly
|
||||||
|
3. enable Cloudflare or Caddy rate limiting if traffic becomes noisy
|
||||||
|
4. monitor logs for unusual request volume
|
||||||
|
5. document that MCP resources must remain safe to publish
|
||||||
|
|
||||||
|
A slightly stricter setup is also reasonable:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/docs public
|
||||||
|
/mcp Cloudflare Access or service token
|
||||||
|
```
|
||||||
|
|
||||||
|
This is the best option if the team wants to reduce drive-by MCP traffic without adding auth code to the application.
|
||||||
|
|
||||||
|
## Upgrade Trigger
|
||||||
|
|
||||||
|
Add real authentication before introducing any MCP capability that can:
|
||||||
|
|
||||||
|
1. read non-public files
|
||||||
|
2. access private notes or credentials
|
||||||
|
3. call upstream APIs
|
||||||
|
4. mutate data
|
||||||
|
5. run commands
|
||||||
|
6. expose environment details
|
||||||
|
7. perform expensive computation
|
||||||
|
|
||||||
|
At that point, prefer edge-level authentication first, such as Cloudflare Access, and consider proper OAuth 2.1 resource-server behavior only if broad public MCP client interoperability becomes a goal.
|
||||||
|
|
||||||
|
## Security Invariant
|
||||||
|
|
||||||
|
Everything exposed by the MCP server must be safe to publish publicly.
|
||||||
|
|
||||||
|
If that invariant stops being true, `/mcp` should be protected before the new capability is deployed.
|
||||||
@@ -0,0 +1,181 @@
|
|||||||
|
---
|
||||||
|
name: async-fastapi-sqlmodel
|
||||||
|
description: 'Explain and apply async database principles for FastAPI, SQLAlchemy 2.x, and SQLModel. Use when: learning or reviewing cached AsyncEngine and session-factory lifecycles, AsyncSession scopes and injection, FastAPI lifespan and yield dependencies, transaction boundaries, concurrency safety, implicit ORM I/O, pooling, testing, or SQLModel integration.'
|
||||||
|
---
|
||||||
|
|
||||||
|
# Async FastAPI, SQLAlchemy, and SQLModel
|
||||||
|
|
||||||
|
Use this skill to explain how an async database layer works, why the recommended patterns exist, and how to evaluate code against them. Teach the runtime model before suggesting implementation changes.
|
||||||
|
|
||||||
|
Primary targets: PostgreSQL with asyncpg and SQLite with aiosqlite.
|
||||||
|
|
||||||
|
Engine and session mechanics mirror the [`nicegui-db` template repository](https://forgejo.john-stream.com/john/nicegui-db). Treat that template as the implementation baseline, then explain the rationale, lifecycle constraints, and tradeoffs behind its cached engines, session factories, context managers, dependency wiring, and `with_session` decorator. Source-specific claims in the references link to the reviewed template commit so behavior remains auditable as the template evolves.
|
||||||
|
|
||||||
|
## When to Use
|
||||||
|
|
||||||
|
- Explain an async engine, session factory, session, connection, or transaction.
|
||||||
|
- Review FastAPI lifespan or dependency-based database management.
|
||||||
|
- Diagnose shared-session concurrency, implicit I/O, cleanup, or transaction problems.
|
||||||
|
- Compare SQLModel's model conveniences with SQLAlchemy's async runtime APIs.
|
||||||
|
- Decide whether a context manager, `AsyncExitStack`, eager loading, pooling option, or explicit transaction is appropriate.
|
||||||
|
|
||||||
|
## Outcome
|
||||||
|
|
||||||
|
Produce a focused technical explanation that:
|
||||||
|
|
||||||
|
- Defines the objects involved and identifies who owns each one.
|
||||||
|
- Traces acquisition, use, transaction behavior, and cleanup.
|
||||||
|
- Separates required invariants from defaults and situational choices.
|
||||||
|
- Explains failure modes and concurrency consequences.
|
||||||
|
- Uses a minimal canonical pattern when code clarifies the mechanics.
|
||||||
|
- Links claims to the relevant reference and upstream documentation.
|
||||||
|
|
||||||
|
Do not default to producing a project plan. Give sequencing advice only when the user explicitly asks for implementation steps.
|
||||||
|
|
||||||
|
## Mental Model
|
||||||
|
|
||||||
|
Keep three ownership scopes distinct:
|
||||||
|
|
||||||
|
| Scope | Object | Purpose | Typical owner |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Application process | Cached `AsyncEngine` and lifespan-owned `async_sessionmaker` | Dialect, connection pool, schema initialization, and repeatable session configuration | FastAPI lifespan |
|
||||||
|
| Request or concurrent task | `AsyncSession` | Mutable ORM identity map and transactional state | A `yield` dependency or explicit unit of work |
|
||||||
|
| Atomic operation | `SessionTransaction` | Commit all changes together or roll them back together | Service or use-case boundary |
|
||||||
|
|
||||||
|
The engine is a long-lived factory and pool, not a single database connection. The session is a mutable unit-of-work object, not a concurrency-safe global. A transaction is a consistency boundary, not merely a call to `commit()`.
|
||||||
|
|
||||||
|
## Core Principles
|
||||||
|
|
||||||
|
### Match lifetime to ownership
|
||||||
|
|
||||||
|
- Resolve one cached `AsyncEngine` per database URL during the active application lifecycle.
|
||||||
|
- Enter one owning engine scope per URL; initialize registered SQLModel metadata by default, then dispose the engine and clear cached resolution on exit.
|
||||||
|
- Configure the application `async_sessionmaker` inside the engine lifecycle; use the template's cached factory resolver only for standalone helpers that cannot receive the application factory.
|
||||||
|
- Close each session deterministically with `async with` or a FastAPI dependency that yields once.
|
||||||
|
|
||||||
|
See [engine lifecycle](references/engine.md) and [session management](references/session.md).
|
||||||
|
|
||||||
|
### Isolate mutable session state
|
||||||
|
|
||||||
|
An `AsyncSession` represents one stateful transaction in progress. Never use one session in multiple concurrent tasks, including branches of `asyncio.gather()`. Give each task its own session. Template-style `@with_session` functions inject one only when the `session` argument is omitted; a supplied session remains caller-owned, and explicit `None` is forwarded unchanged.
|
||||||
|
|
||||||
|
See [session management](references/session.md).
|
||||||
|
|
||||||
|
### Make I/O visible
|
||||||
|
|
||||||
|
Async ORM code must not unexpectedly issue SQL during ordinary attribute access. Load relationships and deferred columns explicitly with eager loader options such as `selectinload()`, use `awaitable_attrs` or `refresh()` for deliberate fallback loading, and consider `lazy="raise"` where accidental access should fail fast. `expire_on_commit=False` is a common async configuration because post-commit expiration can otherwise turn attribute reads into implicit I/O.
|
||||||
|
|
||||||
|
See [implicit ORM I/O](references/implicit_io.md).
|
||||||
|
|
||||||
|
### Put transactions around business invariants
|
||||||
|
|
||||||
|
Use `async with session.begin():` when several operations must commit or roll back as one unit. A successful exit flushes and commits; an exception rolls back. Reads still participate in SQLAlchemy's autobegin behavior unless the connection uses true DBAPI autocommit, so describe a path as read-only because of application intent and permissions, not because a session silently has no transaction.
|
||||||
|
|
||||||
|
Use `begin_nested()` only for a real SAVEPOINT requirement and account for backend-specific behavior. In SQLAlchemy 2.x, calling `session.commit()` commits the outermost transaction, not the current savepoint.
|
||||||
|
|
||||||
|
See [transaction boundaries](references/transactions.md).
|
||||||
|
|
||||||
|
### Keep framework boundaries explicit
|
||||||
|
|
||||||
|
FastAPI lifespan owns resources shared by many requests. A dependency with one `yield` owns request-scoped resources and runs cleanup after use. These are related context-manager mechanisms but solve different lifetime problems.
|
||||||
|
|
||||||
|
Use `AsyncExitStack` when lifespan acquires a variable, conditional, or mixed collection of context-managed resources. It records cleanup as resources are acquired and unwinds callbacks in reverse order. A single engine should use the direct engine context manager; `AsyncExitStack` is a composition tool, not a requirement.
|
||||||
|
|
||||||
|
See [FastAPI database integration](references/fastapi.md).
|
||||||
|
|
||||||
|
### Use SQLModel as the primary modeling layer
|
||||||
|
|
||||||
|
Default to SQLModel for table models and API data models in FastAPI applications. A SQLModel table model is also a SQLAlchemy model, and every SQLModel model is also a Pydantic model, so shared base models can reduce schema duplication while preserving access to SQLAlchemy's full ORM.
|
||||||
|
|
||||||
|
SQLModel does not replace SQLAlchemy's async engine, session, transaction, or loader mechanics. Its main tutorial currently demonstrates synchronous sessions and its advanced guide still lists comprehensive async documentation as future work. For async applications, combine SQLModel models and statements with SQLAlchemy's `AsyncSession` APIs. Use SQLAlchemy declarative models only when a concrete unsupported mapping or library constraint justifies the exception.
|
||||||
|
|
||||||
|
See [SQLModel integration](references/sqlmodel.md).
|
||||||
|
|
||||||
|
### Configure from evidence
|
||||||
|
|
||||||
|
Pool sizing, overflow, recycle, pre-ping, isolation, statement timeouts, and health checks depend on the driver, database, deployment concurrency, and failure model. Explain defaults and tradeoffs before recommending values. Avoid treating pool checkout as proof that a useful query can succeed.
|
||||||
|
|
||||||
|
See [observability and resilience](references/observability.md).
|
||||||
|
|
||||||
|
### Test through the production seam
|
||||||
|
|
||||||
|
Keep the production engine and session-factory construction path intact in tests. Select a dedicated PostgreSQL, local SQLite, or in-memory SQLite URL at that seam, then override the request-session dependency only for the test lifetime. Use a test-scoped outer transaction with SAVEPOINT-backed session commits when application code calls `commit()`; it exercises normal transaction behavior while cleanup remains deterministic.
|
||||||
|
|
||||||
|
In-memory SQLite is suitable for serial tests. For multiple simultaneous sessions, use a named shared-cache SQLite URL or a temporary file, and retain PostgreSQL integration coverage for PostgreSQL-specific behavior.
|
||||||
|
|
||||||
|
See [database testing and fixture data](references/testing.md).
|
||||||
|
|
||||||
|
## Reference Map
|
||||||
|
|
||||||
|
| Concept | Reference |
|
||||||
|
|---|---|
|
||||||
|
| Engine lifecycle and ownership | [Engine lifecycle reference](references/engine.md) |
|
||||||
|
| Session factory and scope | [Session management reference](references/session.md) |
|
||||||
|
| Transaction boundaries | [Transaction boundaries reference](references/transactions.md) |
|
||||||
|
| FastAPI lifespan composition | [FastAPI integration reference](references/fastapi.md) |
|
||||||
|
| FastAPI dependency injection | [FastAPI integration reference](references/fastapi.md) |
|
||||||
|
| Implicit I/O control in ORM | [Implicit I/O reference](references/implicit_io.md) |
|
||||||
|
| Observability and resilience | [Observability reference](references/observability.md) |
|
||||||
|
| SQLModel-first modeling | [SQLModel integration reference](references/sqlmodel.md) |
|
||||||
|
| CRUD repository and standalone functions | [Basic CRUD reference](references/crud.md) |
|
||||||
|
| Test database selection and fixture data | [Database testing reference](references/testing.md) |
|
||||||
|
|
||||||
|
## Canonical Composition Pattern
|
||||||
|
|
||||||
|
The framework-independent primitives live in [engine lifecycle](references/engine.md), [session management](references/session.md), and [transaction boundaries](references/transactions.md). Their canonical FastAPI adaptation, including lifespan state and `Annotated` dependencies, lives in [FastAPI database integration](references/fastapi.md).
|
||||||
|
|
||||||
|
For background work that outlives a request, inject the shared factory and create a new session inside that task instead of retaining the request's session.
|
||||||
|
|
||||||
|
## Explanation Procedure
|
||||||
|
|
||||||
|
1. Identify the exact concept or observed behavior in question.
|
||||||
|
2. Name the owning scope: application, request/task, or transaction.
|
||||||
|
3. Trace what state the object holds and where actual database I/O can occur.
|
||||||
|
4. Explain normal entry, successful exit, exceptional exit, and concurrent use.
|
||||||
|
5. Distinguish an invariant from a recommended default or backend-specific choice.
|
||||||
|
6. Load only the matching reference documents and cite upstream sources.
|
||||||
|
7. Show the smallest useful code pattern or contrast when prose is insufficient.
|
||||||
|
8. End with concrete checks the reader can use to inspect their own code.
|
||||||
|
|
||||||
|
When reviewing code, verify:
|
||||||
|
|
||||||
|
- The URL uses an asyncio-compatible dialect.
|
||||||
|
- Engine creation and disposal have one clear owner.
|
||||||
|
- Every session has a bounded lifetime and is not shared across tasks.
|
||||||
|
- Transaction boundaries match business invariants and exception behavior.
|
||||||
|
- Relationship and deferred-column access cannot surprise the event loop with implicit I/O.
|
||||||
|
- Pool and timeout settings are justified by deployment behavior.
|
||||||
|
- Tests exercise rollback, cleanup, concurrency, and lifespan behavior where relevant.
|
||||||
|
- Tests use a dedicated database target and preserve production session mechanics.
|
||||||
|
|
||||||
|
## Anti-Patterns to Flag
|
||||||
|
|
||||||
|
- Creating engines inside request handlers.
|
||||||
|
- Sharing one AsyncSession across concurrent tasks.
|
||||||
|
- Implicit commit/rollback behavior with unclear ownership.
|
||||||
|
- Global mutable session state.
|
||||||
|
- Lifespan cleanup that depends on implicit garbage collection.
|
||||||
|
- Treating `AsyncExitStack` as mandatory for a fixed single resource.
|
||||||
|
- Treating SQLModel's synchronous tutorial examples as the async runtime pattern.
|
||||||
|
- Allowing lazy relationship access to hide database I/O.
|
||||||
|
- Copying pool settings without relating them to worker count and database capacity.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
Answer in the shape best suited to the question, usually:
|
||||||
|
|
||||||
|
1. Direct explanation.
|
||||||
|
2. Underlying lifecycle or transaction mechanics.
|
||||||
|
3. Required invariants and situational tradeoffs.
|
||||||
|
4. Minimal example or code-review findings when useful.
|
||||||
|
5. Verification questions and source links.
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
|
- [SQLAlchemy transaction management](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html)
|
||||||
|
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
||||||
|
- [FastAPI dependencies with `yield`](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
|
||||||
|
- [Python `AsyncExitStack`](https://docs.python.org/3/library/contextlib.html#contextlib.AsyncExitStack)
|
||||||
|
- [SQLModel session dependency pattern](https://sqlmodel.tiangolo.com/tutorial/fastapi/session-with-dependency/)
|
||||||
@@ -0,0 +1,294 @@
|
|||||||
|
# Basic CRUD Repository and Functions
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [SQLModel create-data tutorial](https://sqlmodel.tiangolo.com/tutorial/fastapi/multiple-models/)
|
||||||
|
- [SQLModel update-data tutorial](https://sqlmodel.tiangolo.com/tutorial/fastapi/update-extra-data/)
|
||||||
|
- [SQLModel select tutorial](https://sqlmodel.tiangolo.com/tutorial/select/)
|
||||||
|
- [SQLAlchemy `AsyncSession` API](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.AsyncSession)
|
||||||
|
- [`nicegui-db` service functions](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/services/my_table.py)
|
||||||
|
|
||||||
|
??? abstract "Decision metadata"
|
||||||
|
- Status: adopted
|
||||||
|
- Decision level: advisory
|
||||||
|
- Applies to: api-runtime, workers, tests
|
||||||
|
- Last reviewed: 2026-08-06
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Show a small SQLModel CRUD layer in two forms:
|
||||||
|
|
||||||
|
- independent functions for convenient standalone or composed operations;
|
||||||
|
- a repository object that groups those functions behind one domain-oriented interface.
|
||||||
|
|
||||||
|
Template-style public functions use `@with_session` and accept an optional `AsyncSession`. When the argument is omitted, the decorator resolves the cached session factory and owns a short-lived session. When supplied, the function borrows the session without controlling its lifetime or transaction. The decorator does not commit, so standalone writes need a visible transaction strategy; repository methods remain explicit-session operations for predictable composition.
|
||||||
|
|
||||||
|
Use the same vocabulary at every layer:
|
||||||
|
|
||||||
|
| Operation | Function | Repository method | Scope when session is omitted | Missing-row result |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| Create | `create_widget()` | `create()` | Owned session; no implicit commit | Not applicable |
|
||||||
|
| Read one | `get_widget()` | `get()` | Owned session | `None` |
|
||||||
|
| Read many | `list_widgets()` | `list()` | Owned session | Empty list |
|
||||||
|
| Update | `update_widget()` | `update()` | Owned session; no implicit commit | `None` |
|
||||||
|
| Delete | `delete_widget()` | `delete()` | Owned session; no implicit commit | `None` |
|
||||||
|
|
||||||
|
Functions and repository methods both put domain arguments first. Database configuration and sessions are keyword-only infrastructure arguments. This keeps call sites analogous and makes ownership choices visible.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Models
|
||||||
|
|
||||||
|
Start with one table model when the application does not need distinct persistence and API schemas.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from sqlmodel import Field
|
||||||
|
from sqlmodel import SQLModel
|
||||||
|
|
||||||
|
|
||||||
|
class Widget(SQLModel, table=True):
|
||||||
|
id: int | None = Field(default=None, primary_key=True)
|
||||||
|
name: str = Field(index=True)
|
||||||
|
description: str | None = None
|
||||||
|
```
|
||||||
|
|
||||||
|
This reference uses direct field arguments and full-update semantics to keep the CRUD mechanics visible. Introduce separate create, update, or public schemas only when an API boundary needs different validation, field visibility, or partial-update behavior. See [SQLModel integration](sqlmodel.md) for that larger modeling pattern.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Independent CRUD Functions
|
||||||
|
|
||||||
|
Functions are the simplest default when grouping state or behavior in an object adds no value. Decorate public service functions when both standalone reads and explicit composition are useful. The assertion narrows the optional type after decorator injection and catches accidental explicit `None` calls.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from sqlmodel import select
|
||||||
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
|
from .session import with_session
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
|
async def create_widget(
|
||||||
|
name: str,
|
||||||
|
description: str | None = None,
|
||||||
|
*,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> Widget:
|
||||||
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
|
widget = Widget(name=name, description=description)
|
||||||
|
session.add(widget)
|
||||||
|
await session.flush()
|
||||||
|
return widget
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
|
async def get_widget(
|
||||||
|
widget_id: int,
|
||||||
|
*,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> Widget | None:
|
||||||
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
|
return await session.get(Widget, widget_id)
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
|
async def list_widgets(
|
||||||
|
*,
|
||||||
|
offset: int = 0,
|
||||||
|
limit: int = 100,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> list[Widget]:
|
||||||
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
|
if offset < 0:
|
||||||
|
raise ValueError("offset must be non-negative")
|
||||||
|
if not 1 <= limit <= 100:
|
||||||
|
raise ValueError("limit must be between 1 and 100")
|
||||||
|
|
||||||
|
statement = select(Widget).order_by(Widget.id).offset(offset).limit(limit)
|
||||||
|
return list(await session.scalars(statement))
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
|
async def update_widget(
|
||||||
|
widget_id: int,
|
||||||
|
name: str,
|
||||||
|
description: str | None,
|
||||||
|
*,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> Widget | None:
|
||||||
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
|
widget = await session.get(Widget, widget_id)
|
||||||
|
if widget is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
widget.name = name
|
||||||
|
widget.description = description
|
||||||
|
await session.flush()
|
||||||
|
return widget
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
|
async def delete_widget(
|
||||||
|
widget_id: int,
|
||||||
|
*,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> Widget | None:
|
||||||
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
|
widget = await session.get(Widget, widget_id)
|
||||||
|
if widget is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
await session.delete(widget)
|
||||||
|
await session.flush()
|
||||||
|
return widget
|
||||||
|
```
|
||||||
|
|
||||||
|
Update and delete load the row through the same session that mutates it. This avoids accepting detached instances from an earlier standalone read and gives both operations an explicit `None` result that the application layer can map to a domain or HTTP error. Delete returns the loaded object for callers that need its values, but that object represents a row scheduled for deletion and must not be reused as persistent state. List operations validate their bounds and order by the primary key so pagination is deterministic. Add a unique tiebreaker whenever ordering by a non-unique field.
|
||||||
|
|
||||||
|
`flush()` sends pending writes and populates ordinary generated primary keys. It does not itself commit. A decorated write called without a session will therefore roll back when its owned session closes unless the function explicitly commits. Prefer passing a transaction-scoped session so several writes compose atomically. Use `await session.refresh(widget)` only when the operation deliberately needs database-generated state that was not returned during the flush; an unconditional refresh adds another query.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Repository Object
|
||||||
|
|
||||||
|
A repository can provide a stable domain-facing interface when several callers need the same grouped operations. It remains stateless here: every method requires a session and delegates to the analogous function.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
|
class WidgetRepository:
|
||||||
|
async def create(
|
||||||
|
self,
|
||||||
|
session: AsyncSession,
|
||||||
|
name: str,
|
||||||
|
description: str | None = None,
|
||||||
|
) -> Widget:
|
||||||
|
return await create_widget(
|
||||||
|
name,
|
||||||
|
description,
|
||||||
|
session=session,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def get(
|
||||||
|
self,
|
||||||
|
session: AsyncSession,
|
||||||
|
widget_id: int,
|
||||||
|
) -> Widget | None:
|
||||||
|
return await get_widget(widget_id, session=session)
|
||||||
|
|
||||||
|
async def list(
|
||||||
|
self,
|
||||||
|
session: AsyncSession,
|
||||||
|
*,
|
||||||
|
offset: int = 0,
|
||||||
|
limit: int = 100,
|
||||||
|
) -> list[Widget]:
|
||||||
|
return await list_widgets(
|
||||||
|
offset=offset,
|
||||||
|
limit=limit,
|
||||||
|
session=session,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def update(
|
||||||
|
self,
|
||||||
|
session: AsyncSession,
|
||||||
|
widget_id: int,
|
||||||
|
name: str,
|
||||||
|
description: str | None,
|
||||||
|
) -> Widget | None:
|
||||||
|
return await update_widget(
|
||||||
|
widget_id,
|
||||||
|
name,
|
||||||
|
description,
|
||||||
|
session=session,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def delete(
|
||||||
|
self,
|
||||||
|
session: AsyncSession,
|
||||||
|
widget_id: int,
|
||||||
|
) -> Widget | None:
|
||||||
|
return await delete_widget(widget_id, session=session)
|
||||||
|
```
|
||||||
|
|
||||||
|
The object is intentionally thin. Tests pass a transaction-scoped test session directly. The caller always owns that session and its transaction, and the repository never closes or commits it.
|
||||||
|
|
||||||
|
If a read participates in a later write, pass the same session and place both operations inside the explicit transaction. This avoids splitting one use case across sessions and keeps SQLAlchemy's autobegin behavior from obscuring transaction ownership. Add a repository only when its naming, shared query policy, dependency substitution, or domain boundary improves the application. Independent functions remain a valid and often clearer design.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Transaction Ownership
|
||||||
|
|
||||||
|
Compose multiple calls under one use-case transaction. `db_transaction_scope()` owns the standalone engine, factory, session, and transaction lifetimes. Decorated CRUD functions detect the supplied session and borrow it; repository methods receive it directly.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from .session import db_transaction_scope
|
||||||
|
|
||||||
|
|
||||||
|
async def replace_widget(
|
||||||
|
repository: WidgetRepository,
|
||||||
|
widget_id: int,
|
||||||
|
replacement_name: str,
|
||||||
|
replacement_description: str | None = None,
|
||||||
|
) -> Widget | None:
|
||||||
|
async with db_transaction_scope() as active_session:
|
||||||
|
deleted_widget = await repository.delete(
|
||||||
|
active_session,
|
||||||
|
widget_id,
|
||||||
|
)
|
||||||
|
if deleted_widget is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
return await repository.create(
|
||||||
|
active_session,
|
||||||
|
replacement_name,
|
||||||
|
replacement_description,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
If creation fails, deletion rolls back with it. Inside an already-running application, prefer `async with session_factory.begin()` or `async with session.begin()` over `db_transaction_scope()` so the application-owned engine and factory remain in use. Do not add direct `commit()` calls to CRUD functions or repository methods because that prevents callers from composing several operations atomically. See [transaction boundaries](transactions.md) and [session management](session.md) for ownership details.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Anti-Patterns
|
||||||
|
|
||||||
|
- Storing one mutable `AsyncSession` on a long-lived repository object.
|
||||||
|
- Creating sessions manually inside functions already using `@with_session`.
|
||||||
|
- Passing database configuration through every CRUD call instead of injecting a session at the data-access boundary.
|
||||||
|
- Assuming decorator-owned write sessions commit on close.
|
||||||
|
- Forwarding explicit `session=None` when decorator injection was intended.
|
||||||
|
- Accepting unbounded list queries.
|
||||||
|
- Accepting detached ORM instances for update or delete when an identifier can be resolved in the active session.
|
||||||
|
- Accessing unloaded attributes after a standalone repository read has closed its owned session.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Operational Checks
|
||||||
|
|
||||||
|
- Every CRUD call receives a task-local `AsyncSession`.
|
||||||
|
- Standalone reads create and close a session at the service or application boundary.
|
||||||
|
- Standalone reads may omit `session`; decorated writes receive a transaction-scoped session or explicitly own their commit policy.
|
||||||
|
- Supplied write sessions remain caller-owned.
|
||||||
|
- Each complete write operation declares a visible transaction boundary.
|
||||||
|
- List operations have pagination and deterministic ordering where required.
|
||||||
|
- Update requires values for both mutable fields; passing `None` explicitly clears the nullable description.
|
||||||
|
- Get, update, and delete use the same identifier and missing-row semantics.
|
||||||
|
- Decorated functions accept an optional keyword-only session; repository methods require one explicitly.
|
||||||
|
- Standalone service reads load all state needed after their owned session closes.
|
||||||
|
- Repository objects hold query policy when useful, never database configuration or request-scoped session state.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing Checks
|
||||||
|
|
||||||
|
- Create tests verify generated identifiers and persisted field values after commit.
|
||||||
|
- Get and list tests cover found, missing, pagination, and ordering behavior.
|
||||||
|
- List tests reject negative offsets and limits outside the supported range.
|
||||||
|
- Update tests cover replacement of both mutable fields, including clearing the nullable description.
|
||||||
|
- Update and delete tests cover missing identifiers without mutating the database.
|
||||||
|
- Delete tests verify the returned row and its absence after commit.
|
||||||
|
- Failure tests verify that a surrounding transaction rolls back all composed CRUD calls.
|
||||||
|
- Optional-session read tests verify borrowed sessions remain open and owned sessions close without committing.
|
||||||
|
- Decorated write tests verify supplied transactions remain caller-owned and omitted sessions do not imply a commit.
|
||||||
|
- Composition tests pass one active session through several CRUD calls and verify one atomic commit or rollback.
|
||||||
@@ -0,0 +1,281 @@
|
|||||||
|
# Async SQLAlchemy Engine
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
|
||||||
|
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||||
|
- [SQLAlchemy connections](https://docs.sqlalchemy.org/en/21/core/connections.html)
|
||||||
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
|
- [SQLAlchemy pooling and multiprocessing](https://docs.sqlalchemy.org/en/21/core/pooling.html#pooling-multiprocessing)
|
||||||
|
- [SQLAlchemy SQLite transaction control](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#enabling-non-legacy-sqlite-transactional-modes-with-the-sqlite3-or-aiosqlite-driver)
|
||||||
|
- [SQLAlchemy SQLite foreign-key support](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#foreign-key-support)
|
||||||
|
- [SQLite PRAGMA reference](https://www.sqlite.org/pragma.html)
|
||||||
|
- [`nicegui-db` engine implementation](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/db/engine.py)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Engine Ownership Model
|
||||||
|
|
||||||
|
Resolve one async engine for each database URL within an application, worker, command, or test lifecycle.
|
||||||
|
|
||||||
|
- SQLAlchemy guidance: the engine is intended as a long-lived, concurrent registry over pooled DB connections, not a per-operation object.
|
||||||
|
- `get_engine(database_url)` owns URL-keyed engine construction and caching.
|
||||||
|
- The composition root enters `engine_scope(database_url)` once and therefore owns initialization and disposal.
|
||||||
|
- Services and repositories receive a session or session factory; they do not resolve an engine.
|
||||||
|
|
||||||
|
!!! tip "Practical rule"
|
||||||
|
- Exactly one cached engine for each database URL during an active application-owned lifecycle.
|
||||||
|
- Exactly one active owning `engine_scope()` for a given URL.
|
||||||
|
- Zero `create_async_engine(...)` calls in feature code.
|
||||||
|
- Zero engine lookup or disposal calls in repository code.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Cached Engine Resolution
|
||||||
|
|
||||||
|
[`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) makes the database URL the engine identity:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from functools import cache
|
||||||
|
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
|
|
||||||
|
|
||||||
|
@cache
|
||||||
|
def get_engine(database_url: str) -> AsyncEngine:
|
||||||
|
engine = create_async_engine(database_url, pool_pre_ping=True)
|
||||||
|
if engine.dialect.name == "sqlite":
|
||||||
|
configure_aiosqlite_engine(engine)
|
||||||
|
return engine
|
||||||
|
```
|
||||||
|
|
||||||
|
Repeated calls with the same exact URL return the same `AsyncEngine`; different URLs produce independent cache entries. Construction configures the dialect and pool but normally does not open a database connection until the first operation. SQLite event listeners are installed only when a new cached engine is constructed, before its first connection.
|
||||||
|
|
||||||
|
Resolve settings into the final URL before calling `get_engine()`. Services and repositories should not call it directly: the cache controls construction identity, not ownership.
|
||||||
|
|
||||||
|
## Owning Engine Scope
|
||||||
|
|
||||||
|
Use one [`asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager) to pair cached resolution and optional schema initialization with disposal:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
from sqlmodel import SQLModel
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def engine_scope(
|
||||||
|
database_url: str,
|
||||||
|
*,
|
||||||
|
initialize: bool = True,
|
||||||
|
) -> AsyncGenerator[AsyncEngine]:
|
||||||
|
engine = get_engine(database_url)
|
||||||
|
if initialize:
|
||||||
|
await initialize_db(database_url)
|
||||||
|
|
||||||
|
try:
|
||||||
|
yield engine
|
||||||
|
finally:
|
||||||
|
await dispose_engine(database_url)
|
||||||
|
|
||||||
|
|
||||||
|
async def initialize_db(database_url: str) -> None:
|
||||||
|
from . import models # noqa: F401
|
||||||
|
|
||||||
|
engine = get_engine(database_url)
|
||||||
|
async with engine.begin() as connection:
|
||||||
|
await connection.run_sync(SQLModel.metadata.create_all)
|
||||||
|
|
||||||
|
|
||||||
|
async def dispose_engine(database_url: str) -> None:
|
||||||
|
engine = get_engine(database_url)
|
||||||
|
try:
|
||||||
|
await engine.dispose()
|
||||||
|
finally:
|
||||||
|
get_engine.cache_clear()
|
||||||
|
```
|
||||||
|
|
||||||
|
The code that enters `engine_scope()` owns the engine. It keeps that scope open for the complete application, worker, command, or test lifecycle and passes the yielded engine into session-factory construction. Successful and exceptional exits both dispose the pool and invalidate cached engine resolution.
|
||||||
|
|
||||||
|
Initialization imports the model package so every table is registered, then runs `SQLModel.metadata.create_all()` in `engine.begin()`. This is suitable for the template and focused tests. Use migrations instead when schema evolution is part of the deployment contract. Pass `initialize=False` only when another owner provisions the schema or a test is directly exercising construction without schema setup.
|
||||||
|
|
||||||
|
`dispose_engine()` clears the complete function cache, not only the requested URL. This matches the template and is safe under its intended single-database lifecycle. Applications that own several simultaneously active database URLs need per-key lifecycle management rather than this global invalidation behavior.
|
||||||
|
|
||||||
|
Workers, scripts, and other composition roots enter `database_scope()` directly:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async with database_scope(settings.database_url) as session_factory:
|
||||||
|
await run_worker(session_factory)
|
||||||
|
```
|
||||||
|
|
||||||
|
`database_scope()` is defined in [session management](session.md). It enters `engine_scope()` and creates the factory bound to the yielded engine.
|
||||||
|
|
||||||
|
Do not overlap two owning scopes for the same URL. Both resolve the same cached engine, and the first scope to exit disposes it and clears the cache while the other still refers to it. For several fixed databases, use one non-overlapping owner per URL and account for global cache invalidation; use [`AsyncExitStack`](https://docs.python.org/3/library/contextlib.html#contextlib.AsyncExitStack) only after adopting lifecycle semantics that support several simultaneous owners.
|
||||||
|
|
||||||
|
When directly testing engine construction or lifecycle behavior, enter `engine_scope()` in the test or fixture. Exiting the context disposes the engine even when the test fails and clears the cache for the next lifecycle.
|
||||||
|
|
||||||
|
See [FastAPI database integration](fastapi.md) for adapting `database_scope()` to application lifespan and dependency injection.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Driver URLs (Project Requirement: asyncpg + aiosqlite)
|
||||||
|
|
||||||
|
Use SQLAlchemy async driver URLs:
|
||||||
|
|
||||||
|
- PostgreSQL: `postgresql+asyncpg://user:pass@host:5432/dbname`
|
||||||
|
- SQLite: `sqlite+aiosqlite:///./app.db`
|
||||||
|
|
||||||
|
!!! warning "Driver compatibility"
|
||||||
|
- Do not mix sync drivers, for example `psycopg2`, with `create_async_engine()`.
|
||||||
|
- Keep URL construction centralized in settings/config, not in feature modules.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## SQLite Connection and Transaction Policy
|
||||||
|
|
||||||
|
SQLite settings do not form one indivisible bundle:
|
||||||
|
|
||||||
|
- `PRAGMA foreign_keys=ON` is a correctness requirement when the schema declares foreign keys. SQLite requires it on every connection, including the connection used by `metadata.create_all()`.
|
||||||
|
- Disabling the driver's implicit `BEGIN` and emitting `BEGIN` from SQLAlchemy provides non-legacy transaction behavior for `aiosqlite`. This makes SELECT, DDL, and SAVEPOINT behavior participate in SQLAlchemy's transaction boundary consistently.
|
||||||
|
- `PRAGMA busy_timeout` is a per-connection lock-wait policy. Choose the duration from the application's latency and contention requirements.
|
||||||
|
- `PRAGMA journal_mode=WAL` is an optional file-database concurrency policy. WAL persists in the database file, cannot be enabled for an in-memory database, and is not a substitute for transaction control.
|
||||||
|
|
||||||
|
Install instance-level listeners exactly once, immediately after constructing an `aiosqlite` engine and before its first connection:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from sqlalchemy import event
|
||||||
|
from sqlalchemy.engine import Connection
|
||||||
|
from sqlalchemy.engine.interfaces import DBAPIConnection
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
|
||||||
|
|
||||||
|
def configure_aiosqlite_engine(
|
||||||
|
engine: AsyncEngine,
|
||||||
|
*,
|
||||||
|
busy_timeout_ms: int | None = 30_000,
|
||||||
|
enable_wal: bool = False,
|
||||||
|
) -> None:
|
||||||
|
if engine.dialect.name != "sqlite" or engine.dialect.driver != "aiosqlite":
|
||||||
|
raise ValueError("Expected a sqlite+aiosqlite engine")
|
||||||
|
if busy_timeout_ms is not None and busy_timeout_ms < 0:
|
||||||
|
raise ValueError("busy_timeout_ms must be non-negative")
|
||||||
|
|
||||||
|
@event.listens_for(engine.sync_engine, "connect")
|
||||||
|
def configure_connection(dbapi_connection: DBAPIConnection, _: object) -> None:
|
||||||
|
dbapi_connection.isolation_level = None
|
||||||
|
cursor = dbapi_connection.cursor()
|
||||||
|
try:
|
||||||
|
cursor.execute("PRAGMA foreign_keys=ON")
|
||||||
|
if busy_timeout_ms is not None:
|
||||||
|
cursor.execute(f"PRAGMA busy_timeout={busy_timeout_ms}")
|
||||||
|
if enable_wal:
|
||||||
|
cursor.execute("PRAGMA journal_mode=WAL")
|
||||||
|
journal_mode = cursor.fetchone()
|
||||||
|
if journal_mode is None or journal_mode[0].lower() != "wal":
|
||||||
|
raise RuntimeError("SQLite could not enable WAL mode")
|
||||||
|
finally:
|
||||||
|
cursor.close()
|
||||||
|
|
||||||
|
@event.listens_for(engine.sync_engine, "begin")
|
||||||
|
def begin_transaction(connection: Connection) -> None:
|
||||||
|
connection.exec_driver_sql("BEGIN")
|
||||||
|
```
|
||||||
|
|
||||||
|
The `connect` listener receives the adapted synchronous DBAPI connection exposed by `engine.sync_engine`; event callbacks themselves are synchronous even though application queries use the async engine. Setting `isolation_level=None` and adding the `begin` listener are one transaction-control strategy and must remain paired. Do not combine this pair with SQLAlchemy's driver-level `AUTOCOMMIT` isolation mode.
|
||||||
|
|
||||||
|
The default above enables foreign keys and modern transaction boundaries for file and in-memory databases. Enable WAL only for a file-backed database after deciding that its read/write concurrency model is appropriate. Treat `30_000` as an example policy, not a universal default; `connect_args={"timeout": 30.0}` at engine construction is another way to configure the underlying SQLite lock timeout.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Pooling Defaults and Tuning
|
||||||
|
|
||||||
|
Default behavior is usually correct first:
|
||||||
|
|
||||||
|
- Async engines use async-compatible pooling (`AsyncAdaptedQueuePool`) by default.
|
||||||
|
- Start with defaults, then tune from observed load (`pool_size`, `max_overflow`, `pool_timeout`, `pool_recycle`).
|
||||||
|
- Enable `pool_pre_ping=True` for safer stale-connection handling in long-running services.
|
||||||
|
|
||||||
|
When to switch pool strategy:
|
||||||
|
|
||||||
|
- `NullPool` if you explicitly need no pooling (special environments, some tests, or strict cross-loop constraints).
|
||||||
|
- Keep in mind this increases connect/disconnect churn.
|
||||||
|
|
||||||
|
### When `StaticPool` Is Appropriate
|
||||||
|
|
||||||
|
Use [`StaticPool`](https://docs.sqlalchemy.org/en/21/core/pooling.html#sqlalchemy.pool.StaticPool) only when every checkout must reuse one DBAPI connection and all database access is serialized. Typical cases are:
|
||||||
|
|
||||||
|
- A serial test suite using a private in-memory SQLite database. The `sqlite+aiosqlite://` URL already selects `StaticPool` automatically, so specifying `poolclass=StaticPool` is normally redundant.
|
||||||
|
- A narrowly scoped SQLite engine that must preserve connection-local state, such as temporary tables, across SQLAlchemy connection or session checkouts.
|
||||||
|
|
||||||
|
When explicit configuration is required:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
|
from sqlalchemy.pool import StaticPool
|
||||||
|
|
||||||
|
engine = create_async_engine(
|
||||||
|
"sqlite+aiosqlite:///./test.db",
|
||||||
|
poolclass=StaticPool,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
`StaticPool` is not a general performance optimization or a way to make SQLite concurrent. All sessions share one underlying connection and its single transaction state, so one session's `COMMIT` or `ROLLBACK` can interfere with another session. Do not use it when several sessions or tasks may access the engine concurrently. For concurrent in-memory work, use a named shared-cache SQLite URL so pooled connections have independent transaction state, or use a temporary file database. See [SQLite test targets](testing.md#sqlite-targets) for those patterns.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Disposal Semantics
|
||||||
|
|
||||||
|
`dispose_engine(database_url)` resolves the cached engine, awaits `engine.dispose()`, and clears the engine cache in a `finally` block. `engine.dispose()` replaces/disposes the pool, but only checked-in connections are immediately closed.
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- Dispose when the app is shutting down.
|
||||||
|
- Clear cached resolution even when disposal raises, so a later lifecycle cannot receive the failed engine object.
|
||||||
|
- Dispose before reusing an engine across event loops.
|
||||||
|
- In forked child-process initialization, use `engine.dispose(close=False)` (sync API guidance) so child processes do not touch parent-held connections.
|
||||||
|
|
||||||
|
Avoid relying on garbage collection for engine cleanup in async code.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Event Loop and Process Boundaries
|
||||||
|
|
||||||
|
Do not share pooled connections across boundaries:
|
||||||
|
|
||||||
|
- Multiple event loops: do not reuse the same pooled async engine across loops unless you intentionally disable pooling (`NullPool`) or dispose before handoff.
|
||||||
|
- Multiprocessing/fork: pooled connections must not be inherited for active use across process boundaries.
|
||||||
|
|
||||||
|
This prevents broken socket state and cross-process connection corruption.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What Not to Do
|
||||||
|
|
||||||
|
- Create an engine inside each operation or unit of work.
|
||||||
|
- Create/dispose engines inside repository methods.
|
||||||
|
- Resolve an engine from repositories instead of injecting a session dependency.
|
||||||
|
- Keep engine creation as a hidden side effect of import-time module globals.
|
||||||
|
- Keep a session factory alive after its bound engine scope exits.
|
||||||
|
- Enter overlapping engine scopes for the same cached URL.
|
||||||
|
- Treat `cache_clear()` as per-URL invalidation when it clears every cached engine.
|
||||||
|
- Use `metadata.create_all()` as a substitute for required production migrations.
|
||||||
|
- Install the same SQLite event listeners more than once on one engine.
|
||||||
|
- Enable WAL blindly for in-memory SQLite or treat a busy timeout as a concurrency guarantee.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Engine Design Checklist
|
||||||
|
|
||||||
|
- One cached engine per exact database URL during an active lifecycle.
|
||||||
|
- One owning engine scope per URL, with no overlapping owners.
|
||||||
|
- Cached resolution, optional initialization, disposal, and cache invalidation follow one framework-independent lifecycle.
|
||||||
|
- The composition root enters the database scope once and keeps it open until shutdown.
|
||||||
|
- Session factory created inside, and never outlives, its engine scope.
|
||||||
|
- Model registration occurs before `metadata.create_all()` when initialization is enabled.
|
||||||
|
- Async driver URL matches backend (`asyncpg` or `aiosqlite`).
|
||||||
|
- `aiosqlite` foreign-key and transaction listeners installed once before first use.
|
||||||
|
- WAL enabled only as an explicit policy for a file-backed SQLite database.
|
||||||
|
- Pooling strategy is explicit for non-default needs.
|
||||||
|
- No feature-path engine creation.
|
||||||
|
- Tests enter the same scope and receive deterministic disposal plus cache cleanup.
|
||||||
@@ -0,0 +1,192 @@
|
|||||||
|
# FastAPI Database Integration
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
||||||
|
- [FastAPI dependencies with `yield`](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
|
||||||
|
- [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
||||||
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
|
- [`nicegui-db` application lifespan](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/ui/app.py)
|
||||||
|
- [`nicegui-db` database dependencies](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/ui/dependency.py)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Connect the framework-independent database tools to FastAPI:
|
||||||
|
|
||||||
|
- lifespan enters one application-owned `database_scope()`,
|
||||||
|
- application state holds settings and the resulting session factory,
|
||||||
|
- dependencies create one session per request,
|
||||||
|
- `Annotated` aliases make route ownership concise and explicit.
|
||||||
|
|
||||||
|
The underlying resource and transaction rules remain in [engine lifecycle](engine.md), [session management](session.md), and [transaction boundaries](transactions.md).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Lifespan Ownership
|
||||||
|
|
||||||
|
Enter `database_scope()` once for the complete application lifecycle. Store the session factory, not the engine, because request code needs sessions rather than direct pool access:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from fastapi import FastAPI
|
||||||
|
|
||||||
|
from .config import Settings
|
||||||
|
from .config import get_database_url
|
||||||
|
from .db import database_scope
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(settings: Settings, app: FastAPI) -> AsyncGenerator[None]:
|
||||||
|
app.state.settings = settings
|
||||||
|
db_url = get_database_url(settings)
|
||||||
|
|
||||||
|
try:
|
||||||
|
async with database_scope(db_url) as session_factory:
|
||||||
|
app.state.session_factory = session_factory
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
del app.state.settings
|
||||||
|
del app.state.session_factory
|
||||||
|
```
|
||||||
|
|
||||||
|
The application factory binds `settings` to lifespan, for example with `partial(lifespan, settings)`. Lifespan does not construct resources per request. It enters the same framework-independent scope used by scripts, workers, and tests, keeps that scope open while requests are served, and lets it dispose the engine and clear cached engine resolution during shutdown.
|
||||||
|
|
||||||
|
The template's unconditional `del app.state.session_factory` mirrors an expected successful startup. If `database_scope()` raises before assignment, cleanup can raise `AttributeError` and obscure the startup error. A production hardening option is to assign a sentinel before the `try` or delete conditionally; that changes failure behavior and is not part of the exact template mechanics.
|
||||||
|
|
||||||
|
Only store the engine too when application-level code genuinely needs direct Core operations, pool instrumentation, or engine-specific diagnostics. Routes and repositories should normally receive an `AsyncSession`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Session Factory Dependency
|
||||||
|
|
||||||
|
A synchronous dependency retrieves the already-created factory from application state:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from typing import Annotated
|
||||||
|
|
||||||
|
from fastapi import Depends
|
||||||
|
from fastapi import Request
|
||||||
|
|
||||||
|
from .session import SessionFactory
|
||||||
|
|
||||||
|
|
||||||
|
def _get_session_factory(request: Request) -> SessionFactory:
|
||||||
|
return request.app.state.session_factory
|
||||||
|
|
||||||
|
|
||||||
|
type SessionFactoryDep = Annotated[SessionFactory, Depends(_get_session_factory)]
|
||||||
|
```
|
||||||
|
|
||||||
|
`Depends()` does not create or cache a factory here. It only exposes the lifespan-owned object. This function is also the narrow seam that tests can override when they need a different factory.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Request Session Dependencies
|
||||||
|
|
||||||
|
Use a session-only dependency for reads and other request conversations that must not commit implicitly:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
|
||||||
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
|
|
||||||
|
async def _get_session(session_factory: SessionFactoryDep) -> AsyncGenerator[AsyncSession]:
|
||||||
|
async with session_factory() as owned_session:
|
||||||
|
yield owned_session
|
||||||
|
|
||||||
|
|
||||||
|
type SessionDep = Annotated[AsyncSession, Depends(_get_session)]
|
||||||
|
```
|
||||||
|
|
||||||
|
The dependency creates and closes one session per request. Closing rolls back any unfinished autobegun transaction; it does not commit.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Route Usage
|
||||||
|
|
||||||
|
Read route:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@router.get("/items/{item_id}")
|
||||||
|
async def get_item(item_id: int, session: SessionDep) -> Item | None:
|
||||||
|
return await find_item(session, item_id)
|
||||||
|
```
|
||||||
|
|
||||||
|
Write route:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@router.post("/items")
|
||||||
|
async def create_item(payload: ItemCreate, session: SessionDep) -> Item:
|
||||||
|
async with session.begin():
|
||||||
|
return await insert_item(session, payload)
|
||||||
|
```
|
||||||
|
|
||||||
|
The template exposes only `SessionDep`; it does not hide commit behavior in dependency teardown. Choose one visible write convention per application:
|
||||||
|
|
||||||
|
- place `async with session.begin():` around a complete write unit, which commits on success and rolls back on exception; or
|
||||||
|
- call `await session.commit()` explicitly after all writes when the route is the complete unit, as the template's simple UI action does.
|
||||||
|
|
||||||
|
The context-manager form scales better to several statements and makes exception rollback visible. Direct `commit()` is concise but requires the route to preserve the single-commit invariant and handle any recovery needs. Do not combine both conventions in one route. Lower-level data-access functions receive the existing session and remain unaware of FastAPI.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Background Work
|
||||||
|
|
||||||
|
A request session belongs to that request and must not be retained by a background task. Inject or otherwise provide the application session factory, then create a new session inside the task:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async def run_background_job(session_factory: SessionFactory) -> None:
|
||||||
|
async with session_factory.begin() as session:
|
||||||
|
await process_pending_items(session)
|
||||||
|
```
|
||||||
|
|
||||||
|
If work must survive application shutdown, it needs an independently owned worker lifecycle rather than the FastAPI lifespan-owned factory.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing and Overrides
|
||||||
|
|
||||||
|
Override the narrow dependency that matches the test objective:
|
||||||
|
|
||||||
|
- Override `_get_session_factory` to preserve production request-session behavior with a test factory.
|
||||||
|
- Override `_get_session` when a test must inject one transaction-scoped session directly.
|
||||||
|
- Verify each lifespan receives a fresh engine and session factory and removes application state during teardown.
|
||||||
|
- Remove overrides during teardown so mutable application state does not leak between tests.
|
||||||
|
|
||||||
|
```python
|
||||||
|
app.dependency_overrides[_get_session] = get_test_session
|
||||||
|
try:
|
||||||
|
yield app
|
||||||
|
finally:
|
||||||
|
app.dependency_overrides.pop(_get_session, None)
|
||||||
|
```
|
||||||
|
|
||||||
|
See [database testing](testing.md) for outer transactions, SAVEPOINT-backed fixtures, and database target selection.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Anti-Patterns
|
||||||
|
|
||||||
|
- Creating an engine or session factory in a request dependency.
|
||||||
|
- Reading settings and constructing database resources from repositories.
|
||||||
|
- Storing one mutable `AsyncSession` on `app.state`.
|
||||||
|
- Sharing a request session with concurrent or background tasks.
|
||||||
|
- Assuming `SessionDep` commits when dependency cleanup runs.
|
||||||
|
- Keeping `app.state.session_factory` after its `database_scope()` exits.
|
||||||
|
- Using deprecated startup and shutdown event handlers alongside lifespan.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Integration Checklist
|
||||||
|
|
||||||
|
- Lifespan enters exactly one `database_scope()` for each application lifecycle.
|
||||||
|
- Application state stores the yielded session factory.
|
||||||
|
- Session dependencies create and close one session per request.
|
||||||
|
- The session dependency owns request session closure but not commit behavior.
|
||||||
|
- Routes use `Annotated` aliases and receive sessions, not engines.
|
||||||
|
- Background tasks create their own sessions from a still-live factory.
|
||||||
|
- Tests override and restore dependencies deterministically.
|
||||||
+13
-18
@@ -1,13 +1,14 @@
|
|||||||
# Preventing Implicit ORM I/O (Asyncio)
|
# Preventing Implicit ORM I/O (Asyncio)
|
||||||
|
|
||||||
Source:
|
!!! info "Primary sources"
|
||||||
- https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#preventing-implicit-io-when-using-asyncsession
|
- [Preventing implicit I/O with AsyncSession](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#preventing-implicit-io-when-using-asyncsession)
|
||||||
- https://docs.sqlalchemy.org/en/21/orm/queryguide/relationships.html
|
- [SQLAlchemy relationship loading](https://docs.sqlalchemy.org/en/21/orm/queryguide/relationships.html)
|
||||||
|
|
||||||
Status: adopted
|
??? abstract "Decision metadata"
|
||||||
Decision level: advisory
|
- Status: adopted
|
||||||
Applies to: api-runtime, workers, tests
|
- Decision level: advisory
|
||||||
Last reviewed: 2026-06-17
|
- Applies to: api-runtime, workers, tests
|
||||||
|
- Last reviewed: 2026-06-17
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -65,13 +66,13 @@ roles = await user.awaitable_attrs.roles
|
|||||||
|
|
||||||
## Practical Enforcement Model
|
## Practical Enforcement Model
|
||||||
|
|
||||||
Use phased enforcement:
|
Require explicit I/O behavior on every async ORM path:
|
||||||
|
|
||||||
1. High-traffic and latency-sensitive routes: enforce explicit eager loading.
|
1. Define loader options for relationships and deferred columns needed by the operation.
|
||||||
2. Background tasks and less critical paths: track and progressively tighten.
|
2. Use `refresh()` or awaitable attributes only when the additional query is deliberate and visible.
|
||||||
3. Add review checks to prevent newly introduced implicit-load hotspots.
|
3. Add review checks that reject unplanned lazy-load paths.
|
||||||
|
|
||||||
This keeps modernization pragmatic while reducing hidden I/O over time.
|
This keeps event-loop behavior predictable and makes query boundaries reviewable from the code.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -98,9 +99,3 @@ This keeps modernization pragmatic while reducing hidden I/O over time.
|
|||||||
- Tests verify expected data is present without hidden secondary query surprises.
|
- Tests verify expected data is present without hidden secondary query surprises.
|
||||||
- Regression tests exist for routes previously affected by implicit-load failures.
|
- Regression tests exist for routes previously affected by implicit-load failures.
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Migration Notes
|
|
||||||
|
|
||||||
- Start advisory: target high-risk paths first.
|
|
||||||
- As coverage improves, elevate selected rules to mandatory in code review policy.
|
|
||||||
+9
-5
@@ -1,6 +1,6 @@
|
|||||||
# FastAPI Async SQLAlchemy References Index
|
# FastAPI Async SQLAlchemy References Index
|
||||||
|
|
||||||
Purpose: concept registry for modernization guidance used by this skill.
|
Purpose: concept registry for the principles, mechanics, and implementation guidance used by this skill.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -8,17 +8,21 @@ Purpose: concept registry for modernization guidance used by this skill.
|
|||||||
|
|
||||||
| Concept | File | Status | Decision Level | Owner | Last Reviewed |
|
| Concept | File | Status | Decision Level | Owner | Last Reviewed |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| Engine lifecycle and ownership | [engine.md](engine.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
| Engine lifecycle and ownership | [engine.md](engine.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||||
| Session factory and scope | [session.md](session.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
| Session factory and scope | [session.md](session.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||||
|
| FastAPI lifespan and dependency injection | [fastapi.md](fastapi.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||||
| Transaction boundaries | [transactions.md](transactions.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
| Transaction boundaries | [transactions.md](transactions.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
||||||
| Implicit ORM I/O under asyncio | [implicit_io.md](implicit_io.md) | adopted | advisory | platform/backend | 2026-06-17 |
|
| Implicit ORM I/O under asyncio | [implicit_io.md](implicit_io.md) | adopted | advisory | platform/backend | 2026-06-17 |
|
||||||
| Observability and resilience | [observability.md](observability.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
| Observability and resilience | [observability.md](observability.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
||||||
|
| SQLModel modeling and async boundaries | [sqlmodel.md](sqlmodel.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||||
|
| Basic CRUD repository and functions | [crud.md](crud.md) | adopted | advisory | platform/backend | 2026-08-06 |
|
||||||
|
| Test database targets and fixture data | [testing.md](testing.md) | adopted | mandatory | platform/backend | 2026-08-06 |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## How to Use This Folder
|
## How to Use This Folder
|
||||||
|
|
||||||
- `SKILL.md` defines the planning workflow and migration procedure.
|
- `SKILL.md` defines the explanatory workflow and shared mental model.
|
||||||
- Each concept doc defines policy-level guidance for one concern.
|
- Each concept doc defines policy-level guidance for one concern.
|
||||||
- Use the template in [template.md](template.md) for new concept docs.
|
- Use the template in [template.md](template.md) for new concept docs.
|
||||||
- Keep references source-linked and implementation snippets minimal.
|
- Keep references source-linked and implementation snippets minimal.
|
||||||
@@ -29,4 +33,4 @@ Purpose: concept registry for modernization guidance used by this skill.
|
|||||||
|
|
||||||
- If a PR changes database lifecycle/session/ORM loading behavior, update the relevant concept file.
|
- If a PR changes database lifecycle/session/ORM loading behavior, update the relevant concept file.
|
||||||
- Keep `Status`, `Decision Level`, and `Last Reviewed` current.
|
- Keep `Status`, `Decision Level`, and `Last Reviewed` current.
|
||||||
- Use `advisory` only when incremental rollout is intended; use `mandatory` for required runtime policy.
|
- Use `advisory` for recommendations that depend on application context; use `mandatory` for required runtime policy.
|
||||||
+10
-16
@@ -1,15 +1,16 @@
|
|||||||
# DB Observability and Resilience
|
# DB Observability and Resilience
|
||||||
|
|
||||||
Source:
|
!!! info "Primary sources"
|
||||||
- https://docs.sqlalchemy.org/en/21/core/pooling.html
|
- [SQLAlchemy pooling](https://docs.sqlalchemy.org/en/21/core/pooling.html)
|
||||||
- https://docs.sqlalchemy.org/en/21/core/engines.html
|
- [SQLAlchemy engine configuration](https://docs.sqlalchemy.org/en/21/core/engines.html)
|
||||||
- https://docs.sqlalchemy.org/en/21/core/events.html
|
- [SQLAlchemy events](https://docs.sqlalchemy.org/en/21/core/events.html)
|
||||||
- https://fastapi.tiangolo.com/advanced/events/
|
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
||||||
|
|
||||||
Status: adopted
|
??? abstract "Decision metadata"
|
||||||
Decision level: mandatory
|
- Status: adopted
|
||||||
Applies to: api-runtime, workers, tests
|
- Decision level: mandatory
|
||||||
Last reviewed: 2026-06-17
|
- Applies to: api-runtime, workers, tests
|
||||||
|
- Last reviewed: 2026-06-17
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -104,10 +105,3 @@ Readiness checks should be lightweight and bounded (timeouts), not heavy diagnos
|
|||||||
- Readiness endpoint test covers healthy and unhealthy DB states.
|
- Readiness endpoint test covers healthy and unhealthy DB states.
|
||||||
- Integration test simulates disconnect/reconnect behavior.
|
- Integration test simulates disconnect/reconnect behavior.
|
||||||
- Load/concurrency tests validate pool behavior under stress.
|
- Load/concurrency tests validate pool behavior under stress.
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Migration Notes
|
|
||||||
|
|
||||||
- Start with resilient defaults (`pool_pre_ping`) and simple health policy.
|
|
||||||
- Add deeper metrics/event hooks incrementally once baseline reliability is in place.
|
|
||||||
@@ -0,0 +1,404 @@
|
|||||||
|
# Async SQLAlchemy Session Management
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
|
||||||
|
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||||
|
- [Python `inspect.signature`](https://docs.python.org/3/library/inspect.html#inspect.signature)
|
||||||
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
|
- [SQLAlchemy session basics](https://docs.sqlalchemy.org/en/21/orm/session_basics.html)
|
||||||
|
- [`nicegui-db` session implementation](https://forgejo.john-stream.com/john/nicegui-db/src/commit/126bc26ad8635a86bacf684d7bda409230347597/src/nicegui_db/db/session.py)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Define one canonical session model for SQLAlchemy asyncio:
|
||||||
|
|
||||||
|
- configure a lifespan-owned factory or resolve a URL-keyed cached factory,
|
||||||
|
- create one AsyncSession per task or unit of work,
|
||||||
|
- let callers supply a session when they already own the scope,
|
||||||
|
- never share one AsyncSession across concurrent tasks.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Scope and Non-Goals
|
||||||
|
|
||||||
|
- In scope: session factory creation, task scoping, and transaction demarcation.
|
||||||
|
- Out of scope: framework dependency wiring, ORM model design, query optimization strategy, and schema migration tooling.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
- Create the application `async_sessionmaker` inside `database_scope()` and store it in application state for request dependencies.
|
||||||
|
- Use `get_session_factory(db_url)` and `resolve_session_factory()` for standalone decorated operations that do not receive the application factory.
|
||||||
|
- Use a fresh AsyncSession for each task or explicit unit of work.
|
||||||
|
- Let reusable service functions accept `AsyncSession | None` and apply `@with_session` when standalone invocation is useful.
|
||||||
|
- Pass an `AsyncSession` directly when composing several calls under one caller-owned scope.
|
||||||
|
- Borrow a caller-provided session without beginning, closing, committing, or rolling it back.
|
||||||
|
- Do not share AsyncSession across `asyncio.gather()` or parallel tasks.
|
||||||
|
- Prefer direct dependency injection over global scoped-session patterns in new code.
|
||||||
|
- Use explicit transaction boundaries (`async with session.begin():`) for writes.
|
||||||
|
- Use `db_transaction_scope()` when a standalone operation must own engine, factory, session, and transaction lifetimes together.
|
||||||
|
- Use `begin_nested()` directly and only when partial rollback through a database SAVEPOINT is required.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Sessions and Transactions
|
||||||
|
|
||||||
|
A session and a transaction solve related but different problems:
|
||||||
|
|
||||||
|
| Concept | Responsibility | Typical lifetime |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `AsyncSession` | Provides the ORM workspace: executes queries, tracks loaded and changed objects in its identity map, and flushes pending changes. It also coordinates access to a database connection. | One task or explicit unit of work. |
|
||||||
|
| Transaction | Defines the atomic database boundary: all work inside it commits together on success or rolls back together on failure. | One complete operation that must have a single outcome. |
|
||||||
|
|
||||||
|
A transaction belongs to a session; it is not an alternative to one. The session is the interface used by application and data-access code, while the transaction determines when that work becomes permanent. A session may coordinate sequential transactions during its lifetime, although short-lived application scopes commonly use one session for one transaction.
|
||||||
|
|
||||||
|
Use a session without a helper-owned commit boundary for independent reads or lower-level functions that must participate in whatever transaction their caller controls:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async with session_factory() as session:
|
||||||
|
item = await find_item(session, item_id)
|
||||||
|
```
|
||||||
|
|
||||||
|
Use an explicit transaction for writes, read-modify-write operations, or several statements that must succeed or fail as one unit:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async with session_factory.begin() as session:
|
||||||
|
order = await create_order(session, order_data)
|
||||||
|
await reserve_inventory(session, order)
|
||||||
|
```
|
||||||
|
|
||||||
|
SQLAlchemy sessions use [autobegin](https://docs.sqlalchemy.org/en/21/orm/session_basics.html#auto-begin), so the first database operation normally starts a transaction even for a read. Therefore, “session-only” means that the surrounding helper owns only session lifetime and does not promise to commit; it does not mean that no database transaction exists. Closing such a session releases its resources and rolls back any unfinished transaction. An explicit `begin()` is valuable when application code must make the atomic boundary and commit ownership visible.
|
||||||
|
|
||||||
|
For most read-only operations, a session context is sufficient. Use an explicit transaction for reads when they need a defined consistency boundary, participate in a larger atomic operation, or use locking such as `SELECT ... FOR UPDATE`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Session Factory Mechanics
|
||||||
|
|
||||||
|
An `async_sessionmaker[AsyncSession]` is a reusable configuration object and callable session producer. It stores how sessions should be created, including the engine binding and options such as `expire_on_commit=False`. It is not itself a session, connection, or transaction, and calling it does not make a shared global `AsyncSession`.
|
||||||
|
|
||||||
|
The template exposes two construction paths with the same session options.
|
||||||
|
|
||||||
|
The application-owned path creates a factory inside the engine lifecycle:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||||
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
|
from .engine import engine_scope
|
||||||
|
|
||||||
|
type SessionFactory = async_sessionmaker[AsyncSession]
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def database_scope(
|
||||||
|
db_url: str,
|
||||||
|
*,
|
||||||
|
auto_flush: bool = True,
|
||||||
|
) -> AsyncGenerator[SessionFactory]:
|
||||||
|
async with engine_scope(db_url) as engine:
|
||||||
|
yield async_sessionmaker(
|
||||||
|
bind=engine,
|
||||||
|
class_=AsyncSession,
|
||||||
|
expire_on_commit=False,
|
||||||
|
autoflush=auto_flush,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
FastAPI lifespan enters this path once and stores the yielded factory on application state. The factory must not outlive the scope because its bound engine is disposed on exit.
|
||||||
|
|
||||||
|
The standalone path caches a factory by URL and `auto_flush` policy:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from functools import cache
|
||||||
|
|
||||||
|
from .engine import get_engine
|
||||||
|
|
||||||
|
|
||||||
|
@cache
|
||||||
|
def get_session_factory(
|
||||||
|
db_url: str,
|
||||||
|
*,
|
||||||
|
auto_flush: bool = True,
|
||||||
|
) -> SessionFactory:
|
||||||
|
return async_sessionmaker(
|
||||||
|
bind=get_engine(db_url),
|
||||||
|
class_=AsyncSession,
|
||||||
|
expire_on_commit=False,
|
||||||
|
autoflush=auto_flush,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_session_factory(settings: Settings | None = None) -> SessionFactory:
|
||||||
|
settings = settings or get_settings()
|
||||||
|
db_url = get_database_url(settings)
|
||||||
|
return get_session_factory(db_url)
|
||||||
|
```
|
||||||
|
|
||||||
|
This path lets framework-independent helpers resolve one stable factory without receiving it through every call. The tradeoff is hidden configuration resolution and a second lifecycle mechanism. `dispose_engine()` clears `get_engine`'s cache but does not clear `get_session_factory`'s cache in the template. A cached factory remains bound to the disposed engine object; SQLAlchemy can create a new pool when that engine is used again, but a later `database_scope()` for the same URL can own a different engine. Treat cached standalone resolution as process-lifetime convenience, avoid repeated application lifecycles in one process, and clear both caches together if the template evolves to support them.
|
||||||
|
|
||||||
|
Each call to `session_factory()` creates a distinct `AsyncSession`. The caller that invokes the factory owns that session lifetime and must close it, normally with `async with`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async with session_factory() as session:
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
The factory can be shared across operations and tasks. Sessions produced by it cannot be shared across concurrent tasks.
|
||||||
|
|
||||||
|
Passing the application factory directly has three useful consequences:
|
||||||
|
|
||||||
|
- Lower layers do not resolve settings or global resources.
|
||||||
|
- Tests can inject a test factory directly through `session_scope(session_factory=...)` or FastAPI state.
|
||||||
|
- Transaction ownership remains independent of engine construction.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Database and Convenience Scopes
|
||||||
|
|
||||||
|
The template provides three framework-independent context managers:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def db_session_scope(
|
||||||
|
db_url: str | None = None,
|
||||||
|
) -> AsyncGenerator[AsyncSession]:
|
||||||
|
db_url = db_url or resolve_database_url()
|
||||||
|
async with database_scope(db_url) as session_factory, session_factory() as session:
|
||||||
|
yield session
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def db_transaction_scope(
|
||||||
|
db_url: str | None = None,
|
||||||
|
) -> AsyncGenerator[AsyncSession]:
|
||||||
|
db_url = db_url or resolve_database_url()
|
||||||
|
async with database_scope(db_url) as session_factory, session_factory.begin() as session:
|
||||||
|
yield session
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def session_scope(
|
||||||
|
*,
|
||||||
|
settings: Settings | None = None,
|
||||||
|
session_factory: SessionFactory | None = None,
|
||||||
|
session: AsyncSession | None = None,
|
||||||
|
) -> AsyncGenerator[AsyncSession]:
|
||||||
|
if session is not None:
|
||||||
|
yield session
|
||||||
|
return
|
||||||
|
|
||||||
|
session_factory = session_factory or resolve_session_factory(settings=settings)
|
||||||
|
async with session_factory() as owned_session:
|
||||||
|
yield owned_session
|
||||||
|
```
|
||||||
|
|
||||||
|
`db_session_scope()` owns a complete temporary database lifecycle and a session but does not commit. `db_transaction_scope()` owns the same resources plus a root transaction that commits on successful exit and rolls back on exception. Both initialize the schema by default because `database_scope()` enters `engine_scope()` with its default `initialize=True`. They are appropriate for scripts, commands, and isolated operations, not per-request use inside an already-running application.
|
||||||
|
|
||||||
|
`session_scope()` is the borrow-or-create helper. Its precedence is supplied session, supplied factory, then settings-based cached factory resolution. A supplied session remains entirely caller-owned; the helper does not require an active transaction and does not begin, commit, roll back, or close it. An owned session is closed on exit, and unfinished autobegun work rolls back.
|
||||||
|
|
||||||
|
Passing `session=None` is the same as omitting the session for `session_scope()` and therefore creates a session. This differs from `with_session`, which tests whether the argument name was bound rather than whether its value is non-null.
|
||||||
|
|
||||||
|
## Signature-Aware Session Injection
|
||||||
|
|
||||||
|
`with_session` allows one async function to support standalone calls and explicit composition:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import Awaitable
|
||||||
|
from collections.abc import Callable
|
||||||
|
from functools import wraps
|
||||||
|
from inspect import signature
|
||||||
|
|
||||||
|
|
||||||
|
def with_session[**P, R](
|
||||||
|
func: Callable[P, Awaitable[R]],
|
||||||
|
) -> Callable[P, Awaitable[R]]:
|
||||||
|
sig = signature(func)
|
||||||
|
|
||||||
|
@wraps(func)
|
||||||
|
async def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
|
||||||
|
bound = sig.bind_partial(*args, **kwargs)
|
||||||
|
|
||||||
|
if "session" in bound.arguments:
|
||||||
|
return await func(*args, **kwargs)
|
||||||
|
|
||||||
|
async with resolve_session_factory()() as session:
|
||||||
|
bound.arguments["session"] = session
|
||||||
|
return await func(*bound.args, **bound.kwargs)
|
||||||
|
|
||||||
|
return wrapper
|
||||||
|
```
|
||||||
|
|
||||||
|
The function must be async and expose a parameter named exactly `session`. The decorator preserves metadata with `wraps()`, binds positional and keyword arguments through the original signature, and injects a fresh session only when the caller omitted that argument.
|
||||||
|
|
||||||
|
The distinction between omitted and explicit `None` is deliberate in the implementation:
|
||||||
|
|
||||||
|
- `await operation()` injects and owns a session.
|
||||||
|
- `await operation(session=existing_session)` borrows the caller's session.
|
||||||
|
- `await operation(None)` or `await operation(session=None)` forwards `None` without injection.
|
||||||
|
|
||||||
|
The decorated function therefore types the parameter as `AsyncSession | None = None` but should assert or guard after decoration. Explicit `None` is not a request for injection. This preserves ordinary Python call binding, but it means wrappers or callers must omit the argument instead of forwarding a nullable value.
|
||||||
|
|
||||||
|
`with_session` owns session lifetime only. It does not begin or commit a transaction, so it is naturally suited to reads. Decorated writes must either manage a visible transaction or be called with a session from `db_transaction_scope()` or another caller-owned transaction. Prefer explicit factory or session injection when lifecycle transparency and test substitution matter more than call-site convenience.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Function and Service Boundaries
|
||||||
|
|
||||||
|
Template service functions support both standalone and composed use by combining `@with_session` with an optional parameter:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from sqlmodel import func
|
||||||
|
from sqlmodel import select
|
||||||
|
|
||||||
|
|
||||||
|
@with_session
|
||||||
|
async def count_items(session: AsyncSession | None = None) -> int:
|
||||||
|
assert session is not None, "Session must be provided by with_session decorator"
|
||||||
|
result = await session.exec(select(func.count()).select_from(Item))
|
||||||
|
return result.one()
|
||||||
|
```
|
||||||
|
|
||||||
|
The standalone call injects and closes a session:
|
||||||
|
|
||||||
|
```python
|
||||||
|
count = await count_items()
|
||||||
|
```
|
||||||
|
|
||||||
|
A larger use case passes one caller-owned session through several decorated functions:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async with session_factory.begin() as session:
|
||||||
|
count = await count_items(session)
|
||||||
|
await create_item(payload, session=session)
|
||||||
|
```
|
||||||
|
|
||||||
|
The decorator sees the bound `session` argument and leaves all ownership with the caller. It never creates a SAVEPOINT or nested transaction.
|
||||||
|
|
||||||
|
For low-level helpers that should never resolve settings, require a non-optional session and leave them undecorated. Application service objects may store the immutable session factory, but they must not store a mutable session:
|
||||||
|
|
||||||
|
```python
|
||||||
|
class ItemService:
|
||||||
|
def __init__(self, session_factory: SessionFactory) -> None:
|
||||||
|
self.session_factory = session_factory
|
||||||
|
|
||||||
|
async def find(self, item_id: int) -> Item | None:
|
||||||
|
async with self.session_factory() as session:
|
||||||
|
return await find_item(session, item_id)
|
||||||
|
```
|
||||||
|
|
||||||
|
Code that already owns a transaction should call the session-required function directly. Repositories should normally remain in that session-required layer; the service or use-case boundary owns standalone session creation. This avoids optional-session APIs spreading into every data-access function.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## SAVEPOINTs and Partial Failure
|
||||||
|
|
||||||
|
Use [`begin_nested()`](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#using-savepoint) only when failure inside one portion of an operation should roll back that portion while preserving the outer transaction:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async with db_transaction_scope() as session:
|
||||||
|
order = await insert_order(session, payload)
|
||||||
|
|
||||||
|
try:
|
||||||
|
async with session.begin_nested():
|
||||||
|
await apply_optional_discount(session, order)
|
||||||
|
except DiscountError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
await reserve_inventory(session, order)
|
||||||
|
```
|
||||||
|
|
||||||
|
Important SAVEPOINT semantics:
|
||||||
|
|
||||||
|
- `begin_nested()` starts a root transaction if one is not already active, so call it inside a visible outer transaction when that ownership matters.
|
||||||
|
- Entering `begin_nested()` unconditionally flushes pending session state, regardless of the `autoflush` setting.
|
||||||
|
- Successful exit releases the SAVEPOINT; it does not commit the outer transaction.
|
||||||
|
- Exceptional exit rolls back to the SAVEPOINT and leaves the outer transaction active.
|
||||||
|
- In SQLAlchemy 2.x, `session.commit()` commits the outermost transaction. Never call it to release a SAVEPOINT; let the nested context manager manage its transaction handle.
|
||||||
|
|
||||||
|
Do not create a SAVEPOINT merely because one service calls another. SAVEPOINTs add database work and alter flush and error-recovery behavior. Use them only for explicit partial-failure requirements such as skipping one conflicting row while retaining the rest of a batch.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Framework Integration
|
||||||
|
|
||||||
|
Keep framework adapters outside these session primitives. See [FastAPI database integration](fastapi.md) for lifespan ownership, `Annotated` dependency aliases, and read-versus-write request sessions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Configuration Guidance
|
||||||
|
|
||||||
|
- `expire_on_commit=False` is commonly preferred in asyncio applications to reduce accidental post-commit reload behavior.
|
||||||
|
- `AsyncSession.refresh()` is preferred over broad expiration patterns when state refresh is needed.
|
||||||
|
- [`async_sessionmaker.begin()`](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin) is a concise option when one scope must create a session, begin a transaction, commit on success, roll back on failure, and close. Do not use it when borrowing a caller's session.
|
||||||
|
|
||||||
|
## SQLModel Alignment
|
||||||
|
|
||||||
|
- Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per task or unit of work.
|
||||||
|
- SQLModel does not replace SQLAlchemy async lifecycle primitives; it provides model declaration, validation, and typing ergonomics on top of them.
|
||||||
|
- Do not mix ad hoc session construction with the canonical session factory.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Concurrency Rules
|
||||||
|
|
||||||
|
- One session per concurrent task.
|
||||||
|
- If work fans out into parallel tasks, each task receives its own AsyncSession.
|
||||||
|
- Pass sessions explicitly to service functions; avoid mutable global session state.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Anti-Patterns
|
||||||
|
|
||||||
|
- A singleton/global AsyncSession reused across tasks or operations.
|
||||||
|
- Sharing one AsyncSession across parallel tasks.
|
||||||
|
- Passing an application-global AsyncSession to a repository constructor.
|
||||||
|
- Creating a new `async_sessionmaker` in each operation.
|
||||||
|
- Retaining a session factory after its bound engine scope exits.
|
||||||
|
- Using cached standalone factory resolution when the application factory is already available.
|
||||||
|
- Assuming `with_session` starts or commits a transaction.
|
||||||
|
- Forwarding `session=None` to a decorated function when injection was intended.
|
||||||
|
- Closing or committing a session supplied by the caller.
|
||||||
|
- Silently starting or committing a transaction on a supplied session.
|
||||||
|
- Creating a SAVEPOINT for ordinary nested service calls.
|
||||||
|
- Hiding root transaction, joined transaction, and SAVEPOINT behavior behind one mode-driven `atomic_scope()` helper.
|
||||||
|
- Calling `session.commit()` inside a SAVEPOINT scope.
|
||||||
|
- Mixing commit/rollback ownership across layers without a declared boundary.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Operational Checks
|
||||||
|
|
||||||
|
- The FastAPI application factory is created inside `database_scope()` and does not outlive its bound engine.
|
||||||
|
- Cached standalone factories are used only where application-state injection is unavailable.
|
||||||
|
- `session_scope()` precedence is supplied session, supplied factory, then settings-based resolution.
|
||||||
|
- Decorated functions receive injection only when the `session` argument is omitted.
|
||||||
|
- No code path creates AsyncSession in module import side effects.
|
||||||
|
- Concurrent jobs and operations each create task-local sessions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing Checks
|
||||||
|
|
||||||
|
- Service constructors accept a test session factory without framework startup.
|
||||||
|
- Session-taking access functions accept a transaction-scoped test session directly.
|
||||||
|
- `session_scope()` tests cover supplied-session, supplied-factory, and settings-resolution precedence.
|
||||||
|
- `db_transaction_scope()` tests verify commit on success, rollback on failure, session closure, engine disposal, and cache cleanup.
|
||||||
|
- `with_session` tests cover omitted, positional, keyword, and explicit-`None` session arguments.
|
||||||
|
- Composition tests verify decorated service calls borrow one caller-owned session without committing it.
|
||||||
|
- SAVEPOINT tests verify local rollback preserves the outer transaction and successful exit does not commit it.
|
||||||
|
- Tests that depend on SAVEPOINT timing account for `begin_nested()` flushing pending state on entry.
|
||||||
|
- Rollback behavior is verified for failed write units.
|
||||||
|
- Parallel-task tests verify no shared AsyncSession instances.
|
||||||
|
- Lifecycle tests confirm schema initialization, factory availability, deterministic teardown, and expected cache behavior.
|
||||||
|
|
||||||
@@ -0,0 +1,124 @@
|
|||||||
|
# SQLModel-First Modeling and Async Boundaries
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [SQLModel documentation](https://sqlmodel.tiangolo.com/)
|
||||||
|
- [SQLModel features](https://sqlmodel.tiangolo.com/features/)
|
||||||
|
- [SQLModel advanced guide](https://sqlmodel.tiangolo.com/advanced/)
|
||||||
|
- [SQLModel FastAPI session dependency tutorial](https://sqlmodel.tiangolo.com/tutorial/fastapi/session-with-dependency/)
|
||||||
|
- [SQLModel release notes](https://sqlmodel.tiangolo.com/release-notes/)
|
||||||
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
|
|
||||||
|
??? abstract "Decision metadata"
|
||||||
|
- Status: adopted
|
||||||
|
- Decision level: mandatory
|
||||||
|
- Applies to: api-runtime, workers, tests
|
||||||
|
- Last reviewed: 2026-08-06
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
Define SQLModel as the primary model layer for async FastAPI applications and explain how it composes with SQLAlchemy's async runtime.
|
||||||
|
|
||||||
|
SQLModel is designed for FastAPI, built on Pydantic and SQLAlchemy, and intended to minimize duplication while preserving the capabilities of both. Async engine, session, transaction, and loading behavior still follow SQLAlchemy's asyncio contract.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Scope and Non-Goals
|
||||||
|
|
||||||
|
- In scope: table models, API data models, SQLAlchemy interoperability, async session usage, and exception criteria.
|
||||||
|
- Out of scope: replacing SQLAlchemy's async runtime primitives or claiming that synchronous tutorial examples are async patterns.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
- Default to SQLModel for new table models and API data models.
|
||||||
|
- Keep SQLAlchemy engine and factory primitives as the runtime base: `create_async_engine` and `async_sessionmaker`. For SQLModel applications, use SQLModel's `AsyncSession` wrapper so its typed `exec()` API remains available.
|
||||||
|
- Keep transaction and session ownership policies identical whether models are SQLAlchemy Declarative or SQLModel.
|
||||||
|
- Use SQLModel inheritance to share validated fields while keeping table, create, update, and public contracts distinct where their semantics differ.
|
||||||
|
- Use SQLAlchemy declarative models only for a concrete unsupported mapping or third-party constraint; document the reason.
|
||||||
|
- Use SQLAlchemy relationship loading options explicitly on async paths.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Recommended Patterns
|
||||||
|
|
||||||
|
### Pattern A: Data model split for API boundaries
|
||||||
|
|
||||||
|
Use distinct models for persistence and external contracts.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from sqlmodel import Field, SQLModel
|
||||||
|
|
||||||
|
|
||||||
|
class UserBase(SQLModel):
|
||||||
|
email: str
|
||||||
|
display_name: str
|
||||||
|
|
||||||
|
|
||||||
|
class User(UserBase, table=True):
|
||||||
|
id: int | None = Field(default=None, primary_key=True)
|
||||||
|
|
||||||
|
|
||||||
|
class UserCreate(UserBase):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class UserRead(UserBase):
|
||||||
|
id: int
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pattern B: Keep SQLModel models with the async runtime
|
||||||
|
|
||||||
|
```python
|
||||||
|
from sqlmodel import select
|
||||||
|
|
||||||
|
async with database_scope(settings.database_url) as session_factory:
|
||||||
|
async with session_factory() as session:
|
||||||
|
users = (await session.exec(select(User))).all()
|
||||||
|
```
|
||||||
|
|
||||||
|
`database_scope()` enters the cached engine lifecycle, initializes registered SQLModel metadata by default, and yields the application session factory while SQLModel supplies the model and statement layer. `sqlmodel.select()` keeps SQLModel's typing-oriented statement construction, and SQLModel's `AsyncSession` adds typed `exec()` results while retaining SQLAlchemy's async lifecycle and transaction behavior. Import `AsyncSession` from `sqlmodel.ext.asyncio.session` when working with SQLModel models; use SQLAlchemy's `AsyncSession` only when the code intentionally has no SQLModel dependency.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Interoperability Notes
|
||||||
|
|
||||||
|
- A SQLModel table model is a SQLAlchemy model and can participate in SQLAlchemy relationships, statements, loader options, and sessions.
|
||||||
|
- A SQLModel model is also a Pydantic model; non-table models are useful for request and response contracts.
|
||||||
|
- SQLModel's official FastAPI dependency tutorial currently uses synchronous `Session`; translate the ownership pattern, not the concrete session type, for async applications.
|
||||||
|
- SQLModel's advanced guide still lists dedicated async documentation as future work, so use SQLAlchemy's asyncio documentation as the authority for runtime mechanics.
|
||||||
|
- Prefer one query style per module to reduce cognitive overhead.
|
||||||
|
- Keep loader strategies explicit in async paths to avoid implicit I/O surprises.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Anti-Patterns
|
||||||
|
|
||||||
|
- Treating SQLModel as an alternative to SQLAlchemy rather than a layer built on it.
|
||||||
|
- Copying a synchronous `Session` example into an async request path.
|
||||||
|
- Constructing sessions in handlers instead of using the application session factory.
|
||||||
|
- Mixing multiple query/session idioms within the same module without clear conventions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Operational Checks
|
||||||
|
|
||||||
|
- New model modules are SQLModel-first; exceptions state the unsupported need or constraint.
|
||||||
|
- Session/transaction ownership remains consistent across both model styles.
|
||||||
|
- Table, create, update, and public models share fields intentionally without exposing persistence-only data.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing Checks
|
||||||
|
|
||||||
|
- Module-level tests verify CRUD semantics for SQLModel models through `AsyncSession`.
|
||||||
|
- API tests verify response/request model behavior for SQLModel-based endpoints.
|
||||||
|
- Relationship tests verify async loader strategies do not depend on implicit I/O.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Version Checks
|
||||||
|
|
||||||
|
- Verify installed SQLModel, SQLAlchemy, and Pydantic versions together when using newly added typing or ORM features.
|
||||||
+8
-12
@@ -1,13 +1,14 @@
|
|||||||
# <Concept Title>
|
# <Concept Title>
|
||||||
|
|
||||||
Source:
|
!!! info "Primary sources"
|
||||||
- <primary source url>
|
- Primary source: `<primary source URL>`
|
||||||
- <secondary source url>
|
- Secondary source: `<secondary source URL>`
|
||||||
|
|
||||||
Status: draft|adopted|deprecated
|
??? abstract "Decision metadata"
|
||||||
Decision level: advisory|mandatory
|
- Status: draft|adopted|deprecated
|
||||||
Applies to: api-runtime|workers|tests
|
- Decision level: advisory|mandatory
|
||||||
Last reviewed: YYYY-MM-DD
|
- Applies to: api-runtime|workers|tests
|
||||||
|
- Last reviewed: YYYY-MM-DD
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -56,8 +57,3 @@ Describe what this concept governs and why it exists.
|
|||||||
- Test 1
|
- Test 1
|
||||||
- Test 2
|
- Test 2
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Migration Notes
|
|
||||||
|
|
||||||
- Staged rollout notes and compatibility caveats.
|
|
||||||
@@ -0,0 +1,172 @@
|
|||||||
|
# Testing Database Targets and Data
|
||||||
|
|
||||||
|
Use the same engine and session primitives in production and tests. Tests select a different URL and, when transaction isolation is required, bind a test session factory to one test-owned connection and outer transaction. They do not replace repositories, services, or SQLAlchemy mechanics with mocks.
|
||||||
|
|
||||||
|
## Decision Table
|
||||||
|
|
||||||
|
| Test need | Database target | Isolation approach | What it proves |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Fast, serial application tests | `sqlite+aiosqlite://` | Per-test engine or connection-bound session factory over an outer transaction | ORM mappings and ordinary application behavior |
|
||||||
|
| Async code using multiple simultaneous sessions | Named SQLite shared-cache URL or temporary SQLite file | Per-test schema or cleanup strategy | Concurrent-session behavior without a database server |
|
||||||
|
| PostgreSQL-specific behavior | Dedicated PostgreSQL test database | Per-test outer transaction and SAVEPOINT | SQL, constraints, types, locking, and migrations that SQLite cannot represent |
|
||||||
|
|
||||||
|
SQLite is a useful fast target, not a drop-in PostgreSQL substitute. Keep a small PostgreSQL integration suite for PostgreSQL-specific queries, extensions, row locking, JSON semantics, collations, isolation, and migration validation.
|
||||||
|
|
||||||
|
## Shared Construction Primitives
|
||||||
|
|
||||||
|
Make the application factory accept a database URL or settings object. Production, workers, and ordinary integration tests enter the same [`database_scope()`](session.md#database-and-convenience-scopes). Tests enter the lower-level [`engine_scope()`](engine.md#owning-engine-scope) only when they need direct engine or connection ownership for schema setup, an outer transaction, or engine-specific assertions:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
|
||||||
|
import pytest_asyncio
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
|
||||||
|
from .engine import engine_scope
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture(scope="session", loop_scope="session")
|
||||||
|
async def test_engine(database_url: str) -> AsyncGenerator[AsyncEngine]:
|
||||||
|
async with engine_scope(database_url) as engine:
|
||||||
|
yield engine
|
||||||
|
```
|
||||||
|
|
||||||
|
Production passes its `postgresql+asyncpg://...` URL to `database_scope()`. A local SQLite run passes `sqlite+aiosqlite:///./app.db`. Tests pass a dedicated test URL to `database_scope()` or `engine_scope()` and receive schema initialization, deterministic disposal, and engine-cache cleanup when the context exits. Do not create an engine during module import: that makes it easy for tests to retain the production URL before an override is applied.
|
||||||
|
|
||||||
|
Use migrations to provision an integration database when migrations are part of the release contract. `metadata.create_all()` is appropriate for focused ORM tests only when it accurately represents the schema under test. Import all table models before creating metadata; [SQLModel documents that model-registration order matters](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/#import-table-models).
|
||||||
|
|
||||||
|
## Transactional Async Fixture
|
||||||
|
|
||||||
|
For tests that exercise code which commits, start an outer transaction on one test connection. Bind a test `SessionFactory` to that connection with `join_transaction_mode="create_savepoint"`. SQLAlchemy documents this as its test-suite pattern: sessions created by the factory resolve their commits through SAVEPOINTs while fixture teardown rolls back the outer transaction.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
|
||||||
|
import pytest_asyncio
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker
|
||||||
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
||||||
|
|
||||||
|
from .session import SessionFactory
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture(scope="function", loop_scope="session")
|
||||||
|
async def session_factory(test_engine: AsyncEngine) -> AsyncGenerator[SessionFactory]:
|
||||||
|
async with test_engine.connect() as connection:
|
||||||
|
transaction = await connection.begin()
|
||||||
|
factory = async_sessionmaker(
|
||||||
|
bind=connection,
|
||||||
|
class_=AsyncSession,
|
||||||
|
expire_on_commit=False,
|
||||||
|
join_transaction_mode="create_savepoint",
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
yield factory
|
||||||
|
finally:
|
||||||
|
await transaction.rollback()
|
||||||
|
```
|
||||||
|
|
||||||
|
Each factory call still creates a distinct `AsyncSession`, matching [session factory mechanics](session.md#session-factory-mechanics). The factory belongs to the fixture's engine and outer transaction and must not escape either scope.
|
||||||
|
|
||||||
|
For service tests that pass a caller-owned session into decorated or undecorated service functions, derive that session from the same factory:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@pytest_asyncio.fixture(scope="function", loop_scope="session")
|
||||||
|
async def session(session_factory: SessionFactory) -> AsyncGenerator[AsyncSession]:
|
||||||
|
async with session_factory() as test_session:
|
||||||
|
await test_session.begin()
|
||||||
|
yield test_session
|
||||||
|
```
|
||||||
|
|
||||||
|
The explicit `begin()` gives test code one visible transaction from the start. Session closure rolls back unfinished work; the outer connection transaction remains the final isolation boundary even if application code commits its SAVEPOINT.
|
||||||
|
|
||||||
|
For FastAPI request tests, override `_get_session_factory` so the production `SessionDep` retains its session-creation and cleanup behavior while receiving the test-bound factory. Always remove the override after the test because [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/) are stored in a mutable application-level dictionary.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import Generator
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi import FastAPI
|
||||||
|
|
||||||
|
from .fastapi import _get_session_factory
|
||||||
|
from .session import SessionFactory
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def app_with_test_database(app: FastAPI, session_factory: SessionFactory) -> Generator[FastAPI]:
|
||||||
|
def get_test_session_factory() -> SessionFactory:
|
||||||
|
return session_factory
|
||||||
|
|
||||||
|
app.dependency_overrides[_get_session_factory] = get_test_session_factory
|
||||||
|
try:
|
||||||
|
yield app
|
||||||
|
finally:
|
||||||
|
app.dependency_overrides.pop(_get_session_factory, None)
|
||||||
|
```
|
||||||
|
|
||||||
|
Construct `app` with test settings before lifespan starts so startup cannot resolve the production URL. The override changes request session creation; it does not prevent lifespan from entering its configured `database_scope()`.
|
||||||
|
|
||||||
|
The connection-bound factory is deliberately serial even though it creates distinct sessions: those sessions still share one connection and outer transaction. A test that verifies concurrently active sessions must use independent connections and a database target that supports them.
|
||||||
|
|
||||||
|
## SQLite Targets
|
||||||
|
|
||||||
|
### Serial in-memory tests
|
||||||
|
|
||||||
|
Use `sqlite+aiosqlite://` for a fresh in-memory database when the test runs all database work serially. SQLAlchemy's `aiosqlite` dialect uses a single-connection `StaticPool` for this target, so all sessions share one SQLite transaction state. One session's rollback can discard another session's uncommitted work.
|
||||||
|
|
||||||
|
`engine_scope()` imports the model package and creates the schema by default, then disposes the engine and clears cached resolution deterministically:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
|
||||||
|
import pytest_asyncio
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
from .engine import engine_scope
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture(scope="session", loop_scope="session")
|
||||||
|
async def test_engine() -> AsyncGenerator[AsyncEngine]:
|
||||||
|
async with engine_scope("sqlite+aiosqlite://") as engine:
|
||||||
|
yield engine
|
||||||
|
```
|
||||||
|
|
||||||
|
### Concurrent in-memory tests
|
||||||
|
|
||||||
|
Do not use the default `:memory:` target for tests that have multiple active sessions or tasks. Use a named shared-cache database instead, with a name unique to the test process:
|
||||||
|
|
||||||
|
```text
|
||||||
|
sqlite+aiosqlite:///file:test-suite?mode=memory&cache=shared&uri=true
|
||||||
|
```
|
||||||
|
|
||||||
|
This lets connections share the same in-memory database while retaining independent transaction state. A temporary file URL such as `sqlite+aiosqlite:////tmp/test.db` is often simpler when test isolation or cleanup tooling already manages files.
|
||||||
|
|
||||||
|
For both SQLite forms, enable and test the constraints your application depends on. SQLite foreign-key enforcement is disabled by default, and its transaction behavior has driver-specific differences. Keep PostgreSQL integration coverage for behavior that SQLite cannot faithfully model.
|
||||||
|
|
||||||
|
## Test Data Practices
|
||||||
|
|
||||||
|
- Build only the data a test needs, through named factory functions or pytest fixtures rather than a large global seed.
|
||||||
|
- Give each fixture a domain meaning, such as `active_account`, `expired_subscription`, or `admin_user`; avoid opaque rows with unexplained defaults.
|
||||||
|
- Set values relevant to the assertion explicitly, including timestamps, permissions, statuses, and unique identifiers. Use fixed clocks or injected clock values instead of the wall clock.
|
||||||
|
- Construct object graphs through relationships, then `await session.flush()` before reading generated identifiers or passing foreign keys onward. `flush()` exercises database constraints without ending the test transaction.
|
||||||
|
- Seed prerequisite data before creating a client request. Let the endpoint own the mutation being asserted; do not pre-insert the row that the endpoint is supposed to create.
|
||||||
|
- Use `commit()` in fixture setup only when the test specifically needs to prove post-commit behavior. With the transactional fixture, this remains isolated through the outer rollback.
|
||||||
|
- Keep shared reference data immutable and explicit. If it must be reused for performance, load it once into a dedicated test database and reset all mutable tables between tests; never depend on test order.
|
||||||
|
- Include both valid and constraint-breaking graphs where a behavior depends on foreign keys, uniqueness, nullability, or cascading deletes. SQLite-only tests should not be the sole evidence for PostgreSQL constraints.
|
||||||
|
|
||||||
|
## Completion Checks
|
||||||
|
|
||||||
|
- A test run cannot reach the production URL; production credentials are absent from the test environment.
|
||||||
|
- Production PostgreSQL, local SQLite, and in-memory SQLite all use `database_scope()` unless a test explicitly needs lower-level engine or connection ownership.
|
||||||
|
- Every test or fixture scope owns its override, session factory, connection, transaction, and session cleanup; the session-scoped engine fixture owns disposal and cache cleanup.
|
||||||
|
- Request tests override `_get_session_factory`, preserving production request-session creation and cleanup behavior.
|
||||||
|
- Test data is deterministic, minimal, and expresses the scenario under test.
|
||||||
|
- PostgreSQL integration tests cover every PostgreSQL-specific contract and run against migrations where migrations are shipped.
|
||||||
|
|
||||||
|
## Sources
|
||||||
|
|
||||||
|
- [SQLAlchemy: joining a session into an external transaction](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites)
|
||||||
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
|
- [SQLAlchemy SQLite dialect and async in-memory pooling](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#using-a-memory-database-with-multiple-coroutines)
|
||||||
|
- [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
||||||
|
- [SQLModel testing with FastAPI](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/)
|
||||||
|
- [pytest-asyncio fixtures](https://pytest-asyncio.readthedocs.io/en/stable/how-to-guides/index.html)
|
||||||
+14
-19
@@ -1,14 +1,15 @@
|
|||||||
# Async Transaction Boundaries
|
# Async Transaction Boundaries
|
||||||
|
|
||||||
Source:
|
!!! info "Primary sources"
|
||||||
- https://docs.sqlalchemy.org/en/21/orm/session_transaction.html
|
- [SQLAlchemy transactions](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html)
|
||||||
- https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||||
- https://docs.sqlalchemy.org/en/21/core/connections.html
|
- [SQLAlchemy connections](https://docs.sqlalchemy.org/en/21/core/connections.html)
|
||||||
|
|
||||||
Status: adopted
|
??? abstract "Decision metadata"
|
||||||
Decision level: mandatory
|
- Status: adopted
|
||||||
Applies to: api-runtime, workers, tests
|
- Decision level: mandatory
|
||||||
Last reviewed: 2026-06-17
|
- Applies to: api-runtime, workers, tests
|
||||||
|
- Last reviewed: 2026-06-17
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -29,7 +30,8 @@ Define consistent transaction demarcation for async SQLAlchemy so write behavior
|
|||||||
|
|
||||||
- Every mutating use case must run inside an explicit transaction boundary.
|
- Every mutating use case must run inside an explicit transaction boundary.
|
||||||
- Prefer `async with session.begin():` for write units.
|
- Prefer `async with session.begin():` for write units.
|
||||||
- Keep transaction ownership at service/use-case boundary, not deep in helper internals.
|
- Keep transaction ownership at a service, use-case, or explicitly documented complete-operation boundary, not deep in helper internals.
|
||||||
|
- An optional-session write may own one transaction when omitting the session clearly means standalone execution; a supplied session must remain caller-owned.
|
||||||
- Read paths should not auto-upgrade into hidden write behavior.
|
- Read paths should not auto-upgrade into hidden write behavior.
|
||||||
- On exception in a transaction block, rely on rollback semantics and propagate or map exceptions intentionally.
|
- On exception in a transaction block, rely on rollback semantics and propagate or map exceptions intentionally.
|
||||||
|
|
||||||
@@ -81,7 +83,7 @@ Use nested transactions only when partial failure semantics are explicitly requi
|
|||||||
## Anti-Patterns
|
## Anti-Patterns
|
||||||
|
|
||||||
- Multiple commits scattered across one logical use case.
|
- Multiple commits scattered across one logical use case.
|
||||||
- Helper functions that commit/rollback without caller awareness.
|
- Helper functions that commit or roll back without an explicit ownership contract.
|
||||||
- Mixing implicit and explicit transaction styles in confusing ways.
|
- Mixing implicit and explicit transaction styles in confusing ways.
|
||||||
- Using savepoints as a default pattern rather than a targeted tool.
|
- Using savepoints as a default pattern rather than a targeted tool.
|
||||||
|
|
||||||
@@ -89,8 +91,8 @@ Use nested transactions only when partial failure semantics are explicitly requi
|
|||||||
|
|
||||||
## Operational Checks
|
## Operational Checks
|
||||||
|
|
||||||
- All mutating service functions declare one clear transaction boundary.
|
- All mutating services and complete operations declare one clear transaction boundary.
|
||||||
- No repository/helper performs hidden commit calls.
|
- No repository or helper performs hidden direct commit calls; standalone ownership is expressed through a documented transaction scope.
|
||||||
- Transaction style is consistent across handlers and workers.
|
- Transaction style is consistent across handlers and workers.
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -101,10 +103,3 @@ Use nested transactions only when partial failure semantics are explicitly requi
|
|||||||
- Failure path test verifies rollback behavior.
|
- Failure path test verifies rollback behavior.
|
||||||
- Tests cover concurrency-sensitive write flows.
|
- Tests cover concurrency-sensitive write flows.
|
||||||
- Savepoint usage (if present) has dedicated behavior tests.
|
- Savepoint usage (if present) has dedicated behavior tests.
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Migration Notes
|
|
||||||
|
|
||||||
- First stabilize session scope, then normalize transaction ownership.
|
|
||||||
- Replace ad hoc commit patterns incrementally with bounded write units.
|
|
||||||
@@ -0,0 +1,119 @@
|
|||||||
|
---
|
||||||
|
name: copilot-customization
|
||||||
|
description: 'Plan, create, review, and debug GitHub Copilot and VS Code agent customizations, including instructions, prompt files, skills, custom agents, hooks, MCP servers, and repo-specific personal-mcp skill integration.'
|
||||||
|
---
|
||||||
|
|
||||||
|
# Copilot Customization
|
||||||
|
|
||||||
|
Use this skill when a task is about changing how GitHub Copilot or VS Code agents behave through customization files or MCP-backed skill resources.
|
||||||
|
|
||||||
|
## When to Use
|
||||||
|
|
||||||
|
- Creating or updating:
|
||||||
|
- `.github/copilot-instructions.md`
|
||||||
|
- `AGENTS.md`
|
||||||
|
- `CLAUDE.md`
|
||||||
|
- `*.instructions.md` files
|
||||||
|
- `*.prompt.md` files
|
||||||
|
- Creating prompt files, custom agents, hooks, or Agent Skills.
|
||||||
|
- Deciding whether behavior belongs in instructions, prompts, skills, agents, hooks, MCP servers, or agent plugins.
|
||||||
|
- Debugging why a customization is not discovered, loaded, or invoked.
|
||||||
|
- Adding a new documentation-backed skill to this `personal-mcp` repository.
|
||||||
|
|
||||||
|
## Start With The Decision
|
||||||
|
|
||||||
|
Choose the smallest customization that matches the desired behavior:
|
||||||
|
|
||||||
|
1. Use always-on instructions for project-wide coding standards, architecture decisions, security rules, and documentation standards that should apply to most requests.
|
||||||
|
2. Use file-based instructions for conventions that only apply to matching files, folders, languages, frameworks, or documentation types.
|
||||||
|
3. Use prompt files for reusable slash commands that package a single recurring prompt.
|
||||||
|
4. Use Agent Skills for portable, task-specific workflows that may include references, scripts, examples, or templates.
|
||||||
|
5. Use custom agents for specialized personas, tool restrictions, model choices, or role-specific workflows.
|
||||||
|
6. Use hooks when a deterministic lifecycle action must enforce a policy, run a command, or block unsafe behavior.
|
||||||
|
7. Use MCP servers when the agent needs live external tools, structured resources, or discoverable data beyond static instruction files.
|
||||||
|
8. Use agent plugins when several related customizations should ship together as an installable package.
|
||||||
|
|
||||||
|
If the request is ambiguous, ask only for the missing axis that changes the file type: scope, trigger, expected output, required tools, or whether it must be portable beyond VS Code.
|
||||||
|
|
||||||
|
## Research Map
|
||||||
|
|
||||||
|
Use [VS Code customization references](./references/vscode-customization.md) for official-source details about locations, frontmatter, discovery behavior, priority, and troubleshooting.
|
||||||
|
|
||||||
|
## Repo Shim Pattern For Personal MCP
|
||||||
|
|
||||||
|
Use a shim when you want another repository to consume this server as a preference and documentation source without duplicating methodology content.
|
||||||
|
|
||||||
|
### What the shim does
|
||||||
|
|
||||||
|
1. Tells the agent when to consult this MCP server.
|
||||||
|
2. Tells the agent how to retrieve relevant guidance.
|
||||||
|
3. Keeps repo-local behavior thin while canonical guidance stays in Personal MCP resources.
|
||||||
|
|
||||||
|
### Shim formats
|
||||||
|
|
||||||
|
Use either:
|
||||||
|
|
||||||
|
1. A repo instruction file (`*.instructions.md`) for always-on or file-scoped behavior.
|
||||||
|
2. A prompt file (`*.prompt.md`) for explicit, on-demand guidance retrieval.
|
||||||
|
|
||||||
|
### Retrieval strategies
|
||||||
|
|
||||||
|
Choose one of these patterns:
|
||||||
|
|
||||||
|
1. Direct URI strategy:
|
||||||
|
- Read `skill://<skill-name>/SKILL.md` when the required skill is known.
|
||||||
|
- Read `skill://<skill-name>/_manifest` only when supporting material may be useful.
|
||||||
|
- Read selected supporting files at `skill://<skill-name>/<supporting-path>`.
|
||||||
|
2. Discovery-first strategy:
|
||||||
|
- List resources, compare native main-resource names and descriptions, then load the best matching `SKILL.md`.
|
||||||
|
|
||||||
|
### Authoring guidance for shims
|
||||||
|
|
||||||
|
1. Keep shim content short and procedural; avoid copying large guidance blocks from Personal MCP.
|
||||||
|
2. State trigger conditions clearly (for example: "when creating a new skill" or "when editing docs contracts").
|
||||||
|
3. Specify whether to use a direct native URI or resource listing for that repo's common workflows.
|
||||||
|
4. Prefer loading only the most relevant main file first; inspect its manifest only when needed.
|
||||||
|
5. For stable repeated workflows, use explicit URIs. For broader or ambiguous requests, use discovery-first.
|
||||||
|
|
||||||
|
### Minimal shim examples
|
||||||
|
|
||||||
|
Instruction-style shim intent:
|
||||||
|
|
||||||
|
1. "For markdown edits (`applyTo: '**/*.md'`), load `skill://zensical-docs/SKILL.md` and apply Zensical-native documentation conventions unless they conflict with expected MkDocs compatibility."
|
||||||
|
|
||||||
|
Prompt-style shim intent:
|
||||||
|
|
||||||
|
1. "For docs authoring tasks, consult `skill://zensical-docs/SKILL.md`, summarize the relevant authoring constraints, then propose the smallest markdown change for this repository."
|
||||||
|
|
||||||
|
### Validation for shim implementation
|
||||||
|
|
||||||
|
1. Confirm the shim triggers in expected contexts.
|
||||||
|
2. Confirm resource loading path is unambiguous (direct URI or discovery).
|
||||||
|
3. Confirm repo-local customization remains thin and references Personal MCP as source of truth.
|
||||||
|
|
||||||
|
## Workspace Customization Workflow
|
||||||
|
|
||||||
|
1. Identify the customization primitive and scope.
|
||||||
|
2. Check existing files before creating a new one.
|
||||||
|
3. Keep the description or frontmatter trigger specific and keyword-rich.
|
||||||
|
4. Keep instructions concise, focused, and self-contained.
|
||||||
|
5. Add examples only when they clarify a non-obvious convention.
|
||||||
|
6. For `*.instructions.md`, set `applyTo` only when automatic file matching is intended.
|
||||||
|
7. For skills, make the folder name match the `name` field exactly and reference any extra files from `SKILL.md` with relative links.
|
||||||
|
8. Validate placement, YAML frontmatter, discovery settings, and whether the customization should be workspace or user scoped.
|
||||||
|
|
||||||
|
## Quality Checks
|
||||||
|
|
||||||
|
Before finishing:
|
||||||
|
|
||||||
|
1. Confirm the customization file is in a supported location for its intended scope.
|
||||||
|
2. Confirm required frontmatter fields are present and valid.
|
||||||
|
3. Confirm names match directory names where VS Code requires it.
|
||||||
|
4. Confirm descriptions include the phrases users are likely to ask for.
|
||||||
|
5. Confirm extra skill resources are linked from `SKILL.md`.
|
||||||
|
6. Confirm native discovery exposes `skill://<skill-name>/SKILL.md`, `_manifest`, and supporting-file reads.
|
||||||
|
7. State any remaining ambiguity or user choice, such as personal vs workspace scope.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
Return the concrete customization created or changed, where it lives, how to invoke or trigger it, and any validation performed.
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
# VS Code Copilot Customization References
|
||||||
|
|
||||||
|
Use these notes as a source map before creating or debugging Copilot customizations.
|
||||||
|
|
||||||
|
## Official Sources
|
||||||
|
|
||||||
|
!!! info "Official sources"
|
||||||
|
- [Customization overview](https://code.visualstudio.com/docs/copilot/customization/overview)
|
||||||
|
- [Custom instructions](https://code.visualstudio.com/docs/copilot/customization/custom-instructions)
|
||||||
|
- [Agent skills](https://code.visualstudio.com/docs/copilot/customization/agent-skills)
|
||||||
|
- [Prompt files](https://code.visualstudio.com/docs/copilot/customization/prompt-files)
|
||||||
|
- [Custom agents](https://code.visualstudio.com/docs/copilot/customization/custom-agents)
|
||||||
|
- [MCP servers](https://code.visualstudio.com/docs/copilot/customization/mcp-servers)
|
||||||
|
- [Hooks](https://code.visualstudio.com/docs/copilot/customization/hooks)
|
||||||
|
|
||||||
|
## Customization Types
|
||||||
|
|
||||||
|
- Instructions describe standards and conventions that apply to every request or to matching files.
|
||||||
|
- Prompt files save reusable slash-command prompts for recurring tasks.
|
||||||
|
- Agent Skills package reusable workflows, scripts, examples, and resources that load on demand.
|
||||||
|
- Custom agents define specialized personas, tool access, model choices, and role-specific workflows.
|
||||||
|
- MCP servers connect the agent to external tools, resources, and data.
|
||||||
|
- Hooks run deterministic actions at defined lifecycle points.
|
||||||
|
- Agent plugins bundle related customization types into an installable package.
|
||||||
|
|
||||||
|
## Instructions
|
||||||
|
|
||||||
|
Use `.github/copilot-instructions.md` for workspace-wide rules that should be included in every chat request. Use `AGENTS.md` when multiple agents should share the same repository guidance, or when nested agent guidance is useful. Use `CLAUDE.md` for Claude-compatible instruction sharing.
|
||||||
|
|
||||||
|
Use `.github/instructions/**/*.instructions.md` for file-based or task-specific rules. Supported frontmatter fields include:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
name: Documentation Standards
|
||||||
|
description: Rules for documentation writing tasks
|
||||||
|
applyTo: '**/*.md'
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
`applyTo` is a workspace-relative glob. If it is omitted, the instruction file can still be manually attached but does not automatically apply by file match.
|
||||||
|
|
||||||
|
## Agent Skills
|
||||||
|
|
||||||
|
Skills live in a directory whose name must match the `name` field in `SKILL.md`. VS Code supports project skills in `.github/skills/`, `.claude/skills/`, and `.agents/skills/`, and personal skills under user-level skill folders.
|
||||||
|
|
||||||
|
Required skill frontmatter:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
name: skill-name
|
||||||
|
description: Description of what the skill does and when to use it.
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
Useful optional fields:
|
||||||
|
|
||||||
|
- `argument-hint`: shown when invoking the skill as a slash command.
|
||||||
|
- `user-invocable`: controls whether it appears in the slash menu.
|
||||||
|
- `disable-model-invocation`: controls whether the model can auto-load it.
|
||||||
|
- `context`: can use `fork` for a separate subagent context when supported.
|
||||||
|
|
||||||
|
Skills load progressively: discovery reads frontmatter, instruction loading reads `SKILL.md`, and extra resources load only when linked from the skill document.
|
||||||
|
|
||||||
|
## Priority And Discovery
|
||||||
|
|
||||||
|
When multiple instruction sources apply, personal instructions have higher priority than repository instructions, and repository instructions have higher priority than organization instructions. If multiple instruction files exist, VS Code combines them; do not rely on ordering between instruction files.
|
||||||
|
|
||||||
|
For monorepos, `chat.useCustomizationsInParentRepositories` can enable discovery from a trusted parent repository root. Skill locations can also be configured with `chat.agentSkillsLocations`, and instruction locations with `chat.instructionsFilesLocations`.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
If a customization is not applied:
|
||||||
|
|
||||||
|
1. Confirm the file is in a supported location.
|
||||||
|
2. Confirm frontmatter is valid YAML.
|
||||||
|
3. Confirm skill `name` matches the parent directory.
|
||||||
|
4. Confirm `applyTo` matches the file path when using `*.instructions.md`.
|
||||||
|
5. Confirm relevant settings are enabled, such as instruction inclusion, referenced instruction inclusion, or AGENTS/CLAUDE support.
|
||||||
|
6. Use the Chat customization diagnostics view or Agent Debug Logs to inspect what VS Code loaded.
|
||||||
|
|
||||||
|
## Writing Effective Instructions
|
||||||
|
|
||||||
|
Keep instructions short, self-contained, and focused on non-obvious rules. Include the reason for a rule when it helps with edge cases. Prefer concrete examples over abstract preferences. Split unrelated rules into separate targeted files when they have different triggers.
|
||||||
+10
-11
@@ -1,7 +1,6 @@
|
|||||||
---
|
---
|
||||||
name: fastapi-uv-docker
|
name: fastapi-uv-docker
|
||||||
description: 'Audit and migrate an existing Python project to best practices for a cloud-native ASGI FastAPI app managed with uv and run with uvicorn in Docker. Use when: conforming a project to production standards, setting up src layout, configuring pyproject.toml, writing multi-stage Dockerfiles, wiring lifespan and settings, adding health endpoints, enforcing non-root container user, migrating from requirements.txt to uv.'
|
description: 'Audit and migrate an existing Python project to best practices for a cloud-native ASGI FastAPI app managed with uv and run with uvicorn in Docker. Use when: conforming a project to production standards, setting up src layout, configuring pyproject.toml, writing multi-stage Dockerfiles, wiring lifespan and settings, adding health endpoints, enforcing non-root container user, migrating from requirements.txt to uv.'
|
||||||
argument-hint: 'What is the current state of the project (bare Python, requirements.txt, pip, etc.)?'
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# FastAPI Project Best Practices
|
# FastAPI Project Best Practices
|
||||||
@@ -22,10 +21,10 @@ Bring an existing Python project into full conformance with cloud-native best pr
|
|||||||
|
|
||||||
Load these references only when needed:
|
Load these references only when needed:
|
||||||
|
|
||||||
- FastAPI patterns and app structure: [./references/fastapi-best-practices.md](./references/fastapi-best-practices.md)
|
- FastAPI patterns and app structure: [FastAPI best practices](./references/fastapi-best-practices.md)
|
||||||
- uv project layout and dependency management: [./references/uv-project-layout.md](./references/uv-project-layout.md)
|
- uv project layout and dependency management: [uv project layout](./references/uv-project-layout.md)
|
||||||
- uvicorn CLI settings reference: [./references/uvicorn-settings.md](./references/uvicorn-settings.md)
|
- uvicorn CLI settings reference: [uvicorn settings](./references/uvicorn-settings.md)
|
||||||
- Docker and cloud-native patterns: [./references/docker-cloud-native.md](./references/docker-cloud-native.md)
|
- Docker and cloud-native patterns: [Docker cloud-native patterns](./references/docker-cloud-native.md)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -44,9 +43,9 @@ Before making changes, map the current state across six areas. Produce a short g
|
|||||||
| **Container** | Is there a `Dockerfile`? Multi-stage? Non-root user? `.dockerignore` present? |
|
| **Container** | Is there a `Dockerfile`? Multi-stage? Non-root user? `.dockerignore` present? |
|
||||||
| **Cloud-native** | Is there a `/healthz` endpoint? Graceful shutdown? Structured logs? |
|
| **Cloud-native** | Is there a `/healthz` endpoint? Graceful shutdown? Structured logs? |
|
||||||
|
|
||||||
Load [./references/fastapi-best-practices.md](./references/fastapi-best-practices.md) for structure rules.
|
Load the [FastAPI best practices reference](./references/fastapi-best-practices.md) for structure rules.
|
||||||
Load [./references/uv-project-layout.md](./references/uv-project-layout.md) for uv migration rules.
|
Load the [uv project layout reference](./references/uv-project-layout.md) for uv migration rules.
|
||||||
Load [./references/uvicorn-settings.md](./references/uvicorn-settings.md) for uvicorn CLI reference.
|
Load the [uvicorn settings reference](./references/uvicorn-settings.md) for uvicorn CLI reference.
|
||||||
|
|
||||||
Completion check: You can name every gap before touching any file.
|
Completion check: You can name every gap before touching any file.
|
||||||
|
|
||||||
@@ -145,7 +144,7 @@ Completion check: `uv run python -m my_app` starts the server.
|
|||||||
|
|
||||||
### Step 3: Wire the FastAPI App Factory
|
### Step 3: Wire the FastAPI App Factory
|
||||||
|
|
||||||
Load [./references/fastapi-best-practices.md](./references/fastapi-best-practices.md) for the full patterns. Key rules:
|
Load the [FastAPI best practices reference](./references/fastapi-best-practices.md) for the full patterns. Key rules:
|
||||||
|
|
||||||
**`src/my_app/main.py`:**
|
**`src/my_app/main.py`:**
|
||||||
|
|
||||||
@@ -211,7 +210,7 @@ Completion check: `uv run uvicorn my_app.main:app --reload` starts with no impor
|
|||||||
|
|
||||||
### Step 4: uvicorn Production Configuration
|
### Step 4: uvicorn Production Configuration
|
||||||
|
|
||||||
Load [./references/uvicorn-settings.md](./references/uvicorn-settings.md) for the full settings reference.
|
Load the [uvicorn settings reference](./references/uvicorn-settings.md) for the full settings reference.
|
||||||
|
|
||||||
**Never** configure uvicorn inside application code. Pass all settings via CLI or environment variables (`UVICORN_*` prefix).
|
**Never** configure uvicorn inside application code. Pass all settings via CLI or environment variables (`UVICORN_*` prefix).
|
||||||
|
|
||||||
@@ -270,7 +269,7 @@ Completion check: `curl http://localhost:8000/healthz` returns `{"status":"ok"}`
|
|||||||
|
|
||||||
### Step 5: Write the Dockerfile
|
### Step 5: Write the Dockerfile
|
||||||
|
|
||||||
Load [./references/docker-cloud-native.md](./references/docker-cloud-native.md) for the full template and cloud-native rules. Key requirements:
|
Load the [Docker cloud-native patterns reference](./references/docker-cloud-native.md) for the full template and cloud-native rules. Key requirements:
|
||||||
|
|
||||||
- Multi-stage build: `builder` stage installs deps; `runtime` stage is slim.
|
- Multi-stage build: `builder` stage installs deps; `runtime` stage is slim.
|
||||||
- Pin uv version (copy from official image, not `latest`).
|
- Pin uv version (copy from official image, not `latest`).
|
||||||
+5
-1
@@ -1,6 +1,10 @@
|
|||||||
# Docker and Cloud-Native Patterns
|
# Docker and Cloud-Native Patterns
|
||||||
|
|
||||||
Source: https://docs.docker.com/build/building/best-practices/ | https://docs.astral.sh/uv/guides/integration/docker/ | https://fastapi.tiangolo.com/deployment/docker/ | https://uvicorn.dev/deployment/
|
!!! info "Primary sources"
|
||||||
|
- [Docker build best practices](https://docs.docker.com/build/building/best-practices/)
|
||||||
|
- [uv Docker integration](https://docs.astral.sh/uv/guides/integration/docker/)
|
||||||
|
- [FastAPI Docker deployment](https://fastapi.tiangolo.com/deployment/docker/)
|
||||||
|
- [uvicorn deployment](https://uvicorn.dev/deployment/)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
+5
-2
@@ -1,6 +1,8 @@
|
|||||||
# FastAPI Best Practices
|
# FastAPI Best Practices
|
||||||
|
|
||||||
Source: https://fastapi.tiangolo.com/deployment/ | https://fastapi.tiangolo.com/advanced/events/
|
!!! info "Primary sources"
|
||||||
|
- [FastAPI deployment](https://fastapi.tiangolo.com/deployment/)
|
||||||
|
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -44,7 +46,8 @@ def create_app(settings: Settings | None = None) -> FastAPI:
|
|||||||
app = create_app()
|
app = create_app()
|
||||||
```
|
```
|
||||||
|
|
||||||
**Never use `@app.on_event("startup")` / `@app.on_event("shutdown")`** — these are deprecated. The `asynccontextmanager` lifespan is the canonical approach since FastAPI 0.95.
|
!!! warning "Prefer lifespan handlers"
|
||||||
|
Never use `@app.on_event("startup")` / `@app.on_event("shutdown")`. These are deprecated. The `asynccontextmanager` lifespan is the canonical approach since FastAPI 0.95.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
+6
-2
@@ -1,6 +1,9 @@
|
|||||||
# uv Project Layout and Dependency Management
|
# uv Project Layout and Dependency Management
|
||||||
|
|
||||||
Source: https://docs.astral.sh/uv/guides/projects/ | https://docs.astral.sh/uv/concepts/projects/layout/ | https://docs.astral.sh/uv/guides/integration/docker/
|
!!! info "Primary sources"
|
||||||
|
- [uv project guide](https://docs.astral.sh/uv/guides/projects/)
|
||||||
|
- [uv project layout](https://docs.astral.sh/uv/concepts/projects/layout/)
|
||||||
|
- [uv Docker integration](https://docs.astral.sh/uv/guides/integration/docker/)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -13,7 +16,8 @@ Source: https://docs.astral.sh/uv/guides/projects/ | https://docs.astral.sh/uv/c
|
|||||||
| `.python-version` | Default Python version for the project | Yes |
|
| `.python-version` | Default Python version for the project | Yes |
|
||||||
| `.venv/` | Local virtual environment | No (`.gitignore`) |
|
| `.venv/` | Local virtual environment | No (`.gitignore`) |
|
||||||
|
|
||||||
**`uv.lock` must be committed.** It is the source of truth for reproducible installs in CI and Docker. Never edit it by hand.
|
!!! warning "Commit the lockfile"
|
||||||
|
`uv.lock` must be committed. It is the source of truth for reproducible installs in CI and Docker. Never edit it by hand.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
+5
-2
@@ -1,6 +1,8 @@
|
|||||||
# uvicorn Settings Reference
|
# uvicorn Settings Reference
|
||||||
|
|
||||||
Source: https://uvicorn.dev/settings/ | https://uvicorn.dev/deployment/
|
!!! info "Primary sources"
|
||||||
|
- [uvicorn settings](https://uvicorn.dev/settings/)
|
||||||
|
- [uvicorn deployment](https://uvicorn.dev/deployment/)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -21,7 +23,8 @@ uvicorn main:app
|
|||||||
uvicorn.run("main:app", host="0.0.0.0", port=8000)
|
uvicorn.run("main:app", host="0.0.0.0", port=8000)
|
||||||
```
|
```
|
||||||
|
|
||||||
**Note:** `UVICORN_*` env vars cannot be used from within an `--env-file`. The `--env-file` flag is for the ASGI *application's* config, not uvicorn's own config.
|
!!! note "Environment file scope"
|
||||||
|
`UVICORN_*` env vars cannot be used from within an `--env-file`. The `--env-file` flag is for the ASGI *application's* config, not uvicorn's own config.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
---
|
||||||
|
name: mcp-details
|
||||||
|
description: "Reference hub for MCP and FastMCP source documentation links. Use when you need authoritative protocol, SDK, transport, and deployment docs without loading broad implementation guidance."
|
||||||
|
---
|
||||||
|
|
||||||
|
# MCP Details
|
||||||
|
|
||||||
|
This skill is a reference index only. It is optimized for fast retrieval of upstream documentation links for MCP and FastMCP.
|
||||||
|
|
||||||
|
## When to Use
|
||||||
|
|
||||||
|
- You need official MCP protocol and architecture docs.
|
||||||
|
- You need MCP SDK links for Python or TypeScript.
|
||||||
|
- You need FastMCP docs and source references.
|
||||||
|
- You need ecosystem links for tooling, inspection, and client configuration.
|
||||||
|
|
||||||
|
## How To Use This Skill
|
||||||
|
|
||||||
|
1. Classify the request by intent: protocol, SDK usage, FastMCP, or ecosystem integration.
|
||||||
|
2. Open only the matching reference page first.
|
||||||
|
3. Load at most one additional reference page if the request spans multiple areas.
|
||||||
|
4. Return links grouped by category, with a one-line reason for each group.
|
||||||
|
|
||||||
|
## Intent Router
|
||||||
|
|
||||||
|
1. MCP fundamentals, protocol architecture, resources, tools, prompts, transports, security: [mcp-protocol-and-spec.md](./references/mcp-protocol-and-spec.md)
|
||||||
|
2. MCP SDK and FastMCP implementation references for Python and TypeScript: [sdk-and-fastmcp.md](./references/sdk-and-fastmcp.md)
|
||||||
|
3. MCP client integration and operational tooling references: [ecosystem-and-tooling.md](./references/ecosystem-and-tooling.md)
|
||||||
|
|
||||||
|
## Load Order
|
||||||
|
|
||||||
|
1. Start with the single best-match reference page from the Intent Router.
|
||||||
|
2. If the question includes both protocol and implementation details, load [mcp-protocol-and-spec.md](./references/mcp-protocol-and-spec.md) then [sdk-and-fastmcp.md](./references/sdk-and-fastmcp.md).
|
||||||
|
3. Load [ecosystem-and-tooling.md](./references/ecosystem-and-tooling.md) only when the request includes client setup, inspector usage, or deployment/operations context.
|
||||||
|
|
||||||
|
## Load Budget
|
||||||
|
|
||||||
|
1. Single-focus request: 1 reference page.
|
||||||
|
2. Mixed protocol and implementation request: 2 reference pages.
|
||||||
|
3. Broad audit or migration planning request: up to 3 reference pages.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
When this skill is applied, return:
|
||||||
|
1. Which reference files were consulted.
|
||||||
|
2. The discovery path used (intent classification and load order).
|
||||||
|
3. Curated source-document links grouped by topic.
|
||||||
|
4. Any notable gaps or ambiguities in the currently indexed links.
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
# Ecosystem and Tooling
|
||||||
|
|
||||||
|
Use this page for MCP client setup, operational tools, and integration references.
|
||||||
|
|
||||||
|
## VS Code and Copilot MCP Integration
|
||||||
|
|
||||||
|
!!! info "VS Code MCP docs"
|
||||||
|
- [VS Code MCP servers overview](https://code.visualstudio.com/docs/agent-customization/mcp-servers)
|
||||||
|
- [VS Code MCP configuration reference](https://code.visualstudio.com/docs/agents/reference/mcp-configuration)
|
||||||
|
- [VS Code MCP developer guide](https://code.visualstudio.com/docs/agents/guides/mcp-developer-guide)
|
||||||
|
- [VS Code MCP Apps support](https://code.visualstudio.com/blogs/2026/01/26/mcp-apps-support)
|
||||||
|
- [VS Code Copilot customization overview](https://code.visualstudio.com/docs/copilot/customization/overview)
|
||||||
|
|
||||||
|
## Debugging and Inspection
|
||||||
|
|
||||||
|
!!! info "Inspector and diagnostics"
|
||||||
|
- [MCP inspector repository](https://github.com/modelcontextprotocol/inspector)
|
||||||
|
- [MCP protocol repository issues](https://github.com/modelcontextprotocol/spec/issues)
|
||||||
|
- [Python logging configuration docs](https://docs.python.org/3/library/logging.config.html)
|
||||||
|
|
||||||
|
## Runtime and API Framework References
|
||||||
|
|
||||||
|
!!! info "Runtime references"
|
||||||
|
- [FastAPI documentation](https://fastapi.tiangolo.com/)
|
||||||
|
- [Uvicorn settings](https://www.uvicorn.org/settings/)
|
||||||
|
- [AnyIO documentation](https://anyio.readthedocs.io/en/stable/)
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Use these links when tasks include IDE wiring, MCP server runtime setup, or production operations.
|
||||||
|
- Keep protocol and SDK references separate to avoid overloading implementation prompts.
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# MCP Protocol and Specification
|
||||||
|
|
||||||
|
Use this page for authoritative links about MCP concepts, protocol shape, and official specification assets.
|
||||||
|
|
||||||
|
## Official Documentation
|
||||||
|
|
||||||
|
!!! info "MCP docs"
|
||||||
|
- [MCP introduction](https://modelcontextprotocol.io/docs/getting-started/intro)
|
||||||
|
- [Architecture overview](https://modelcontextprotocol.io/docs/learn/architecture)
|
||||||
|
- [Server concepts](https://modelcontextprotocol.io/docs/learn/server-concepts)
|
||||||
|
- [Client concepts](https://modelcontextprotocol.io/docs/learn/client-concepts)
|
||||||
|
- [Security overview](https://modelcontextprotocol.io/docs/learn/security-overview)
|
||||||
|
|
||||||
|
## Protocol and Schema Sources
|
||||||
|
|
||||||
|
!!! info "Specification repositories"
|
||||||
|
- [MCP specification repository](https://github.com/modelcontextprotocol/spec)
|
||||||
|
- [Specification schema directory](https://github.com/modelcontextprotocol/spec/tree/main/schema)
|
||||||
|
- [Specification issues and proposals](https://github.com/modelcontextprotocol/spec/issues)
|
||||||
|
|
||||||
|
## Core Capability References
|
||||||
|
|
||||||
|
!!! info "Capability details"
|
||||||
|
- [Resources concept docs](https://modelcontextprotocol.io/docs/learn/server-concepts#resources)
|
||||||
|
- [Tools concept docs](https://modelcontextprotocol.io/docs/learn/server-concepts#tools)
|
||||||
|
- [Prompt objects concept docs](https://modelcontextprotocol.io/docs/learn/server-concepts#prompts)
|
||||||
|
- [Sampling concept docs](https://modelcontextprotocol.io/docs/learn/client-concepts)
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Prefer these links when the user asks about protocol correctness, transport semantics, capability naming, or compatibility.
|
||||||
|
- For implementation-level examples, use [sdk-and-fastmcp.md](./sdk-and-fastmcp.md).
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
# SDK and FastMCP
|
||||||
|
|
||||||
|
Use this page for implementation-oriented links across MCP SDKs and FastMCP.
|
||||||
|
|
||||||
|
## MCP SDKs
|
||||||
|
|
||||||
|
!!! info "SDK sources"
|
||||||
|
- [Python SDK repository](https://github.com/modelcontextprotocol/python-sdk)
|
||||||
|
- [TypeScript SDK repository](https://github.com/modelcontextprotocol/typescript-sdk)
|
||||||
|
- [Python SDK documentation](https://modelcontextprotocol.github.io/python-sdk/)
|
||||||
|
|
||||||
|
## FastMCP
|
||||||
|
|
||||||
|
!!! info "FastMCP sources"
|
||||||
|
- [FastMCP project documentation](https://gofastmcp.com/)
|
||||||
|
- [FastMCP server identity and behavior](https://gofastmcp.com/servers/server)
|
||||||
|
- [FastMCP tools and annotations](https://gofastmcp.com/servers/tools)
|
||||||
|
- [FastMCP resources and templates](https://gofastmcp.com/servers/resources)
|
||||||
|
- [FastMCP prompts](https://gofastmcp.com/servers/prompts)
|
||||||
|
- [FastMCP argument completion](https://gofastmcp.com/servers/completions)
|
||||||
|
- [FastMCP component icons](https://gofastmcp.com/servers/icons)
|
||||||
|
- [FastMCP Apps](https://gofastmcp.com/apps/overview)
|
||||||
|
- [FastMCP GitHub repository](https://github.com/jlowin/fastmcp)
|
||||||
|
- [FastMCP examples directory](https://github.com/jlowin/fastmcp/tree/main/examples)
|
||||||
|
- [FastMCP PyPI package](https://pypi.org/project/fastmcp/)
|
||||||
|
|
||||||
|
## Server Implementation Patterns
|
||||||
|
|
||||||
|
!!! info "Implementation references"
|
||||||
|
- [MCP server concepts](https://modelcontextprotocol.io/docs/learn/server-concepts)
|
||||||
|
- [MCP architecture patterns](https://modelcontextprotocol.io/docs/learn/architecture)
|
||||||
|
- [Python packaging and resources](https://docs.python.org/3/library/importlib.resources.html)
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- Prefer official SDK repositories for API shape and compatibility checks.
|
||||||
|
- Use FastMCP references for rapid server scaffolding and implementation examples.
|
||||||
|
- For protocol-first questions, start from [mcp-protocol-and-spec.md](./mcp-protocol-and-spec.md).
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
---
|
||||||
|
name: nicegui
|
||||||
|
description: 'Reference hub for NiceGUI and FastAPI application structure, typed configuration, ASGI and Uvicorn startup, UI composition, styling, bindable state, interactions, troubleshooting, testing, and source documentation. Use when planning, implementing, reviewing, deploying, or debugging NiceGUI applications; load only the references relevant to the task.'
|
||||||
|
---
|
||||||
|
|
||||||
|
# NiceGUI Reference
|
||||||
|
|
||||||
|
Use this skill as a progressive reference for NiceGUI applications built with FastAPI. Start with the routing map, load only the material needed for the current question, and reconcile it with the target project's NiceGUI version and established conventions.
|
||||||
|
|
||||||
|
## When to Use
|
||||||
|
|
||||||
|
- Planning or reviewing NiceGUI application structure and FastAPI composition.
|
||||||
|
- Building or refactoring pages, components, layouts, and static assets.
|
||||||
|
- Modeling UI state with bindings or bindable dataclasses.
|
||||||
|
- Implementing forms, uploads, refreshes, live updates, or background work.
|
||||||
|
- Diagnosing UI state, concurrency, navigation, or asset problems.
|
||||||
|
- Verifying framework behavior against primary documentation.
|
||||||
|
|
||||||
|
## How to Use This Skill
|
||||||
|
|
||||||
|
1. Classify the request using the discovery map below.
|
||||||
|
2. Load the smallest relevant reference, or at most two references for a mixed concern.
|
||||||
|
3. Inspect the target repository before applying guidance; preserve its sound local patterns.
|
||||||
|
4. Check the pinned NiceGUI and integration versions before relying on version-specific APIs.
|
||||||
|
5. Validate the changed behavior with focused tests and, for UI work, relevant viewport checks.
|
||||||
|
|
||||||
|
## Progressive Discovery Map
|
||||||
|
|
||||||
|
### Application Architecture
|
||||||
|
|
||||||
|
Load [application architecture](./references/architecture.md) for:
|
||||||
|
|
||||||
|
- FastAPI app factories and lifespan ownership
|
||||||
|
- package boundaries and dependency direction
|
||||||
|
- page registration and health routes
|
||||||
|
- optional persistence, LangGraph, or mounted documentation
|
||||||
|
- async responsiveness and baseline tests
|
||||||
|
|
||||||
|
### FastAPI And Uvicorn Startup
|
||||||
|
|
||||||
|
Load [FastAPI and Uvicorn startup](./references/fastapi-uvicorn-startup.md) for:
|
||||||
|
|
||||||
|
- choosing between `ui.run()` and `ui.run_with()`
|
||||||
|
- understanding the parent FastAPI app and NiceGUI's internal app
|
||||||
|
- composing ASGI lifespan and mounted routes
|
||||||
|
- loading one typed settings snapshot for server and application configuration
|
||||||
|
- serving an app instance or factory with Uvicorn
|
||||||
|
- exposing programmatic startup through `[project.scripts]`
|
||||||
|
- reload, worker, and process-local state constraints
|
||||||
|
|
||||||
|
### Styling And Customization
|
||||||
|
|
||||||
|
Load [styling and customization](./references/styling-and-customization.md) for:
|
||||||
|
|
||||||
|
- progressive discovery through NiceGUI docs, constructors, and Quasar docs
|
||||||
|
- Quasar props, slots, events, and NiceGUI customization methods
|
||||||
|
- app-wide and page-level color themes, dark mode, and semantic CSS tokens
|
||||||
|
- Tailwind for structural styling and static stylesheets for fine tuning
|
||||||
|
- responsive layout and static asset conventions
|
||||||
|
- Tailwind and Quasar breakpoint scales, container queries, and responsive testing
|
||||||
|
- uniformly scaling dialogs on mobile
|
||||||
|
- preserving Quasar field proportions
|
||||||
|
- keeping detached `QSelect` menus anchored
|
||||||
|
- sizing scrollable dialog cards under CSS `zoom`
|
||||||
|
- validating zoomed controls with Playwright or a browser
|
||||||
|
|
||||||
|
### Special Component Customization
|
||||||
|
|
||||||
|
Load [special component customization](./references/special-component-customization.md) for:
|
||||||
|
|
||||||
|
- the required source-research gate before generating component customizations
|
||||||
|
- `ui.select` constructors, Quasar props, slots, detached popups, and option caveats
|
||||||
|
- `ui.icon` names, icon families, sizing, colors, assets, and Material Symbol variants
|
||||||
|
- component-specific accessibility, sanitization, and validation checks
|
||||||
|
|
||||||
|
### Bindable State
|
||||||
|
|
||||||
|
Load [bindable dataclasses](./references/binding-dataclasses.md) for:
|
||||||
|
|
||||||
|
- typed local UI state
|
||||||
|
- propagation and refresh behavior
|
||||||
|
- nested structures and strict bindings
|
||||||
|
- mutable defaults, performance, and version notes
|
||||||
|
|
||||||
|
### Interaction Patterns
|
||||||
|
|
||||||
|
Load [interaction patterns](./references/interaction-patterns.md) for:
|
||||||
|
|
||||||
|
- uploads and form submission
|
||||||
|
- explicit refreshes
|
||||||
|
- server-sent events and WebSockets
|
||||||
|
- background work and duplicate-submission guards
|
||||||
|
|
||||||
|
### Troubleshooting And Quality
|
||||||
|
|
||||||
|
Load [troubleshooting and quality gates](./references/troubleshooting-and-quality-gates.md) for:
|
||||||
|
|
||||||
|
- upload failures and UI race conditions
|
||||||
|
- stale assets and navigation drift
|
||||||
|
- responsiveness, accessibility, reliability, and maintainability checks
|
||||||
|
|
||||||
|
### Primary Sources
|
||||||
|
|
||||||
|
Load [source documentation](./references/source-documentation.md) when:
|
||||||
|
|
||||||
|
- behavior is version-sensitive or uncertain
|
||||||
|
- an integration recommendation needs verification
|
||||||
|
- upstream NiceGUI, FastAPI, Tailwind, Quasar, SQLAlchemy, Pydantic, or LangGraph documentation is required
|
||||||
|
|
||||||
|
## Common Discovery Paths
|
||||||
|
|
||||||
|
### New Application Or Architecture Review
|
||||||
|
|
||||||
|
1. Load [application architecture](./references/architecture.md).
|
||||||
|
2. Add [FastAPI and Uvicorn startup](./references/fastapi-uvicorn-startup.md) when FastAPI owns the application or startup must be exposed as a project command.
|
||||||
|
3. Add [styling and customization](./references/styling-and-customization.md) only when page layout or visual customization is in scope.
|
||||||
|
|
||||||
|
### Page Or Component Work
|
||||||
|
|
||||||
|
1. Load [application architecture](./references/architecture.md) for page and component ownership decisions.
|
||||||
|
2. Load [styling and customization](./references/styling-and-customization.md) for layout, responsive behavior, or visual customization.
|
||||||
|
3. Add [special component customization](./references/special-component-customization.md) when the work targets `ui.select`, `ui.icon`, or another component with specialized Quasar behavior.
|
||||||
|
4. Add [interaction patterns](./references/interaction-patterns.md) or [bindable dataclasses](./references/binding-dataclasses.md) according to the page behavior.
|
||||||
|
|
||||||
|
### Debugging Or Production Review
|
||||||
|
|
||||||
|
1. Start with [troubleshooting and quality gates](./references/troubleshooting-and-quality-gates.md).
|
||||||
|
2. Follow the symptom to one detailed reference.
|
||||||
|
3. Confirm uncertain behavior in [source documentation](./references/source-documentation.md).
|
||||||
|
|
||||||
|
## General Defaults
|
||||||
|
|
||||||
|
- Keep composition, transport, services, pages, and components directionally separated.
|
||||||
|
- Keep business logic out of UI components and event handlers.
|
||||||
|
- Avoid blocking I/O and CPU-heavy work in the UI event loop.
|
||||||
|
- Prefer event-driven updates and explicit refreshes over unrelated polling.
|
||||||
|
- Discover component capabilities through NiceGUI docs and constructors, then the wrapped Quasar API.
|
||||||
|
- Research the current NiceGUI and Quasar source documentation before generating component-specific code or CSS.
|
||||||
|
- Prefer constructor arguments and native Quasar features through NiceGUI; use Tailwind for structure and scoped static CSS for stable fine tuning.
|
||||||
|
- Provide loading, success, and failure states for user-triggered work.
|
||||||
|
- Treat version-specific guidance as a prompt to verify the project's dependency version.
|
||||||
|
|
||||||
|
## Reference Use Contract
|
||||||
|
|
||||||
|
When applying this skill:
|
||||||
|
|
||||||
|
- return only guidance relevant to the current task
|
||||||
|
- distinguish repository facts from reference recommendations
|
||||||
|
- cite the appropriate source reference for framework-level claims
|
||||||
|
- state assumptions when application requirements are missing
|
||||||
|
- report the focused checks used to validate implementation changes
|
||||||
@@ -0,0 +1,144 @@
|
|||||||
|
# NiceGUI Application Architecture
|
||||||
|
|
||||||
|
Load this reference for application composition, package boundaries, and optional subsystem decisions.
|
||||||
|
|
||||||
|
## Baseline Package Boundaries
|
||||||
|
|
||||||
|
- `main.py`: process entry point and app factory exposure.
|
||||||
|
- `bootstrap.py`: app composition, router wiring, page registration, and lifespan orchestration.
|
||||||
|
- `config.py`: typed settings and environment parsing.
|
||||||
|
- `logging.py`: centralized logging setup.
|
||||||
|
- `api/`: HTTP transport that delegates to services.
|
||||||
|
- `services/`: business and use-case logic.
|
||||||
|
- `ui/pages/`: route-level NiceGUI pages.
|
||||||
|
- `ui/components/`: shared presentation building blocks.
|
||||||
|
|
||||||
|
Recommended base shape:
|
||||||
|
|
||||||
|
```text
|
||||||
|
.
|
||||||
|
├─ pyproject.toml
|
||||||
|
├─ .env.example
|
||||||
|
├─ src/
|
||||||
|
│ └─ app/
|
||||||
|
│ ├─ __init__.py
|
||||||
|
│ ├─ main.py
|
||||||
|
│ ├─ config.py
|
||||||
|
│ ├─ logging.py
|
||||||
|
│ ├─ api/
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ └─ health.py
|
||||||
|
│ ├─ services/
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ └─ example_service.py
|
||||||
|
│ └─ ui/
|
||||||
|
│ ├─ __init__.py
|
||||||
|
│ ├─ components/
|
||||||
|
│ │ ├─ __init__.py
|
||||||
|
│ │ └─ nav.py
|
||||||
|
│ └─ pages/
|
||||||
|
│ ├─ __init__.py
|
||||||
|
│ ├─ home.py
|
||||||
|
│ ├─ dashboard.py
|
||||||
|
│ └─ about.py
|
||||||
|
└─ tests/
|
||||||
|
├─ test_health.py
|
||||||
|
└─ test_pages_registration.py
|
||||||
|
```
|
||||||
|
|
||||||
|
## Required Baseline Behavior
|
||||||
|
|
||||||
|
- FastAPI is the base ASGI app.
|
||||||
|
- `create_app()` composes routes, resources, and NiceGUI.
|
||||||
|
- Lifespan owns startup and shutdown resources.
|
||||||
|
- NiceGUI pages are modular and explicitly registered.
|
||||||
|
- FastAPI exposes a health route such as `/healthz`.
|
||||||
|
- Imports do not trigger runtime global side effects.
|
||||||
|
|
||||||
|
For the ownership relationship between a caller-created FastAPI app, `nicegui.app`, `ui.run_with()`, Uvicorn, and a packaged startup command, load [FastAPI and Uvicorn startup](./fastapi-uvicorn-startup.md).
|
||||||
|
|
||||||
|
## Dependency Direction
|
||||||
|
|
||||||
|
Prefer:
|
||||||
|
|
||||||
|
- `main/bootstrap` -> `config/logging` + `api` + `ui/pages` + `services`
|
||||||
|
- `api` -> `services`
|
||||||
|
- `ui/pages` -> `ui/components` + `services`
|
||||||
|
- `services` -> helpers, clients, and `db/` when enabled
|
||||||
|
|
||||||
|
Avoid imports from services back into API or UI modules.
|
||||||
|
|
||||||
|
## Page And Component Ownership
|
||||||
|
|
||||||
|
Page modules compose routes from presentation components and service calls. They should not own domain rules, persistence, or long-running synchronous work.
|
||||||
|
|
||||||
|
Extract a presentation pattern to `ui/components/` when it appears on two or more pages or owns a meaningful interaction boundary. Keep one-off route composition in the page module. Reusable components should accept data and event callbacks instead of importing page state or business services implicitly.
|
||||||
|
|
||||||
|
For page composition, responsive layout, Quasar props, and CSS customization, load [styling and customization](./styling-and-customization.md).
|
||||||
|
|
||||||
|
## Optional Persistence
|
||||||
|
|
||||||
|
Use only when the product requires durable data.
|
||||||
|
|
||||||
|
```text
|
||||||
|
src/app/db/
|
||||||
|
├─ __init__.py
|
||||||
|
├─ base.py
|
||||||
|
├─ session.py
|
||||||
|
├─ models/
|
||||||
|
└─ repositories/
|
||||||
|
```
|
||||||
|
|
||||||
|
- Create one engine and sessionmaker per process.
|
||||||
|
- Provide request- or operation-scoped sessions with `yield`.
|
||||||
|
- Keep transaction boundaries explicit in service or repository flows.
|
||||||
|
- Never share sessions across concurrent tasks.
|
||||||
|
- Use Alembic as the schema migration source of truth.
|
||||||
|
|
||||||
|
## Optional LangGraph AI
|
||||||
|
|
||||||
|
Use only for multi-step orchestration, resumable work, streaming, or human approval.
|
||||||
|
|
||||||
|
```text
|
||||||
|
src/app/ai/
|
||||||
|
├─ state.py
|
||||||
|
├─ nodes/
|
||||||
|
├─ graphs/
|
||||||
|
├─ runtime.py
|
||||||
|
└─ contracts.py
|
||||||
|
```
|
||||||
|
|
||||||
|
- Keep graph internals outside API and UI modules.
|
||||||
|
- Invoke graphs through a service such as `services/ai_service.py`.
|
||||||
|
- Use stable thread or session IDs for resumable flows.
|
||||||
|
- Keep interrupt payloads JSON-serializable.
|
||||||
|
|
||||||
|
## Optional Mounted Docs
|
||||||
|
|
||||||
|
Use only when generated docs must be served by the application.
|
||||||
|
|
||||||
|
Suggested settings:
|
||||||
|
|
||||||
|
- `docs_enabled`
|
||||||
|
- `docs_mount_path`
|
||||||
|
- `docs_site_dir`
|
||||||
|
- `docs_require_build`
|
||||||
|
|
||||||
|
Mount docs in the composition layer, normalize the mount path, avoid route conflicts, and define behavior for missing build artifacts.
|
||||||
|
|
||||||
|
## Async And Responsiveness
|
||||||
|
|
||||||
|
- Use `async def` where a handler or service path performs I/O.
|
||||||
|
- Prefer non-blocking clients and libraries.
|
||||||
|
- Offload CPU-heavy work to worker or background execution.
|
||||||
|
- Define progress, cancellation, timeout, completion, and error states for long actions.
|
||||||
|
- Stream or chunk results when workflows are long-running or multi-step.
|
||||||
|
|
||||||
|
## Testing Minimums
|
||||||
|
|
||||||
|
- Test the FastAPI health route.
|
||||||
|
- Test page registration wiring.
|
||||||
|
- If persistence is enabled, test session lifecycle and rollback behavior.
|
||||||
|
- If AI is enabled, test happy paths and interrupt/resume behavior.
|
||||||
|
- If docs are enabled, test the mounted index route.
|
||||||
|
- For long actions, test loading, completion, and error states.
|
||||||
@@ -0,0 +1,100 @@
|
|||||||
|
# Binding Dataclasses Deep Dive
|
||||||
|
|
||||||
|
Use this reference to model NiceGUI state with bindable dataclasses and avoid common propagation and performance pitfalls.
|
||||||
|
|
||||||
|
## Primary Sources
|
||||||
|
|
||||||
|
- NiceGUI binding docs: [binding properties](https://www.nicegui.io/documentation/section_binding_properties)
|
||||||
|
- Python dataclass docs: [dataclasses module](https://docs.python.org/3/library/dataclasses.html)
|
||||||
|
- Data class design rationale: [PEP 557](https://peps.python.org/pep-0557/)
|
||||||
|
|
||||||
|
## Bindable Dataclass Behavior
|
||||||
|
|
||||||
|
`@binding.bindable_dataclass` extends standard dataclasses by turning fields into bindable properties, allowing UI bindings to propagate when a field is assigned.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from nicegui import binding, ui
|
||||||
|
|
||||||
|
|
||||||
|
@binding.bindable_dataclass
|
||||||
|
class Profile:
|
||||||
|
name: str = "Ada"
|
||||||
|
age: int = 37
|
||||||
|
|
||||||
|
|
||||||
|
profile = Profile()
|
||||||
|
|
||||||
|
ui.input("Name").bind_value(profile, "name")
|
||||||
|
ui.number("Age", min=0).bind_value(profile, "age")
|
||||||
|
ui.label().bind_text_from(profile, "name", backward=lambda name: f"User: {name}")
|
||||||
|
```
|
||||||
|
|
||||||
|
## Propagation And Performance
|
||||||
|
|
||||||
|
NiceGUI distinguishes between two link types:
|
||||||
|
|
||||||
|
- Bindable properties propagate efficiently when values are assigned.
|
||||||
|
- Active links are checked in a refresh loop.
|
||||||
|
|
||||||
|
Prefer bindable dataclasses for frequently updated form state. Keep binding transforms pure and inexpensive. If an application has many active links, tune `binding_refresh_interval` in `ui.run(...)` only after measuring the impact.
|
||||||
|
|
||||||
|
## Dataclass Modeling Rules
|
||||||
|
|
||||||
|
- Use `field(default_factory=...)` for mutable defaults.
|
||||||
|
- Avoid `frozen=True` for models edited by UI controls.
|
||||||
|
- Use `slots=True` only after confirming compatibility with inheritance and extension needs.
|
||||||
|
- Keep UI-editable fields explicit and typed.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from dataclasses import field
|
||||||
|
|
||||||
|
from nicegui import binding
|
||||||
|
|
||||||
|
|
||||||
|
@binding.bindable_dataclass
|
||||||
|
class Filters:
|
||||||
|
query: str = ""
|
||||||
|
tags: list[str] = field(default_factory=list)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Nested Structures
|
||||||
|
|
||||||
|
NiceGUI supports tuple paths for nested data structures.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from nicegui import ui
|
||||||
|
|
||||||
|
data = {"user": {"name": "Ada"}}
|
||||||
|
|
||||||
|
ui.input("Name").bind_value(data, ("user", "name"))
|
||||||
|
ui.label().bind_text_from(data, ("user", "name"))
|
||||||
|
```
|
||||||
|
|
||||||
|
Keep nested dataclass updates explicit and predictable at the field level.
|
||||||
|
|
||||||
|
## Strictness And Refactor Safety
|
||||||
|
|
||||||
|
- Object attributes are checked by default.
|
||||||
|
- Dictionary keys are not checked by default.
|
||||||
|
- Use `strict=True` when missing dictionary keys should produce warnings.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from nicegui import app, ui
|
||||||
|
|
||||||
|
ui.input().bind_value(app.storage.user, "display_name", strict=True)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Common Pitfalls
|
||||||
|
|
||||||
|
- In-place mutation may not produce immediate UI synchronization. Assign the updated value back to the bound field.
|
||||||
|
- Heavy binding transforms can degrade refresh performance. Move expensive work to event handlers or services.
|
||||||
|
- State shared across unrelated pages or users can leak data. Scope models to the appropriate page, client, or user context.
|
||||||
|
|
||||||
|
## Version Checks
|
||||||
|
|
||||||
|
- `bindable_dataclass` was added in NiceGUI 2.11.0.
|
||||||
|
- Depth-first binding propagation was documented in NiceGUI 2.16.0.
|
||||||
|
- Binding `strict` behavior was documented in NiceGUI 3.0.0.
|
||||||
|
- Tuple paths for nested properties were documented in NiceGUI 3.10.0.
|
||||||
|
|
||||||
|
Verify these behaviors against the NiceGUI version pinned by the target project.
|
||||||
@@ -0,0 +1,315 @@
|
|||||||
|
# FastAPI And Uvicorn Startup
|
||||||
|
|
||||||
|
Use this reference when FastAPI owns the application and NiceGUI is one part of it. The central distinction is between **composing an ASGI application** and **starting an ASGI server**:
|
||||||
|
|
||||||
|
- [`ui.run_with()`](https://github.com/zauberzeug/nicegui/blob/main/nicegui/ui_run_with.py) composes NiceGUI with a caller-owned FastAPI application. It does not start Uvicorn.
|
||||||
|
- [`uvicorn.run()`](https://www.uvicorn.org/#running-programmatically) starts the server and tells it which ASGI application to serve.
|
||||||
|
|
||||||
|
## Ownership Model
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
flowchart TD
|
||||||
|
E["Project script: my-app"] --> M["main()"]
|
||||||
|
M --> S["get_settings()"]
|
||||||
|
M --> U["uvicorn.run()"]
|
||||||
|
U --> F["create_app()"]
|
||||||
|
F --> S
|
||||||
|
F --> P["Parent FastAPI app"]
|
||||||
|
P --> A["API routes and middleware"]
|
||||||
|
P -->|"mount_path=/gui"| N["NiceGUI App"]
|
||||||
|
U -->|"ASGI requests and lifespan"| P
|
||||||
|
```
|
||||||
|
|
||||||
|
The objects have separate responsibilities:
|
||||||
|
|
||||||
|
| Object | Owner | Responsibility |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Parent `FastAPI` instance | Application code | Root ASGI app, API routes, middleware, lifespan, and mounted applications |
|
||||||
|
| `Settings` instance | Application code | Immutable, process-local configuration snapshot shared by startup and composition |
|
||||||
|
| `nicegui.app` | NiceGUI | A process-local [`App`](https://github.com/zauberzeug/nicegui/blob/main/nicegui/app/app.py) instance that subclasses `FastAPI` |
|
||||||
|
| `ui.run_with(parent_app)` | NiceGUI integration | Configures NiceGUI, mounts `nicegui.app` into `parent_app`, and integrates lifecycle handling |
|
||||||
|
| Uvicorn | Server process | Imports or receives the root ASGI app, opens sockets, drives lifespan, and serves requests |
|
||||||
|
|
||||||
|
Uvicorn must serve the **parent FastAPI app** when using `ui.run_with()`. Passing `nicegui.app` to `ui.run_with()` is rejected because it would mount NiceGUI into itself and recurse on unmatched routes.
|
||||||
|
|
||||||
|
## Choose One Startup Mode
|
||||||
|
|
||||||
|
### Let NiceGUI Own Startup
|
||||||
|
|
||||||
|
Use `ui.run()` when NiceGUI is the main application. Add ordinary FastAPI routes to the exported `nicegui.app` object:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from nicegui import app, ui
|
||||||
|
|
||||||
|
|
||||||
|
@app.get('/healthz')
|
||||||
|
def health() -> dict[str, str]:
|
||||||
|
return {'status': 'ok'}
|
||||||
|
|
||||||
|
|
||||||
|
@ui.page('/')
|
||||||
|
def home() -> None:
|
||||||
|
ui.label('Home')
|
||||||
|
|
||||||
|
|
||||||
|
ui.run()
|
||||||
|
```
|
||||||
|
|
||||||
|
In this mode, NiceGUI configures and starts its own [Uvicorn-derived server](https://github.com/zauberzeug/nicegui/blob/main/nicegui/server.py). Do not also call `uvicorn.run()`.
|
||||||
|
|
||||||
|
### Let FastAPI Own The Application
|
||||||
|
|
||||||
|
Use `ui.run_with()` when an existing FastAPI application owns middleware, API routers, OpenAPI configuration, lifespan resources, or deployment startup. The [official NiceGUI FastAPI example](https://github.com/zauberzeug/nicegui/blob/main/examples/fastapi/main.py) follows this model.
|
||||||
|
|
||||||
|
`mount_path` controls where the NiceGUI application appears externally. A NiceGUI page declared as `/` is reachable at `/gui/` when mounted at `/gui`, while parent routes such as `/healthz` remain at the root. A dedicated UI prefix usually makes ownership and route conflicts clearer than mounting both applications at `/`.
|
||||||
|
|
||||||
|
## Canonical Factory Layout
|
||||||
|
|
||||||
|
Keep application composition importable and server startup explicit:
|
||||||
|
|
||||||
|
```text
|
||||||
|
.
|
||||||
|
├─ pyproject.toml
|
||||||
|
└─ src/
|
||||||
|
└─ my_app/
|
||||||
|
├─ __init__.py
|
||||||
|
├─ config.py
|
||||||
|
└─ main.py
|
||||||
|
```
|
||||||
|
|
||||||
|
```python title="src/my_app/config.py"
|
||||||
|
from functools import cache
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field, SecretStr
|
||||||
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
|
|
||||||
|
|
||||||
|
class ServerSettings(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
host: str = '0.0.0.0'
|
||||||
|
port: int = 8000
|
||||||
|
log_level: Literal['critical', 'error', 'warning', 'info', 'debug', 'trace'] = (
|
||||||
|
'info'
|
||||||
|
)
|
||||||
|
reload: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class GuiSettings(BaseModel):
|
||||||
|
model_config = ConfigDict(frozen=True)
|
||||||
|
|
||||||
|
mount_path: str = '/gui'
|
||||||
|
storage_secret: SecretStr | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
model_config = SettingsConfigDict(
|
||||||
|
env_prefix='MY_APP_',
|
||||||
|
env_nested_delimiter='__',
|
||||||
|
env_file='.env',
|
||||||
|
env_file_encoding='utf-8',
|
||||||
|
frozen=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
server: ServerSettings = Field(default_factory=ServerSettings)
|
||||||
|
gui: GuiSettings = Field(default_factory=GuiSettings)
|
||||||
|
|
||||||
|
|
||||||
|
@cache
|
||||||
|
def get_settings() -> Settings:
|
||||||
|
return Settings()
|
||||||
|
```
|
||||||
|
|
||||||
|
`ServerSettings` and `GuiSettings` inherit from `BaseModel` because they share one application owner, source policy, and process lifecycle. The root `BaseSettings` reads the sources once and validates one atomic snapshot. Environment variables use names such as `MY_APP_SERVER__PORT`, `MY_APP_SERVER__RELOAD`, `MY_APP_GUI__MOUNT_PATH`, and `MY_APP_GUI__STORAGE_SECRET`.
|
||||||
|
|
||||||
|
The argument-free [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) provider is appropriate here because both the project entry point and Uvicorn's zero-argument factory need process-lifetime access. Each reload or worker process gets its own settings instance. Do not add override arguments to `get_settings()`; inject a `Settings` instance directly into `create_app()` in tests or alternate composition roots. See the [Pydantic settings implementation guide](../../pydantic-settings/SKILL.md) for source precedence, independent settings boundaries, cache clearing, and runtime reload guidance.
|
||||||
|
|
||||||
|
```python title="src/my_app/main.py"
|
||||||
|
from collections.abc import AsyncGenerator
|
||||||
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
|
import uvicorn
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from nicegui import ui
|
||||||
|
|
||||||
|
from my_app.config import Settings, get_settings
|
||||||
|
|
||||||
|
|
||||||
|
@asynccontextmanager
|
||||||
|
async def lifespan(app: FastAPI) -> AsyncGenerator[None]:
|
||||||
|
app.state.ready = True
|
||||||
|
try:
|
||||||
|
yield
|
||||||
|
finally:
|
||||||
|
app.state.ready = False
|
||||||
|
|
||||||
|
|
||||||
|
def register_pages() -> None:
|
||||||
|
@ui.page('/')
|
||||||
|
def dashboard() -> None:
|
||||||
|
ui.label('Dashboard')
|
||||||
|
|
||||||
|
|
||||||
|
def create_app(settings: Settings | None = None) -> FastAPI:
|
||||||
|
settings = settings or get_settings()
|
||||||
|
app = FastAPI(lifespan=lifespan)
|
||||||
|
app.state.settings = settings
|
||||||
|
|
||||||
|
@app.get('/healthz')
|
||||||
|
def health() -> dict[str, str]:
|
||||||
|
return {'status': 'ok'}
|
||||||
|
|
||||||
|
register_pages()
|
||||||
|
ui.run_with(
|
||||||
|
app,
|
||||||
|
mount_path=settings.gui.mount_path,
|
||||||
|
storage_secret=(
|
||||||
|
settings.gui.storage_secret.get_secret_value()
|
||||||
|
if settings.gui.storage_secret is not None
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
)
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
settings = get_settings()
|
||||||
|
uvicorn.run(
|
||||||
|
'my_app.main:create_app',
|
||||||
|
factory=True,
|
||||||
|
host=settings.server.host,
|
||||||
|
port=settings.server.port,
|
||||||
|
log_level=settings.server.log_level,
|
||||||
|
reload=settings.server.reload,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
|
```
|
||||||
|
|
||||||
|
The `storage_secret` is optional unless the application uses `ui.storage.user` or `ui.storage.browser`. `SecretStr` prevents accidental plaintext display in logs and model representations, while `get_secret_value()` unwraps it only at the NiceGUI integration boundary. Supply production secrets through environment variables or a supported settings secret source rather than committing them.
|
||||||
|
|
||||||
|
The example passes an [import string and `factory=True`](https://www.uvicorn.org/settings/#application) to Uvicorn. Uvicorn imports `my_app.main`, calls the zero-argument `create_app` factory, and serves the returned parent FastAPI app. Import strings are also required when Uvicorn creates reload or worker subprocesses; passing `create_app()` directly only supports the simple single-process case.
|
||||||
|
|
||||||
|
NiceGUI keeps framework state in its process-local app singleton. Treat `create_app()` as a once-per-worker factory. Calling it repeatedly in one interpreter can register the same pages and lifecycle handlers more than once; tests that create multiple apps must isolate or reset NiceGUI state.
|
||||||
|
|
||||||
|
## Lifespan Ordering
|
||||||
|
|
||||||
|
The [ASGI lifespan protocol](https://asgi.readthedocs.io/en/latest/specs/lifespan.html) is driven by the server. Uvicorn sends startup before accepting requests and sends shutdown while terminating the process. Lifespan runs once per event loop, including once in each worker process.
|
||||||
|
|
||||||
|
Current NiceGUI source integrates with the parent application by:
|
||||||
|
|
||||||
|
1. Capturing the parent FastAPI lifespan context.
|
||||||
|
2. Mounting NiceGUI's internal app on the parent.
|
||||||
|
3. Replacing the parent lifespan with a wrapper.
|
||||||
|
4. Starting NiceGUI before entering the original parent lifespan.
|
||||||
|
5. Exiting the original parent lifespan before shutting down NiceGUI.
|
||||||
|
|
||||||
|
This exact ordering comes from the current [`ui.run_with` implementation](https://github.com/zauberzeug/nicegui/blob/main/nicegui/ui_run_with.py) and is version-sensitive. Check the pinned NiceGUI version before making one startup handler depend on another framework's internal ordering.
|
||||||
|
|
||||||
|
Create database pools, HTTP clients, and similar resources in the parent [FastAPI lifespan](https://fastapi.tiangolo.com/advanced/events/), then close them after `yield`. Do not create event-loop-bound resources at import time or assume that globals are shared between workers.
|
||||||
|
|
||||||
|
## Expose The Server As A Project Script
|
||||||
|
|
||||||
|
Map a command name to the no-argument startup function:
|
||||||
|
|
||||||
|
```toml title="pyproject.toml"
|
||||||
|
[project]
|
||||||
|
name = "my-app"
|
||||||
|
version = "0.1.0"
|
||||||
|
requires-python = ">=3.12"
|
||||||
|
dependencies = [
|
||||||
|
"fastapi",
|
||||||
|
"nicegui",
|
||||||
|
"pydantic-settings",
|
||||||
|
"uvicorn[standard]",
|
||||||
|
]
|
||||||
|
|
||||||
|
[project.scripts]
|
||||||
|
my-app = "my_app.main:main"
|
||||||
|
|
||||||
|
[build-system]
|
||||||
|
requires = ["hatchling"]
|
||||||
|
build-backend = "hatchling.build"
|
||||||
|
|
||||||
|
[tool.hatch.build.targets.wheel]
|
||||||
|
packages = ["src/my_app"]
|
||||||
|
```
|
||||||
|
|
||||||
|
Run the installed command through uv:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uv run my-app
|
||||||
|
```
|
||||||
|
|
||||||
|
The uv [project entry-point documentation](https://docs.astral.sh/uv/concepts/projects/config/#entry-points) requires a build system so uv installs the project and generates its command. The `[project.scripts]` target follows the [PyPA entry-point specification](https://packaging.python.org/en/latest/specifications/entry-points/#use-for-scripts): its generated wrapper imports `main`, calls it without arguments, and uses the return value as the process exit status. Returning `None` means successful completion.
|
||||||
|
|
||||||
|
The settings model now owns host, port, logging, reload, mount path, and storage-secret configuration. Add an explicit CLI settings source or another CLI parser only when the project command needs user-supplied arguments; the entry-point callable itself still receives no arguments.
|
||||||
|
|
||||||
|
## Development Reload
|
||||||
|
|
||||||
|
Because `main()` supplies an import string, it can enable Uvicorn reload for local development:
|
||||||
|
|
||||||
|
```dotenv title=".env"
|
||||||
|
MY_APP_SERVER__HOST=127.0.0.1
|
||||||
|
MY_APP_SERVER__RELOAD=true
|
||||||
|
```
|
||||||
|
|
||||||
|
The cached settings object is a process-start snapshot. Changing an environment variable or dotenv file does not mutate a running instance; restart the process, or let the development reloader create a new worker when a watched file changes. Keep reload disabled in production. Uvicorn documents [`reload` and `workers` as mutually exclusive](https://www.uvicorn.org/settings/#production), and each worker would have independent settings, NiceGUI state, lifespan resources, and WebSocket connections. Use one worker by default unless the application has explicitly validated session affinity and externalized every stateful dependency needed across processes.
|
||||||
|
|
||||||
|
## Anti-Patterns
|
||||||
|
|
||||||
|
| Anti-pattern | Why it fails | Preferred approach |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `ui.run_with(nicegui.app)` | Mounts NiceGUI into itself | Pass a separately created `FastAPI()` instance |
|
||||||
|
| Calling both `ui.run()` and `ui.run_with()` | Gives two paths responsibility for startup | Choose one ownership model |
|
||||||
|
| `uvicorn.run(create_app(), reload=True)` | Reload subprocesses cannot import the app object | Use an import string with `factory=True` |
|
||||||
|
| Calling `uvicorn.run()` at module import time | Importing the module starts a blocking server and breaks subprocess startup | Call it from `main()` |
|
||||||
|
| Top-level `ui.label(...)` with `ui.run_with()` | Script-mode elements are discarded by this integration | Register UI in `@ui.page` functions or a root callable |
|
||||||
|
| Multiple workers by default | Process-local UI state and WebSockets are not automatically shared | Start with one worker and validate a distributed design explicitly |
|
||||||
|
| Reconstructing `Settings()` throughout the app | Re-reads sources and obscures the active configuration lifecycle | Inject the startup snapshot or use the argument-free provider at framework boundaries |
|
||||||
|
| Adding kwargs to cached `get_settings()` | Retains one hidden process-lifetime instance per argument combination | Construct explicit `Settings(...)` overrides and inject them |
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
Use `TestClient` as a context manager so the parent ASGI lifespan runs:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from my_app.config import GuiSettings, Settings
|
||||||
|
from my_app.main import create_app
|
||||||
|
|
||||||
|
|
||||||
|
def test_application_routes() -> None:
|
||||||
|
settings = Settings(
|
||||||
|
gui=GuiSettings(storage_secret='test-storage-secret'),
|
||||||
|
)
|
||||||
|
|
||||||
|
with TestClient(create_app(settings)) as client:
|
||||||
|
assert client.get('/healthz').json() == {'status': 'ok'}
|
||||||
|
assert client.get('/gui/').status_code == 200
|
||||||
|
```
|
||||||
|
|
||||||
|
Also verify:
|
||||||
|
|
||||||
|
- startup resources exist while the client context is active and are released afterward
|
||||||
|
- the mounted UI returns HTML and parent API failures retain FastAPI's JSON responses
|
||||||
|
- `uv run my-app` starts the server and responds on both the API and UI paths
|
||||||
|
- shutdown signals complete without orphaned background tasks
|
||||||
|
|
||||||
|
## Primary Sources
|
||||||
|
|
||||||
|
- [NiceGUI pages, routing, and FastAPI integration](https://www.nicegui.io/documentation/section_pages_routing)
|
||||||
|
- [NiceGUI `ui.run_with` implementation](https://github.com/zauberzeug/nicegui/blob/main/nicegui/ui_run_with.py)
|
||||||
|
- [NiceGUI FastAPI example](https://github.com/zauberzeug/nicegui/blob/main/examples/fastapi/main.py)
|
||||||
|
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
||||||
|
- [ASGI lifespan protocol](https://asgi.readthedocs.io/en/latest/specs/lifespan.html)
|
||||||
|
- [Uvicorn settings](https://www.uvicorn.org/settings/)
|
||||||
|
- [Uvicorn programmatic startup](https://www.uvicorn.org/#running-programmatically)
|
||||||
|
- [Pydantic settings management](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/)
|
||||||
|
- [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||||
|
- [uv project entry points](https://docs.astral.sh/uv/concepts/projects/config/#entry-points)
|
||||||
|
- [PyPA entry points specification](https://packaging.python.org/en/latest/specifications/entry-points/)
|
||||||
+4
-3
@@ -104,6 +104,7 @@ ui.button("Refresh").on_click(lambda: item_list.refresh())
|
|||||||
|
|
||||||
## Links
|
## Links
|
||||||
|
|
||||||
- NiceGUI action events: https://nicegui.io/documentation/section_action_events
|
!!! info "Primary sources"
|
||||||
- FastAPI SSE: https://fastapi.tiangolo.com/advanced/server-sent-events/
|
- [NiceGUI action events](https://nicegui.io/documentation/section_action_events)
|
||||||
- FastAPI WebSockets: https://fastapi.tiangolo.com/advanced/websockets/
|
- [FastAPI server-sent events](https://fastapi.tiangolo.com/advanced/server-sent-events/)
|
||||||
|
- [FastAPI WebSockets](https://fastapi.tiangolo.com/advanced/websockets/)
|
||||||
@@ -0,0 +1,73 @@
|
|||||||
|
# Source Documentation
|
||||||
|
|
||||||
|
Use these links to verify framework-specific behavior before relying on version-sensitive or integration-specific guidance.
|
||||||
|
|
||||||
|
## NiceGUI
|
||||||
|
|
||||||
|
!!! info "NiceGUI sources"
|
||||||
|
- [Component documentation](https://nicegui.io/documentation)
|
||||||
|
- [Element styling, props, and events](https://nicegui.io/documentation/element)
|
||||||
|
- [NiceGUI element source](https://github.com/zauberzeug/nicegui/tree/main/nicegui/elements)
|
||||||
|
- [Pages, routing, and FastAPI integration](https://www.nicegui.io/documentation/section_pages_routing)
|
||||||
|
- [`ui.run_with` implementation](https://github.com/zauberzeug/nicegui/blob/main/nicegui/ui_run_with.py)
|
||||||
|
- [FastAPI integration example](https://github.com/zauberzeug/nicegui/blob/main/examples/fastapi/main.py)
|
||||||
|
- [Binding properties and bindable dataclasses](https://www.nicegui.io/documentation/section_binding_properties)
|
||||||
|
- [Action events](https://www.nicegui.io/documentation/section_action_events)
|
||||||
|
- [Security best practices](https://www.nicegui.io/documentation/section_security)
|
||||||
|
|
||||||
|
## FastAPI
|
||||||
|
|
||||||
|
!!! info "FastAPI sources"
|
||||||
|
- [Lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
||||||
|
- [Settings and environment variables](https://fastapi.tiangolo.com/advanced/settings/)
|
||||||
|
- [Dependencies with yield](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
|
||||||
|
- [Server-sent events](https://fastapi.tiangolo.com/advanced/server-sent-events/)
|
||||||
|
- [WebSockets](https://fastapi.tiangolo.com/advanced/websockets/)
|
||||||
|
|
||||||
|
## ASGI And Uvicorn
|
||||||
|
|
||||||
|
!!! info "Server and lifespan sources"
|
||||||
|
- [ASGI lifespan protocol](https://asgi.readthedocs.io/en/latest/specs/lifespan.html)
|
||||||
|
- [Uvicorn settings](https://www.uvicorn.org/settings/)
|
||||||
|
- [Uvicorn programmatic startup](https://www.uvicorn.org/#running-programmatically)
|
||||||
|
- [Uvicorn deployment](https://www.uvicorn.org/deployment/)
|
||||||
|
|
||||||
|
## uv And Project Scripts
|
||||||
|
|
||||||
|
!!! info "Packaging and command sources"
|
||||||
|
- [uv project entry points](https://docs.astral.sh/uv/concepts/projects/config/#entry-points)
|
||||||
|
- [uv project packaging](https://docs.astral.sh/uv/concepts/projects/config/#project-packaging)
|
||||||
|
- [PyPA entry points specification](https://packaging.python.org/en/latest/specifications/entry-points/)
|
||||||
|
|
||||||
|
## Styling
|
||||||
|
|
||||||
|
!!! info "Styling sources"
|
||||||
|
- [Tailwind utility-first styling](https://tailwindcss.com/docs/utility-first)
|
||||||
|
- [Tailwind responsive design and container queries](https://tailwindcss.com/docs/responsive-design)
|
||||||
|
- [Quasar components](https://quasar.dev/vue-components)
|
||||||
|
- [Quasar Screen plugin documentation source](https://github.com/quasarframework/quasar/blob/dev/docs/src/pages/options/screen-plugin.md)
|
||||||
|
- [CSS media queries](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_media_queries/Using_media_queries)
|
||||||
|
- [CSS container queries](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_containment/Container_queries)
|
||||||
|
|
||||||
|
## Persistence
|
||||||
|
|
||||||
|
!!! info "Persistence sources"
|
||||||
|
- [SQLAlchemy engine configuration and pooling](https://docs.sqlalchemy.org/en/20/core/engines.html)
|
||||||
|
- [SQLAlchemy session lifecycle](https://docs.sqlalchemy.org/en/20/orm/session_basics.html)
|
||||||
|
- [Alembic tutorial](https://alembic.sqlalchemy.org/en/latest/tutorial.html)
|
||||||
|
|
||||||
|
## Configuration And Dataclasses
|
||||||
|
|
||||||
|
!!! info "Python and Pydantic sources"
|
||||||
|
- [Pydantic settings management](https://docs.pydantic.dev/latest/concepts/pydantic_settings/)
|
||||||
|
- [Python dataclasses](https://docs.python.org/3/library/dataclasses.html)
|
||||||
|
- [PEP 557: Data Classes](https://peps.python.org/pep-0557/)
|
||||||
|
|
||||||
|
## LangGraph
|
||||||
|
|
||||||
|
!!! info "LangGraph sources"
|
||||||
|
- [Overview](https://docs.langchain.com/oss/python/langgraph/overview)
|
||||||
|
- [Workflows and agents](https://docs.langchain.com/oss/python/langgraph/workflows-agents)
|
||||||
|
- [Persistence](https://docs.langchain.com/oss/python/langgraph/persistence)
|
||||||
|
- [Streaming](https://docs.langchain.com/oss/python/langgraph/streaming)
|
||||||
|
- [Interrupts and human-in-the-loop](https://docs.langchain.com/oss/python/langgraph/interrupts)
|
||||||
@@ -0,0 +1,144 @@
|
|||||||
|
# NiceGUI Special Component Customization
|
||||||
|
|
||||||
|
Use this reference for components whose NiceGUI wrapper, Quasar implementation, popup behavior, slots, or external assets require component-specific handling. Start with [styling and customization](./styling-and-customization.md) for the general escalation workflow.
|
||||||
|
|
||||||
|
## Source Research Gate
|
||||||
|
|
||||||
|
Research the target component before generating code or CSS. Do not rely on a remembered NiceGUI or Quasar API.
|
||||||
|
|
||||||
|
For each component:
|
||||||
|
|
||||||
|
1. Read its current NiceGUI documentation page.
|
||||||
|
2. Inspect the constructor and implementation in the target project's installed NiceGUI package.
|
||||||
|
3. Confirm the wrapped Quasar component in the NiceGUI source.
|
||||||
|
4. Read the matching Quasar guide and API definition for props, slots, events, and methods.
|
||||||
|
5. Check the target project's pinned NiceGUI version before using current upstream behavior.
|
||||||
|
6. Record which layer owns each proposed customization before writing it.
|
||||||
|
|
||||||
|
Use current upstream source only as a fallback when the target environment is unavailable. If installed and upstream behavior differ, follow the installed version and state the difference.
|
||||||
|
|
||||||
|
## `ui.select`
|
||||||
|
|
||||||
|
### Source Map
|
||||||
|
|
||||||
|
- [NiceGUI `ui.select` documentation](https://nicegui.io/documentation/select)
|
||||||
|
- [NiceGUI `Select` source](https://github.com/zauberzeug/nicegui/blob/main/nicegui/elements/select.py)
|
||||||
|
- [Quasar `QSelect` guide](https://quasar.dev/vue-components/select/)
|
||||||
|
- [Quasar `QSelect` API source](https://github.com/quasarframework/quasar/blob/dev/ui/src/components/select/QSelect.json)
|
||||||
|
|
||||||
|
NiceGUI's `Select` wraps Quasar `QSelect` but owns important Python-side behavior. Its constructor handles options, labels, values, change callbacks, input filtering, new-value modes, multiple selection, clearing, validation, and key generation. Use those constructor parameters before adding equivalent Quasar props manually.
|
||||||
|
|
||||||
|
### Customization Order
|
||||||
|
|
||||||
|
1. Use `options`, `label`, `value`, `on_change`, `with_input`, `new_value_mode`, `multiple`, `clearable`, `validation`, and `key_generator` through the NiceGUI constructor.
|
||||||
|
2. Use `.props()` for additional documented `QSelect` behavior such as field design, chips, option density, popup classes, popup positioning, or menu/dialog behavior.
|
||||||
|
3. Use `.classes()` and Tailwind for the field's structural width and placement.
|
||||||
|
4. Use named slots for prepend, append, loading, no-option, selected, or option content when props are insufficient.
|
||||||
|
5. Use `popup-content-class` to attach an application class to the detached options popup, then fine-tune it in a static stylesheet.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from nicegui import ui
|
||||||
|
|
||||||
|
item_select = ui.select(
|
||||||
|
options={"chair": "Chair", "desk": "Desk", "lamp": "Lamp"},
|
||||||
|
label="Items",
|
||||||
|
multiple=True,
|
||||||
|
clearable=True,
|
||||||
|
with_input=True,
|
||||||
|
).props(
|
||||||
|
"outlined use-chips options-dense "
|
||||||
|
"popup-content-class=app-item-select-menu"
|
||||||
|
).classes(
|
||||||
|
"w-full md:max-w-md"
|
||||||
|
)
|
||||||
|
|
||||||
|
with item_select.add_slot("prepend"):
|
||||||
|
ui.icon("inventory_2")
|
||||||
|
```
|
||||||
|
|
||||||
|
```css
|
||||||
|
.app-item-select-menu {
|
||||||
|
max-height: min(24rem, 60dvh);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Select-Specific Caveats
|
||||||
|
|
||||||
|
- NiceGUI accepts a list of values or a dictionary mapping values to labels. Do not assume the Python options model is the same as Quasar's JavaScript object-array examples.
|
||||||
|
- After mutating `options`, call `update()` or use `set_options()` so the client receives the change.
|
||||||
|
- `new_value_mode` enables input automatically. For dictionary options with `add`, NiceGUI requires a `key_generator`.
|
||||||
|
- A multiple select has a list value. NiceGUI normalizes a non-list initial value, but application state should still use the intended list shape.
|
||||||
|
- `map-options` has a Quasar performance cost. Do not add it to NiceGUI's mapped options without confirming that the wrapper's value translation requires it.
|
||||||
|
- `display-value-html` and `options-html` can create cross-site scripting risk. When using `selected`, `selected-item`, or `option` slots, the application owns sanitization.
|
||||||
|
- Custom option slots use virtual scrolling. When one option renders multiple sibling elements, Quasar requires `q-virtual-scroll--with-prev` on every additional sibling.
|
||||||
|
- Buttons placed in `before`, `after`, `prepend`, or `append` field slots do not propagate clicks to the parent. A submit button in one of those slots needs its own submit handler.
|
||||||
|
- `QSelect` renders its popup outside the field. Style it through `popup-content-class`; do not assume a descendant selector beneath the field will reach it.
|
||||||
|
- Quasar switches between menu and dialog popup behavior by platform. Verify forced `behavior=menu` carefully on iOS when input filtering is enabled.
|
||||||
|
|
||||||
|
Use `.on()` or `run_method()` only after confirming the event or method in the installed Quasar API. Prefer NiceGUI's `on_change`, `set_options()`, value bindings, and `is_showing_popup` when they cover the behavior.
|
||||||
|
|
||||||
|
## `ui.icon`
|
||||||
|
|
||||||
|
### Source Map
|
||||||
|
|
||||||
|
- [NiceGUI `ui.icon` documentation](https://nicegui.io/documentation/icon)
|
||||||
|
- [NiceGUI `Icon` source](https://github.com/zauberzeug/nicegui/blob/main/nicegui/elements/icon.py)
|
||||||
|
- [Quasar `QIcon` guide](https://quasar.dev/vue-components/icon/)
|
||||||
|
- [Quasar `QIcon` API source](https://github.com/quasarframework/quasar/blob/dev/ui/src/components/icon/QIcon.json)
|
||||||
|
- [Google Material Symbols and Icons](https://fonts.google.com/icons)
|
||||||
|
|
||||||
|
NiceGUI's `Icon` is a thin `QIcon` wrapper. Its constructor exposes `name`, `size`, and `color`; the source forwards these to a `q-icon` element. Use Quasar's icon naming and asset rules for anything beyond those parameters.
|
||||||
|
|
||||||
|
### Customization Order
|
||||||
|
|
||||||
|
1. Choose an icon family that is actually loaded by the application.
|
||||||
|
2. Pass the documented icon name, size, and color to `ui.icon()`.
|
||||||
|
3. Use `.props()` for supported `QIcon` props such as `left`, `right`, or a custom render tag.
|
||||||
|
4. Use `.classes()` for structural placement and an application class for stable visual variants.
|
||||||
|
5. Use a static stylesheet for Material Symbol axes, state variants, custom webfonts, or repeated effects.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from nicegui import ui
|
||||||
|
|
||||||
|
ui.icon(
|
||||||
|
"sym_o_home",
|
||||||
|
size="1.5rem",
|
||||||
|
color="primary",
|
||||||
|
).classes(
|
||||||
|
"app-symbol-filled shrink-0"
|
||||||
|
).tooltip(
|
||||||
|
"Home"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
```css
|
||||||
|
.app-symbol-filled {
|
||||||
|
font-variation-settings:
|
||||||
|
"FILL" 1,
|
||||||
|
"wght" 400,
|
||||||
|
"GRAD" 0,
|
||||||
|
"opsz" 24;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Icon-Specific Caveats
|
||||||
|
|
||||||
|
- Material icon names use snake case. Material variants use prefixes such as `o_`, `r_`, `s_`, `sym_o_`, `sym_r_`, and `sym_s_`.
|
||||||
|
- Other icon families have their own prefixes and require their webfont or stylesheet to be loaded. A valid name does not load the corresponding asset.
|
||||||
|
- `size` accepts CSS units or Quasar sizes such as `xs`, `sm`, `md`, `lg`, and `xl`. Quasar implements icon sizing through `font-size`.
|
||||||
|
- Icon color inherits text color unless the `color` prop or a CSS color overrides it.
|
||||||
|
- Material Symbol variable axes apply to webfont icons, not static SVG icon exports.
|
||||||
|
- Quasar also supports SVG path strings, `svguse:` references, and `img:` URLs. Confirm the exact `QIcon` name format and mount path before generating one of these forms.
|
||||||
|
- For an action, use a semantic control such as `ui.button(icon=..., on_click=...)` and give it an accessible label or tooltip. Do not turn a bare decorative icon into an unlabeled control.
|
||||||
|
- Prefer `ui.icon(...).tooltip(...)` over manually constructing tooltip slot markup when NiceGUI's method covers the requirement.
|
||||||
|
|
||||||
|
## Completion Check
|
||||||
|
|
||||||
|
Before accepting a special-component customization:
|
||||||
|
|
||||||
|
1. Cite the NiceGUI component page and implementation that were inspected.
|
||||||
|
2. Cite the matching Quasar guide or API source.
|
||||||
|
3. Identify constructor arguments, Quasar props, slots, Tailwind classes, and stylesheet rules separately.
|
||||||
|
4. Confirm detached popup or external asset behavior where applicable.
|
||||||
|
5. Test keyboard interaction, focus, labels, and tooltips.
|
||||||
|
6. Test the supported mobile, landscape desktop, and portrait desktop viewports.
|
||||||
@@ -0,0 +1,438 @@
|
|||||||
|
# NiceGUI Styling And Customization
|
||||||
|
|
||||||
|
Use this reference to discover how a NiceGUI component can be customized, apply the least invasive supported mechanism, and introduce CSS without fighting Quasar's internal geometry.
|
||||||
|
|
||||||
|
For package boundaries, dependency direction, and page or component ownership, load [application architecture](./architecture.md).
|
||||||
|
|
||||||
|
## Progressive Customization Workflow
|
||||||
|
|
||||||
|
Increase the customization level only when the previous source does not expose what the design requires:
|
||||||
|
|
||||||
|
1. Read the NiceGUI documentation page for the component.
|
||||||
|
2. Inspect the NiceGUI element function or class constructor.
|
||||||
|
3. Identify the wrapped Quasar component and read its documentation.
|
||||||
|
4. Use Quasar props, slots, and events through NiceGUI's native customization APIs.
|
||||||
|
5. Use Tailwind classes for structural layout.
|
||||||
|
6. Add a scoped static stylesheet for stable visual fine tuning.
|
||||||
|
|
||||||
|
Stop as soon as the required behavior is supported. Do not begin by targeting Quasar's generated DOM or internal selectors.
|
||||||
|
|
||||||
|
### 1. Start With The NiceGUI Component Page
|
||||||
|
|
||||||
|
Find the component in the [NiceGUI documentation](https://nicegui.io/documentation). Check its examples, parameters, methods, events, bindings, and inheritance before writing CSS. The component page establishes the public NiceGUI API and often demonstrates the intended Quasar integration.
|
||||||
|
|
||||||
|
Confirm the target project's installed NiceGUI version because the current online documentation can differ from the pinned release.
|
||||||
|
|
||||||
|
### 2. Inspect The NiceGUI Constructor
|
||||||
|
|
||||||
|
Read the signature and implementation of the imported NiceGUI function or element class. The constructor reveals accepted Python parameters, defaults, event callbacks, validation, and values NiceGUI forwards to the frontend.
|
||||||
|
|
||||||
|
Use editor navigation or runtime inspection against the project's selected environment:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from inspect import getsource, signature
|
||||||
|
|
||||||
|
from nicegui import ui
|
||||||
|
|
||||||
|
print(signature(ui.select))
|
||||||
|
print(getsource(ui.select))
|
||||||
|
```
|
||||||
|
|
||||||
|
When `ui.<name>` is a factory or alias, follow it to the element class in the [NiceGUI element sources](https://github.com/zauberzeug/nicegui/tree/main/nicegui/elements). Prefer the installed package source when behavior may differ by version.
|
||||||
|
|
||||||
|
### 3. Read The Underlying Quasar Component Docs
|
||||||
|
|
||||||
|
NiceGUI wraps Quasar components such as [`QInput`](https://quasar.dev/vue-components/input/), [`QSelect`](https://quasar.dev/vue-components/select/), and [`QDialog`](https://quasar.dev/vue-components/dialog/). Use the matching Quasar component page to discover its complete props, slots, events, methods, and behavior notes.
|
||||||
|
|
||||||
|
Map Quasar's Vue API onto the NiceGUI wrapper instead of copying a Vue template. Verify that a prop or slot exists in the Quasar version used by the installed NiceGUI release.
|
||||||
|
|
||||||
|
### 4. Apply Native Quasar Features Through NiceGUI
|
||||||
|
|
||||||
|
Use the NiceGUI element customization methods to reach the supported Quasar surface:
|
||||||
|
|
||||||
|
- `.props(...)` for Quasar properties and boolean flags
|
||||||
|
- `.classes(...)` for Tailwind utilities and stable application class names
|
||||||
|
- `.style(...)` for dynamic inline values or a quick, local probe
|
||||||
|
- `.on(...)` for events that are not represented by a constructor callback
|
||||||
|
- slots or child elements for Quasar extension points exposed by the wrapper
|
||||||
|
|
||||||
|
```python
|
||||||
|
with ui.select(
|
||||||
|
options=items,
|
||||||
|
label="Item",
|
||||||
|
).props(
|
||||||
|
"outlined clearable options-dense popup-content-class=app-item-menu"
|
||||||
|
).classes(
|
||||||
|
"w-full md:max-w-md"
|
||||||
|
) as item_select:
|
||||||
|
with item_select.add_slot("prepend"):
|
||||||
|
ui.icon("inventory_2")
|
||||||
|
```
|
||||||
|
|
||||||
|
Prefer constructor arguments when NiceGUI exposes the behavior directly. Use `.props()` for supported Quasar features that are not constructor parameters. Use slots when the Quasar docs define a semantic insertion point; do not reproduce that content with absolute positioning.
|
||||||
|
|
||||||
|
## Application Themes With NiceGUI And Quasar
|
||||||
|
|
||||||
|
Treat a theme as three related layers with different owners:
|
||||||
|
|
||||||
|
1. Configure Quasar's named color roles through NiceGUI.
|
||||||
|
2. Let Quasar own light, dark, and automatic mode state.
|
||||||
|
3. Define application semantic tokens for surfaces and content not covered by Quasar components.
|
||||||
|
|
||||||
|
Do not implement a parallel theme switch by replacing Quasar classes or directly restyling each component. NiceGUI's color APIs set the supported Quasar `--q-*` custom properties, so Quasar components, `color=` arguments, and classes such as `text-primary` and `bg-positive` stay aligned.
|
||||||
|
|
||||||
|
### Set The App-Wide Palette Once
|
||||||
|
|
||||||
|
Use [`app.colors()`](https://nicegui.io/documentation/colors#app-wide-colors) in the composition layer for the default palette. Prefer Quasar's semantic roles over shade names: `primary`, `secondary`, `accent`, `positive`, `negative`, `info`, and `warning`. The `dark` and `dark_page` arguments configure dark surface colors; they do not enable dark mode.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from nicegui import app, ui
|
||||||
|
|
||||||
|
app.colors(
|
||||||
|
primary="#176b5b",
|
||||||
|
secondary="#52645f",
|
||||||
|
accent="#c05a32",
|
||||||
|
dark="#202523",
|
||||||
|
dark_page="#151917",
|
||||||
|
positive="#2e7d32",
|
||||||
|
negative="#b3261e",
|
||||||
|
info="#276b8e",
|
||||||
|
warning="#a86600",
|
||||||
|
brand="#176b5b",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@ui.page("/")
|
||||||
|
def index() -> None:
|
||||||
|
ui.button("Save")
|
||||||
|
ui.label("Current workspace").classes("text-brand")
|
||||||
|
|
||||||
|
|
||||||
|
ui.run()
|
||||||
|
```
|
||||||
|
|
||||||
|
Custom names such as `brand` become Quasar color names and can be used through `color="brand"`, `text-brand`, or `bg-brand`. Register them before any component uses them. `app.colors()` was added in NiceGUI 3.6.0; for an older pinned version, centralize the same `ui.colors(...)` call in a shared page shell.
|
||||||
|
|
||||||
|
Use [`ui.colors()`](https://nicegui.io/documentation/colors) only when one page intentionally overrides the app palette. It is page-scoped and takes precedence over `app.colors()`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
@ui.page("/operations")
|
||||||
|
def operations_page() -> None:
|
||||||
|
ui.colors(primary="#8f3d2c")
|
||||||
|
ui.button("Operations action")
|
||||||
|
```
|
||||||
|
|
||||||
|
Avoid scattering `ui.colors()` calls among reusable components. A component should consume semantic roles from its owning page rather than silently changing the palette for the whole page.
|
||||||
|
|
||||||
|
### Let Quasar Control Light And Dark Mode
|
||||||
|
|
||||||
|
Use [`ui.dark_mode()`](https://nicegui.io/documentation/dark_mode) for page mode. Its value is tri-state: `True` enables dark mode, `False` disables it, and `None` follows the client's `prefers-color-scheme` setting. It overrides the `dark` default supplied to `ui.run()` or `@ui.page` for that page.
|
||||||
|
|
||||||
|
```python
|
||||||
|
dark_mode = ui.dark_mode(None)
|
||||||
|
|
||||||
|
with ui.button_group():
|
||||||
|
ui.button("System", on_click=dark_mode.auto)
|
||||||
|
ui.button("Light", on_click=dark_mode.disable)
|
||||||
|
ui.button("Dark", on_click=dark_mode.enable)
|
||||||
|
```
|
||||||
|
|
||||||
|
Quasar applies `body--light` or `body--dark`, updates its dark-aware components, and tracks system changes while mode is automatic. Use the NiceGUI element instead of invoking Quasar's JavaScript Dark plugin directly. Persist an explicit user preference separately when it must survive navigation or a new browser session.
|
||||||
|
|
||||||
|
### Add Semantic Tokens For Application Surfaces
|
||||||
|
|
||||||
|
Quasar's brand roles cover framework components, not every application-specific surface. Define a small set of semantic CSS variables in the static stylesheet and change their values under Quasar's documented `.body--dark` class:
|
||||||
|
|
||||||
|
```css
|
||||||
|
:root {
|
||||||
|
--app-page: #f6f8f7;
|
||||||
|
--app-surface: #ffffff;
|
||||||
|
--app-text: #202623;
|
||||||
|
--app-border: #cbd4d0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.body--dark {
|
||||||
|
--app-page: var(--q-dark-page);
|
||||||
|
--app-surface: var(--q-dark);
|
||||||
|
--app-text: #eef3f0;
|
||||||
|
--app-border: #46504b;
|
||||||
|
}
|
||||||
|
|
||||||
|
body {
|
||||||
|
background: var(--app-page);
|
||||||
|
color: var(--app-text);
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-panel {
|
||||||
|
background: var(--app-surface);
|
||||||
|
border: 1px solid var(--app-border);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Name tokens by purpose, such as `--app-surface` or `--app-muted-text`, rather than by a fixed color such as `--app-gray-100`. Reuse `--q-primary` and the other Quasar variables when the meaning matches. Check text, icon, border, focus, hover, disabled, positive, warning, and negative contrast in both modes; a palette is not complete merely because the page background changes.
|
||||||
|
|
||||||
|
## Structural Styling With Tailwind
|
||||||
|
|
||||||
|
Use standard [Tailwind utility classes](https://tailwindcss.com/docs/utility-first) for page and component structure:
|
||||||
|
|
||||||
|
- display, flex, and grid behavior
|
||||||
|
- width, height, and maximum-width constraints
|
||||||
|
- spacing, gaps, padding, and alignment
|
||||||
|
- wrapping, overflow, and responsive variants
|
||||||
|
- typography and common visual utilities when they fully express the design
|
||||||
|
|
||||||
|
Build the outer layout before fine-tuning individual controls:
|
||||||
|
|
||||||
|
1. Define the page shell and width constraints.
|
||||||
|
2. Establish responsive rows, columns, gaps, and wrapping.
|
||||||
|
3. Add semantic sections and repeated visual patterns.
|
||||||
|
4. Configure component appearance and behavior with constructor arguments and Quasar props.
|
||||||
|
5. Add stable application classes for any remaining stylesheet rules.
|
||||||
|
|
||||||
|
```python
|
||||||
|
with ui.column().classes("w-full max-w-6xl mx-auto gap-6 px-4"):
|
||||||
|
page_header(title="Inventory")
|
||||||
|
|
||||||
|
with ui.row().classes("w-full gap-4 flex-wrap lg:flex-nowrap items-start"):
|
||||||
|
filters_panel().classes("w-full lg:w-72 shrink-0")
|
||||||
|
item_grid().classes("w-full flex-1 min-w-0")
|
||||||
|
```
|
||||||
|
|
||||||
|
Use stable width, minimum-width, and flex constraints so labels, icons, validation messages, and loaded content do not shift the surrounding layout.
|
||||||
|
|
||||||
|
## Fine Tuning With Static Stylesheets
|
||||||
|
|
||||||
|
Move stable fine tuning into a static stylesheet after the structure and native component configuration are correct. Static stylesheets provide reusable selectors, media queries, pseudo-classes, CSS variables, and a clear cascade that inline declarations cannot provide.
|
||||||
|
|
||||||
|
Attach an application-owned class with `.classes()` or a Quasar popup prop, then scope stylesheet rules beneath it:
|
||||||
|
|
||||||
|
```python
|
||||||
|
ui.select(...).props("popup-content-class=app-item-menu").classes(
|
||||||
|
"app-item-select w-full md:max-w-md"
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
```css
|
||||||
|
.app-item-select {
|
||||||
|
--app-field-accent: #176b5b;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-item-select:focus-within {
|
||||||
|
filter: drop-shadow(0 0 0.25rem rgb(23 107 91 / 20%));
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-item-menu {
|
||||||
|
max-height: min(24rem, 60dvh);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `.style()` when a value is calculated at runtime or while testing a local hypothesis. Once a declaration becomes stable or repeated, move it to the stylesheet and keep only the application class in Python.
|
||||||
|
|
||||||
|
Avoid overriding Quasar internals such as `.q-field__label`, `.q-field__native`, `.q-field__control`, and `.q-field__input` unless the public props, slots, and application-level selectors cannot express the requirement.
|
||||||
|
|
||||||
|
Quasar coordinates field height, padding, labels, values, icons, and floating-label transforms. Changing only one internal part tends to cause clipping or overlap.
|
||||||
|
|
||||||
|
## Responsive Layout
|
||||||
|
|
||||||
|
Support these layouts only:
|
||||||
|
|
||||||
|
- mobile: a single-column layout with wrapping toolbars and full-width controls
|
||||||
|
- landscape desktop: $1920 \times 1080$ with side-by-side panels where they improve scanning
|
||||||
|
- portrait desktop: $1080 \times 1920$ with stacked panels or a narrow fixed sidebar
|
||||||
|
|
||||||
|
Build the mobile layout first, then add one desktop breakpoint when a row or grid needs more space. Prefer flex wrapping and fluid grids before adding another breakpoint. Use Tailwind classes for page layout and Quasar props for component behavior.
|
||||||
|
|
||||||
|
```python
|
||||||
|
with ui.row().classes("w-full flex-wrap gap-4 lg:flex-nowrap items-start"):
|
||||||
|
filters_panel().classes("w-full lg:w-72 shrink-0")
|
||||||
|
item_grid().classes("w-full flex-1 min-w-0")
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `min-w-0` for flexible children, `flex-wrap` for toolbars, and `max-w-* mx-auto` to keep portrait layouts readable. Do not add device-specific component trees, container queries, or custom breakpoints unless a supported layout demonstrates a concrete failure.
|
||||||
|
|
||||||
|
## Loading Stylesheets And Static Assets
|
||||||
|
|
||||||
|
- Mount and link static stylesheets once from the composition layer rather than injecting CSS from individual pages.
|
||||||
|
- Keep custom CSS tokenized with variables and scoped to application classes.
|
||||||
|
- Avoid broad rules against Quasar internals.
|
||||||
|
- Mount referenced assets in the composition layer.
|
||||||
|
- Verify mount paths, reverse-proxy rewrites, and cache behavior.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from fastapi.staticfiles import StaticFiles
|
||||||
|
|
||||||
|
STATIC_DIR = Path(__file__).parent / "ui" / "static"
|
||||||
|
|
||||||
|
app.mount("/static", StaticFiles(directory=STATIC_DIR), name="static")
|
||||||
|
ui.add_head_html(
|
||||||
|
'<link rel="stylesheet" href="/static/css/base.css">',
|
||||||
|
shared=True,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Worked Example: Responsive Dialog Customization
|
||||||
|
|
||||||
|
This example begins with normal field density and Quasar popup props, then uses an application class and static stylesheet for the remaining responsive fine tuning. Use whole-card scaling when a form dialog must become uniformly larger on mobile while preserving Quasar's internal proportions. Keep detached select menus unscaled and make the card itself scrollable.
|
||||||
|
|
||||||
|
### Use Normal Field Density
|
||||||
|
|
||||||
|
Normal Quasar fields are approximately `56px` high, while dense fields are approximately `40px` high. Remove `dense` when larger controls are needed.
|
||||||
|
|
||||||
|
```python
|
||||||
|
ui.input("Name").props("outlined")
|
||||||
|
ui.number("Quantity").props("outlined")
|
||||||
|
ui.select(...).props("outlined popup-content-class=app-item-detail-menu")
|
||||||
|
ui.textarea("Description").props("outlined autogrow")
|
||||||
|
```
|
||||||
|
|
||||||
|
Add a scoped class to the dialog card:
|
||||||
|
|
||||||
|
```python
|
||||||
|
ui.card().classes("app-detail-card app-item-detail-card")
|
||||||
|
```
|
||||||
|
|
||||||
|
### Scale The Complete Card
|
||||||
|
|
||||||
|
```css
|
||||||
|
:root {
|
||||||
|
--item-dialog-scale: 1;
|
||||||
|
--item-dialog-max-height: calc(100dvh - 3rem);
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-item-detail-card {
|
||||||
|
width: min(50rem, 50vw);
|
||||||
|
max-height: var(--item-dialog-max-height);
|
||||||
|
overflow-y: auto;
|
||||||
|
overscroll-behavior: contain;
|
||||||
|
zoom: var(--item-dialog-scale);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Restore Quasar's baseline if a global rule overrides it. */
|
||||||
|
.app-item-detail-card .q-field,
|
||||||
|
.app-item-detail-menu {
|
||||||
|
font-size: 14px;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (max-width: 599px) {
|
||||||
|
:root {
|
||||||
|
--item-dialog-scale: 1.2;
|
||||||
|
/* 75dvh becomes 90dvh after 1.2x zoom. */
|
||||||
|
--item-dialog-max-height: 75dvh;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-item-detail-card {
|
||||||
|
width: 80vw;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-item-detail-menu {
|
||||||
|
font-size: 16.8px;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The main mobile tuning knob is:
|
||||||
|
|
||||||
|
```css
|
||||||
|
--item-dialog-scale: 1.2;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Keep Detached Popups Unscaled
|
||||||
|
|
||||||
|
Do not apply `zoom` or `transform: scale()` to a `QSelect` popup menu. Quasar renders menus outside the dialog and positions them from the unscaled anchor geometry. Scaling the menu container afterward separates it from its field.
|
||||||
|
|
||||||
|
Avoid:
|
||||||
|
|
||||||
|
```css
|
||||||
|
.app-item-detail-card,
|
||||||
|
.app-item-detail-menu {
|
||||||
|
zoom: 1.2;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Use:
|
||||||
|
|
||||||
|
```css
|
||||||
|
.app-item-detail-card {
|
||||||
|
zoom: 1.2;
|
||||||
|
}
|
||||||
|
|
||||||
|
.app-item-detail-menu {
|
||||||
|
font-size: 16.8px;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `popup-content-class=app-item-detail-menu` to target the detached menu and enlarge its text without changing its coordinate system.
|
||||||
|
|
||||||
|
### Account For Zoom When Scrolling
|
||||||
|
|
||||||
|
The card's pre-zoom maximum height must account for the scale:
|
||||||
|
|
||||||
|
\[
|
||||||
|
\begin{aligned}
|
||||||
|
h_{\mathrm{pre}} &= \frac{h_{\mathrm{visible}}}{s} \\
|
||||||
|
\text{where } s &= \text{the zoom scale}
|
||||||
|
\end{aligned}
|
||||||
|
\]
|
||||||
|
|
||||||
|
For a desired visual height of `90dvh` at \(1.2\times\):
|
||||||
|
|
||||||
|
\[
|
||||||
|
\frac{90\,\mathrm{dvh}}{1.2} = 75\,\mathrm{dvh}
|
||||||
|
\]
|
||||||
|
|
||||||
|
Therefore:
|
||||||
|
|
||||||
|
```css
|
||||||
|
--item-dialog-max-height: 75dvh;
|
||||||
|
```
|
||||||
|
|
||||||
|
Apply scrolling to the card itself:
|
||||||
|
|
||||||
|
```css
|
||||||
|
.app-item-detail-card {
|
||||||
|
max-height: var(--item-dialog-max-height);
|
||||||
|
overflow-y: auto;
|
||||||
|
overscroll-behavior: contain;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
This keeps the dimmed page stationary while the form scrolls.
|
||||||
|
|
||||||
|
### Match The Quasar Breakpoint
|
||||||
|
|
||||||
|
Quasar's extra-small breakpoint ends at `599.98px`. A mobile-only rule can use:
|
||||||
|
|
||||||
|
```css
|
||||||
|
@media (max-width: 599px) {
|
||||||
|
/* Mobile rules. */
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Confirm custom breakpoint values against the target application's Quasar configuration.
|
||||||
|
|
||||||
|
## Validation Checklist
|
||||||
|
|
||||||
|
Check each completed page at these three viewports:
|
||||||
|
|
||||||
|
1. A representative mobile viewport, such as $390 \times 844$.
|
||||||
|
2. Landscape desktop at $1920 \times 1080$.
|
||||||
|
3. Portrait desktop at $1080 \times 1920$.
|
||||||
|
|
||||||
|
Confirm that page sections do not overlap, toolbars wrap on mobile, desktop panels use the available space without becoming excessively wide, and dialogs remain visible and scroll to their final field.
|
||||||
|
|
||||||
|
## Sources
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [NiceGUI element styling and props](https://nicegui.io/documentation/element)
|
||||||
|
- [NiceGUI binding properties](https://nicegui.io/documentation/section_binding_properties)
|
||||||
|
- [NiceGUI color theming](https://nicegui.io/documentation/colors)
|
||||||
|
- [NiceGUI dark mode](https://nicegui.io/documentation/dark_mode)
|
||||||
|
- [Quasar components](https://quasar.dev/vue-components)
|
||||||
|
- [Quasar color palette and runtime brand variables](https://quasar.dev/style/color-palette)
|
||||||
|
- [Quasar dark mode](https://quasar.dev/style/dark-mode)
|
||||||
|
- [Quasar field](https://quasar.dev/vue-components/field/)
|
||||||
|
- [Quasar select](https://quasar.dev/vue-components/select/)
|
||||||
|
- [Tailwind responsive design](https://tailwindcss.com/docs/responsive-design)
|
||||||
|
- [MDN `zoom`](https://developer.mozilla.org/en-US/docs/Web/CSS/zoom)
|
||||||
@@ -0,0 +1,442 @@
|
|||||||
|
---
|
||||||
|
name: pydantic-settings
|
||||||
|
description: "Practical guide for implementing typed application configuration with pydantic-settings. Use when designing BaseSettings models, choosing nested or independent settings boundaries, managing settings lifecycles, configuring dotenv or secrets, and customizing source priority safely."
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pydantic Settings Implementation Guide
|
||||||
|
|
||||||
|
Use this skill to implement robust, typed application configuration with `pydantic-settings` in production Python services.
|
||||||
|
|
||||||
|
## When to Use
|
||||||
|
|
||||||
|
- You need a single typed configuration model for app settings.
|
||||||
|
- You are migrating from ad-hoc `os.getenv(...)` calls.
|
||||||
|
- You need predictable precedence across init args, env vars, dotenv files, and secrets.
|
||||||
|
- You need nested settings models and reliable parsing behavior.
|
||||||
|
- You need to choose between one nested application settings object and independently owned settings objects.
|
||||||
|
- You need a deliberate construction, caching, or reload lifecycle.
|
||||||
|
- You need to customize settings sources or source order safely.
|
||||||
|
|
||||||
|
## Procedure
|
||||||
|
|
||||||
|
### 1. Baseline Model
|
||||||
|
|
||||||
|
Create a single settings model for the service boundary:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
|
|
||||||
|
|
||||||
|
class DatabaseSettings(BaseModel):
|
||||||
|
host: str = "localhost"
|
||||||
|
port: int = 5432
|
||||||
|
user: str
|
||||||
|
password: str
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
model_config = SettingsConfigDict(
|
||||||
|
env_prefix="APP_",
|
||||||
|
env_file=".env",
|
||||||
|
env_file_encoding="utf-8",
|
||||||
|
extra="ignore",
|
||||||
|
frozen=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
debug: bool = False
|
||||||
|
log_level: str = "info"
|
||||||
|
database: DatabaseSettings
|
||||||
|
api_key: str = Field(validation_alias="MY_API_KEY")
|
||||||
|
```
|
||||||
|
|
||||||
|
Quality gate:
|
||||||
|
|
||||||
|
1. Required fields fail fast when missing.
|
||||||
|
2. Defaults are intentional and safe.
|
||||||
|
|
||||||
|
### 2. Pick Env Naming Rules
|
||||||
|
|
||||||
|
1. Choose one prefix and apply it consistently.
|
||||||
|
2. Use aliases only for compatibility or external contracts.
|
||||||
|
3. Document whether env names are case-sensitive.
|
||||||
|
|
||||||
|
Quality gate:
|
||||||
|
|
||||||
|
1. Team can derive env variable names without guessing.
|
||||||
|
2. Legacy names are supported only where needed.
|
||||||
|
|
||||||
|
### 3. Decide Nested Parsing
|
||||||
|
|
||||||
|
For nested models via env vars, configure delimiters intentionally:
|
||||||
|
|
||||||
|
```python
|
||||||
|
model_config = SettingsConfigDict(
|
||||||
|
env_prefix="APP_",
|
||||||
|
env_nested_delimiter="__",
|
||||||
|
env_nested_max_split=1,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Typical vars:
|
||||||
|
|
||||||
|
1. `APP_DATABASE={"host": "db", "port": 5432, "user": "svc", "password": "pw"}`
|
||||||
|
2. `APP_DATABASE__HOST=db.internal`
|
||||||
|
|
||||||
|
Quality gate:
|
||||||
|
|
||||||
|
1. Nested overrides behave as expected.
|
||||||
|
2. Delimiter choice does not collide with field names.
|
||||||
|
|
||||||
|
### 4. Confirm Source Priority
|
||||||
|
|
||||||
|
Default priority (higher first):
|
||||||
|
|
||||||
|
1. CLI args (if enabled)
|
||||||
|
2. init kwargs
|
||||||
|
3. env vars
|
||||||
|
4. dotenv
|
||||||
|
5. secrets dir
|
||||||
|
6. defaults
|
||||||
|
|
||||||
|
Only customize when required:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from pydantic_settings import PydanticBaseSettingsSource
|
||||||
|
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def settings_customise_sources(
|
||||||
|
cls,
|
||||||
|
settings_cls: type[BaseSettings],
|
||||||
|
init_settings: PydanticBaseSettingsSource,
|
||||||
|
env_settings: PydanticBaseSettingsSource,
|
||||||
|
dotenv_settings: PydanticBaseSettingsSource,
|
||||||
|
file_secret_settings: PydanticBaseSettingsSource,
|
||||||
|
) -> tuple[PydanticBaseSettingsSource, ...]:
|
||||||
|
return (init_settings, env_settings, dotenv_settings, file_secret_settings)
|
||||||
|
```
|
||||||
|
|
||||||
|
Quality gate:
|
||||||
|
|
||||||
|
1. Priority order is explicit in code.
|
||||||
|
2. Tests verify conflict resolution.
|
||||||
|
|
||||||
|
### 5. Add Secrets Strategy
|
||||||
|
|
||||||
|
1. In local development, dotenv is acceptable for non-production values.
|
||||||
|
2. In deployed environments, prefer env vars or secret managers.
|
||||||
|
3. For file-mounted secrets, use `secrets_dir`.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```python
|
||||||
|
model_config = SettingsConfigDict(
|
||||||
|
env_prefix="APP_",
|
||||||
|
env_file=".env",
|
||||||
|
secrets_dir="/run/secrets",
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Quality gate:
|
||||||
|
|
||||||
|
1. No secret literals in repository code.
|
||||||
|
2. Missing secrets behavior is understood per environment.
|
||||||
|
|
||||||
|
### 6. Choose Nested Or Independent Settings Boundaries
|
||||||
|
|
||||||
|
Prefer one root `BaseSettings` object with nested `BaseModel` sections when the configuration belongs to one application lifecycle:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
|
|
||||||
|
|
||||||
|
class DatabaseSettings(BaseModel):
|
||||||
|
host: str = "localhost"
|
||||||
|
port: int = 5432
|
||||||
|
|
||||||
|
|
||||||
|
class ObservabilitySettings(BaseModel):
|
||||||
|
log_level: str = "INFO"
|
||||||
|
json_logs: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
model_config = SettingsConfigDict(
|
||||||
|
env_prefix="APP_",
|
||||||
|
env_nested_delimiter="__",
|
||||||
|
frozen=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
database: DatabaseSettings = Field(default_factory=DatabaseSettings)
|
||||||
|
observability: ObservabilitySettings = Field(
|
||||||
|
default_factory=ObservabilitySettings
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
This produces names such as `APP_DATABASE__HOST` and gives the application one validated, atomic configuration snapshot. Nested sections should normally inherit from `BaseModel`, not `BaseSettings`; otherwise each nested settings model can collect sources independently and produce surprising results.
|
||||||
|
|
||||||
|
Use independent `BaseSettings` classes when the objects have genuinely independent ownership:
|
||||||
|
|
||||||
|
1. Different packages or deployable components own the schemas.
|
||||||
|
2. Each object needs its own env prefix or source policy.
|
||||||
|
3. A component is optional or loaded lazily.
|
||||||
|
4. Components need different reload lifecycles.
|
||||||
|
5. The same component must run outside the application.
|
||||||
|
|
||||||
|
Construct independent objects explicitly at the composition root and inject each dependency. Do not nest one `BaseSettings` class inside another merely to reuse its fields. Extract a shared `BaseModel` schema when models need common structure.
|
||||||
|
|
||||||
|
### Alternative Database Backends
|
||||||
|
|
||||||
|
When one application can run against one of several database backends, model the selected backend as a [discriminated union](https://docs.pydantic.dev/latest/concepts/unions/#discriminated-unions). Pydantic validates only the variant selected by `driver`, so required PostgreSQL values do not make a SQLite configuration fail, and vice versa.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from typing import Annotated, Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field, SecretStr
|
||||||
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
|
|
||||||
|
|
||||||
|
class SqliteSettings(BaseModel):
|
||||||
|
driver: Literal["sqlite"] = "sqlite"
|
||||||
|
path: str = "app.db"
|
||||||
|
|
||||||
|
|
||||||
|
class PostgresSettings(BaseModel):
|
||||||
|
driver: Literal["postgres"] = "postgres"
|
||||||
|
host: str
|
||||||
|
port: int = 5432
|
||||||
|
database: str
|
||||||
|
user: str
|
||||||
|
password: SecretStr
|
||||||
|
|
||||||
|
|
||||||
|
DatabaseSettings = Annotated[
|
||||||
|
SqliteSettings | PostgresSettings,
|
||||||
|
Field(discriminator="driver"),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
model_config = SettingsConfigDict(
|
||||||
|
env_prefix="APP_",
|
||||||
|
env_nested_delimiter="__",
|
||||||
|
env_file=".env",
|
||||||
|
extra="ignore",
|
||||||
|
frozen=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
database: DatabaseSettings
|
||||||
|
```
|
||||||
|
|
||||||
|
Choose one configuration. A SQLite deployment requires no PostgreSQL variables:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
APP_DATABASE__DRIVER=sqlite
|
||||||
|
APP_DATABASE__PATH=./data/app.db
|
||||||
|
```
|
||||||
|
|
||||||
|
A PostgreSQL deployment requires only the PostgreSQL branch:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
APP_DATABASE__DRIVER=postgres
|
||||||
|
APP_DATABASE__HOST=db.internal
|
||||||
|
APP_DATABASE__PORT=5432
|
||||||
|
APP_DATABASE__DATABASE=app
|
||||||
|
APP_DATABASE__USER=app_user
|
||||||
|
APP_DATABASE__PASSWORD=provided-by-the-runtime
|
||||||
|
```
|
||||||
|
|
||||||
|
After settings validation, select an async SQLAlchemy driver URL. This is a pure configuration step; create the engine, session factory, and sessions in their own lifecycle-managed providers:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from functools import cache
|
||||||
|
|
||||||
|
from sqlalchemy import URL
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncEngine
|
||||||
|
from sqlalchemy.ext.asyncio import create_async_engine
|
||||||
|
|
||||||
|
|
||||||
|
def get_database_url(settings: Settings) -> str:
|
||||||
|
match settings.database:
|
||||||
|
case SqliteSettings(path=path):
|
||||||
|
url = URL.create(
|
||||||
|
drivername="sqlite+aiosqlite",
|
||||||
|
database=path,
|
||||||
|
)
|
||||||
|
case PostgresSettings() as database:
|
||||||
|
url = URL.create(
|
||||||
|
drivername="postgresql+asyncpg",
|
||||||
|
host=database.host,
|
||||||
|
port=database.port,
|
||||||
|
database=database.database,
|
||||||
|
username=database.user,
|
||||||
|
password=database.password.get_secret_value(),
|
||||||
|
)
|
||||||
|
return url.render_as_string(hide_password=False)
|
||||||
|
|
||||||
|
|
||||||
|
@cache
|
||||||
|
def get_engine(database_url: str) -> AsyncEngine:
|
||||||
|
return create_async_engine(database_url, pool_pre_ping=True)
|
||||||
|
|
||||||
|
|
||||||
|
async def dispose_engine(database_url: str) -> None:
|
||||||
|
engine = get_engine(database_url)
|
||||||
|
try:
|
||||||
|
await engine.dispose()
|
||||||
|
finally:
|
||||||
|
get_engine.cache_clear()
|
||||||
|
|
||||||
|
|
||||||
|
async def refresh_engine(database_url: str) -> AsyncEngine:
|
||||||
|
await dispose_engine(database_url)
|
||||||
|
return get_engine(database_url)
|
||||||
|
```
|
||||||
|
|
||||||
|
At the composition boundary, resolve the URL once with `get_database_url(settings)` and use it to retrieve the cached engine. In FastAPI, expose that engine through lifespan and build one `async_sessionmaker` from it; each request or unit of work then creates its own `AsyncSession`. Do not call `aiosqlite.connect()` or `asyncpg.create_pool()` directly: `aiosqlite` and `asyncpg` are selected as SQLAlchemy drivers by the URL, while SQLAlchemy owns pooling, disposal, and session integration.
|
||||||
|
|
||||||
|
The nested variants remain `BaseModel` classes. `Settings` is the only `BaseSettings` model and therefore the only object that reads environment variables, dotenv files, or secrets. This keeps one source policy and validated configuration snapshot while keeping the engine, session factory, and sessions in their distinct lifecycles. See the [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html), the [engine lifecycle guidance](../async-fastapi-sqlmodel/references/engine.md), and the [session lifecycle guidance](../async-fastapi-sqlmodel/references/session.md).
|
||||||
|
|
||||||
|
Quality gate:
|
||||||
|
|
||||||
|
1. Nested sections share one source policy and lifecycle.
|
||||||
|
2. Independent settings have distinct owners, prefixes, or lifecycles.
|
||||||
|
3. The application does not repeatedly scan the same sources through accidental nested `BaseSettings` construction.
|
||||||
|
4. Each backend configuration validates without values required only by another backend.
|
||||||
|
5. One cached `AsyncEngine` exists per configured driver URL, while each request or unit of work receives a new `AsyncSession`.
|
||||||
|
|
||||||
|
### 7. Own The Settings Lifecycle
|
||||||
|
|
||||||
|
For most applications, construct settings once at the composition root and pass the validated object to services:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def main() -> None:
|
||||||
|
settings = Settings()
|
||||||
|
application = Application(settings=settings)
|
||||||
|
application.run()
|
||||||
|
```
|
||||||
|
|
||||||
|
This makes ownership, startup failure, and test overrides explicit. Treat the object as a snapshot: environment variables and files changing later do not update an existing instance. Prefer `frozen=True` for shared settings so consumers cannot silently mutate process-wide configuration.
|
||||||
|
|
||||||
|
Use [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) only when process-lifetime singleton access is intentional and explicit injection is awkward, such as a framework dependency provider:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from functools import cache
|
||||||
|
|
||||||
|
|
||||||
|
@cache
|
||||||
|
def get_settings() -> Settings:
|
||||||
|
return Settings()
|
||||||
|
```
|
||||||
|
|
||||||
|
Keep the cached factory argument-free. Passing override kwargs creates one cached instance per argument combination, retains those values for the process lifetime, and obscures which configuration is active. In tests, instantiate `Settings(...)` directly or override the dependency; when a test must exercise the cached getter, isolate environment changes with `get_settings.cache_clear()` before and after the assertion.
|
||||||
|
|
||||||
|
`cache` is process-local. Every worker process gets its own instance, and concurrent first calls can construct more than one instance before the cache is populated. Settings construction must therefore be side-effect free; create engines, clients, and sessions in their own lifecycle-managed providers.
|
||||||
|
|
||||||
|
Quality gate:
|
||||||
|
|
||||||
|
1. Settings are created once per intended application or worker lifecycle.
|
||||||
|
2. Cached factories are argument-free and side-effect free.
|
||||||
|
3. Tests do not leak cached settings or environment changes.
|
||||||
|
4. Resource construction is separate from configuration parsing.
|
||||||
|
|
||||||
|
### 8. Reload Deliberately
|
||||||
|
|
||||||
|
Static service configuration should normally require a process restart. If runtime reload is a real requirement, construct a fresh settings instance and atomically replace the owned reference. Do not call `__init__()` on a shared instance: readers can observe mutation in progress, and resources derived from old values may remain alive.
|
||||||
|
|
||||||
|
Settings sources are synchronous. In an async application, construction or reload that reads dotenv, secrets, JSON, TOML, or YAML files should run in a worker thread:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
|
||||||
|
async def load_settings() -> Settings:
|
||||||
|
return await asyncio.to_thread(Settings)
|
||||||
|
```
|
||||||
|
|
||||||
|
Clearing `get_settings` is sufficient for controlled tests or single-threaded administration, but it is not an atomic live-reload protocol. Concurrent applications should own the current reference behind an application-specific lock or lifecycle manager, swap in a fully validated replacement, and then rebuild dependent resources.
|
||||||
|
|
||||||
|
Quality gate:
|
||||||
|
|
||||||
|
1. Reload creates and validates a replacement before publication.
|
||||||
|
2. Readers cannot observe a partially mutated object.
|
||||||
|
3. Dependent resources are recreated after the settings reference changes.
|
||||||
|
4. File-backed source reads do not block an async event loop.
|
||||||
|
|
||||||
|
### 9. Add Focused Lifecycle Tests
|
||||||
|
|
||||||
|
Do not add tests that re-validate baseline `pydantic-settings` functionality unless custom behavior is layered on top. Test the application-owned behavior instead:
|
||||||
|
|
||||||
|
1. Repeated cached getter calls return the same instance.
|
||||||
|
2. Cache clearing after an environment change returns a newly validated instance.
|
||||||
|
3. Explicitly injected settings bypass global cached state.
|
||||||
|
4. Reload swaps the settings snapshot and rebuilds dependent resources, when reload is supported.
|
||||||
|
|
||||||
|
Suggested invocation:
|
||||||
|
|
||||||
|
1. `uv run pytest -q`
|
||||||
|
|
||||||
|
## Completion Checks
|
||||||
|
|
||||||
|
1. Settings ownership matches the application or component lifecycle.
|
||||||
|
2. Source precedence is documented and tested.
|
||||||
|
3. Env naming conventions and aliases are explicit and stable.
|
||||||
|
4. Nested parsing behavior is tested when custom parsing behavior is added.
|
||||||
|
5. Secrets and dotenv usage are environment-appropriate and do not leak sensitive defaults.
|
||||||
|
6. Validation errors are actionable and fail fast for required values.
|
||||||
|
7. Cached factories are argument-free, process-local, and cleared deliberately in tests.
|
||||||
|
8. Nested models share one source policy; independent settings have an explicit ownership reason.
|
||||||
|
9. Runtime reload, if supported, replaces a validated snapshot and rebuilds dependent resources.
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
|
||||||
|
When this skill is applied, return:
|
||||||
|
|
||||||
|
1. Which references were consulted.
|
||||||
|
2. The chosen source-precedence model and why.
|
||||||
|
3. The exact parsing and alias decisions made.
|
||||||
|
4. Any deferred choices and their risk.
|
||||||
|
5. The validation commands or tests run to confirm behavior.
|
||||||
|
|
||||||
|
Use these upstream docs when implementing or reviewing `pydantic-settings` behavior.
|
||||||
|
|
||||||
|
## Source Docs
|
||||||
|
|
||||||
|
### Primary
|
||||||
|
|
||||||
|
- [Settings Management](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/)
|
||||||
|
- [pydantic-settings package repository](https://github.com/pydantic/pydantic-settings)
|
||||||
|
|
||||||
|
### Core Concepts
|
||||||
|
|
||||||
|
- [Field aliases](https://pydantic.dev/docs/validation/latest/concepts/fields/#field-aliases)
|
||||||
|
- [Alias choices](https://pydantic.dev/docs/validation/latest/concepts/alias#aliaspath-and-aliaschoices)
|
||||||
|
- [Validation default behavior](https://pydantic.dev/docs/validation/latest/concepts/fields#validate-default-values)
|
||||||
|
- [ImportString type](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.ImportString)
|
||||||
|
|
||||||
|
### Priority And Sources
|
||||||
|
|
||||||
|
- [Field value priority](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#field-value-priority)
|
||||||
|
- [Customise settings sources](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#customise-settings-sources)
|
||||||
|
- [Other settings source types](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#other-settings-source)
|
||||||
|
|
||||||
|
### Environment And Parsing
|
||||||
|
|
||||||
|
- [Environment variable names and prefix behavior](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#environment-variable-names)
|
||||||
|
- [Case sensitivity behavior](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#case-sensitivity)
|
||||||
|
- [Parsing environment variable values](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#parsing-environment-variable-values)
|
||||||
|
- [Nested model default partial updates](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#nested-model-default-partial-updates)
|
||||||
|
|
||||||
|
### Lifecycle And Reloading
|
||||||
|
|
||||||
|
- [In-place reloading](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#in-place-reloading)
|
||||||
|
- [Async environments](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#async-environments)
|
||||||
|
- [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
||||||
|
|
||||||
|
### Dotenv And Secrets
|
||||||
|
|
||||||
|
- [Dotenv support](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#dotenv-env-support)
|
||||||
|
- [Secrets](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#secrets)
|
||||||
|
- [Nested secrets](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#nested-secrets)
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
---
|
||||||
|
name: pytesting
|
||||||
|
description: "Reference hub for pytest suite structure, naming, markers, and stack-specific testing patterns. Optimized for progressive discovery so naming and hierarchy guidance are loaded first when shaping or reorganizing tests."
|
||||||
|
---
|
||||||
|
|
||||||
|
# Pytesting
|
||||||
|
|
||||||
|
This skill is a collection of preferences and links to source documentation for building and maintaining pytest suites.
|
||||||
|
|
||||||
|
Use it to quickly find the right guidance for:
|
||||||
|
1. Baseline pytest structure and marker strategy.
|
||||||
|
2. Naming conventions and test hierarchy organization.
|
||||||
|
3. FastAPI route, dependency override, and lifespan testing patterns.
|
||||||
|
4. SQLAlchemy transaction and session testing patterns.
|
||||||
|
5. AsyncIO loop-scope, fixture-lifecycle, and cancellation-safe testing patterns.
|
||||||
|
|
||||||
|
Repository defaults:
|
||||||
|
- `uv run pytest` is the canonical invocation.
|
||||||
|
- pytest settings live in `pyproject.toml` under `[tool.pytest.ini_options]`.
|
||||||
|
- strict marker checking is expected (`--strict-markers`).
|
||||||
|
|
||||||
|
## Progressive Discovery Start
|
||||||
|
|
||||||
|
Use this load order by default so guidance stays targeted and naming conventions are pulled in early:
|
||||||
|
|
||||||
|
1. Classify intent first: naming and organization, baseline pytest mechanics, FastAPI testing, SQLAlchemy testing, or mixed.
|
||||||
|
2. For create/restructure/rename tasks, load [naming-and-organization.md](./references/naming-and-organization.md) first.
|
||||||
|
3. Load [pytest-docs.md](./references/pytest-docs.md) next for fixture and marker defaults.
|
||||||
|
4. Load at most one stack-specific reference unless the request is explicitly mixed stack.
|
||||||
|
5. If confidence is low after two references, ask one clarifying question before loading more.
|
||||||
|
|
||||||
|
Load budget defaults:
|
||||||
|
|
||||||
|
1. Single-stack task: 1 to 2 references.
|
||||||
|
2. Mixed-stack task: up to 3 references.
|
||||||
|
3. Avoid loading all references unless the user explicitly asks for a broad audit.
|
||||||
|
|
||||||
|
## Intent Router
|
||||||
|
|
||||||
|
Open only the reference that matches the immediate task.
|
||||||
|
|
||||||
|
1. Naming, file layout, discovery prefixes, class/function naming: [naming-and-organization.md](./references/naming-and-organization.md)
|
||||||
|
2. Fixture layering, marker policy, collect-only and fast-path commands: [pytest-docs.md](./references/pytest-docs.md)
|
||||||
|
3. Route tests, dependency overrides, lifespan handling: [fastapi-testing.md](./references/fastapi-testing.md)
|
||||||
|
4. Session and transaction fixtures, async ORM behavior: [sqlalchemy-testing.md](./references/sqlalchemy-testing.md)
|
||||||
|
5. Async test mode selection, event loop scope, cancel-scope teardown issues: [asyncio-testing.md](./references/asyncio-testing.md)
|
||||||
|
|
||||||
|
## Naming Pull-In Triggers
|
||||||
|
|
||||||
|
Always consult [naming-and-organization.md](./references/naming-and-organization.md) before recommending structure when any of these are true:
|
||||||
|
|
||||||
|
1. New tests are being added.
|
||||||
|
2. Existing tests are being reorganized or renamed.
|
||||||
|
3. The request mentions conventions, readability, hierarchy, or discoverability.
|
||||||
|
4. The task introduces parametrization where case naming affects failure readability.
|
||||||
|
|
||||||
|
## Guiding Principles
|
||||||
|
|
||||||
|
These principles are abstract, but are the highest priority to follow.
|
||||||
|
|
||||||
|
- Much of testing is very well-trodden. In general, tests should follow whatever conventions there are.
|
||||||
|
- Tests will be run very frequently, so it's important that they run quickly and deterministically.
|
||||||
|
- When tests fail, it should be easy to determine what failed and fix it.
|
||||||
|
- Always be on guard against tests that are tautological. Every test should provide specific value by capturing something about the intent of the program.
|
||||||
|
|
||||||
|
## Pytest Best Practices
|
||||||
|
|
||||||
|
These are stable defaults regardless of stack:
|
||||||
|
|
||||||
|
1. Apply pytest naming and hierarchy conventions first so discovery and ownership stay predictable; see [naming-and-organization.md](./references/naming-and-organization.md).
|
||||||
|
2. Mirror `src/` into `tests/` so ownership and coverage are obvious.
|
||||||
|
3. Keep fixtures explicit and layered (`tests/conftest.py` globally, subtree `conftest.py` for domain-specific fixtures).
|
||||||
|
4. Register markers up front (`unit`, `integration`, `smoke`, `slow`, `external`) and keep strict marker checks enabled.
|
||||||
|
5. Separate fast feedback (`-m unit`) from broader integration/external lanes.
|
||||||
|
6. Validate structure early with collection checks before expanding assertions.
|
||||||
|
7. Keep test scope tight and count intentional; add tests only when each case protects a distinct behavior.
|
||||||
|
8. Start with the single core-intent behavior path, then add edge cases based on real risk.
|
||||||
|
9. Prefer parametrized tests for behavior variants instead of cloning near-identical test functions.
|
||||||
|
10. Reject low-signal assertions (for example `assert True` patterns) and avoid tests that only assert a mock was called.
|
||||||
|
11. Prefer behavior-first tests that exercise real code paths and concrete inputs over patching internals.
|
||||||
|
12. Use monkeypatching, mocks, and fakes extremely sparingly, only when no practical real-input alternative exists, and only after explicit user confirmation.
|
||||||
|
|
||||||
|
## Universal Test Double Policy (Repo-Local Placement)
|
||||||
|
|
||||||
|
To avoid over-using monkeypatching, mocks, fakes, etc, apply this policy whenever a test change introduces one of them:
|
||||||
|
|
||||||
|
1. Attempt a real-input, real-object test design first.
|
||||||
|
2. If that approach is impractical, explain why and request user confirmation before adding monkeypatching, mocks, or fakes.
|
||||||
|
3. Keep any approved test double narrowly scoped and document the exact boundary it replaces.
|
||||||
|
4. Do not treat call-only verification as sufficient; pair any test double with assertions on observable behavior or outputs.
|
||||||
|
5. Revisit approved test doubles when implementation seams improve so they can be removed.
|
||||||
|
|
||||||
|
## Stack-Specific Guidance
|
||||||
|
|
||||||
|
- For FastAPI, prefer dependency overrides and clear lifecycle handling; see [fastapi-testing.md](./references/fastapi-testing.md).
|
||||||
|
- For SQLAlchemy, prefer transaction-safe session fixtures and explicit async loading strategy; see [sqlalchemy-testing.md](./references/sqlalchemy-testing.md).
|
||||||
|
- For async fixtures, loop-scope selection, and cancellation-safe teardown, see [asyncio-testing.md](./references/asyncio-testing.md).
|
||||||
|
- For naming and tree organization, use the conventions in [naming-and-organization.md](./references/naming-and-organization.md).
|
||||||
|
|
||||||
|
## Source Documentation Entry Points
|
||||||
|
|
||||||
|
Primary upstream docs are curated in each reference page. Start with:
|
||||||
|
|
||||||
|
1. Pytest good practices: [pytest docs](https://docs.pytest.org/en/stable/explanation/goodpractices.html)
|
||||||
|
2. Pytest fixtures: [fixture how-to](https://docs.pytest.org/en/stable/how-to/fixtures.html)
|
||||||
|
3. Pytest markers: [marker examples](https://docs.pytest.org/en/stable/example/markers.html)
|
||||||
|
4. FastAPI testing: [FastAPI testing tutorial](https://fastapi.tiangolo.com/tutorial/testing/)
|
||||||
|
5. SQLAlchemy transaction testing: [SQLAlchemy external transaction pattern](https://docs.sqlalchemy.org/en/20/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites)
|
||||||
|
6. Pytest monkeypatch usage and limits: [monkeypatch how-to](https://docs.pytest.org/en/stable/how-to/monkeypatch.html)
|
||||||
|
7. pytest-asyncio configuration: [pytest-asyncio config](https://pytest-asyncio.readthedocs.io/en/stable/reference/configuration.html)
|
||||||
|
8. AnyIO cancellation semantics: [AnyIO cancellation and timeouts](https://anyio.readthedocs.io/en/stable/cancellation.html)
|
||||||
|
|
||||||
|
## Quick Validation Commands
|
||||||
|
|
||||||
|
Use these commands to check structure and execution lanes:
|
||||||
|
|
||||||
|
1. `uv run pytest --collect-only -q`
|
||||||
|
2. `uv run pytest -m unit -q`
|
||||||
|
3. `uv run pytest -m "not external" -q`
|
||||||
|
4. `uv run pytest -q`
|
||||||
|
|
||||||
|
## Output Contract
|
||||||
|
When this skill is applied, return:
|
||||||
|
1. Which references were consulted.
|
||||||
|
2. The discovery path used (intent classification, load order, and why).
|
||||||
|
3. Recommended structure, naming, fixture, and marker decisions.
|
||||||
|
4. Concrete naming outcomes: file/module naming pattern, class usage decision, and any parametrization `ids` conventions.
|
||||||
|
5. Exact validation commands.
|
||||||
|
6. Relevant source-doc links for any non-trivial recommendation.
|
||||||
|
7. Risks, assumptions, or open questions.
|
||||||
|
8. Explicit confirmation status if monkeypatching, mocks, or fakes were requested or used.
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
# AsyncIO Testing Patterns (Pytest, FastAPI, AnyIO)
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [pytest-asyncio configuration](https://pytest-asyncio.readthedocs.io/en/stable/reference/configuration.html)
|
||||||
|
- [pytest-asyncio concepts](https://pytest-asyncio.readthedocs.io/en/stable/concepts.html)
|
||||||
|
- [pytest-asyncio fixture loop scope how-to](https://pytest-asyncio.readthedocs.io/en/stable/how-to-guides/change_fixture_loop.html)
|
||||||
|
- [pytest-asyncio default fixture loop scope how-to](https://pytest-asyncio.readthedocs.io/en/stable/how-to-guides/change_default_fixture_loop.html)
|
||||||
|
- [AnyIO cancellation and cancel-scope safety](https://anyio.readthedocs.io/en/stable/cancellation.html)
|
||||||
|
- [FastAPI async tests](https://fastapi.tiangolo.com/advanced/async-tests/)
|
||||||
|
|
||||||
|
## Agent Quick Path
|
||||||
|
Use this reference when tests involve asynchronous fixtures, HTTP clients, task groups, or teardown failures.
|
||||||
|
|
||||||
|
1. Confirm async plugin mode in pytest config (`asyncio_mode`).
|
||||||
|
2. Keep async fixture loop scope predictable, defaulting to `function` unless there is a measured need to broaden it.
|
||||||
|
3. Prefer one async testing model per lane (pytest-asyncio or AnyIO-style markers), and keep it consistent.
|
||||||
|
4. Keep async fixtures small and isolate stateful resources to the narrowest useful scope.
|
||||||
|
5. If teardown errors mention cancel scopes or task groups, validate that setup and teardown run in the same task context.
|
||||||
|
|
||||||
|
## Baseline Configuration
|
||||||
|
|
||||||
|
Recommended defaults for most projects using `pytest-asyncio`:
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[tool.pytest.ini_options]
|
||||||
|
asyncio_mode = "auto"
|
||||||
|
asyncio_default_fixture_loop_scope = "function"
|
||||||
|
```
|
||||||
|
|
||||||
|
Why:
|
||||||
|
- [Strict mode](https://pytest-asyncio.readthedocs.io/en/stable/concepts.html#test-discovery-modes) is safer for multi-plugin environments, but [auto mode](https://pytest-asyncio.readthedocs.io/en/stable/concepts.html#test-discovery-modes) is often simpler when the suite is primarily asyncio-based.
|
||||||
|
- [Function loop scope](https://pytest-asyncio.readthedocs.io/en/stable/reference/configuration.html#asyncio-default-fixture-loop-scope) minimizes cross-test coupling and avoids many lifecycle surprises.
|
||||||
|
|
||||||
|
If a fixture or test needs broader loop sharing, make it explicit instead of changing suite-wide defaults:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import pytest
|
||||||
|
import pytest_asyncio
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture(loop_scope="module")
|
||||||
|
async def shared_resource():
|
||||||
|
...
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio(loop_scope="module")
|
||||||
|
async def test_uses_shared_loop(shared_resource):
|
||||||
|
...
|
||||||
|
```
|
||||||
|
|
||||||
|
## FastAPI Endpoint Test Patterns
|
||||||
|
|
||||||
|
Use [FastAPI's async testing guidance](https://fastapi.tiangolo.com/advanced/async-tests/) as the default:
|
||||||
|
|
||||||
|
1. Use `httpx.AsyncClient` with `ASGITransport` for async endpoint tests.
|
||||||
|
2. Mark async tests with one consistent marker style for the suite.
|
||||||
|
3. If app lifespan hooks matter, add [LifespanManager](https://fastapi.tiangolo.com/advanced/async-tests/#httpx) support because `AsyncClient` alone does not trigger lifespan events.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import pytest
|
||||||
|
from httpx import ASGITransport, AsyncClient
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_healthz(app):
|
||||||
|
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
|
||||||
|
response = await client.get("/healthz")
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
```
|
||||||
|
|
||||||
|
## Fixture Design For Async Reliability
|
||||||
|
|
||||||
|
Apply these patterns first:
|
||||||
|
|
||||||
|
1. Keep async fixtures narrow (`function` scope by default).
|
||||||
|
2. Keep one responsibility per fixture when possible.
|
||||||
|
3. Prefer yield fixtures and pair each setup step with teardown in the same fixture.
|
||||||
|
4. Avoid mixing many independent event-loop lifecycles in one fixture chain.
|
||||||
|
|
||||||
|
When using transports that manage internal task groups (for example, streaming clients), avoid patterns that risk splitting lifecycle across different task contexts.
|
||||||
|
|
||||||
|
## Troubleshooting Cancel-Scope Teardown Failures
|
||||||
|
|
||||||
|
When you see errors like `Attempted to exit cancel scope in a different task than it was entered in`, treat it as an async lifecycle-ownership issue first.
|
||||||
|
|
||||||
|
Checklist:
|
||||||
|
|
||||||
|
1. Verify fixture and test loop scopes are compatible and explicit.
|
||||||
|
2. Confirm async resource setup and teardown are owned by the same fixture context.
|
||||||
|
3. Reduce fixture scope (`module` or `session` -> `function`) to test for loop/task ownership drift.
|
||||||
|
4. Ensure the suite uses one primary async plugin model for the failing lane.
|
||||||
|
5. Re-run with focused selection and skip reasons to isolate first failing fixture:
|
||||||
|
- `uv run --group test python -m pytest -m smoke tests/web -q -rs`
|
||||||
|
|
||||||
|
Relevant references:
|
||||||
|
- [Avoiding cancel scope stack corruption](https://anyio.readthedocs.io/en/stable/cancellation.html#avoiding-cancel-scope-stack-corruption)
|
||||||
|
- [pytest-asyncio configuration](https://pytest-asyncio.readthedocs.io/en/stable/reference/configuration.html)
|
||||||
|
- [pytest fixture teardown behavior](https://docs.pytest.org/en/stable/how-to/fixtures.html#teardown-cleanup-aka-fixture-finalization)
|
||||||
|
|
||||||
|
## Commands Worth Remembering
|
||||||
|
|
||||||
|
- `uv run --group test python -m pytest --collect-only -q`
|
||||||
|
- `uv run --group test python -m pytest -m smoke tests/web -q -rs`
|
||||||
|
- `uv run --group test python -m pytest -m integration -q`
|
||||||
|
- `uv run --group test python -m pytest -q`
|
||||||
@@ -0,0 +1,236 @@
|
|||||||
|
# [FastAPI Testing](https://fastapi.tiangolo.com/tutorial/testing/)
|
||||||
|
|
||||||
|
Best practices for testing FastAPI applications with pytest.
|
||||||
|
|
||||||
|
## Agent Quick Path
|
||||||
|
Use this sequence before reading the full reference:
|
||||||
|
|
||||||
|
1. If test is pure route behavior, use `TestClient` and plain `def` tests.
|
||||||
|
2. If test must `await` other async work, use `AsyncClient` + `@pytest.mark.anyio`.
|
||||||
|
3. Prefer `app.dependency_overrides` over `mock.patch`.
|
||||||
|
4. Reset overrides after each test/fixture teardown.
|
||||||
|
5. For startup/shutdown logic, use `TestClient` as context manager or `LifespanManager` with async client.
|
||||||
|
|
||||||
|
Decision rules:
|
||||||
|
- Need DB contract verification: choose integration tests and override `get_db`/`get_session`.
|
||||||
|
- Need pure business logic checks: keep tests HTTP-free (`unit`).
|
||||||
|
- Need one critical path sanity check: one endpoint per `smoke` test.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tools
|
||||||
|
|
||||||
|
| Tool | Purpose |
|
||||||
|
|------|---------|
|
||||||
|
| `fastapi.testclient.TestClient` | Synchronous HTTP test client (wraps HTTPX, built on Starlette) |
|
||||||
|
| `httpx.AsyncClient` + `ASGITransport` | Async client for tests that `await` other async code |
|
||||||
|
| `app.dependency_overrides` | Replace any `Depends()` dependency for the duration of a test |
|
||||||
|
| `anyio` / `pytest-anyio` | Run async test functions with `@pytest.mark.anyio` |
|
||||||
|
|
||||||
|
Install deps: `httpx`, `anyio` (or `pytest-anyio`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Synchronous Tests (Preferred Default)
|
||||||
|
|
||||||
|
Use `TestClient` for route tests that don't need to `await` anything else.
|
||||||
|
Test functions are plain `def` — no `async def`, no `await`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from myapp.main import app
|
||||||
|
|
||||||
|
client = TestClient(app)
|
||||||
|
|
||||||
|
def test_read_item_returns_200():
|
||||||
|
response = client.get("/items/foo", headers={"X-Token": "secret"})
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert response.json()["id"] == "foo"
|
||||||
|
|
||||||
|
def test_read_item_bad_token_returns_400():
|
||||||
|
response = client.get("/items/foo", headers={"X-Token": "wrong"})
|
||||||
|
assert response.status_code == 400
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- One `TestClient` per test module is fine (stateless between calls).
|
||||||
|
- Pass headers, query params, JSON body, or form data the same way as HTTPX/requests.
|
||||||
|
- Do not pass Pydantic models directly; use `.model_dump()` or `jsonable_encoder`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Async Tests
|
||||||
|
|
||||||
|
Use `AsyncClient` only when the test itself needs to `await` other coroutines
|
||||||
|
(e.g. querying a real async DB after an API call to verify side effects).
|
||||||
|
|
||||||
|
```python
|
||||||
|
import pytest
|
||||||
|
from httpx import ASGITransport, AsyncClient
|
||||||
|
from myapp.main import app
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_root_async():
|
||||||
|
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as ac:
|
||||||
|
response = await ac.get("/")
|
||||||
|
assert response.status_code == 200
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- Mark with `@pytest.mark.anyio`; register the `anyio` marker in `pyproject.toml`.
|
||||||
|
- `AsyncClient` does **not** trigger lifespan events by default; use `asgi-lifespan`'s
|
||||||
|
`LifespanManager` when startup/shutdown matters.
|
||||||
|
- Instantiate objects that require an event loop (e.g. async DB clients) inside
|
||||||
|
async functions, not at module level.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dependency Overrides (Preferred Over Mocking)
|
||||||
|
|
||||||
|
`app.dependency_overrides` is the idiomatic FastAPI seam — use it instead of
|
||||||
|
patching internals with `unittest.mock`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from myapp.main import app
|
||||||
|
from myapp.deps import get_current_user
|
||||||
|
|
||||||
|
client = TestClient(app)
|
||||||
|
|
||||||
|
def fake_user():
|
||||||
|
return {"id": 1, "name": "Test User"}
|
||||||
|
|
||||||
|
def test_protected_route_with_fake_user():
|
||||||
|
app.dependency_overrides[get_current_user] = fake_user
|
||||||
|
response = client.get("/me")
|
||||||
|
app.dependency_overrides = {} # always reset after the test
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert response.json()["name"] == "Test User"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or reset cleanly with `autouse=False` fixture teardown:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import pytest
|
||||||
|
from myapp.main import app
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def override_user():
|
||||||
|
app.dependency_overrides[get_current_user] = lambda: {"id": 1, "name": "Test User"}
|
||||||
|
yield
|
||||||
|
app.dependency_overrides = {}
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- Override at the lowest-level dependency that owns the external boundary
|
||||||
|
(e.g. `get_db`, `get_current_user`, `get_settings`).
|
||||||
|
- Always reset `app.dependency_overrides` after each test or fixture teardown.
|
||||||
|
- Prefer a real in-process fake (e.g. in-memory SQLite session) over a mock object.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Database Testing
|
||||||
|
|
||||||
|
The preferred pattern is a real SQLite (or test Postgres) session injected via
|
||||||
|
dependency override, not a mock.
|
||||||
|
|
||||||
|
```python
|
||||||
|
# tests/conftest.py
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import create_engine
|
||||||
|
from sqlalchemy.orm import sessionmaker
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from myapp.main import app
|
||||||
|
from myapp.db.session import get_db
|
||||||
|
from myapp.db.models import Base
|
||||||
|
|
||||||
|
TEST_DB_URL = "sqlite:///./test.db"
|
||||||
|
|
||||||
|
@pytest.fixture(scope="session")
|
||||||
|
def engine():
|
||||||
|
e = create_engine(TEST_DB_URL, connect_args={"check_same_thread": False})
|
||||||
|
Base.metadata.create_all(bind=e)
|
||||||
|
yield e
|
||||||
|
Base.metadata.drop_all(bind=e)
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def db_session(engine):
|
||||||
|
Session = sessionmaker(bind=engine)
|
||||||
|
session = Session()
|
||||||
|
yield session
|
||||||
|
session.rollback()
|
||||||
|
session.close()
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def client(db_session):
|
||||||
|
app.dependency_overrides[get_db] = lambda: db_session
|
||||||
|
yield TestClient(app)
|
||||||
|
app.dependency_overrides = {}
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- Prefer `session`-scoped engine creation; `function`-scoped session with rollback per test.
|
||||||
|
- Keep unit tests DB-free; use this pattern only in `integration`-marked tests.
|
||||||
|
- For async SQLAlchemy, mirror the same pattern using `AsyncEngine` / `AsyncSession`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Lifespan and Startup Events
|
||||||
|
|
||||||
|
`TestClient` triggers lifespan events (startup/shutdown) when used as a context manager:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def test_with_lifespan():
|
||||||
|
with TestClient(app) as client:
|
||||||
|
response = client.get("/health")
|
||||||
|
assert response.status_code == 200
|
||||||
|
```
|
||||||
|
|
||||||
|
For `AsyncClient`, use `asgi-lifespan`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from asgi_lifespan import LifespanManager
|
||||||
|
|
||||||
|
@pytest.mark.anyio
|
||||||
|
async def test_with_async_lifespan():
|
||||||
|
async with LifespanManager(app):
|
||||||
|
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as ac:
|
||||||
|
response = await ac.get("/health")
|
||||||
|
assert response.status_code == 200
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Marker Strategy for FastAPI Tests
|
||||||
|
|
||||||
|
| Marker | When to use |
|
||||||
|
|--------|------------|
|
||||||
|
| `unit` | Pure service/utility logic with no HTTP or DB calls |
|
||||||
|
| `integration` | `TestClient` + real DB session via dependency override |
|
||||||
|
| `smoke` | One `TestClient` call per critical user path, no DB reset |
|
||||||
|
| `external` | Tests that call real third-party APIs (skip in CI by default) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Reference: Sending Data
|
||||||
|
|
||||||
|
| What to send | Parameter |
|
||||||
|
|---|---|
|
||||||
|
| Path / query param | Part of the URL string |
|
||||||
|
| JSON body | `json={"key": "value"}` |
|
||||||
|
| Form data | `data={"field": "value"}` |
|
||||||
|
| Headers | `headers={"X-Token": "..."}` |
|
||||||
|
| Cookies | `cookies={"session": "..."}` |
|
||||||
|
| File upload | `files={"file": ("name.txt", b"content", "text/plain")}` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Official Docs
|
||||||
|
|
||||||
|
- [Testing tutorial](https://fastapi.tiangolo.com/tutorial/testing/)
|
||||||
|
- [Async tests](https://fastapi.tiangolo.com/advanced/async-tests/)
|
||||||
|
- [Testing dependencies with overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
||||||
|
- [Testing a database (SQLModel)](https://fastapi.tiangolo.com/how-to/testing-database/)
|
||||||
|
- [Testing lifespan events](https://fastapi.tiangolo.com/advanced/testing-events/)
|
||||||
|
- [Testing WebSockets](https://fastapi.tiangolo.com/advanced/testing-websockets/)
|
||||||
|
- [HTTPX docs](https://www.python-httpx.org/)
|
||||||
@@ -0,0 +1,118 @@
|
|||||||
|
# Pytest Naming Conventions and Test Organization
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [Good integration practices](https://docs.pytest.org/en/stable/explanation/goodpractices.html)
|
||||||
|
- [Changing standard (Python) test discovery](https://docs.pytest.org/en/stable/example/pythoncollection.html)
|
||||||
|
- [How to use fixtures](https://docs.pytest.org/en/stable/how-to/fixtures.html)
|
||||||
|
- [How to parametrize fixtures and test functions](https://docs.pytest.org/en/stable/how-to/parametrize.html)
|
||||||
|
- [Marker examples](https://docs.pytest.org/en/stable/example/markers.html)
|
||||||
|
|
||||||
|
## Agent Quick Path
|
||||||
|
Use this when creating or reorganizing test modules so naming and hierarchy stay predictable.
|
||||||
|
|
||||||
|
1. Mirror the product domain structure in `tests/` so ownership is obvious.
|
||||||
|
2. Encode broad context in module and class names (`test_*.py`, `Test*`).
|
||||||
|
3. Keep leaf test names short and behavior-focused (`test_*`).
|
||||||
|
4. Use `class Test<Subject>:` only for grouping related scenarios.
|
||||||
|
5. Place fixtures in the nearest `conftest.py` needed by scope.
|
||||||
|
6. Separate expensive tests with markers first, directories second.
|
||||||
|
|
||||||
|
## Naming Conventions
|
||||||
|
|
||||||
|
### File and directory naming
|
||||||
|
- Use lowercase snake_case for test file names: `test_user_service.py`.
|
||||||
|
- Keep directories domain-oriented and stable over time: `tests/orders/`, `tests/billing/`.
|
||||||
|
- Prefer descriptive test names over internal ticket numbers or implementation details.
|
||||||
|
|
||||||
|
### Test function naming
|
||||||
|
- Prefer hierarchical naming: put broad context in folder/module/class, and keep the function name focused on the final assertion.
|
||||||
|
- Keep pytest discovery prefixes intact:
|
||||||
|
- modules start with `test_`
|
||||||
|
- classes start with `Test`
|
||||||
|
- functions start with `test_`
|
||||||
|
- Start with user-visible behavior or contract, not private helper names.
|
||||||
|
|
||||||
|
Recommended pattern:
|
||||||
|
- module: `test_<subject>.py`
|
||||||
|
- class: `Test<Operation>` or `Test<Scenario>`
|
||||||
|
- function: `test_<expected_outcome>`
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
- Flat (still valid): `test_create_order_rejects_invalid_currency`
|
||||||
|
- Class-context: `TestOrder -> TestCreate -> test_rejects_invalid_currency`
|
||||||
|
- Module-context: `test_order.py -> TestCreate -> test_rejects_invalid_currency`
|
||||||
|
- Module + class context can similarly shorten:
|
||||||
|
- `test_token.py -> TestRefresh -> test_rotates_session_id`
|
||||||
|
- `test_user_list.py -> TestListUsers -> test_returns_empty_for_new_tenant`
|
||||||
|
|
||||||
|
### Test class naming
|
||||||
|
- Use `class Test<SubjectOrScenario>:` for scenario grouping and context reduction.
|
||||||
|
- Keep class names noun-focused (`TestOrderService`) rather than action-focused.
|
||||||
|
- Avoid xUnit style setup inheritance when fixtures can express dependencies directly.
|
||||||
|
|
||||||
|
## Hierarchy and Organization Patterns
|
||||||
|
|
||||||
|
Two patterns work well; choose one and apply it consistently.
|
||||||
|
|
||||||
|
### Pattern A: Source-mirror hierarchy (default for product code ownership)
|
||||||
|
|
||||||
|
```text
|
||||||
|
src/
|
||||||
|
app/
|
||||||
|
orders/service.py
|
||||||
|
billing/invoice.py
|
||||||
|
|
||||||
|
tests/
|
||||||
|
app/
|
||||||
|
orders/test_service.py
|
||||||
|
billing/test_invoice.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Use this when teams own modules by source path and want direct test-to-source mapping.
|
||||||
|
|
||||||
|
### Pattern B: Cost-lane hierarchy (default for CI policy clarity)
|
||||||
|
|
||||||
|
```text
|
||||||
|
tests/
|
||||||
|
unit/
|
||||||
|
orders/test_service.py
|
||||||
|
integration/
|
||||||
|
api/test_orders.py
|
||||||
|
persistence/test_order_repository.py
|
||||||
|
smoke/
|
||||||
|
test_health.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Use this when CI gating is based on cost lanes and marker filtering.
|
||||||
|
|
||||||
|
### Hybrid rule (recommended)
|
||||||
|
- Keep a source-mirror tree for local ownership.
|
||||||
|
- Add markers (`unit`, `integration`, `smoke`, `external`) for runtime policy.
|
||||||
|
- Avoid duplicating both trees unless the repository already requires it.
|
||||||
|
|
||||||
|
## Fixture Placement Strategy
|
||||||
|
- Put universal lightweight fixtures in `tests/conftest.py`.
|
||||||
|
- Put domain fixtures in subtree `conftest.py` files close to where they are used.
|
||||||
|
- Keep fixtures composable and explicit; avoid large fixture "god objects".
|
||||||
|
- Use `yield` fixtures for teardown so cleanup is always paired with setup.
|
||||||
|
|
||||||
|
## Parametrize and ID Naming
|
||||||
|
- Use `pytest.mark.parametrize` for behavior matrices instead of copy/paste tests.
|
||||||
|
- Provide explicit `ids=` labels when case names are not obvious.
|
||||||
|
- Keep IDs business-meaningful (`"expired-token"`, `"zero-balance"`) so failures are readable.
|
||||||
|
|
||||||
|
## Collection and Structure Checks
|
||||||
|
Use these checks after introducing new test files or renaming modules:
|
||||||
|
|
||||||
|
- `uv run pytest --collect-only -q`
|
||||||
|
- `uv run pytest -m unit -q`
|
||||||
|
- `uv run pytest -m "not external" -q`
|
||||||
|
|
||||||
|
If collection surprises appear, verify file names, marker registration, and directory placement first.
|
||||||
|
|
||||||
|
## Common Anti-Patterns
|
||||||
|
- Mixed naming styles (`testFoo.py`, `test_foo.py`, `foo_test.py`) in one repository.
|
||||||
|
- Deep fixture chains that hide setup behavior.
|
||||||
|
- Test names that encode implementation details instead of behavior.
|
||||||
|
- Moving slow tests into `unit` directories without marker updates.
|
||||||
|
- Sharing mutable module-level state across tests.
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# Pytest Documentation Notes
|
||||||
|
|
||||||
|
!!! info "Primary sources"
|
||||||
|
- [Good integration practices](https://docs.pytest.org/en/stable/explanation/goodpractices.html)
|
||||||
|
- [Fixture how-to](https://docs.pytest.org/en/stable/how-to/fixtures.html)
|
||||||
|
- [Marker examples](https://docs.pytest.org/en/stable/example/markers.html)
|
||||||
|
- [Configuration reference](https://docs.pytest.org/en/stable/reference/customize.html)
|
||||||
|
- [Flaky tests](https://docs.pytest.org/en/stable/explanation/flaky.html)
|
||||||
|
|
||||||
|
## Agent Quick Path
|
||||||
|
Use this file when you need fast pytest scaffolding defaults without framework-specific details.
|
||||||
|
|
||||||
|
1. Mirror source layout under `tests/`.
|
||||||
|
2. Keep fixtures small and explicit; default to `function` scope.
|
||||||
|
3. Register markers up front in `pyproject.toml`.
|
||||||
|
4. Validate structure first with `uv run pytest --collect-only -q`.
|
||||||
|
5. Run fast lane with `uv run pytest -m unit -q`.
|
||||||
|
|
||||||
|
Load other references only when needed:
|
||||||
|
- FastAPI routes/dependency injection/lifespan: `fastapi-testing.md`
|
||||||
|
- SQLAlchemy sessions/transactions/DB fixtures: `sqlalchemy-testing.md`
|
||||||
|
- Naming conventions and test hierarchy: `naming-and-organization.md`
|
||||||
|
|
||||||
|
## Practical Guidance For This Skill
|
||||||
|
- Use src-aligned test layout and keep test discovery conventional.
|
||||||
|
- Keep fixtures small, composable, and explicit; use `yield` for teardown.
|
||||||
|
- Register custom markers and keep strict marker validation on.
|
||||||
|
- Separate quick unit runs from slower integration/external runs.
|
||||||
|
- Minimize flakiness by controlling shared state and avoiding hidden dependencies.
|
||||||
|
- Use `--collect-only` and marker-filtered runs to validate scaffold quality early.
|
||||||
|
|
||||||
|
## Commands Worth Remembering
|
||||||
|
- `uv run pytest --collect-only -q`
|
||||||
|
- `uv run pytest -m unit -q`
|
||||||
|
- `uv run pytest -m "not external" -q`
|
||||||
|
- `uv run pytest -q`
|
||||||
@@ -0,0 +1,246 @@
|
|||||||
|
# [SQLAlchemy 2.x Testing](https://docs.sqlalchemy.org/en/20/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites)
|
||||||
|
|
||||||
|
Best practices for testing SQLAlchemy ORM code (sync and async) with pytest.
|
||||||
|
|
||||||
|
## Agent Quick Path
|
||||||
|
Use this path first; read deeper sections only when needed.
|
||||||
|
|
||||||
|
1. Create engine once per test session.
|
||||||
|
2. Open connection + outer transaction per test function.
|
||||||
|
3. Bind `Session`/`AsyncSession` to that connection with `join_transaction_mode="create_savepoint"`.
|
||||||
|
4. Let code under test call `commit()` safely; rollback outer transaction after test.
|
||||||
|
5. Inject session into FastAPI via dependency override and always clear overrides.
|
||||||
|
|
||||||
|
Branching logic:
|
||||||
|
- Sync stack: use `create_engine` + `Session` fixtures.
|
||||||
|
- Async stack: use `create_async_engine` + `AsyncSession` fixtures + `pytest.mark.anyio`.
|
||||||
|
- SQLite in-memory with threaded client: use `StaticPool` when required by framework threading behavior.
|
||||||
|
- Async relationship access fails (`MissingGreenlet`): eager load (`selectinload`) or explicit refresh.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Core Concepts
|
||||||
|
|
||||||
|
| Concept | Preferred approach |
|
||||||
|
|---|---|
|
||||||
|
| Isolate tests from production DB | Use an in-memory SQLite engine per test |
|
||||||
|
| Prevent data leaking between tests | Roll back at the connection level after each test |
|
||||||
|
| Inject test DB into FastAPI | Override the `get_db` / `get_session` dependency |
|
||||||
|
| Create schema for tests | Call `Base.metadata.create_all(engine)` once per session |
|
||||||
|
| Avoid lazy-load errors in async | Use `expire_on_commit=False`; use `selectinload()` for relationships |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Sync SQLAlchemy + FastAPI (Recommended Pattern)
|
||||||
|
|
||||||
|
The canonical 2.0 pattern joins a test `Session` into an external transaction on a shared `Connection`, then rolls back after each test. This means `session.commit()` calls within the code under test are "committed" to a savepoint, not the real transaction, and are fully undone after the test.
|
||||||
|
|
||||||
|
```python
|
||||||
|
# tests/conftest.py
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import create_engine
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from myapp.main import app
|
||||||
|
from myapp.db.session import get_db
|
||||||
|
from myapp.db.models import Base
|
||||||
|
|
||||||
|
@pytest.fixture(scope="session")
|
||||||
|
def engine():
|
||||||
|
e = create_engine("sqlite://", connect_args={"check_same_thread": False})
|
||||||
|
Base.metadata.create_all(bind=e)
|
||||||
|
yield e
|
||||||
|
Base.metadata.drop_all(bind=e)
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def db_session(engine):
|
||||||
|
connection = engine.connect()
|
||||||
|
transaction = connection.begin()
|
||||||
|
session = Session(bind=connection, join_transaction_mode="create_savepoint")
|
||||||
|
yield session
|
||||||
|
session.close()
|
||||||
|
transaction.rollback()
|
||||||
|
connection.close()
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
def client(db_session):
|
||||||
|
app.dependency_overrides[get_db] = lambda: db_session
|
||||||
|
yield TestClient(app)
|
||||||
|
app.dependency_overrides = {}
|
||||||
|
```
|
||||||
|
|
||||||
|
Key points:
|
||||||
|
- `join_transaction_mode="create_savepoint"` means each `session.commit()` inside code under test issues a SAVEPOINT release, not a real COMMIT — everything is rolled back when the test ends.
|
||||||
|
- `scope="session"` engine + `scope="function"` session/connection gives fast table creation with full per-test isolation.
|
||||||
|
- SQLite in-memory (`sqlite://`) is preferred: no files, no cleanup, fast.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Async SQLAlchemy + FastAPI
|
||||||
|
|
||||||
|
For `AsyncSession` / `AsyncEngine`, the setup mirrors the sync version but uses async fixtures and `pytest-anyio`.
|
||||||
|
|
||||||
|
```python
|
||||||
|
# tests/conftest.py
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine, async_sessionmaker
|
||||||
|
from fastapi.testclient import TestClient # sync client still works
|
||||||
|
from httpx import AsyncClient, ASGITransport
|
||||||
|
|
||||||
|
from myapp.main import app
|
||||||
|
from myapp.db.session import get_db
|
||||||
|
from myapp.db.models import Base
|
||||||
|
|
||||||
|
@pytest.fixture(scope="session")
|
||||||
|
def anyio_backend():
|
||||||
|
return "asyncio"
|
||||||
|
|
||||||
|
@pytest.fixture(scope="session")
|
||||||
|
async def async_engine():
|
||||||
|
e = create_async_engine("sqlite+aiosqlite://", connect_args={"check_same_thread": False})
|
||||||
|
async with e.begin() as conn:
|
||||||
|
await conn.run_sync(Base.metadata.create_all)
|
||||||
|
yield e
|
||||||
|
async with e.begin() as conn:
|
||||||
|
await conn.run_sync(Base.metadata.drop_all)
|
||||||
|
await e.dispose()
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
async def async_session(async_engine):
|
||||||
|
async with async_engine.connect() as conn:
|
||||||
|
await conn.begin()
|
||||||
|
session = AsyncSession(bind=conn, join_transaction_mode="create_savepoint",
|
||||||
|
expire_on_commit=False)
|
||||||
|
yield session
|
||||||
|
await session.close()
|
||||||
|
await conn.rollback()
|
||||||
|
|
||||||
|
@pytest.fixture()
|
||||||
|
async def async_client(async_session):
|
||||||
|
async def override_get_db():
|
||||||
|
yield async_session
|
||||||
|
|
||||||
|
app.dependency_overrides[get_db] = override_get_db
|
||||||
|
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as ac:
|
||||||
|
yield ac
|
||||||
|
app.dependency_overrides = {}
|
||||||
|
```
|
||||||
|
|
||||||
|
Notes:
|
||||||
|
- Async fixtures need `@pytest.mark.anyio` on the test or `anyio_backend` session fixture.
|
||||||
|
- `expire_on_commit=False` prevents expired-attribute access on objects after `await session.commit()`.
|
||||||
|
- Install: `aiosqlite`, `anyio[asyncio]`, `httpx`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## SQLModel Pattern (FastAPI + SQLModel)
|
||||||
|
|
||||||
|
SQLModel's official test pattern uses `StaticPool` + in-memory SQLite, with separate `session` and `client` pytest fixtures.
|
||||||
|
|
||||||
|
```python
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
from sqlmodel import Session, SQLModel, create_engine
|
||||||
|
from sqlmodel.pool import StaticPool
|
||||||
|
|
||||||
|
from myapp.main import app, get_session # get_session is the SQLModel dependency
|
||||||
|
|
||||||
|
@pytest.fixture(name="session")
|
||||||
|
def session_fixture():
|
||||||
|
engine = create_engine(
|
||||||
|
"sqlite://",
|
||||||
|
connect_args={"check_same_thread": False},
|
||||||
|
poolclass=StaticPool,
|
||||||
|
)
|
||||||
|
SQLModel.metadata.create_all(engine)
|
||||||
|
with Session(engine) as session:
|
||||||
|
yield session
|
||||||
|
|
||||||
|
@pytest.fixture(name="client")
|
||||||
|
def client_fixture(session: Session):
|
||||||
|
app.dependency_overrides[get_session] = lambda: session
|
||||||
|
yield TestClient(app)
|
||||||
|
app.dependency_overrides.clear()
|
||||||
|
|
||||||
|
# --- Tests ---
|
||||||
|
|
||||||
|
def test_create_hero(client: TestClient):
|
||||||
|
response = client.post("/heroes/", json={"name": "Deadpond", "secret_name": "Dive Wilson"})
|
||||||
|
assert response.status_code == 200
|
||||||
|
|
||||||
|
def test_read_heroes(session: Session, client: TestClient):
|
||||||
|
# Directly insert test data via the session — no HTTP call needed for setup
|
||||||
|
from myapp.models import Hero
|
||||||
|
session.add(Hero(name="Test Hero", secret_name="Hidden"))
|
||||||
|
session.commit()
|
||||||
|
response = client.get("/heroes/")
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert len(response.json()) == 1
|
||||||
|
```
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- Use `StaticPool` so a single in-memory SQLite connection is shared across threads (required by `TestClient`'s threading model).
|
||||||
|
- Both the `client` fixture and test functions can receive the same `session` — insert data directly for controlled setup rather than via the API.
|
||||||
|
- Always call `app.dependency_overrides.clear()` in fixture teardown (after `yield`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Async Session: Avoiding Implicit I/O
|
||||||
|
|
||||||
|
SQLAlchemy async has strict rules about lazy loading — attributes that would trigger IO on access will raise an error.
|
||||||
|
|
||||||
|
```python
|
||||||
|
# WRONG — will raise MissingGreenlet / lazy load error
|
||||||
|
result = await session.execute(select(User))
|
||||||
|
user = result.scalars().one()
|
||||||
|
print(user.posts) # lazy load, fails in async context
|
||||||
|
|
||||||
|
# RIGHT — use selectinload() to load relationships eagerly
|
||||||
|
from sqlalchemy.orm import selectinload
|
||||||
|
|
||||||
|
result = await session.execute(
|
||||||
|
select(User).options(selectinload(User.posts))
|
||||||
|
)
|
||||||
|
user = result.scalars().one()
|
||||||
|
print(user.posts) # already loaded, no IO needed
|
||||||
|
```
|
||||||
|
|
||||||
|
Other strategies:
|
||||||
|
- `AsyncAttrs` mixin: access any attribute as an awaitable via `await obj.awaitable_attrs.relationship_name`.
|
||||||
|
- `write_only` relationships: never loaded implicitly; queried explicitly.
|
||||||
|
- `await session.refresh(obj, ["attribute_name"])`: force-load a specific attribute after the fact.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Fixture Scope Decision Table
|
||||||
|
|
||||||
|
| What to scope | `scope` | Reason |
|
||||||
|
|---|---|---|
|
||||||
|
| Engine + DDL (`create_all`) | `session` | Expensive; shared across all tests |
|
||||||
|
| Connection + Transaction | `function` | Rolled back per test for isolation |
|
||||||
|
| Session | `function` | One transaction per test |
|
||||||
|
| TestClient / AsyncClient | `function` | Depends on session; recreated per test |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Pitfalls
|
||||||
|
|
||||||
|
| Problem | Cause | Fix |
|
||||||
|
|---|---|---|
|
||||||
|
| `MissingGreenlet` in async | Lazy-loaded relationship accessed outside awaitable context | Use `selectinload()` or `AsyncAttrs.awaitable_attrs` |
|
||||||
|
| `RuntimeError: Event loop is closed` | `AsyncEngine` not disposed | Call `await engine.dispose()` in fixture teardown |
|
||||||
|
| Tests share state / data bleeds | Session not rolled back | Use `join_transaction_mode="create_savepoint"` + rollback pattern |
|
||||||
|
| `StaticPool` not used with SQLite in-memory | TestClient spawns threads that get separate in-memory DBs | Always add `poolclass=StaticPool` for in-memory SQLite |
|
||||||
|
| `expire_on_commit=True` (default) breaks async | Accessing attributes after commit triggers lazy IO | Set `expire_on_commit=False` on AsyncSession |
|
||||||
|
| Not resetting `dependency_overrides` | Override persists into next test | Always clear in fixture teardown, after `yield` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Official Docs
|
||||||
|
|
||||||
|
- [Joining a Session into an External Transaction (test suites)](https://docs.sqlalchemy.org/en/20/orm/session_transaction.html#joining-a-session-into-an-external-transaction-such-as-for-test-suites)
|
||||||
|
- [Asynchronous I/O (asyncio)](https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html)
|
||||||
|
- [Preventing Implicit IO when Using AsyncSession](https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html#preventing-implicit-io-when-using-asyncsession)
|
||||||
|
- [SQLModel: Test Applications with FastAPI](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/)
|
||||||
|
- [FastAPI: Testing Dependencies with Overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user