improving async fastapi sqlmodel skill
This commit is contained in:
+1
-1
@@ -125,7 +125,7 @@ Use the personal-mcp catalog tools to search for the most relevant skill for Fas
|
||||
Example direct-load prompt:
|
||||
|
||||
```text
|
||||
Call get_skill_document_by_id for fastapi-async-sqlalchemy-modernization and use that document as the main context for this task.
|
||||
Call get_skill_document_by_id for async-fastapi-sqlmodel and use that document as the main context for this task.
|
||||
```
|
||||
|
||||
Example bounded-selection prompt:
|
||||
|
||||
@@ -1,72 +1,116 @@
|
||||
---
|
||||
name: async-fastapi-sqlmodel
|
||||
description: 'Create a step-by-step modernization plan for an existing FastAPI app using SQLAlchemy async patterns, context managers, and AsyncExitStack. Use when: planning migration from legacy DB setup, standardizing async engine/session lifecycles, defining transaction boundaries, and aligning with SQLAlchemy 2.x best practices.'
|
||||
description: 'Explain and apply async database principles for FastAPI, SQLAlchemy 2.x, and SQLModel. Use when: learning or reviewing AsyncEngine and AsyncSession lifecycles, FastAPI lifespan and yield dependencies, transaction boundaries, concurrency safety, implicit ORM I/O, AsyncExitStack, pooling, testing, or SQLModel integration.'
|
||||
x-personal-mcp:
|
||||
id: async-fastapi-sqlmodel
|
||||
version: 1.0.0
|
||||
tags:
|
||||
- fastapi
|
||||
- sqlalchemy
|
||||
- async
|
||||
- asyncio
|
||||
- modernization
|
||||
capabilities:
|
||||
- resource://skills/async-fastapi-sqlmodel/document
|
||||
id: async-fastapi-sqlmodel
|
||||
version: 1.1.0
|
||||
tags:
|
||||
- fastapi
|
||||
- sqlalchemy
|
||||
- sqlmodel
|
||||
- async
|
||||
- asyncio
|
||||
- database
|
||||
- transactions
|
||||
- resource-lifecycle
|
||||
- architecture
|
||||
capabilities:
|
||||
- resource://skills/async-fastapi-sqlmodel/document
|
||||
---
|
||||
|
||||
# FastAPI Async SQLAlchemy Modernization Plan
|
||||
# Async FastAPI, SQLAlchemy, and SQLModel
|
||||
|
||||
Create an implementation-ready plan that brings an existing FastAPI application in line with modern async SQLAlchemy practices, with explicit resource lifecycles and deterministic cleanup using async context managers and AsyncExitStack.
|
||||
Use this skill to explain how an async database layer works, why the recommended patterns exist, and how to evaluate code against them. Teach the runtime model before suggesting implementation changes.
|
||||
|
||||
Primary targets: PostgreSQL with asyncpg and SQLite with aiosqlite.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Existing FastAPI app has ad hoc database setup or mixed sync/async access.
|
||||
- Session management is inconsistent across routes/services.
|
||||
- Lifespan startup and shutdown work is spread across globals and side effects.
|
||||
- Team needs a migration plan first, not immediate large-scale rewrites.
|
||||
- Explain an async engine, session factory, session, connection, or transaction.
|
||||
- Review FastAPI lifespan or dependency-based database management.
|
||||
- Diagnose shared-session concurrency, implicit I/O, cleanup, or transaction problems.
|
||||
- Compare SQLModel's model conveniences with SQLAlchemy's async runtime APIs.
|
||||
- Decide whether a context manager, `AsyncExitStack`, eager loading, pooling option, or explicit transaction is appropriate.
|
||||
|
||||
## Outcome
|
||||
|
||||
Produce a practical modernization plan with:
|
||||
Produce a focused technical explanation that:
|
||||
|
||||
- Current-state gap assessment.
|
||||
- Target architecture for engine/session/transaction lifecycle.
|
||||
- Branch-based migration path (low-risk staged rollout).
|
||||
- Quality gates and completion checks.
|
||||
- Risks, rollback strategy, and test plan.
|
||||
- Defines the objects involved and identifies who owns each one.
|
||||
- Traces acquisition, use, transaction behavior, and cleanup.
|
||||
- Separates required invariants from defaults and situational choices.
|
||||
- Explains failure modes and concurrency consequences.
|
||||
- Uses a minimal canonical pattern when code clarifies the mechanics.
|
||||
- Links claims to the relevant reference and upstream documentation.
|
||||
|
||||
## Top-Level Concepts
|
||||
Do not default to producing a project plan. Give sequencing advice only when the user explicitly asks for implementation steps.
|
||||
|
||||
Use these concepts as the planning backbone:
|
||||
## Mental Model
|
||||
|
||||
1. Engine lifecycle and ownership:
|
||||
One AsyncEngine per process for each DB URL, created once and disposed explicitly when the app lifecycle ends.
|
||||
See the [engine lifecycle reference](references/engine.md).
|
||||
2. Session factory and scope:
|
||||
Use async_sessionmaker for configuration; create one AsyncSession per request or unit-of-work, never shared across concurrent tasks.
|
||||
See the [session management reference](references/session.md).
|
||||
3. Transaction boundaries:
|
||||
Prefer context-managed begin blocks for write units and explicit read-only sessions for queries.
|
||||
See the [transaction boundaries reference](references/transactions.md).
|
||||
4. Lifespan composition:
|
||||
Compose startup/shutdown resources with AsyncExitStack so cleanup is deterministic and ordered.
|
||||
See the [engine lifecycle reference](references/engine.md).
|
||||
5. Dependency injection:
|
||||
Provide sessions via FastAPI dependencies with async generators/context managers, not globals.
|
||||
See the [session management reference](references/session.md).
|
||||
6. Implicit I/O control in ORM:
|
||||
Avoid accidental lazy loads; use explicit eager-loading/refresh strategies for asyncio safety.
|
||||
See the [implicit I/O reference](references/implicit_io.md).
|
||||
7. Observability and resilience:
|
||||
Add pool/connection settings, logging, timeout, and health checks as first-class plan items.
|
||||
See the [observability reference](references/observability.md).
|
||||
8. SQLModel adoption where appropriate:
|
||||
Prefer SQLModel for typed ORM models and API-facing data models when it reduces duplication, while preserving SQLAlchemy async lifecycle patterns.
|
||||
See the [SQLModel integration reference](references/sqlmodel.md).
|
||||
Keep three ownership scopes distinct:
|
||||
|
||||
### Concept Reference Map
|
||||
| Scope | Object | Purpose | Typical owner |
|
||||
|---|---|---|---|
|
||||
| Application process | `AsyncEngine` and `async_sessionmaker` | Dialect, connection pool, and repeatable session configuration | FastAPI lifespan |
|
||||
| Request or concurrent task | `AsyncSession` | Mutable ORM identity map and transactional state | A `yield` dependency or explicit unit of work |
|
||||
| Atomic operation | `SessionTransaction` | Commit all changes together or roll them back together | Service or use-case boundary |
|
||||
|
||||
The engine is a long-lived factory and pool, not a single database connection. The session is a mutable unit-of-work object, not a concurrency-safe global. A transaction is a consistency boundary, not merely a call to `commit()`.
|
||||
|
||||
## Core Principles
|
||||
|
||||
### Match lifetime to ownership
|
||||
|
||||
- Create one `AsyncEngine` per process and database configuration in the normal case.
|
||||
- Dispose it explicitly in an awaitable shutdown path; garbage collection cannot reliably await async driver cleanup.
|
||||
- Configure `async_sessionmaker` once and call it to create short-lived sessions.
|
||||
- Close each session deterministically with `async with` or a FastAPI dependency that yields once.
|
||||
|
||||
See [engine lifecycle](references/engine.md) and [session management](references/session.md).
|
||||
|
||||
### Isolate mutable session state
|
||||
|
||||
An `AsyncSession` represents one stateful transaction in progress. Never use one session in multiple concurrent tasks, including branches of `asyncio.gather()`. Give each task its own session and pass sessions explicitly rather than relying on mutable scoped globals.
|
||||
|
||||
See [session management](references/session.md).
|
||||
|
||||
### Make I/O visible
|
||||
|
||||
Async ORM code must not unexpectedly issue SQL during ordinary attribute access. Load relationships and deferred columns explicitly with eager loader options such as `selectinload()`, use `awaitable_attrs` or `refresh()` for deliberate fallback loading, and consider `lazy="raise"` where accidental access should fail fast. `expire_on_commit=False` is a common async configuration because post-commit expiration can otherwise turn attribute reads into implicit I/O.
|
||||
|
||||
See [implicit ORM I/O](references/implicit_io.md).
|
||||
|
||||
### Put transactions around business invariants
|
||||
|
||||
Use `async with session.begin():` when several operations must commit or roll back as one unit. A successful exit flushes and commits; an exception rolls back. Reads still participate in SQLAlchemy's autobegin behavior unless the connection uses true DBAPI autocommit, so describe a path as read-only because of application intent and permissions, not because a session silently has no transaction.
|
||||
|
||||
Use `begin_nested()` only for a real SAVEPOINT requirement and account for backend-specific behavior. In SQLAlchemy 2.x, calling `session.commit()` commits the outermost transaction, not the current savepoint.
|
||||
|
||||
See [transaction boundaries](references/transactions.md).
|
||||
|
||||
### Keep framework boundaries explicit
|
||||
|
||||
FastAPI lifespan owns resources shared by many requests. A dependency with one `yield` owns request-scoped resources and runs cleanup after use. These are related context-manager mechanisms but solve different lifetime problems.
|
||||
|
||||
Use `AsyncExitStack` when lifespan acquires a variable, conditional, or mixed collection of context-managed resources. It records cleanup as resources are acquired and unwinds callbacks in reverse order. A single engine with one cleanup callback can use a plain `try/finally`; `AsyncExitStack` is a composition tool, not a requirement.
|
||||
|
||||
See [engine lifecycle](references/engine.md).
|
||||
|
||||
### Use SQLModel as the primary modeling layer
|
||||
|
||||
Default to SQLModel for table models and API data models in FastAPI applications. A SQLModel table model is also a SQLAlchemy model, and every SQLModel model is also a Pydantic model, so shared base models can reduce schema duplication while preserving access to SQLAlchemy's full ORM.
|
||||
|
||||
SQLModel does not replace SQLAlchemy's async engine, session, transaction, or loader mechanics. Its main tutorial currently demonstrates synchronous sessions and its advanced guide still lists comprehensive async documentation as future work. For async applications, combine SQLModel models and statements with SQLAlchemy's `AsyncSession` APIs. Use SQLAlchemy declarative models only when a concrete unsupported mapping or library constraint justifies the exception.
|
||||
|
||||
See [SQLModel integration](references/sqlmodel.md).
|
||||
|
||||
### Configure from evidence
|
||||
|
||||
Pool sizing, overflow, recycle, pre-ping, isolation, statement timeouts, and health checks depend on the driver, database, deployment concurrency, and failure model. Explain defaults and tradeoffs before recommending values. Avoid treating pool checkout as proof that a useful query can succeed.
|
||||
|
||||
See [observability and resilience](references/observability.md).
|
||||
|
||||
## Reference Map
|
||||
|
||||
| Concept | Reference |
|
||||
|---|---|
|
||||
@@ -77,171 +121,57 @@ Use these concepts as the planning backbone:
|
||||
| Dependency injection | [Session management reference](references/session.md) |
|
||||
| Implicit I/O control in ORM | [Implicit I/O reference](references/implicit_io.md) |
|
||||
| Observability and resilience | [Observability reference](references/observability.md) |
|
||||
| SQLModel adoption where appropriate | [SQLModel integration reference](references/sqlmodel.md) |
|
||||
| SQLModel-first modeling | [SQLModel integration reference](references/sqlmodel.md) |
|
||||
|
||||
## Decision Points
|
||||
## Canonical Composition Pattern
|
||||
|
||||
Use these branching decisions before proposing migration steps.
|
||||
|
||||
| Decision | Branch A | Branch B |
|
||||
|---|---|---|
|
||||
| DB driver | Already async driver (e.g. asyncpg, aiosqlite): modernize in place | Sync driver: plan driver migration first |
|
||||
| ORM usage | Already ORM 2.x style (`select`, `session.execute`) | Legacy Query API: add compatibility stage and refactor incrementally |
|
||||
| Session scope | Request-scoped already | Global/shared sessions found: prioritize session-scope fix first |
|
||||
| Lifespan | Existing FastAPI lifespan hook | No lifespan hook: introduce lifespan before broader DB changes |
|
||||
| Model layer | Existing SQLModel models fit roadmap | SQLAlchemy-only models: evaluate SQLModel adoption by bounded module |
|
||||
| Concurrency | Background jobs/tasks use DB | No background DB use |
|
||||
| Transaction style | Explicit context-managed transactions | Implicit/autobegin side effects |
|
||||
|
||||
## Procedure
|
||||
|
||||
### Step 0: Audit Current State
|
||||
|
||||
Inventory the app and write a concise gap list.
|
||||
|
||||
- Engine creation location(s) and count.
|
||||
- Driver URL(s) and async compatibility.
|
||||
- Session creation patterns in routes/services/background tasks.
|
||||
- Transaction handling style (explicit begin/commit/rollback vs implicit).
|
||||
- Lifespan startup/shutdown and cleanup behavior.
|
||||
- ORM loading patterns that may trigger implicit I/O.
|
||||
|
||||
Completion check: every DB touchpoint is mapped to its engine, session, and transaction source.
|
||||
|
||||
### Step 1: Define the Target Runtime Model
|
||||
|
||||
Define one canonical model to migrate toward.
|
||||
|
||||
- Create AsyncEngine once per process.
|
||||
- Configure async_sessionmaker once.
|
||||
- Use per-request AsyncSession dependency.
|
||||
- Keep one AsyncSession per concurrent task.
|
||||
- Use context-managed transactions for writes.
|
||||
|
||||
Completion check: architecture diagram can explain where engine/session are created, used, and closed.
|
||||
|
||||
### Step 1.5: Decide SQLModel Adoption Scope
|
||||
|
||||
Decide where SQLModel should be introduced during modernization.
|
||||
|
||||
- Prefer SQLModel when it reduces duplicated schema definitions between ORM entities and API data models.
|
||||
- Keep SQLAlchemy async engine/session lifecycle as the runtime foundation.
|
||||
- Use bounded adoption first (one module or feature area), then expand after validation.
|
||||
- If project is already heavily SQLAlchemy-only and stable, document rationale for staying SQLAlchemy-only.
|
||||
|
||||
Completion check: plan includes an explicit SQLModel branch with target modules and non-goals.
|
||||
|
||||
### Step 2: Plan Engine Modernization
|
||||
|
||||
Plan engine creation and pool behavior.
|
||||
|
||||
- Use `create_async_engine()` with async dialect URL.
|
||||
- Standardize pool settings and pre-ping strategy where relevant.
|
||||
- Decide isolation level strategy at engine level (avoid ad hoc per-operation switching unless justified).
|
||||
- Define explicit disposal policy for short-lived scopes and tests.
|
||||
|
||||
Completion check: engine configuration is centralized and no per-request engine creation remains.
|
||||
|
||||
### Step 3: Plan Session Lifecycle Modernization
|
||||
|
||||
Define session factory and request dependency pattern.
|
||||
|
||||
- Build `async_sessionmaker(engine, expire_on_commit=False)` unless a strict reason says otherwise.
|
||||
- Provide session via dependency that yields exactly one AsyncSession.
|
||||
- Explicitly prohibit sharing a single AsyncSession across concurrent tasks.
|
||||
- Prefer direct dependency passing over async_scoped_session for new designs.
|
||||
|
||||
Completion check: all route/service entry points receive a session from one canonical dependency.
|
||||
|
||||
### Step 4: Plan Transaction Demarcation
|
||||
|
||||
Establish consistent write and read behavior.
|
||||
|
||||
- Writes: `async with session.begin(): ...` for atomic units.
|
||||
- Reads: execute in managed session context with explicit loader options.
|
||||
- Nested/SAVEPOINT use only where required; call out backend caveats.
|
||||
- Define rollback behavior for service-layer exceptions.
|
||||
|
||||
Completion check: every mutating use case has a declared transaction boundary.
|
||||
|
||||
### Step 5: Compose Lifespan with AsyncExitStack
|
||||
|
||||
Use async context composition as the preferred orchestration pattern.
|
||||
This example shows the ownership boundaries. Adapt state storage and dependency wiring to the application's conventions.
|
||||
|
||||
```python
|
||||
from contextlib import AsyncExitStack, asynccontextmanager
|
||||
from collections.abc import AsyncIterator
|
||||
|
||||
from fastapi import FastAPI
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
||||
async with AsyncExitStack() as stack:
|
||||
# Compose resources in acquisition order; cleanup is automatic in reverse order.
|
||||
engine = create_async_engine(settings.database_url)
|
||||
stack.push_async_callback(engine.dispose)
|
||||
|
||||
session_factory = async_sessionmaker(engine, expire_on_commit=False)
|
||||
app.state.session_factory = session_factory
|
||||
|
||||
# Add other async resources with stack.enter_async_context(...) as needed.
|
||||
yield
|
||||
|
||||
|
||||
async def get_session() -> AsyncIterator[AsyncSession]:
|
||||
async with app.state.session_factory() as session:
|
||||
yield session
|
||||
```
|
||||
|
||||
Planning rules:
|
||||
For direct construction without `AsyncExitStack`, put `await engine.dispose()` in a `finally` block. For background work that outlives a request, create a new session inside that task instead of retaining the request's session.
|
||||
|
||||
- Register every acquired resource with AsyncExitStack at acquisition time.
|
||||
- Prefer `enter_async_context()` for resources that already expose async context managers.
|
||||
- Prefer `push_async_callback()` for async cleanup callables.
|
||||
- Keep resource ownership in lifespan, not in route handlers.
|
||||
## Explanation Procedure
|
||||
|
||||
Completion check: startup/shutdown ordering is explicit and deterministic.
|
||||
1. Identify the exact concept or observed behavior in question.
|
||||
2. Name the owning scope: application, request/task, or transaction.
|
||||
3. Trace what state the object holds and where actual database I/O can occur.
|
||||
4. Explain normal entry, successful exit, exceptional exit, and concurrent use.
|
||||
5. Distinguish an invariant from a recommended default or backend-specific choice.
|
||||
6. Load only the matching reference documents and cite upstream sources.
|
||||
7. Show the smallest useful code pattern or contrast when prose is insufficient.
|
||||
8. End with concrete checks the reader can use to inspect their own code.
|
||||
|
||||
### Step 6: Prevent Implicit ORM I/O Under Asyncio (Advisory Mode)
|
||||
When reviewing code, verify:
|
||||
|
||||
Plan for explicit loading behavior, but treat this as progressive guidance rather than a hard gate.
|
||||
|
||||
- Recommend eager-loading strategies (for example selectin-style loading) where relationship access is required.
|
||||
- For lazy/deferred attributes, define explicit awaitable or refresh paths on high-risk and high-traffic paths first.
|
||||
- Document model-level defaults and known exceptions so teams can migrate incrementally.
|
||||
|
||||
Completion check: critical request paths have explicit loading plans; non-critical paths have tracked follow-up items.
|
||||
|
||||
### Step 7: Testing and Verification Plan
|
||||
|
||||
Create modernization quality gates.
|
||||
|
||||
- Unit tests for session dependency and transaction behavior.
|
||||
- Integration tests for commit/rollback semantics.
|
||||
- Concurrency tests confirming one-session-per-task behavior.
|
||||
- Lifespan tests verifying cleanup calls and ordering.
|
||||
- Health/readiness tests including DB connectivity checks.
|
||||
- If SQLModel is adopted, model-validation tests cover SQLModel table models and API models at module boundaries.
|
||||
|
||||
Completion check: all quality gates pass under the target async configuration.
|
||||
|
||||
### Step 8: Rollout Strategy
|
||||
|
||||
Plan low-risk migration phases.
|
||||
|
||||
1. Introduce centralized engine/session factory and lifespan orchestration.
|
||||
2. Migrate read paths to new session dependency.
|
||||
3. Migrate write paths to explicit transaction blocks.
|
||||
4. Remove legacy globals/helpers and dead code.
|
||||
5. Enable stricter linting/review checks for forbidden patterns.
|
||||
|
||||
Completion check: no legacy session/engine creation path remains in production code.
|
||||
|
||||
## Quality Criteria
|
||||
|
||||
A plan is complete only when it includes:
|
||||
|
||||
- Clear current vs target architecture.
|
||||
- Branch decisions with rationale.
|
||||
- Explicit context-manager patterns for resource ownership.
|
||||
- AsyncExitStack composition strategy.
|
||||
- Transaction policy and exception behavior.
|
||||
- SQLModel adoption branch (use/adopt/defer) with rationale.
|
||||
- Concrete tests and rollout checkpoints.
|
||||
- A documented advisory backlog for non-critical implicit I/O improvements.
|
||||
- The URL uses an asyncio-compatible dialect.
|
||||
- Engine creation and disposal have one clear owner.
|
||||
- Every session has a bounded lifetime and is not shared across tasks.
|
||||
- Transaction boundaries match business invariants and exception behavior.
|
||||
- Relationship and deferred-column access cannot surprise the event loop with implicit I/O.
|
||||
- Pool and timeout settings are justified by deployment behavior.
|
||||
- Tests exercise rollback, cleanup, concurrency, and lifespan behavior where relevant.
|
||||
|
||||
## Anti-Patterns to Flag
|
||||
|
||||
@@ -250,21 +180,27 @@ A plan is complete only when it includes:
|
||||
- Implicit commit/rollback behavior with unclear ownership.
|
||||
- Global mutable session state.
|
||||
- Lifespan cleanup that depends on implicit garbage collection.
|
||||
- Forcing SQLModel rewrites across the entire codebase in one phase without module-level rollout.
|
||||
- Treating `AsyncExitStack` as mandatory for a fixed single resource.
|
||||
- Treating SQLModel's synchronous tutorial examples as the async runtime pattern.
|
||||
- Allowing lazy relationship access to hide database I/O.
|
||||
- Copying pool settings without relating them to worker count and database capacity.
|
||||
|
||||
## Output Contract
|
||||
|
||||
Return the plan as:
|
||||
Answer in the shape best suited to the question, usually:
|
||||
|
||||
1. Current-state gap summary.
|
||||
2. Target architecture summary.
|
||||
3. Phased migration checklist with branch notes.
|
||||
4. Risk register and rollback approach.
|
||||
5. Verification matrix (tests + operational checks).
|
||||
1. Direct explanation.
|
||||
2. Underlying lifecycle or transaction mechanics.
|
||||
3. Required invariants and situational tradeoffs.
|
||||
4. Minimal example or code-review findings when useful.
|
||||
5. Verification questions and source links.
|
||||
|
||||
## References
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [SQLAlchemy engine and connections](https://docs.sqlalchemy.org/en/21/core/connections.html)
|
||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||
- [Python async context managers and AsyncExitStack](https://docs.python.org/3/library/contextlib.html)
|
||||
- [SQLAlchemy transaction management](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html)
|
||||
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
||||
- [FastAPI dependencies with `yield`](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
|
||||
- [Python `AsyncExitStack`](https://docs.python.org/3/library/contextlib.html#contextlib.AsyncExitStack)
|
||||
- [SQLModel session dependency pattern](https://sqlmodel.tiangolo.com/tutorial/fastapi/session-with-dependency/)
|
||||
|
||||
@@ -66,13 +66,13 @@ roles = await user.awaitable_attrs.roles
|
||||
|
||||
## Practical Enforcement Model
|
||||
|
||||
Use phased enforcement:
|
||||
Require explicit I/O behavior on every async ORM path:
|
||||
|
||||
1. High-traffic and latency-sensitive routes: enforce explicit eager loading.
|
||||
2. Background tasks and less critical paths: track and progressively tighten.
|
||||
3. Add review checks to prevent newly introduced implicit-load hotspots.
|
||||
1. Define loader options for relationships and deferred columns needed by the operation.
|
||||
2. Use `refresh()` or awaitable attributes only when the additional query is deliberate and visible.
|
||||
3. Add review checks that reject unplanned lazy-load paths.
|
||||
|
||||
This keeps modernization pragmatic while reducing hidden I/O over time.
|
||||
This keeps event-loop behavior predictable and makes query boundaries reviewable from the code.
|
||||
|
||||
---
|
||||
|
||||
@@ -99,9 +99,3 @@ This keeps modernization pragmatic while reducing hidden I/O over time.
|
||||
- Tests verify expected data is present without hidden secondary query surprises.
|
||||
- Regression tests exist for routes previously affected by implicit-load failures.
|
||||
|
||||
---
|
||||
|
||||
## Migration Notes
|
||||
|
||||
- Start advisory: target high-risk paths first.
|
||||
- As coverage improves, elevate selected rules to mandatory in code review policy.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# FastAPI Async SQLAlchemy References Index
|
||||
|
||||
Purpose: concept registry for modernization guidance used by this skill.
|
||||
Purpose: concept registry for the principles, mechanics, and implementation guidance used by this skill.
|
||||
|
||||
---
|
||||
|
||||
@@ -13,13 +13,13 @@ Purpose: concept registry for modernization guidance used by this skill.
|
||||
| Transaction boundaries | [transactions.md](transactions.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
||||
| Implicit ORM I/O under asyncio | [implicit_io.md](implicit_io.md) | adopted | advisory | platform/backend | 2026-06-17 |
|
||||
| Observability and resilience | [observability.md](observability.md) | adopted | mandatory | platform/backend | 2026-06-17 |
|
||||
| SQLModel adoption and boundaries | [sqlmodel.md](sqlmodel.md) | adopted | advisory | platform/backend | 2026-06-26 |
|
||||
| SQLModel modeling and async boundaries | [sqlmodel.md](sqlmodel.md) | adopted | mandatory | platform/backend | 2026-07-26 |
|
||||
|
||||
---
|
||||
|
||||
## How to Use This Folder
|
||||
|
||||
- `SKILL.md` defines the planning workflow and migration procedure.
|
||||
- `SKILL.md` defines the explanatory workflow and shared mental model.
|
||||
- Each concept doc defines policy-level guidance for one concern.
|
||||
- Use the template in [template.md](template.md) for new concept docs.
|
||||
- Keep references source-linked and implementation snippets minimal.
|
||||
@@ -30,4 +30,4 @@ Purpose: concept registry for modernization guidance used by this skill.
|
||||
|
||||
- If a PR changes database lifecycle/session/ORM loading behavior, update the relevant concept file.
|
||||
- Keep `Status`, `Decision Level`, and `Last Reviewed` current.
|
||||
- Use `advisory` only when incremental rollout is intended; use `mandatory` for required runtime policy.
|
||||
- Use `advisory` for recommendations that depend on application context; use `mandatory` for required runtime policy.
|
||||
@@ -105,10 +105,3 @@ Readiness checks should be lightweight and bounded (timeouts), not heavy diagnos
|
||||
- Readiness endpoint test covers healthy and unhealthy DB states.
|
||||
- Integration test simulates disconnect/reconnect behavior.
|
||||
- Load/concurrency tests validate pool behavior under stress.
|
||||
|
||||
---
|
||||
|
||||
## Migration Notes
|
||||
|
||||
- Start with resilient defaults (`pool_pre_ping`) and simple health policy.
|
||||
- Add deeper metrics/event hooks incrementally once baseline reliability is in place.
|
||||
@@ -100,9 +100,9 @@ Notes:
|
||||
|
||||
## SQLModel Alignment
|
||||
|
||||
- If using SQLModel, keep the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per request/unit-of-work.
|
||||
- SQLModel does not replace SQLAlchemy async lifecycle primitives; it complements model declaration and typed data handling.
|
||||
- During migration, avoid mixed ad hoc patterns where some handlers create SQLAlchemy sessions directly while others use SQLModel-specific wrappers.
|
||||
- Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per request/unit-of-work.
|
||||
- SQLModel does not replace SQLAlchemy async lifecycle primitives; it provides model declaration, validation, and typing ergonomics on top of them.
|
||||
- Do not mix ad hoc session construction with the canonical async dependency.
|
||||
|
||||
---
|
||||
|
||||
@@ -139,9 +139,3 @@ Notes:
|
||||
- Parallel-task tests verify no shared AsyncSession instances.
|
||||
- Lifespan tests confirm session factory is initialized and teardown-safe.
|
||||
|
||||
---
|
||||
|
||||
## Migration Notes
|
||||
|
||||
- If current code uses global/shared sessions, fix scope first before refactoring query style.
|
||||
- If legacy sync patterns are present, keep session boundary rules stable while migrating incrementally.
|
||||
|
||||
@@ -1,53 +1,50 @@
|
||||
# SQLModel Adoption and Boundaries
|
||||
# SQLModel-First Modeling and Async Boundaries
|
||||
|
||||
!!! info "Primary sources"
|
||||
- [SQLModel documentation](https://sqlmodel.tiangolo.com/)
|
||||
- [SQLModel features](https://sqlmodel.tiangolo.com/features/)
|
||||
- [SQLModel advanced guide](https://sqlmodel.tiangolo.com/advanced/)
|
||||
- [SQLModel FastAPI session dependency tutorial](https://sqlmodel.tiangolo.com/tutorial/fastapi/session-with-dependency/)
|
||||
- [SQLModel release notes](https://sqlmodel.tiangolo.com/release-notes/)
|
||||
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
||||
|
||||
??? abstract "Decision metadata"
|
||||
- Status: adopted
|
||||
- Decision level: advisory
|
||||
- Decision level: mandatory
|
||||
- Applies to: api-runtime, workers, tests
|
||||
- Last reviewed: 2026-06-26
|
||||
- Last reviewed: 2026-07-26
|
||||
|
||||
---
|
||||
|
||||
## Purpose
|
||||
|
||||
Define when and how to use SQLModel in an async FastAPI + SQLAlchemy modernization effort.
|
||||
Define SQLModel as the primary model layer for async FastAPI applications and explain how it composes with SQLAlchemy's async runtime.
|
||||
|
||||
The goal is pragmatic adoption: use SQLModel where it reduces model duplication and improves typing ergonomics, without disrupting established async engine/session lifecycle rules.
|
||||
SQLModel is designed for FastAPI, built on Pydantic and SQLAlchemy, and intended to minimize duplication while preserving the capabilities of both. Async engine, session, transaction, and loading behavior still follow SQLAlchemy's asyncio contract.
|
||||
|
||||
---
|
||||
|
||||
## Scope and Non-Goals
|
||||
|
||||
- In scope: model-layer decisions, integration boundaries, phased adoption strategy.
|
||||
- Out of scope: full framework rewrites and all-at-once model migration.
|
||||
- In scope: table models, API data models, SQLAlchemy interoperability, async session usage, and exception criteria.
|
||||
- Out of scope: replacing SQLAlchemy's async runtime primitives or claiming that synchronous tutorial examples are async patterns.
|
||||
|
||||
---
|
||||
|
||||
## Rules
|
||||
|
||||
- Default to SQLModel for new table models and API data models.
|
||||
- Keep SQLAlchemy async primitives as the runtime base: `create_async_engine`, `async_sessionmaker`, and `AsyncSession`.
|
||||
- Prefer SQLModel for new domain modules where table models and API schemas would otherwise be duplicated.
|
||||
- Migrate by bounded module or feature area; do not force whole-repo conversion in one phase.
|
||||
- Keep transaction and session ownership policies identical whether models are SQLAlchemy Declarative or SQLModel.
|
||||
- Document explicit reasons when SQLModel is deferred for a module.
|
||||
- Use SQLModel inheritance to share validated fields while keeping table, create, update, and public contracts distinct where their semantics differ.
|
||||
- Use SQLAlchemy declarative models only for a concrete unsupported mapping or third-party constraint; document the reason.
|
||||
- Use SQLAlchemy relationship loading options explicitly on async paths.
|
||||
|
||||
---
|
||||
|
||||
## Recommended Patterns
|
||||
|
||||
### Pattern A: Bounded module adoption
|
||||
|
||||
- Choose one feature slice (for example, billing, projects, or auth profile data).
|
||||
- Introduce SQLModel models for that slice only.
|
||||
- Keep unchanged modules on existing SQLAlchemy models until a dedicated migration phase.
|
||||
|
||||
### Pattern B: Data model split for API boundaries
|
||||
### Pattern A: Data model split for API boundaries
|
||||
|
||||
Use distinct models for persistence and external contracts.
|
||||
|
||||
@@ -72,20 +69,29 @@ class UserRead(UserBase):
|
||||
id: int
|
||||
```
|
||||
|
||||
### Pattern C: Keep async lifecycle unchanged
|
||||
### Pattern B: Keep SQLModel models with the async runtime
|
||||
|
||||
```python
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
|
||||
from sqlmodel import select
|
||||
|
||||
engine = create_async_engine(settings.database_url, pool_pre_ping=True)
|
||||
session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
|
||||
|
||||
async with session_factory() as session:
|
||||
users = (await session.scalars(select(User))).all()
|
||||
```
|
||||
|
||||
`sqlmodel.select()` keeps SQLModel's typing-oriented statement construction, while `AsyncSession.scalars()` and the surrounding lifecycle come from SQLAlchemy.
|
||||
|
||||
---
|
||||
|
||||
## Interoperability Notes
|
||||
|
||||
- SQLModel is designed as a thin layer over SQLAlchemy and Pydantic, so mixed codebases are expected during migration.
|
||||
- A SQLModel table model is a SQLAlchemy model and can participate in SQLAlchemy relationships, statements, loader options, and sessions.
|
||||
- A SQLModel model is also a Pydantic model; non-table models are useful for request and response contracts.
|
||||
- SQLModel's official FastAPI dependency tutorial currently uses synchronous `Session`; translate the ownership pattern, not the concrete session type, for async applications.
|
||||
- SQLModel's advanced guide still lists dedicated async documentation as future work, so use SQLAlchemy's asyncio documentation as the authority for runtime mechanics.
|
||||
- Prefer one query style per module to reduce cognitive overhead.
|
||||
- Keep loader strategies explicit in async paths to avoid implicit I/O surprises.
|
||||
|
||||
@@ -93,31 +99,29 @@ session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_comm
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- Treating SQLModel adoption as equivalent to async-session modernization.
|
||||
- Rewriting all models at once without rollback checkpoints.
|
||||
- Introducing SQLModel in handlers while keeping old global/shared session patterns.
|
||||
- Treating SQLModel as an alternative to SQLAlchemy rather than a layer built on it.
|
||||
- Copying a synchronous `Session` example into an async request path.
|
||||
- Constructing sessions in handlers instead of using the application session factory.
|
||||
- Mixing multiple query/session idioms within the same module without clear conventions.
|
||||
|
||||
---
|
||||
|
||||
## Operational Checks
|
||||
|
||||
- Modernized module documents whether it is SQLModel-first or SQLAlchemy-only.
|
||||
- New model modules are SQLModel-first; exceptions state the unsupported need or constraint.
|
||||
- Session/transaction ownership remains consistent across both model styles.
|
||||
- New model modules use explicit API boundary models where needed.
|
||||
- Table, create, update, and public models share fields intentionally without exposing persistence-only data.
|
||||
|
||||
---
|
||||
|
||||
## Testing Checks
|
||||
|
||||
- Module-level tests verify CRUD semantics for adopted SQLModel models.
|
||||
- Module-level tests verify CRUD semantics for SQLModel models through `AsyncSession`.
|
||||
- API tests verify response/request model behavior for SQLModel-based endpoints.
|
||||
- Regression tests confirm unchanged modules continue to function during phased rollout.
|
||||
- Relationship tests verify async loader strategies do not depend on implicit I/O.
|
||||
|
||||
---
|
||||
|
||||
## Migration Notes
|
||||
## Version Checks
|
||||
|
||||
- Start with low-risk bounded domains.
|
||||
- Expand only after validation of session lifecycle, transaction behavior, and endpoint correctness.
|
||||
- Maintain a tracked backlog of deferred modules with rationale and planned phase.
|
||||
- Verify installed SQLModel, SQLAlchemy, and Pydantic versions together when using newly added typing or ORM features.
|
||||
@@ -57,8 +57,3 @@ Describe what this concept governs and why it exists.
|
||||
- Test 1
|
||||
- Test 2
|
||||
|
||||
---
|
||||
|
||||
## Migration Notes
|
||||
|
||||
- Staged rollout notes and compatibility caveats.
|
||||
|
||||
@@ -102,10 +102,3 @@ Use nested transactions only when partial failure semantics are explicitly requi
|
||||
- Failure path test verifies rollback behavior.
|
||||
- Tests cover concurrency-sensitive write flows.
|
||||
- Savepoint usage (if present) has dedicated behavior tests.
|
||||
|
||||
---
|
||||
|
||||
## Migration Notes
|
||||
|
||||
- First stabilize session scope, then normalize transaction ownership.
|
||||
- Replace ad hoc commit patterns incrementally with bounded write units.
|
||||
@@ -22,9 +22,6 @@ build-backend = "hatchling.build"
|
||||
[tool.hatch.build.targets.wheel]
|
||||
packages = ["src/personal_mcp"]
|
||||
|
||||
[tool.hatch.build.targets.wheel.force-include]
|
||||
docs = "personal_mcp/docs"
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"ipywidgets>=8.1.8",
|
||||
|
||||
@@ -12,7 +12,7 @@ REGISTRY = get_docs_registry()
|
||||
# it relies on so search_patterns query terms map to discoverable skills.
|
||||
REQUIRED_LIBRARY_TAGS_BY_SKILL = {
|
||||
"copilot-customization": {"copilot", "vscode", "mcp"},
|
||||
"fastapi-async-sqlalchemy-modernization": {"fastapi", "sqlalchemy", "asyncio"},
|
||||
"async-fastapi-sqlmodel": {"fastapi", "sqlalchemy", "asyncio"},
|
||||
"fastapi-uv-docker": {"fastapi", "uv", "uvicorn", "docker"},
|
||||
"mcp-details": {"mcp", "fastmcp"},
|
||||
"nicegui": {"nicegui", "fastapi"},
|
||||
|
||||
@@ -38,7 +38,7 @@ SEARCH_QUERY_PARAMETERS = (
|
||||
),
|
||||
pytest.param(
|
||||
"asyncio",
|
||||
{"pytesting", "fastapi-async-sqlalchemy-modernization"},
|
||||
{"pytesting", "async-fastapi-sqlmodel"},
|
||||
id="query-asyncio",
|
||||
),
|
||||
pytest.param(
|
||||
|
||||
+9
-9
@@ -89,15 +89,15 @@ nav = [
|
||||
{ "Docker" = "skills/fastapi-uv-docker/references/docker-cloud-native.md" },
|
||||
] },
|
||||
{ "Async SQLA" = [
|
||||
{ "Overview" = "skills/fastapi-async-sqlalchemy-modernization/SKILL.md" },
|
||||
{ "Index" = "skills/fastapi-async-sqlalchemy-modernization/references/index.md" },
|
||||
{ "Engine" = "skills/fastapi-async-sqlalchemy-modernization/references/engine.md" },
|
||||
{ "Session" = "skills/fastapi-async-sqlalchemy-modernization/references/session.md" },
|
||||
{ "Tx" = "skills/fastapi-async-sqlalchemy-modernization/references/transactions.md" },
|
||||
{ "SQLModel" = "skills/fastapi-async-sqlalchemy-modernization/references/sqlmodel.md" },
|
||||
{ "IO" = "skills/fastapi-async-sqlalchemy-modernization/references/implicit_io.md" },
|
||||
{ "Obs" = "skills/fastapi-async-sqlalchemy-modernization/references/observability.md" },
|
||||
{ "Template" = "skills/fastapi-async-sqlalchemy-modernization/references/template.md" },
|
||||
{ "Overview" = "skills/async-fastapi-sqlmodel/SKILL.md" },
|
||||
{ "Index" = "skills/async-fastapi-sqlmodel/references/index.md" },
|
||||
{ "Engine" = "skills/async-fastapi-sqlmodel/references/engine.md" },
|
||||
{ "Session" = "skills/async-fastapi-sqlmodel/references/session.md" },
|
||||
{ "Tx" = "skills/async-fastapi-sqlmodel/references/transactions.md" },
|
||||
{ "SQLModel" = "skills/async-fastapi-sqlmodel/references/sqlmodel.md" },
|
||||
{ "IO" = "skills/async-fastapi-sqlmodel/references/implicit_io.md" },
|
||||
{ "Obs" = "skills/async-fastapi-sqlmodel/references/observability.md" },
|
||||
{ "Template" = "skills/async-fastapi-sqlmodel/references/template.md" },
|
||||
] },
|
||||
{ "NiceGUI" = [
|
||||
{ "Overview" = "skills/nicegui/SKILL.md" },
|
||||
|
||||
Reference in New Issue
Block a user