223 lines
12 KiB
Markdown
223 lines
12 KiB
Markdown
---
|
|
name: async-fastapi-sqlmodel
|
|
description: 'Explain and apply async database principles for FastAPI, SQLAlchemy 2.x, and SQLModel. Use when: learning or reviewing AsyncEngine and AsyncSession lifecycles, FastAPI lifespan and yield dependencies, transaction boundaries, concurrency safety, implicit ORM I/O, AsyncExitStack, pooling, testing, or SQLModel integration.'
|
|
x-personal-mcp:
|
|
id: async-fastapi-sqlmodel
|
|
version: 1.1.0
|
|
tags:
|
|
- fastapi
|
|
- sqlalchemy
|
|
- sqlmodel
|
|
- async
|
|
- asyncio
|
|
- database
|
|
- transactions
|
|
- resource-lifecycle
|
|
- architecture
|
|
capabilities:
|
|
- resource://skills/async-fastapi-sqlmodel/document
|
|
---
|
|
|
|
# Async FastAPI, SQLAlchemy, and SQLModel
|
|
|
|
Use this skill to explain how an async database layer works, why the recommended patterns exist, and how to evaluate code against them. Teach the runtime model before suggesting implementation changes.
|
|
|
|
Primary targets: PostgreSQL with asyncpg and SQLite with aiosqlite.
|
|
|
|
## When to Use
|
|
|
|
- Explain an async engine, session factory, session, connection, or transaction.
|
|
- Review FastAPI lifespan or dependency-based database management.
|
|
- Diagnose shared-session concurrency, implicit I/O, cleanup, or transaction problems.
|
|
- Compare SQLModel's model conveniences with SQLAlchemy's async runtime APIs.
|
|
- Decide whether a context manager, `AsyncExitStack`, eager loading, pooling option, or explicit transaction is appropriate.
|
|
|
|
## Outcome
|
|
|
|
Produce a focused technical explanation that:
|
|
|
|
- Defines the objects involved and identifies who owns each one.
|
|
- Traces acquisition, use, transaction behavior, and cleanup.
|
|
- Separates required invariants from defaults and situational choices.
|
|
- Explains failure modes and concurrency consequences.
|
|
- Uses a minimal canonical pattern when code clarifies the mechanics.
|
|
- Links claims to the relevant reference and upstream documentation.
|
|
|
|
Do not default to producing a project plan. Give sequencing advice only when the user explicitly asks for implementation steps.
|
|
|
|
## Mental Model
|
|
|
|
Keep three ownership scopes distinct:
|
|
|
|
| Scope | Object | Purpose | Typical owner |
|
|
|---|---|---|---|
|
|
| Application process | `AsyncEngine` and `async_sessionmaker` | Dialect, connection pool, and repeatable session configuration | FastAPI lifespan |
|
|
| Request or concurrent task | `AsyncSession` | Mutable ORM identity map and transactional state | A `yield` dependency or explicit unit of work |
|
|
| Atomic operation | `SessionTransaction` | Commit all changes together or roll them back together | Service or use-case boundary |
|
|
|
|
The engine is a long-lived factory and pool, not a single database connection. The session is a mutable unit-of-work object, not a concurrency-safe global. A transaction is a consistency boundary, not merely a call to `commit()`.
|
|
|
|
## Core Principles
|
|
|
|
### Match lifetime to ownership
|
|
|
|
- Create one `AsyncEngine` per process and database configuration in the normal case.
|
|
- Dispose it explicitly in an awaitable shutdown path; garbage collection cannot reliably await async driver cleanup.
|
|
- Configure `async_sessionmaker` once and call it to create short-lived sessions.
|
|
- Close each session deterministically with `async with` or a FastAPI dependency that yields once.
|
|
|
|
See [engine lifecycle](references/engine.md) and [session management](references/session.md).
|
|
|
|
### Isolate mutable session state
|
|
|
|
An `AsyncSession` represents one stateful transaction in progress. Never use one session in multiple concurrent tasks, including branches of `asyncio.gather()`. Give each task its own session and pass sessions explicitly rather than relying on mutable scoped globals.
|
|
|
|
See [session management](references/session.md).
|
|
|
|
### Make I/O visible
|
|
|
|
Async ORM code must not unexpectedly issue SQL during ordinary attribute access. Load relationships and deferred columns explicitly with eager loader options such as `selectinload()`, use `awaitable_attrs` or `refresh()` for deliberate fallback loading, and consider `lazy="raise"` where accidental access should fail fast. `expire_on_commit=False` is a common async configuration because post-commit expiration can otherwise turn attribute reads into implicit I/O.
|
|
|
|
See [implicit ORM I/O](references/implicit_io.md).
|
|
|
|
### Put transactions around business invariants
|
|
|
|
Use `async with session.begin():` when several operations must commit or roll back as one unit. A successful exit flushes and commits; an exception rolls back. Reads still participate in SQLAlchemy's autobegin behavior unless the connection uses true DBAPI autocommit, so describe a path as read-only because of application intent and permissions, not because a session silently has no transaction.
|
|
|
|
Use `begin_nested()` only for a real SAVEPOINT requirement and account for backend-specific behavior. In SQLAlchemy 2.x, calling `session.commit()` commits the outermost transaction, not the current savepoint.
|
|
|
|
See [transaction boundaries](references/transactions.md).
|
|
|
|
### Keep framework boundaries explicit
|
|
|
|
FastAPI lifespan owns resources shared by many requests. A dependency with one `yield` owns request-scoped resources and runs cleanup after use. These are related context-manager mechanisms but solve different lifetime problems.
|
|
|
|
Use `AsyncExitStack` when lifespan acquires a variable, conditional, or mixed collection of context-managed resources. It records cleanup as resources are acquired and unwinds callbacks in reverse order. A single engine with one cleanup callback can use a plain `try/finally`; `AsyncExitStack` is a composition tool, not a requirement.
|
|
|
|
See [engine lifecycle](references/engine.md).
|
|
|
|
### Use SQLModel as the primary modeling layer
|
|
|
|
Default to SQLModel for table models and API data models in FastAPI applications. A SQLModel table model is also a SQLAlchemy model, and every SQLModel model is also a Pydantic model, so shared base models can reduce schema duplication while preserving access to SQLAlchemy's full ORM.
|
|
|
|
SQLModel does not replace SQLAlchemy's async engine, session, transaction, or loader mechanics. Its main tutorial currently demonstrates synchronous sessions and its advanced guide still lists comprehensive async documentation as future work. For async applications, combine SQLModel models and statements with SQLAlchemy's `AsyncSession` APIs. Use SQLAlchemy declarative models only when a concrete unsupported mapping or library constraint justifies the exception.
|
|
|
|
See [SQLModel integration](references/sqlmodel.md).
|
|
|
|
### Configure from evidence
|
|
|
|
Pool sizing, overflow, recycle, pre-ping, isolation, statement timeouts, and health checks depend on the driver, database, deployment concurrency, and failure model. Explain defaults and tradeoffs before recommending values. Avoid treating pool checkout as proof that a useful query can succeed.
|
|
|
|
See [observability and resilience](references/observability.md).
|
|
|
|
### Test through the production seam
|
|
|
|
Keep the production engine and session-factory construction path intact in tests. Select a dedicated PostgreSQL, local SQLite, or in-memory SQLite URL at that seam, then override the request-session dependency only for the test lifetime. Use a test-scoped outer transaction with SAVEPOINT-backed session commits when application code calls `commit()`; it exercises normal transaction behavior while cleanup remains deterministic.
|
|
|
|
In-memory SQLite is suitable for serial tests. For multiple simultaneous sessions, use a named shared-cache SQLite URL or a temporary file, and retain PostgreSQL integration coverage for PostgreSQL-specific behavior.
|
|
|
|
See [database testing and fixture data](references/testing.md).
|
|
|
|
## Reference Map
|
|
|
|
| Concept | Reference |
|
|
|---|---|
|
|
| Engine lifecycle and ownership | [Engine lifecycle reference](references/engine.md) |
|
|
| Session factory and scope | [Session management reference](references/session.md) |
|
|
| Transaction boundaries | [Transaction boundaries reference](references/transactions.md) |
|
|
| Lifespan composition | [Engine lifecycle reference](references/engine.md) |
|
|
| Dependency injection | [Session management reference](references/session.md) |
|
|
| Implicit I/O control in ORM | [Implicit I/O reference](references/implicit_io.md) |
|
|
| Observability and resilience | [Observability reference](references/observability.md) |
|
|
| SQLModel-first modeling | [SQLModel integration reference](references/sqlmodel.md) |
|
|
| CRUD repository and standalone functions | [Basic CRUD reference](references/crud.md) |
|
|
| Test database selection and fixture data | [Database testing reference](references/testing.md) |
|
|
|
|
## Canonical Composition Pattern
|
|
|
|
This example shows the ownership boundaries. Adapt state storage and dependency wiring to the application's conventions.
|
|
|
|
```python
|
|
from contextlib import AsyncExitStack, asynccontextmanager
|
|
from collections.abc import AsyncGeneratorr
|
|
|
|
from fastapi import FastAPI
|
|
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
|
from sqlmodel.ext.asyncio.session import AsyncSession
|
|
|
|
@asynccontextmanager
|
|
async def lifespan(app: FastAPI) -> AsyncGenerator[None]:
|
|
async with AsyncExitStack() as stack:
|
|
engine = create_async_engine(settings.database_url)
|
|
stack.push_async_callback(engine.dispose)
|
|
|
|
session_factory = async_sessionmaker(
|
|
engine,
|
|
class_=AsyncSession,
|
|
expire_on_commit=False,
|
|
)
|
|
app.state.session_factory = session_factory
|
|
yield
|
|
|
|
|
|
async def get_session() -> AsyncGenerator[AsyncSession]:
|
|
async with app.state.session_factory() as session:
|
|
yield session
|
|
```
|
|
|
|
For direct construction without `AsyncExitStack`, put `await engine.dispose()` in a `finally` block. For background work that outlives a request, create a new session inside that task instead of retaining the request's session.
|
|
|
|
## Explanation Procedure
|
|
|
|
1. Identify the exact concept or observed behavior in question.
|
|
2. Name the owning scope: application, request/task, or transaction.
|
|
3. Trace what state the object holds and where actual database I/O can occur.
|
|
4. Explain normal entry, successful exit, exceptional exit, and concurrent use.
|
|
5. Distinguish an invariant from a recommended default or backend-specific choice.
|
|
6. Load only the matching reference documents and cite upstream sources.
|
|
7. Show the smallest useful code pattern or contrast when prose is insufficient.
|
|
8. End with concrete checks the reader can use to inspect their own code.
|
|
|
|
When reviewing code, verify:
|
|
|
|
- The URL uses an asyncio-compatible dialect.
|
|
- Engine creation and disposal have one clear owner.
|
|
- Every session has a bounded lifetime and is not shared across tasks.
|
|
- Transaction boundaries match business invariants and exception behavior.
|
|
- Relationship and deferred-column access cannot surprise the event loop with implicit I/O.
|
|
- Pool and timeout settings are justified by deployment behavior.
|
|
- Tests exercise rollback, cleanup, concurrency, and lifespan behavior where relevant.
|
|
- Tests use a dedicated database target and preserve production session mechanics.
|
|
|
|
## Anti-Patterns to Flag
|
|
|
|
- Creating engines inside request handlers.
|
|
- Sharing one AsyncSession across concurrent tasks.
|
|
- Implicit commit/rollback behavior with unclear ownership.
|
|
- Global mutable session state.
|
|
- Lifespan cleanup that depends on implicit garbage collection.
|
|
- Treating `AsyncExitStack` as mandatory for a fixed single resource.
|
|
- Treating SQLModel's synchronous tutorial examples as the async runtime pattern.
|
|
- Allowing lazy relationship access to hide database I/O.
|
|
- Copying pool settings without relating them to worker count and database capacity.
|
|
|
|
## Output Contract
|
|
|
|
Answer in the shape best suited to the question, usually:
|
|
|
|
1. Direct explanation.
|
|
2. Underlying lifecycle or transaction mechanics.
|
|
3. Required invariants and situational tradeoffs.
|
|
4. Minimal example or code-review findings when useful.
|
|
5. Verification questions and source links.
|
|
|
|
## References
|
|
|
|
!!! info "Primary sources"
|
|
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
|
|
- [SQLAlchemy transaction management](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html)
|
|
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
|
|
- [FastAPI dependencies with `yield`](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
|
|
- [Python `AsyncExitStack`](https://docs.python.org/3/library/contextlib.html#contextlib.AsyncExitStack)
|
|
- [SQLModel session dependency pattern](https://sqlmodel.tiangolo.com/tutorial/fastapi/session-with-dependency/)
|