engine/session updates

This commit is contained in:
John Lancaster
2026-07-31 22:15:51 -05:00
parent cd11ea8255
commit 0dc06f72ca
12 changed files with 767 additions and 457 deletions
+10 -10
View File
@@ -3,7 +3,7 @@ name: jsfiddle-page-layout
description: Create a responsive sample page layout for a user-supplied domain and return paste-ready HTML and CSS for JSFiddle. description: Create a responsive sample page layout for a user-supplied domain and return paste-ready HTML and CSS for JSFiddle.
x-personal-mcp: x-personal-mcp:
id: jsfiddle-page-layout id: jsfiddle-page-layout
version: 1.0.0 version: 1.1.0
tags: tags:
- frontend - frontend
- html - html
@@ -14,15 +14,6 @@ x-personal-mcp:
- prompts - prompts
capabilities: capabilities:
- resource://prompts/jsfiddle-page-layout/document - resource://prompts/jsfiddle-page-layout/document
arguments:
domain:
title: Domain
description: The product, service, organization, or subject the sample page should represent, including its audience when known.
required: true
layout_brief:
title: Layout brief
description: Optional page type, required sections, content priorities, visual direction, or constraints.
required: false
--- ---
# JSFiddle Page Layout # JSFiddle Page Layout
@@ -46,6 +37,15 @@ Create a polished sample page layout for the supplied domain. The result must ru
8. Include accessible landmarks, heading order, labels, focus styles, color contrast, and reduced-motion handling when animation is present. 8. Include accessible landmarks, heading order, labels, focus styles, color contrast, and reduced-motion handling when animation is present.
9. Use CSS custom properties for the color, typography, spacing, border, and shadow system. Avoid generic framework styling and tailor the visual language to the domain. 9. Use CSS custom properties for the color, typography, spacing, border, and shadow system. Avoid generic framework styling and tailor the visual language to the domain.
## Design References
Use these references as comparative guidance, not as templates to copy. Select principles that fit the domain and layout brief, and do not reproduce a vendor's visual language unless the user requests it.
1. [Material Design 3 foundations](https://m3.material.io/foundations) for current approaches to layout, interaction states, design tokens, and adaptable UI systems.
2. [Apple Human Interface Guidelines](https://developer.apple.com/design/human-interface-guidelines/) for contemporary principles covering hierarchy, typography, controls, and platform-aware interaction.
3. [web.dev responsive web design basics](https://web.dev/articles/responsive-web-design-basics) for content-led breakpoints, flexible layouts, and input-aware responsiveness.
4. [Web Content Accessibility Guidelines (WCAG) 2.2](https://www.w3.org/TR/WCAG22/) as the accessibility baseline for structure, contrast, focus, reflow, and target sizing.
## Output Contract ## Output Contract
Return exactly two fenced code blocks in this order: Return exactly two fenced code blocks in this order:
@@ -0,0 +1,95 @@
---
name: nicegui-component-extraction
description: Extract a user-selected component from a JSFiddle page layout and implement it as a reusable NiceGUI render function with responsive styling and typed bindable state where needed.
x-personal-mcp:
id: nicegui-component-extraction
version: 1.0.0
tags:
- nicegui
- components
- frontend
- refactoring
- jsfiddle
- prompts
capabilities:
- resource://prompts/nicegui-component-extraction/document
arguments:
component:
title: Component
description: Component or page region to extract, identified by its visible label, semantic role, or selector.
required: true
source_layout:
title: Source layout
description: Optional HTML and CSS from the JSFiddle page layout prompt; when omitted, use the latest applicable output in the conversation.
required: false
target_location:
title: Target location
description: Optional target NiceGUI page, module, or package in which to create and integrate the component.
required: false
behavior_requirements:
title: Behavior requirements
description: Optional interactions, state, callbacks, or content variations the extracted component must support.
required: false
---
# NiceGUI Component Extraction
Extract one user-selected component from the output of the [JSFiddle Page Layout](../jsfiddle-page-layout/PROMPT.md) prompt and implement it as a reusable NiceGUI component in the target repository.
## Inputs
1. `component`: required visible label, semantic role, or selector identifying the component to extract
2. `source_layout`: optional HTML and CSS; when omitted, use the latest applicable JSFiddle page layout output in the conversation
3. `target_location`: optional target page, module, or package; infer it from the repository when omitted
4. `behavior_requirements`: optional interactions, state, callbacks, or content variations
If the selected component or source layout cannot be identified unambiguously, ask one concise clarification question before editing.
## Required References
Apply both references before implementation:
1. Component boundaries, responsive layout, Quasar props, Tailwind utilities, and shared CSS: [NiceGUI Page Layout and Styling](../../skills/nicegui/references/architecture-and-styling.md)
2. Typed UI state, propagation, mutable defaults, binding strictness, and version checks: [Binding Dataclasses Deep Dive](../../skills/nicegui/references/binding-dataclasses.md)
## Workflow
1. Locate the selected region in the source HTML and CSS, including its responsive rules, states, and dependencies on surrounding layout.
2. Inspect the target repository's NiceGUI version, package structure, component conventions, shared CSS loading, and nearest page call site.
3. Define the smallest reusable API for the component:
- name the public function `render_<component_name>` using snake_case
- accept content, typed state, and event callbacks as explicit parameters
- keep business rules, persistence, and service access outside the component
- preserve an established return-value convention; otherwise return the component's root NiceGUI element
4. Translate semantic HTML into native NiceGUI and Quasar elements. Do not embed the original page wholesale with `ui.html` when standard components express the structure.
5. Recreate only the CSS needed by the extracted component:
- use Quasar props for component appearance and behavior
- use NiceGUI classes and Tailwind utilities for spacing, sizing, alignment, and responsive layout
- use scoped shared CSS only where props and utilities are insufficient
- do not override Quasar field internals or duplicate globally loaded styles
6. Model editable or shared component state with a typed `@binding.bindable_dataclass` only when binding improves the interaction:
- use `field(default_factory=...)` for mutable defaults
- scope state to the appropriate page, client, or user
- keep binding transforms pure and inexpensive
- assign updated collections back to bound fields instead of relying on in-place mutation
7. Integrate the render function at the nearest target page or call site without moving unrelated page composition or domain logic into the component.
8. Preserve accessibility, focus behavior, text wrapping, stable dimensions, and the source layout's visual hierarchy.
9. Run the narrowest available tests, lint, and type checks for the changed files. For visual components, verify representative mobile, landscape desktop, and portrait desktop viewports when browser tooling is available.
## Output Contract
Complete the implementation in the target repository, then report:
1. Files created or updated.
2. The `render_*` function signature and its state or callback contract.
3. Any deliberate visual or interaction differences from the JSFiddle source.
4. Validation commands and outcomes, including viewport checks when performed.
## Quality Rules
1. Extract exactly the requested component and its necessary local dependencies.
2. Prefer the target repository's established patterns over introducing a new abstraction style.
3. Keep the component presentation-focused and reusable across pages with compatible data.
4. Do not add a bindable dataclass for static content or event-local state that is clearer as ordinary parameters.
5. Do not create a second component tree for mobile; use responsive classes and stable layout constraints.
6. Keep custom CSS tokenized, scoped to the component, and loaded once by the application's composition layer.
+12 -20
View File
@@ -63,7 +63,7 @@ The engine is a long-lived factory and pool, not a single database connection. T
- Create one `AsyncEngine` per process and database configuration in the normal case. - Create one `AsyncEngine` per process and database configuration in the normal case.
- Dispose it explicitly in an awaitable shutdown path; garbage collection cannot reliably await async driver cleanup. - Dispose it explicitly in an awaitable shutdown path; garbage collection cannot reliably await async driver cleanup.
- Configure `async_sessionmaker` once and call it to create short-lived sessions. - Configure `async_sessionmaker` once inside the engine lifecycle and call it to create short-lived sessions.
- Close each session deterministically with `async with` or a FastAPI dependency that yields once. - Close each session deterministically with `async with` or a FastAPI dependency that yields once.
See [engine lifecycle](references/engine.md) and [session management](references/session.md). See [engine lifecycle](references/engine.md) and [session management](references/session.md).
@@ -92,7 +92,7 @@ See [transaction boundaries](references/transactions.md).
FastAPI lifespan owns resources shared by many requests. A dependency with one `yield` owns request-scoped resources and runs cleanup after use. These are related context-manager mechanisms but solve different lifetime problems. FastAPI lifespan owns resources shared by many requests. A dependency with one `yield` owns request-scoped resources and runs cleanup after use. These are related context-manager mechanisms but solve different lifetime problems.
Use `AsyncExitStack` when lifespan acquires a variable, conditional, or mixed collection of context-managed resources. It records cleanup as resources are acquired and unwinds callbacks in reverse order. A single engine with one cleanup callback can use a plain `try/finally`; `AsyncExitStack` is a composition tool, not a requirement. Use `AsyncExitStack` when lifespan acquires a variable, conditional, or mixed collection of context-managed resources. It records cleanup as resources are acquired and unwinds callbacks in reverse order. A single engine should use the direct engine context manager; `AsyncExitStack` is a composition tool, not a requirement.
See [engine lifecycle](references/engine.md). See [engine lifecycle](references/engine.md).
@@ -138,34 +138,26 @@ See [database testing and fixture data](references/testing.md).
This example shows the ownership boundaries. Adapt state storage and dependency wiring to the application's conventions. This example shows the ownership boundaries. Adapt state storage and dependency wiring to the application's conventions.
```python ```python
from contextlib import AsyncExitStack, asynccontextmanager from collections.abc import AsyncGenerator
from collections.abc import AsyncGeneratorr from contextlib import asynccontextmanager
from fastapi import FastAPI from fastapi import FastAPI, Request
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
@asynccontextmanager @asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None]: async def lifespan(app: FastAPI) -> AsyncGenerator[None]:
async with AsyncExitStack() as stack: async with engine_scope(settings.database_url) as engine:
engine = create_async_engine(settings.database_url) app.state.session_factory = create_session_factory(engine)
stack.push_async_callback(engine.dispose)
session_factory = async_sessionmaker(
engine,
class_=AsyncSession,
expire_on_commit=False,
)
app.state.session_factory = session_factory
yield yield
async def get_session() -> AsyncGenerator[AsyncSession]: async def get_session(request: Request) -> AsyncGenerator[AsyncSession]:
async with app.state.session_factory() as session: async with request.app.state.session_factory() as session:
yield session yield session
``` ```
For direct construction without `AsyncExitStack`, put `await engine.dispose()` in a `finally` block. For background work that outlives a request, create a new session inside that task instead of retaining the request's session. `engine_scope()` and `create_session_factory()` are defined in the engine and session references. For background work that outlives a request, inject the shared factory and create a new session inside that task instead of retaining the request's session.
## Explanation Procedure ## Explanation Procedure
@@ -58,194 +58,143 @@ This reference uses direct field arguments and full-update semantics to keep the
## Independent CRUD Functions ## Independent CRUD Functions
Functions are the simplest default when grouping state or behavior in an object adds no value. Each function is a complete operation boundary: it can run standalone by resolving the cached factory from `database_url`, or compose into a caller-owned scope through `session`. Functions are the simplest default when grouping state or behavior in an object adds no value. Keep them in the session-required data-access layer so transaction ownership remains external and several calls can compose under one boundary.
```python ```python
from sqlmodel import select from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from .session import session_scope
from .session import transaction_scope
async def create_widget( async def create_widget(
session: AsyncSession,
name: str, name: str,
description: str | None = None, description: str | None = None,
*,
database_url: str,
session: AsyncSession | None = None,
) -> Widget: ) -> Widget:
async with transaction_scope( widget = Widget(name=name, description=description)
database_url=database_url, session.add(widget)
session=session, await session.flush()
) as active_session: return widget
widget = Widget(name=name, description=description)
active_session.add(widget)
await active_session.flush()
return widget
async def get_widget( async def get_widget(
session: AsyncSession,
widget_id: int, widget_id: int,
*,
database_url: str,
session: AsyncSession | None = None,
) -> Widget | None: ) -> Widget | None:
async with session_scope( return await session.get(Widget, widget_id)
database_url=database_url,
session=session,
) as active_session:
return await active_session.get(Widget, widget_id)
async def list_widgets( async def list_widgets(
session: AsyncSession,
*, *,
database_url: str,
offset: int = 0, offset: int = 0,
limit: int = 100, limit: int = 100,
session: AsyncSession | None = None,
) -> list[Widget]: ) -> list[Widget]:
if offset < 0: if offset < 0:
raise ValueError("offset must be non-negative") raise ValueError("offset must be non-negative")
if not 1 <= limit <= 100: if not 1 <= limit <= 100:
raise ValueError("limit must be between 1 and 100") raise ValueError("limit must be between 1 and 100")
async with session_scope( statement = select(Widget).order_by(Widget.id).offset(offset).limit(limit)
database_url=database_url, return list(await session.scalars(statement))
session=session,
) as active_session:
statement = select(Widget).order_by(Widget.id).offset(offset).limit(limit)
return list(await active_session.scalars(statement))
async def update_widget( async def update_widget(
session: AsyncSession,
widget_id: int, widget_id: int,
name: str, name: str,
description: str | None, description: str | None,
*,
database_url: str,
session: AsyncSession | None = None,
) -> Widget | None: ) -> Widget | None:
async with transaction_scope( widget = await session.get(Widget, widget_id)
database_url=database_url, if widget is None:
session=session, return None
) as active_session:
widget = await active_session.get(Widget, widget_id)
if widget is None:
return None
widget.name = name widget.name = name
widget.description = description widget.description = description
await active_session.flush() await session.flush()
return widget return widget
async def delete_widget( async def delete_widget(
session: AsyncSession,
widget_id: int, widget_id: int,
*,
database_url: str,
session: AsyncSession | None = None,
) -> Widget | None: ) -> Widget | None:
async with transaction_scope( widget = await session.get(Widget, widget_id)
database_url=database_url, if widget is None:
session=session, return None
) as active_session:
widget = await active_session.get(Widget, widget_id)
if widget is None:
return None
await active_session.delete(widget) await session.delete(widget)
await active_session.flush() await session.flush()
return widget return widget
``` ```
Update and delete load the row through the same session that mutates it. This avoids accepting detached instances from an earlier standalone read and gives both operations an explicit `None` result that the application layer can map to a domain or HTTP error. Delete returns the loaded object for callers that need its values, but that object represents a row scheduled for deletion and must not be reused as persistent state. List operations validate their bounds and order by the primary key so pagination is deterministic. Add a unique tiebreaker whenever ordering by a non-unique field. Update and delete load the row through the same session that mutates it. This avoids accepting detached instances from an earlier standalone read and gives both operations an explicit `None` result that the application layer can map to a domain or HTTP error. Delete returns the loaded object for callers that need its values, but that object represents a row scheduled for deletion and must not be reused as persistent state. List operations validate their bounds and order by the primary key so pagination is deterministic. Add a unique tiebreaker whenever ordering by a non-unique field.
`flush()` sends pending writes and populates ordinary generated primary keys. It does not itself commit. For a standalone write, the surrounding owned `transaction_scope()` commits after the function body succeeds. For a supplied session, the caller's outer transaction retains commit and rollback ownership. Use `await active_session.refresh(widget)` only when the operation deliberately needs database-generated state that was not returned during the flush; an unconditional refresh adds another query. `flush()` sends pending writes and populates ordinary generated primary keys. It does not itself commit. The caller's transaction retains commit and rollback ownership. Use `await session.refresh(widget)` only when the operation deliberately needs database-generated state that was not returned during the flush; an unconditional refresh adds another query.
--- ---
## Repository Object ## Repository Object
A repository can provide a stable domain-facing interface when several callers need the same grouped operations. It stores repeatable database configuration, never a mutable session. Every method delegates to the analogous function and exposes the same optional-session contract. A repository can provide a stable domain-facing interface when several callers need the same grouped operations. It remains stateless here: every method requires a session and delegates to the analogous function.
```python ```python
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
class WidgetRepository: class WidgetRepository:
def __init__(self, database_url: str) -> None:
self.database_url = database_url
async def create( async def create(
self, self,
session: AsyncSession,
name: str, name: str,
description: str | None = None, description: str | None = None,
*,
session: AsyncSession | None = None,
) -> Widget: ) -> Widget:
return await create_widget( return await create_widget(
session,
name, name,
description, description,
database_url=self.database_url,
session=session,
) )
async def get( async def get(
self, self,
session: AsyncSession,
widget_id: int, widget_id: int,
*,
session: AsyncSession | None = None,
) -> Widget | None: ) -> Widget | None:
return await get_widget( return await get_widget(session, widget_id)
widget_id,
database_url=self.database_url,
session=session,
)
async def list( async def list(
self, self,
session: AsyncSession,
*, *,
offset: int = 0, offset: int = 0,
limit: int = 100, limit: int = 100,
session: AsyncSession | None = None,
) -> list[Widget]: ) -> list[Widget]:
return await list_widgets( return await list_widgets(
database_url=self.database_url, session,
offset=offset, offset=offset,
limit=limit, limit=limit,
session=session,
) )
async def update( async def update(
self, self,
session: AsyncSession,
widget_id: int, widget_id: int,
name: str, name: str,
description: str | None, description: str | None,
*,
session: AsyncSession | None = None,
) -> Widget | None: ) -> Widget | None:
return await update_widget( return await update_widget(
session,
widget_id, widget_id,
name, name,
description, description,
database_url=self.database_url,
session=session,
) )
async def delete( async def delete(
self, self,
session: AsyncSession,
widget_id: int, widget_id: int,
*,
session: AsyncSession | None = None,
) -> Widget | None: ) -> Widget | None:
return await delete_widget( return await delete_widget(session, widget_id)
widget_id,
database_url=self.database_url,
session=session,
)
``` ```
The object is intentionally thin. Tests can construct it with a test database URL or pass a transaction-scoped test session to individual methods. A caller-provided session always wins and remains open after the method returns. A standalone operation closes its owned session before returning, so returned objects are detached; load every required scalar, deferred column, and relationship explicitly before the scope exits, and do not mutate those objects expecting persistence. The object is intentionally thin. Tests pass a transaction-scoped test session directly. The caller always owns that session and its transaction, and the repository never closes or commits it.
If a read participates in a later write, pass the same session and place both operations inside the explicit transaction. This avoids splitting one use case across sessions and keeps SQLAlchemy's autobegin behavior from obscuring transaction ownership. Add a repository only when its naming, shared query policy, dependency substitution, or domain boundary improves the application. Independent functions remain a valid and often clearer design. If a read participates in a later write, pass the same session and place both operations inside the explicit transaction. This avoids splitting one use case across sessions and keeps SQLAlchemy's autobegin behavior from obscuring transaction ownership. Add a repository only when its naming, shared query policy, dependency substitution, or domain boundary improves the application. Independent functions remain a valid and often clearer design.
@@ -253,15 +202,18 @@ If a read participates in a later write, pass the same session and place both op
## Transaction Ownership ## Transaction Ownership
Compose multiple calls under one use-case transaction. At this boundary, a supplied session joins its already-active caller-owned transaction, while omitting the session creates a standalone session and transaction. Each nested CRUD write receives `active_session`, detects that transaction, and borrows it instead of committing independently. Compose multiple calls under one use-case transaction. Only the complete use case accepts an optional session: a supplied session joins its caller's active transaction, while omitting the session creates and owns a standalone session and transaction. CRUD functions and repository methods simply use the resulting `active_session`.
The scope names describe exactly what they own: `session_scope()` manages session lifetime but never commits, while `transaction_scope()` manages a complete transaction only when it also creates the session. Both yield the name `active_session` because downstream CRUD code does not need to know whether the session was borrowed or owned.
```python ```python
from sqlalchemy.ext.asyncio import async_sessionmaker
from .session import transaction_scope from .session import transaction_scope
type SessionFactory = async_sessionmaker[AsyncSession]
async def replace_widget( async def replace_widget(
session_factory: SessionFactory,
repository: WidgetRepository, repository: WidgetRepository,
widget_id: int, widget_id: int,
replacement_name: str, replacement_name: str,
@@ -270,20 +222,20 @@ async def replace_widget(
session: AsyncSession | None = None, session: AsyncSession | None = None,
) -> Widget | None: ) -> Widget | None:
async with transaction_scope( async with transaction_scope(
database_url=repository.database_url, session_factory,
session=session, session=session,
) as active_session: ) as active_session:
deleted_widget = await repository.delete( deleted_widget = await repository.delete(
active_session,
widget_id, widget_id,
session=active_session,
) )
if deleted_widget is None: if deleted_widget is None:
return None return None
return await repository.create( return await repository.create(
active_session,
replacement_name, replacement_name,
replacement_description, replacement_description,
session=active_session,
) )
``` ```
@@ -294,8 +246,8 @@ If creation fails, deletion rolls back with it. For a caller-owned transaction,
## Anti-Patterns ## Anti-Patterns
- Storing one mutable `AsyncSession` on a long-lived repository object. - Storing one mutable `AsyncSession` on a long-lived repository object.
- Constructing ad hoc factories or sessions instead of resolving the cached factory through the scope helpers. - Creating sessions or transactions inside CRUD functions and repository methods.
- Using `session_scope()` for an optional write, which would close an owned session without committing. - Passing database configuration through every CRUD call instead of injecting a session at the data-access boundary.
- Accepting a supplied session for a write without requiring an active caller-owned transaction. - Accepting a supplied session for a write without requiring an active caller-owned transaction.
- Calling `commit()` or `rollback()` directly instead of expressing ownership through `transaction_scope()`. - Calling `commit()` or `rollback()` directly instead of expressing ownership through `transaction_scope()`.
- Accepting unbounded list queries. - Accepting unbounded list queries.
@@ -307,16 +259,16 @@ If creation fails, deletion rolls back with it. For a caller-owned transaction,
## Operational Checks ## Operational Checks
- Every CRUD call receives a task-local `AsyncSession`. - Every CRUD call receives a task-local `AsyncSession`.
- Standalone reads resolve the cached factory by database URL and close their owned session. - Standalone reads create and close a session at the service or application boundary.
- Standalone writes resolve the cached factory and own commit, rollback, and session cleanup through `transaction_scope()`. - Standalone writes own commit, rollback, and session cleanup through `transaction_scope()` at the service or use-case boundary.
- Supplied write sessions already have an active caller-owned transaction. - Supplied write sessions already have an active caller-owned transaction.
- Each complete operation, service, or use-case boundary borrows an active transaction or owns a complete session-and-transaction scope. - Each complete operation, service, or use-case boundary borrows an active transaction or owns a complete session-and-transaction scope.
- List operations have pagination and deterministic ordering where required. - List operations have pagination and deterministic ordering where required.
- Update requires values for both mutable fields; passing `None` explicitly clears the nullable description. - Update requires values for both mutable fields; passing `None` explicitly clears the nullable description.
- Get, update, and delete use the same identifier and missing-row semantics. - Get, update, and delete use the same identifier and missing-row semantics.
- Functions and repository methods use domain arguments first and keyword-only infrastructure arguments consistently. - Functions and repository methods take the session explicitly and do not accept database configuration.
- Standalone reads load all state needed after their owned session closes. - Standalone service reads load all state needed after their owned session closes.
- Repository objects hold configuration or policy, never request-scoped session state. - Repository objects hold query policy when useful, never database configuration or request-scoped session state.
--- ---
@@ -1,99 +1,87 @@
# Async SQLAlchemy Engine # Async SQLAlchemy Engine
!!! info "Primary sources" !!! info "Primary sources"
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) - [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
- [SQLAlchemy connections](https://docs.sqlalchemy.org/en/21/core/connections.html) - [SQLAlchemy connections](https://docs.sqlalchemy.org/en/21/core/connections.html)
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html) - [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
- [SQLAlchemy pooling and multiprocessing](https://docs.sqlalchemy.org/en/21/core/pooling.html#pooling-multiprocessing) - [SQLAlchemy pooling and multiprocessing](https://docs.sqlalchemy.org/en/21/core/pooling.html#pooling-multiprocessing)
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/) - [SQLAlchemy SQLite transaction control](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#enabling-non-legacy-sqlite-transactional-modes-with-the-sqlite3-or-aiosqlite-driver)
- [SQLAlchemy SQLite foreign-key support](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#foreign-key-support)
- [SQLite PRAGMA reference](https://www.sqlite.org/pragma.html)
--- ---
## Engine Ownership Model ## Engine Ownership Model
Create one async engine per process per database URL and keep engine construction independent from FastAPI. Create one async engine for each application, worker, command, or test lifecycle.
- SQLAlchemy guidance: the engine is intended as a long-lived, concurrent registry over pooled DB connections, not a per-request object. - SQLAlchemy guidance: the engine is intended as a long-lived, concurrent registry over pooled DB connections, not a per-operation object.
- A cached function provides stable process-local engine identity without making framework state the only way to obtain it. - The composition root owns engine creation and disposal.
- FastAPI lifespan starts and stops that independently defined resource; it does not contain the construction policy. - Services and repositories receive a session or session factory; they do not resolve an engine.
!!! tip "Practical rule" !!! tip "Practical rule"
- Exactly one `create_async_engine(...)` call in the cached engine factory. - Exactly one `create_async_engine(...)` call for each application-owned engine lifecycle.
- Zero `create_async_engine(...)` calls in request handlers. - Zero `create_async_engine(...)` calls in feature code.
- Zero calls to the cached factory from repository code. - Zero engine lookup or disposal calls in repository code.
--- ---
## Cached Engine Factory ## One Engine Context Manager
Use [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) on a synchronous factory. Creating an `AsyncEngine` configures the dialect and pool; it does not need to await a database connection. Use one [`asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager) to pair engine creation with disposal:
```python ```python
from functools import cache from collections.abc import AsyncGenerator
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
@cache
def get_engine(database_url: str) -> AsyncEngine:
return create_async_engine(
database_url,
pool_pre_ping=True,
)
async def dispose_engine(database_url: str) -> None:
engine = get_engine(database_url)
try:
await engine.dispose()
finally:
get_engine.cache_clear()
async def refresh_engine(database_url: str) -> AsyncEngine:
await dispose_engine(database_url)
return get_engine(database_url)
```
The database URL is an explicit, hashable cache key. Calls with the same URL return the same engine; a different URL receives a different engine. If engine options vary at runtime, make them explicit hashable arguments too.
Resolve settings at the composition boundary and call `get_engine(settings.database_url)`. Do not hide settings lookup or engine creation inside feature code.
## Thin FastAPI Lifespan Wrapper
The lifespan context manager only connects the cached resource to FastAPI ownership:
```python
from collections.abc import AsyncGeneratorr
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
from fastapi import FastAPI from sqlalchemy.ext.asyncio import AsyncEngine
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlalchemy.ext.asyncio import create_async_engine
from sqlmodel.ext.asyncio.session import AsyncSession
type SessionFactory = async_sessionmaker[AsyncSession]
@asynccontextmanager @asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None]: async def database_scope(database_url: str) -> AsyncGenerator[SessionFactory]:
database_url = app.state.settings.database_url async with engine_scope(database_url) as engine:
engine = get_engine(database_url) yield async_sessionmaker(
app.state.engine = engine bind=engine,
class_=AsyncSession,
expire_on_commit=False,
)
@asynccontextmanager
async def engine_scope(database_url: str) -> AsyncGenerator[AsyncEngine]:
engine = create_async_engine(database_url, pool_pre_ping=True)
if engine.dialect.name == "sqlite":
configure_aiosqlite_engine(engine)
try: try:
yield yield engine
finally: finally:
await dispose_engine(database_url) await engine.dispose()
app = FastAPI(lifespan=lifespan)
``` ```
`dispose()` closes checked-in connections and replaces the pool, but it does not remove the Python object from `functools.cache`. `dispose_engine()` clears the cache even if driver cleanup raises, preventing a later lifespan run or test from retrieving that engine instance. The code that enters `engine_scope()` owns the engine. It keeps that scope open for the complete application, worker, command, or test lifecycle and passes the yielded engine into session-factory construction. Successful and exceptional exits both dispose the pool.
This simple cleanup assumes one configured database URL per process. If a process intentionally owns several cached engines, use a small registry with per-key removal instead of clearing the whole cache. For a fixed engine, `try/finally` is sufficient; use `AsyncExitStack` when lifespan composes multiple conditional or dynamically acquired resources. Creating an `AsyncEngine` configures its dialect and pool; the first database operation normally establishes a connection. No cache is required when the application composition root enters this context exactly once. Removing the cache also removes cache-key, refresh, and invalidation behavior that otherwise must remain synchronized with the session factory.
When directly testing engine construction or lifespan behavior: Resolve settings before entering the scope. Do not hide settings lookup or engine creation inside feature code.
- Call `get_engine.cache_clear()` before the test to remove process-local state. Workers, scripts, and other composition roots enter `database_scope()` directly:
- Dispose any engine the test creates.
- Clear the cache again during teardown, even when the test fails. ```python
async with database_scope(settings.database_url) as session_factory:
await run_worker(session_factory)
```
For several fixed databases, nest one scope per engine. Use [`AsyncExitStack`](https://docs.python.org/3/library/contextlib.html#contextlib.AsyncExitStack) only when the number of engines is dynamic or conditional.
When directly testing engine construction or lifecycle behavior, enter `engine_scope()` in the test or fixture. Exiting the context disposes the engine even when the test fails; no global cache reset is needed.
See [FastAPI database integration](fastapi.md) for adapting `database_scope()` to application lifespan and dependency injection.
--- ---
@@ -110,6 +98,62 @@ Use SQLAlchemy async driver URLs:
--- ---
## SQLite Connection and Transaction Policy
SQLite settings do not form one indivisible bundle:
- `PRAGMA foreign_keys=ON` is a correctness requirement when the schema declares foreign keys. SQLite requires it on every connection, including the connection used by `metadata.create_all()`.
- Disabling the driver's implicit `BEGIN` and emitting `BEGIN` from SQLAlchemy provides non-legacy transaction behavior for `aiosqlite`. This makes SELECT, DDL, and SAVEPOINT behavior participate in SQLAlchemy's transaction boundary consistently.
- `PRAGMA busy_timeout` is a per-connection lock-wait policy. Choose the duration from the application's latency and contention requirements.
- `PRAGMA journal_mode=WAL` is an optional file-database concurrency policy. WAL persists in the database file, cannot be enabled for an in-memory database, and is not a substitute for transaction control.
Install instance-level listeners exactly once, immediately after constructing an `aiosqlite` engine and before its first connection:
```python
from sqlalchemy import event
from sqlalchemy.engine import Connection
from sqlalchemy.engine.interfaces import DBAPIConnection
from sqlalchemy.ext.asyncio import AsyncEngine
def configure_aiosqlite_engine(
engine: AsyncEngine,
*,
busy_timeout_ms: int | None = 30_000,
enable_wal: bool = False,
) -> None:
if engine.dialect.name != "sqlite" or engine.dialect.driver != "aiosqlite":
raise ValueError("Expected a sqlite+aiosqlite engine")
if busy_timeout_ms is not None and busy_timeout_ms < 0:
raise ValueError("busy_timeout_ms must be non-negative")
@event.listens_for(engine.sync_engine, "connect")
def configure_connection(dbapi_connection: DBAPIConnection, _: object) -> None:
dbapi_connection.isolation_level = None
cursor = dbapi_connection.cursor()
try:
cursor.execute("PRAGMA foreign_keys=ON")
if busy_timeout_ms is not None:
cursor.execute(f"PRAGMA busy_timeout={busy_timeout_ms}")
if enable_wal:
cursor.execute("PRAGMA journal_mode=WAL")
journal_mode = cursor.fetchone()
if journal_mode is None or journal_mode[0].lower() != "wal":
raise RuntimeError("SQLite could not enable WAL mode")
finally:
cursor.close()
@event.listens_for(engine.sync_engine, "begin")
def begin_transaction(connection: Connection) -> None:
connection.exec_driver_sql("BEGIN")
```
The `connect` listener receives the adapted synchronous DBAPI connection exposed by `engine.sync_engine`; event callbacks themselves are synchronous even though application queries use the async engine. Setting `isolation_level=None` and adding the `begin` listener are one transaction-control strategy and must remain paired. Do not combine this pair with SQLAlchemy's driver-level `AUTOCOMMIT` isolation mode.
The default above enables foreign keys and modern transaction boundaries for file and in-memory databases. Enable WAL only for a file-backed database after deciding that its read/write concurrency model is appropriate. Treat `30_000` as an example policy, not a universal default; `connect_args={"timeout": 30.0}` at engine construction is another way to configure the underlying SQLite lock timeout.
---
## Pooling Defaults and Tuning ## Pooling Defaults and Tuning
Default behavior is usually correct first: Default behavior is usually correct first:
@@ -123,6 +167,27 @@ When to switch pool strategy:
- `NullPool` if you explicitly need no pooling (special environments, some tests, or strict cross-loop constraints). - `NullPool` if you explicitly need no pooling (special environments, some tests, or strict cross-loop constraints).
- Keep in mind this increases connect/disconnect churn. - Keep in mind this increases connect/disconnect churn.
### When `StaticPool` Is Appropriate
Use [`StaticPool`](https://docs.sqlalchemy.org/en/21/core/pooling.html#sqlalchemy.pool.StaticPool) only when every checkout must reuse one DBAPI connection and all database access is serialized. Typical cases are:
- A serial test suite using a private in-memory SQLite database. The `sqlite+aiosqlite://` URL already selects `StaticPool` automatically, so specifying `poolclass=StaticPool` is normally redundant.
- A narrowly scoped SQLite engine that must preserve connection-local state, such as temporary tables, across SQLAlchemy connection or session checkouts.
When explicit configuration is required:
```python
from sqlalchemy.ext.asyncio import create_async_engine
from sqlalchemy.pool import StaticPool
engine = create_async_engine(
"sqlite+aiosqlite:///./test.db",
poolclass=StaticPool,
)
```
`StaticPool` is not a general performance optimization or a way to make SQLite concurrent. All sessions share one underlying connection and its single transaction state, so one session's `COMMIT` or `ROLLBACK` can interfere with another session. Do not use it when several sessions or tasks may access the engine concurrently. For concurrent in-memory work, use a named shared-cache SQLite URL so pooled connections have independent transaction state, or use a temporary file database. See [SQLite test targets](testing.md#sqlite-targets) for those patterns.
--- ---
## Disposal Semantics ## Disposal Semantics
@@ -151,21 +216,26 @@ This prevents broken socket state and cross-process connection corruption.
## What Not to Do ## What Not to Do
- Create an engine inside every request dependency. - Create an engine inside each operation or unit of work.
- Create/dispose engines inside repository methods. - Create/dispose engines inside repository methods.
- Call `get_engine()` from repositories instead of injecting their engine or session dependency. - Resolve an engine from repositories instead of injecting a session dependency.
- Keep engine creation as a hidden side effect of import-time module globals. - Keep engine creation as a hidden side effect of import-time module globals.
- Dispose a cached engine without clearing the cache during final teardown. - Keep a session factory alive after its bound engine scope exits.
- Use deprecated FastAPI startup/shutdown events together with lifespan. - Add process-global engine caching when one composition root already owns the lifecycle.
- Install the same SQLite event listeners more than once on one engine.
- Enable WAL blindly for in-memory SQLite or treat a busy timeout as a concurrency guarantee.
--- ---
## Engine Design Checklist ## Engine Design Checklist
- One engine per process per DB URL. - One engine scope per application-owned database lifecycle.
- Engine created by one cached, framework-independent factory. - Engine creation and disposal paired by one framework-independent context manager.
- Lifespan only retrieves, exposes, disposes, and uncaches the engine. - The composition root enters the database scope once and keeps it open until shutdown.
- Session factory created inside, and never outlives, its engine scope.
- Async driver URL matches backend (`asyncpg` or `aiosqlite`). - Async driver URL matches backend (`asyncpg` or `aiosqlite`).
- `aiosqlite` foreign-key and transaction listeners installed once before first use.
- WAL enabled only as an explicit policy for a file-backed SQLite database.
- Pooling strategy is explicit for non-default needs. - Pooling strategy is explicit for non-default needs.
- No request-path engine creation. - No feature-path engine creation.
- Tests dispose engines and clear cached state deterministically. - Tests enter the same scope and receive deterministic disposal without global cache cleanup.
@@ -0,0 +1,202 @@
# FastAPI Database Integration
!!! info "Primary sources"
- [FastAPI lifespan events](https://fastapi.tiangolo.com/advanced/events/)
- [FastAPI dependencies with `yield`](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
- [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
---
## Purpose
Connect the framework-independent database tools to FastAPI:
- lifespan enters one application-owned `database_scope()`,
- application state holds the resulting session factory,
- dependencies create one session per request,
- `Annotated` aliases make route ownership concise and explicit.
The underlying resource and transaction rules remain in [engine lifecycle](engine.md), [session management](session.md), and [transaction boundaries](transactions.md).
---
## Lifespan Ownership
Enter `database_scope()` once for the complete application lifecycle. Store the session factory, not the engine, because request code needs sessions rather than direct pool access:
```python
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from fastapi import FastAPI
from .engine import database_scope
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None]:
database_url = app.state.settings.database_url
async with database_scope(database_url) as session_factory:
app.state.session_factory = session_factory
try:
yield
finally:
del app.state.session_factory
app = FastAPI(lifespan=lifespan)
```
Lifespan does not construct resources per request. It enters the same framework-independent scope used by scripts, workers, and tests, keeps that scope open while requests are served, and lets it dispose the engine during shutdown.
Only store the engine too when application-level code genuinely needs direct Core operations, pool instrumentation, or engine-specific diagnostics. Routes and repositories should normally receive an `AsyncSession`.
---
## Session Factory Dependency
A synchronous dependency retrieves the already-created factory from application state:
```python
from typing import Annotated
from fastapi import Depends
from fastapi import Request
from .session import SessionFactory
from .session import transaction_scope
def get_session_factory(request: Request) -> SessionFactory:
return request.app.state.session_factory
type SessionFactoryDep = Annotated[SessionFactory, Depends(get_session_factory)]
```
`Depends()` does not create or cache a factory here. It only exposes the lifespan-owned object. This function is also the narrow seam that tests can override when they need a different factory.
---
## Request Session Dependencies
Use a session-only dependency for reads and other request conversations that must not commit implicitly:
```python
from collections.abc import AsyncGenerator
from sqlmodel.ext.asyncio.session import AsyncSession
async def get_session(session_factory: SessionFactoryDep) -> AsyncGenerator[AsyncSession]:
async with session_factory() as session:
yield session
type SessionDep = Annotated[AsyncSession, Depends(get_session)]
```
The dependency creates and closes one session per request. Closing rolls back any unfinished autobegun transaction; it does not commit.
Use a separate dependency when the whole route is one write transaction:
```python
async def get_transaction_session(session_factory: SessionFactoryDep) -> AsyncGenerator[AsyncSession]:
async with transaction_scope(session_factory) as session:
yield session
type TransactionSessionDep = Annotated[AsyncSession, Depends(get_transaction_session)]
```
This adapter uses `transaction_scope()` from [session management](session.md), so the same root transaction ownership applies inside and outside FastAPI.
Successful dependency exit commits and closes the session. Exceptional exit rolls back and closes it. Route and service code using `TransactionSessionDep` must not call `commit()`, `rollback()`, or `close()`.
---
## Route Usage
Read route:
```python
@router.get("/items/{item_id}")
async def get_item(item_id: int, session: SessionDep) -> Item | None:
return await find_item(session, item_id)
```
Write route:
```python
@router.post("/items")
async def create_item(payload: ItemCreate, session: TransactionSessionDep) -> Item:
return await insert_item(session, payload)
```
Choose one write convention per application:
- inject `TransactionSessionDep` when the route itself is the complete transaction boundary, or
- inject `SessionDep` and place `async with session.begin():` visibly around the service call.
Do not combine both conventions in one route. Lower-level data-access functions continue to require an existing session and remain unaware of FastAPI.
---
## Background Work
A request session belongs to that request and must not be retained by a background task. Inject or otherwise provide the application session factory, then create a new session inside the task:
```python
async def run_background_job(session_factory: SessionFactory) -> None:
async with session_factory.begin() as session:
await process_pending_items(session)
```
If work must survive application shutdown, it needs an independently owned worker lifecycle rather than the FastAPI lifespan-owned factory.
---
## Testing and Overrides
Override the narrow dependency that matches the test objective:
- Override `get_session_factory` to preserve production request-session behavior with a test factory.
- Override `get_session` when a test must inject one transaction-scoped session directly.
- Verify each lifespan receives a fresh engine and session factory and removes application state during teardown.
- Remove overrides during teardown so mutable application state does not leak between tests.
```python
app.dependency_overrides[get_session] = get_test_session
try:
yield app
finally:
app.dependency_overrides.pop(get_session, None)
```
See [database testing](testing.md) for outer transactions, SAVEPOINT-backed fixtures, and database target selection.
---
## Anti-Patterns
- Creating an engine or session factory in a request dependency.
- Reading settings and constructing database resources from repositories.
- Storing one mutable `AsyncSession` on `app.state`.
- Sharing a request session with concurrent or background tasks.
- Calling `commit()` inside a route that uses `TransactionSessionDep`.
- Keeping `app.state.session_factory` after its `database_scope()` exits.
- Using deprecated startup and shutdown event handlers alongside lifespan.
---
## Integration Checklist
- Lifespan enters exactly one `database_scope()` for each application lifecycle.
- Application state stores the yielded session factory.
- Session dependencies create and close one session per request.
- Read and transactional dependencies have distinct commit semantics.
- Routes use `Annotated` aliases and receive sessions, not engines.
- Background tasks create their own sessions from a still-live factory.
- Tests override and restore dependencies deterministically.
@@ -10,6 +10,7 @@ Purpose: concept registry for the principles, mechanics, and implementation guid
|---|---|---|---|---|---| |---|---|---|---|---|---|
| Engine lifecycle and ownership | [engine.md](engine.md) | adopted | mandatory | platform/backend | 2026-06-17 | | Engine lifecycle and ownership | [engine.md](engine.md) | adopted | mandatory | platform/backend | 2026-06-17 |
| Session factory and scope | [session.md](session.md) | adopted | mandatory | platform/backend | 2026-06-17 | | Session factory and scope | [session.md](session.md) | adopted | mandatory | platform/backend | 2026-06-17 |
| FastAPI lifespan and dependency injection | [fastapi.md](fastapi.md) | adopted | mandatory | platform/backend | 2026-07-31 |
| Transaction boundaries | [transactions.md](transactions.md) | adopted | mandatory | platform/backend | 2026-06-17 | | Transaction boundaries | [transactions.md](transactions.md) | adopted | mandatory | platform/backend | 2026-06-17 |
| Implicit ORM I/O under asyncio | [implicit_io.md](implicit_io.md) | adopted | advisory | platform/backend | 2026-06-17 | | Implicit ORM I/O under asyncio | [implicit_io.md](implicit_io.md) | adopted | advisory | platform/backend | 2026-06-17 |
| Observability and resilience | [observability.md](observability.md) | adopted | mandatory | platform/backend | 2026-06-17 | | Observability and resilience | [observability.md](observability.md) | adopted | mandatory | platform/backend | 2026-06-17 |
@@ -1,42 +1,43 @@
# Async SQLAlchemy Session Management # Async SQLAlchemy Session Management
!!! info "Primary sources" !!! info "Primary sources"
- [Python `functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
- [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager) - [Python `asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager)
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html) - [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
- [SQLAlchemy session basics](https://docs.sqlalchemy.org/en/21/orm/session_basics.html) - [SQLAlchemy session basics](https://docs.sqlalchemy.org/en/21/orm/session_basics.html)
- [FastAPI dependencies with yield](https://fastapi.tiangolo.com/tutorial/dependencies/dependencies-with-yield/)
--- ---
## Purpose ## Purpose
Define one canonical session model for FastAPI + SQLAlchemy asyncio: Define one canonical session model for SQLAlchemy asyncio:
- configure one shared session factory, - configure one shared session factory,
- create one AsyncSession per request or per unit-of-work, - create one AsyncSession per task or unit of work,
- never share one AsyncSession across concurrent tasks. - never share one AsyncSession across concurrent tasks.
--- ---
## Scope and Non-Goals ## Scope and Non-Goals
- In scope: session factory creation, FastAPI dependency wiring, request/task scoping, transaction demarcation. - In scope: session factory creation, task scoping, and transaction demarcation.
- Out of scope: ORM model design, query optimization strategy, schema migration tooling. - Out of scope: framework dependency wiring, ORM model design, query optimization strategy, and schema migration tooling.
--- ---
## Rules ## Rules
- Create one cached `async_sessionmaker` per app-owned AsyncEngine. - Create one `async_sessionmaker` inside each app-owned engine scope.
- Let repositories resolve the cached maker by database URL. - Resolve the configured `async_sessionmaker` at the application composition boundary and inject it where standalone operations begin.
- Use a fresh AsyncSession for each request or explicit unit-of-work. - Use a fresh AsyncSession for each task or explicit unit of work.
- Pass an `AsyncSession` directly to data-access functions. - Pass an `AsyncSession` directly to data-access functions.
- Borrow a caller-provided session without closing or committing it. - Require lower-level data-access functions to receive an `AsyncSession`; they must not create sessions or control transactions.
- Treat a supplied session as an explicit declaration that the caller owns an active transaction.
- Borrow a caller-provided session without beginning, closing, committing, or rolling it back.
- Do not share AsyncSession across `asyncio.gather()` or parallel tasks. - Do not share AsyncSession across `asyncio.gather()` or parallel tasks.
- Prefer direct dependency injection over global scoped-session patterns in new code. - Prefer direct dependency injection over global scoped-session patterns in new code.
- Use explicit transaction boundaries (`async with session.begin():`) for writes. - Use explicit transaction boundaries (`async with session.begin():`) for writes.
- When a use case accepts an optional session, borrow only an active caller-owned transaction or own the complete session-and-transaction scope. - When a complete operation accepts an optional session, borrow the caller's active transaction or own the complete session-and-transaction scope.
- Use `begin_nested()` directly and only when partial rollback through a database SAVEPOINT is required.
--- ---
@@ -46,7 +47,7 @@ A session and a transaction solve related but different problems:
| Concept | Responsibility | Typical lifetime | | Concept | Responsibility | Typical lifetime |
| --- | --- | --- | | --- | --- | --- |
| `AsyncSession` | Provides the ORM workspace: executes queries, tracks loaded and changed objects in its identity map, and flushes pending changes. It also coordinates access to a database connection. | One request, task, or explicit unit of work. | | `AsyncSession` | Provides the ORM workspace: executes queries, tracks loaded and changed objects in its identity map, and flushes pending changes. It also coordinates access to a database connection. | One task or explicit unit of work. |
| Transaction | Defines the atomic database boundary: all work inside it commits together on success or rolls back together on failure. | One complete operation that must have a single outcome. | | Transaction | Defines the atomic database boundary: all work inside it commits together on success or rolls back together on failure. | One complete operation that must have a single outcome. |
A transaction belongs to a session; it is not an alternative to one. The session is the interface used by application and data-access code, while the transaction determines when that work becomes permanent. A session may coordinate sequential transactions during its lifetime, although short-lived application scopes commonly use one session for one transaction. A transaction belongs to a session; it is not an alternative to one. The session is the interface used by application and data-access code, while the transaction determines when that work becomes permanent. A session may coordinate sequential transactions during its lifetime, although short-lived application scopes commonly use one session for one transaction.
@@ -76,33 +77,25 @@ For most read-only operations, a session context is sufficient. Use an explicit
An `async_sessionmaker[AsyncSession]` is a reusable configuration object and callable session producer. It stores how sessions should be created, including the engine binding and options such as `expire_on_commit=False`. It is not itself a session, connection, or transaction, and calling it does not make a shared global `AsyncSession`. An `async_sessionmaker[AsyncSession]` is a reusable configuration object and callable session producer. It stores how sessions should be created, including the engine binding and options such as `expire_on_commit=False`. It is not itself a session, connection, or transaction, and calling it does not make a shared global `AsyncSession`.
Cache it by the application-owned engine so repeated composition calls return the same maker: Create it once from the application-owned engine and inject it into application services and dependencies:
```python ```python
from functools import cache from sqlalchemy.ext.asyncio import AsyncEngine
from sqlalchemy.ext.asyncio import async_sessionmaker from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from .engine import dispose_engine type SessionFactory = async_sessionmaker[AsyncSession]
from .engine import get_engine
@cache def create_session_factory(engine: AsyncEngine) -> SessionFactory:
def get_session_factory(database_url: str) -> async_sessionmaker[AsyncSession]:
return async_sessionmaker( return async_sessionmaker(
bind=get_engine(database_url), bind=engine,
class_=AsyncSession, class_=AsyncSession,
expire_on_commit=False, expire_on_commit=False,
) )
async def dispose_session_factory(database_url: str) -> None:
get_session_factory.cache_clear()
await dispose_engine(database_url)
``` ```
`functools.cache` caches by argument equality and requires hashable arguments. The database URL is an explicit string key shared with the cached engine factory. The cache retains the returned maker until `get_session_factory.cache_clear()` runs. Cache the synchronous maker function, never an async function and never a produced `AsyncSession`. The maker is cheap configuration and has no independent connection pool or async cleanup method. Its bound engine owns the pool, so construct the maker inside that engine's lifecycle and do not retain it after the engine scope exits. A global cache adds no value when the composition root creates both resources once.
Each call to `session_factory()` creates a distinct `AsyncSession`. The caller that invokes the factory owns that session lifetime and must close it, normally with `async with`: Each call to `session_factory()` creates a distinct `AsyncSession`. The caller that invokes the factory owns that session lifetime and must close it, normally with `async with`:
@@ -111,123 +104,84 @@ async with session_factory() as session:
... ...
``` ```
The factory can be shared across requests and tasks. Sessions produced by it cannot be shared across concurrent tasks. The factory can be shared across operations and tasks. Sessions produced by it cannot be shared across concurrent tasks.
An `async_sessionmaker` has no connection pool or async `dispose()` method of its own. `dispose_session_factory()` means "invalidate the cached maker, then dispose its engine." Clearing the maker first ensures no subsequent composition call can retrieve a maker bound to the engine being shut down. Passing the factory directly has three useful consequences:
Use the helper when shutting down or replacing the database resources: - Lower layers do not resolve settings or global resources.
- Tests can inject a test factory directly.
```python - Transaction ownership remains independent of engine construction.
await dispose_session_factory(database_url)
```
Otherwise, a later call can return a maker that still references the old engine object. This matters in lifespan tests, application restarts within one process, and test suites that replace engines.
--- ---
## Optional Session Ownership ## Minimal Scope Model
A small [`asynccontextmanager`](https://docs.python.org/3/library/contextlib.html#contextlib.asynccontextmanager) can make repository methods composable. It borrows an existing session when supplied; otherwise it creates and closes one from a supplied factory: Most applications need only these three forms:
1. `session_factory()` for a standalone read or other session-only conversation.
2. One `transaction_scope()` helper for a complete operation that may either own a transaction or join its caller's transaction.
3. `session.begin_nested()` at the exact call site that needs partial rollback through a SAVEPOINT.
Do not add a general `atomic_scope()` abstraction. The word "atomic" does not reveal whether the scope joins an outer transaction, creates and commits a root transaction, or creates a SAVEPOINT. Those behaviors have different failure and ownership semantics and should remain visible.
### One optional-ownership helper
```python ```python
from collections.abc import AsyncGeneratorr from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.ext.asyncio import async_sessionmaker
@asynccontextmanager type SessionFactory = async_sessionmaker[AsyncSession]
async def session_scope(
*,
database_url: str,
session: AsyncSession | None = None,
) -> AsyncGenerator[AsyncSession]:
if session is not None:
yield session
return
async with get_session_factory(database_url)() as owned_session:
yield owned_session
```
The branch is intentionally explicit. Python's [`nullcontext`](https://docs.python.org/3/library/contextlib.html#contextlib.nullcontext) can express the same borrow-or-own idea, but the branch keeps ownership and typing obvious.
This helper manages session lifetime only:
- It does not close, commit, or roll back a supplied session; the caller owns it.
- It closes a session that it creates. Closing releases resources and rolls back an unfinished transaction; it does not commit.
- It does not start a transaction. Put `session.begin()` at the use-case boundary.
- A supplied session wins; the cached factory is not resolved.
- Otherwise, `database_url` selects the cached factory returned by `get_session_factory()`.
Do not turn this into an implicit unit-of-work helper that sometimes commits. Whether work joins an existing transaction or creates a new one must remain visible to the caller.
---
## Optional Transaction Ownership
Use a separate context manager when a service or use-case function must support both a caller-owned transaction and a standalone transaction. A supplied session must already be inside a transaction; otherwise the helper creates a session and transaction together with `async_sessionmaker.begin()`:
```python
@asynccontextmanager @asynccontextmanager
async def transaction_scope( async def transaction_scope(
session_factory: SessionFactory,
*, *,
database_url: str,
session: AsyncSession | None = None, session: AsyncSession | None = None,
) -> AsyncGenerator[AsyncSession]: ) -> AsyncGenerator[AsyncSession]:
if session is not None: if session is not None:
if not session.in_transaction(): if not session.in_transaction():
raise RuntimeError("A supplied session must have an active transaction") raise RuntimeError("A supplied session must have an active transaction")
yield session yield session
return return
session_factory = get_session_factory(database_url)
async with session_factory.begin() as owned_session: async with session_factory.begin() as owned_session:
yield owned_session yield owned_session
``` ```
Here, `begin()` is intentionally called on the [`async_sessionmaker`](https://docs.sqlalchemy.org/en/20/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin), not on an existing `AsyncSession`. The related APIs have different ownership semantics: The explicit branch is preferable to compressing both paths through `nullcontext()` or a mode-driven helper. It makes the ownership transition obvious and keeps type narrowing straightforward. Its runtime cost is negligible compared with database I/O.
- `session_factory()` creates a session whose lifetime the surrounding code must manage; it does not commit automatically. The two paths have deliberately different responsibilities:
- `session_factory.begin()` creates a new session and transaction together, commits on successful exit or rolls back on exceptional exit, and then closes the session.
- `session.begin()` manages a transaction on an existing session but does not own or close that session.
The factory form is equivalent in ownership terms to creating a session and then entering that session's transaction: | Input | Session owner | Transaction owner | Successful exit | Exceptional exit |
| --- | --- | --- | --- | --- |
| `session=None` | Helper | Helper | Flush, commit, then close | Roll back, then close |
| Existing `session` | Caller | Caller | Yield control back to caller | Propagate to caller without cleanup |
```python [`async_sessionmaker.begin()`](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html#sqlalchemy.ext.asyncio.async_sessionmaker.begin) is the right primitive for the owned path because it creates the session and root transaction together, commits on successful exit, rolls back on exceptional exit, and closes the session. It is equivalent in ownership terms to nesting `session_factory()` and `owned_session.begin()` context managers.
async with session_factory() as owned_session:
async with owned_session.begin():
yield owned_session
```
This helper makes transaction ownership follow the same explicit borrow-or-own mechanics as session ownership: Do not call `session.begin()` when a session is supplied. A supplied session means the caller has already chosen the transaction boundary. Silently beginning a transaction would make commit ownership depend on hidden branch behavior and would fail when the session was already active.
- A supplied session and its active transaction remain caller-owned. The helper does not commit, roll back, or close them. ### Autobegin and the defensive check
- Without a supplied session, the helper owns the session and transaction. Successful exit commits; exceptional exit rolls back; either path closes the session.
- Use this helper only at a complete operation, service, or use-case boundary. A public CRUD function or repository method may be such a boundary when its optional-session contract explicitly states that omitting the session owns and commits one transaction. Never use it inside a lower-level session-required helper.
- Do not silently begin a transaction on a supplied session. That would make commit ownership depend on hidden helper behavior.
Callers that supply a session make their ownership visible with an outer transaction: The `session.in_transaction()` check catches the obvious contract violation of passing an unused session without an outer transaction. It does not prove that the caller intentionally opened a transaction.
```python SQLAlchemy's [autobegin](https://docs.sqlalchemy.org/en/21/orm/session_basics.html#auto-begin) behavior starts transactional state after operations such as `execute()`, `add()`, or modifying a persistent object. A preceding read can therefore make `in_transaction()` return `True`. The real ownership signal is the API call itself: passing `session=` declares that the caller owns the active transaction.
async with session_factory() as session:
async with session.begin():
await run_use_case(..., session=session)
```
Standalone callers omit the session and let the use case own the complete unit of work: Applications that require mechanical enforcement can construct sessions with `autobegin=False`, but then every database conversation, including reads and every post-commit reuse, must begin explicitly. That stricter policy is valid but is not the minimalist default.
```python
await run_use_case(...)
```
--- ---
## Repository and Function Boundaries ## Function and Service Boundaries
Pass the database URL to repository constructors. The repository stores repeatable database configuration, not mutable session state, and `session_scope()` resolves the cached factory when a standalone operation needs a session: Lower-level functions should require a session and contain only data-access behavior:
```python ```python
from sqlalchemy import select from sqlalchemy import select
from sqlmodel.ext.asyncio.session import AsyncSession
async def find_item(session: AsyncSession, item_id: int) -> Item | None: async def find_item(session: AsyncSession, item_id: int) -> Item | None:
@@ -235,88 +189,109 @@ async def find_item(session: AsyncSession, item_id: int) -> Item | None:
return await session.scalar(statement) return await session.scalar(statement)
class ItemRepository: async def insert_order(session: AsyncSession, payload: OrderCreate) -> Order:
def __init__(self, database_url: str) -> None: order = Order.model_validate(payload)
self.database_url = database_url session.add(order)
await session.flush()
async def find( return order
self,
item_id: int,
*,
session: AsyncSession | None = None,
) -> Item | None:
async with session_scope(
database_url=self.database_url,
session=session,
) as active_session:
return await find_item(active_session, item_id)
``` ```
This split gives each layer one job: These functions do not create, close, commit, roll back, or nest transactions. This keeps them composable and makes transaction behavior a property of the calling use case rather than the query helper.
- The repository object identifies its database configuration and creates a session only for a standalone call. A complete write operation may accept an optional session and use `transaction_scope()`:
- Standalone calls reuse the cached factory selected by database URL.
- A caller can pass a session to join an existing unit of work; the repository borrows it.
- The access function owns only the query and requires an existing `AsyncSession`.
- Application wiring supplies the production factory.
- Tests can use a test database URL or call `find_item()` with a transaction-scoped test session.
When several repository operations must share one transaction, pass the same session through each call. Put the transaction at the use-case boundary: ```python
async def create_order(
session_factory: SessionFactory,
payload: OrderCreate,
*,
session: AsyncSession | None = None,
) -> Order:
async with transaction_scope(
session_factory,
session=session,
) as active_session:
return await insert_order(active_session, payload)
```
The standalone call owns and commits its work:
```python
order = await create_order(session_factory, payload)
```
A larger use case owns one transaction and passes the same session through every operation:
```python
async with transaction_scope(session_factory) as session:
order = await create_order(
session_factory,
payload,
session=session,
)
await reserve_inventory(session, order)
await create_audit_entry(session, order)
```
The inner `create_order()` scope joins the existing transaction; it does not commit and does not create a SAVEPOINT. If inventory reservation or audit creation fails, the outer scope rolls back all three operations together. This is ordinary service composition, not a nested database transaction.
For standalone reads, use the factory directly rather than routing through a transaction-owning helper:
```python ```python
async with session_factory() as session: async with session_factory() as session:
async with session.begin(): item = await find_item(session, item_id)
item = await repository.find(item_id, session=session)
await update_item(session, item, changes)
``` ```
This preserves atomicity without making repository objects hold mutable `AsyncSession` instances across calls. The session context closes the session and rolls back any unfinished autobegun transaction. It does not commit. If a public read operation supports a caller-supplied session, keep the small borrow-or-create branch in that operation; do not disguise it as transaction ownership.
Application service objects that represent standalone operations may store the immutable session factory, but they must not store a mutable session:
```python
class ItemService:
def __init__(self, session_factory: SessionFactory) -> None:
self.session_factory = session_factory
async def find(self, item_id: int) -> Item | None:
async with self.session_factory() as session:
return await find_item(session, item_id)
```
Code that already owns a transaction should call the session-required function directly. Repositories should normally remain in that session-required layer; the service or use-case boundary owns standalone session creation. This avoids optional-session APIs spreading into every data-access function.
--- ---
## Canonical FastAPI Dependency Pattern ## SAVEPOINTs and Partial Failure
Use [`begin_nested()`](https://docs.sqlalchemy.org/en/21/orm/session_transaction.html#using-savepoint) only when failure inside one portion of an operation should roll back that portion while preserving the outer transaction:
```python ```python
from collections.abc import AsyncGenerator async with transaction_scope(session_factory) as session:
order = await insert_order(session, payload)
from fastapi import Depends try:
from fastapi import Request async with session.begin_nested():
from sqlalchemy.ext.asyncio import async_sessionmaker await apply_optional_discount(session, order)
from sqlmodel.ext.asyncio.session import AsyncSession except DiscountError:
pass
await reserve_inventory(session, order)
type SessionFactory = async_sessionmaker[AsyncSession]
def resolve_session_factory(request: Request) -> SessionFactory:
return get_session_factory(request.app.state.settings.database_url)
async def get_db_session(
session_factory: SessionFactory = Depends(resolve_session_factory),
) -> AsyncGenerator[AsyncSession]:
async with session_factory() as session:
yield session
``` ```
Route usage: Important SAVEPOINT semantics:
```python - `begin_nested()` starts a root transaction if one is not already active, so call it inside a visible outer transaction when that ownership matters.
from fastapi import APIRouter, Depends - Entering `begin_nested()` unconditionally flushes pending session state, regardless of the `autoflush` setting.
from sqlmodel.ext.asyncio.session import AsyncSession - Successful exit releases the SAVEPOINT; it does not commit the outer transaction.
- Exceptional exit rolls back to the SAVEPOINT and leaves the outer transaction active.
- In SQLAlchemy 2.x, `session.commit()` commits the outermost transaction. Never call it to release a SAVEPOINT; let the nested context manager manage its transaction handle.
from .session import get_db_session Do not create a SAVEPOINT merely because one service calls another. SAVEPOINTs add database work and alter flush and error-recovery behavior. Use them only for explicit partial-failure requirements such as skipping one conflicting row while retaining the rest of a batch.
router = APIRouter() ---
## Framework Integration
@router.post("/items") Keep framework adapters outside these session primitives. See [FastAPI database integration](fastapi.md) for lifespan ownership, `Annotated` dependency aliases, and read-versus-write request sessions.
async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
async with session.begin():
# write operations here
...
return {"status": "ok"}
```
--- ---
@@ -328,9 +303,9 @@ async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
## SQLModel Alignment ## SQLModel Alignment
- Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per request/unit-of-work. - Use SQLModel as the default model and statement layer while keeping the same session ownership model: one `async_sessionmaker`, one `AsyncSession` per task or unit of work.
- SQLModel does not replace SQLAlchemy async lifecycle primitives; it provides model declaration, validation, and typing ergonomics on top of them. - SQLModel does not replace SQLAlchemy async lifecycle primitives; it provides model declaration, validation, and typing ergonomics on top of them.
- Do not mix ad hoc session construction with the canonical async dependency. - Do not mix ad hoc session construction with the canonical session factory.
--- ---
@@ -344,41 +319,44 @@ async def create_item(session: AsyncSession = Depends(get_db_session)) -> dict:
## Anti-Patterns ## Anti-Patterns
- A singleton/global AsyncSession reused across requests. - A singleton/global AsyncSession reused across tasks or operations.
- Sharing one AsyncSession across parallel tasks. - Sharing one AsyncSession across parallel tasks.
- Passing an application-global AsyncSession to a repository constructor. - Passing an application-global AsyncSession to a repository constructor.
- Caching an `AsyncSession` instead of caching `async_sessionmaker`. - Creating a new `async_sessionmaker` in each operation.
- Leaving a cached maker pointing at a disposed or replaced engine. - Retaining a session factory after its bound engine scope exits.
- Calling the session factory inside low-level access functions such as `find_item()`. - Calling the session factory inside low-level access functions such as `find_item()`.
- Hidden session creation in lower access functions with no caller control. - Hidden session creation in lower access functions with no caller control.
- Closing or committing a session supplied by the caller. - Closing or committing a session supplied by the caller.
- Starting a new transaction inside a helper that may receive a session already in a transaction. - Starting a new transaction inside a helper that may receive a session already in a transaction.
- Silently starting or committing a transaction on a supplied session. - Silently starting or committing a transaction on a supplied session.
- Treating `in_transaction()` as proof that a caller intentionally owns the transaction.
- Creating a SAVEPOINT for ordinary nested service calls.
- Hiding root transaction, joined transaction, and SAVEPOINT behavior behind one mode-driven `atomic_scope()` helper.
- Calling `session.commit()` inside a SAVEPOINT scope.
- Mixing commit/rollback ownership across layers without a declared boundary. - Mixing commit/rollback ownership across layers without a declared boundary.
--- ---
## Operational Checks ## Operational Checks
- Exactly one cached `async_sessionmaker` exists per application engine. - Exactly one `async_sessionmaker` is configured inside each application engine scope.
- Session factory caches are cleared before their engines are disposed or replaced. - The session factory does not outlive its bound engine.
- Request handlers receive sessions from one canonical dependency. - Application operations receive sessions from one canonical session factory.
- No code path creates AsyncSession in module import side effects. - No code path creates AsyncSession in module import side effects.
- Background jobs and API handlers each create task-local sessions. - Concurrent jobs and operations each create task-local sessions.
--- ---
## Testing Checks ## Testing Checks
- Repository constructors accept a test database URL without FastAPI startup. - Service constructors accept a test session factory without framework startup.
- Session-taking access functions accept a transaction-scoped test session directly. - Session-taking access functions accept a transaction-scoped test session directly.
- Optional-session tests verify that borrowed sessions remain open and created sessions close. - Transaction-scope tests verify supplied sessions require an active transaction and remain caller-owned.
- Optional-session tests verify that neither path commits implicitly. - Transaction-scope tests verify owned transactions commit on success, roll back on failure, and close their sessions.
- Optional-transaction tests verify supplied sessions require an active transaction and remain caller-owned. - Composition tests verify nested service calls join one outer transaction without committing it.
- Optional-transaction tests verify owned transactions commit on success, roll back on failure, and close their sessions. - SAVEPOINT tests verify local rollback preserves the outer transaction and successful exit does not commit it.
- Cache tests clear `get_session_factory` before and after replacing engines. - Tests that depend on SAVEPOINT timing account for `begin_nested()` flushing pending state on entry.
- Dependency override exists for the FastAPI session factory.
- Rollback behavior is verified for failed write units. - Rollback behavior is verified for failed write units.
- Parallel-task tests verify no shared AsyncSession instances. - Parallel-task tests verify no shared AsyncSession instances.
- Lifespan tests confirm session factory is initialized and teardown-safe. - Lifecycle tests confirm the session factory is initialized and teardown-safe.
@@ -72,18 +72,15 @@ class UserRead(UserBase):
### Pattern B: Keep SQLModel models with the async runtime ### Pattern B: Keep SQLModel models with the async runtime
```python ```python
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlmodel import select from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
engine = create_async_engine(settings.database_url, pool_pre_ping=True) async with engine_scope(settings.database_url) as engine:
session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False) session_factory = create_session_factory(engine)
async with session_factory() as session:
async with session_factory() as session: users = (await session.scalars(select(User))).all()
users = (await session.scalars(select(User))).all()
``` ```
`sqlmodel.select()` keeps SQLModel's typing-oriented statement construction, and SQLModel's `AsyncSession` adds typed `exec()` results while retaining SQLAlchemy's async lifecycle and transaction behavior. Import `AsyncSession` from `sqlmodel.ext.asyncio.session` when working with SQLModel models; use SQLAlchemy's `AsyncSession` only when the code intentionally has no SQLModel dependency. `engine_scope()` and `create_session_factory()` preserve the canonical lifecycle while SQLModel supplies the model and statement layer. `sqlmodel.select()` keeps SQLModel's typing-oriented statement construction, and SQLModel's `AsyncSession` adds typed `exec()` results while retaining SQLAlchemy's async lifecycle and transaction behavior. Import `AsyncSession` from `sqlmodel.ext.asyncio.session` when working with SQLModel models; use SQLAlchemy's `AsyncSession` only when the code intentionally has no SQLModel dependency.
--- ---
@@ -1,94 +1,112 @@
# Testing Database Targets and Data # Testing Database Targets and Data
Use the same application database construction path in production and tests. Tests select a different URL and bind their request-session dependency to a test-scoped transaction; they do not replace repositories, services, or SQLAlchemy mechanics with mocks. Use the same engine and session primitives in production and tests. Tests select a different URL and, when transaction isolation is required, bind a test session factory to one test-owned connection and outer transaction. They do not replace repositories, services, or SQLAlchemy mechanics with mocks.
## Decision Table ## Decision Table
| Test need | Database target | Isolation approach | What it proves | | Test need | Database target | Isolation approach | What it proves |
|---|---|---|---| |---|---|---|---|
| Fast, serial application tests | `sqlite+aiosqlite://` | Per-test engine or outer transaction | ORM mappings and ordinary application behavior | | Fast, serial application tests | `sqlite+aiosqlite://` | Per-test engine or connection-bound session factory over an outer transaction | ORM mappings and ordinary application behavior |
| Async code using multiple simultaneous sessions | Named SQLite shared-cache URL or temporary SQLite file | Per-test schema or cleanup strategy | Concurrent-session behavior without a database server | | Async code using multiple simultaneous sessions | Named SQLite shared-cache URL or temporary SQLite file | Per-test schema or cleanup strategy | Concurrent-session behavior without a database server |
| PostgreSQL-specific behavior | Dedicated PostgreSQL test database | Per-test outer transaction and SAVEPOINT | SQL, constraints, types, locking, and migrations that SQLite cannot represent | | PostgreSQL-specific behavior | Dedicated PostgreSQL test database | Per-test outer transaction and SAVEPOINT | SQL, constraints, types, locking, and migrations that SQLite cannot represent |
SQLite is a useful fast target, not a drop-in PostgreSQL substitute. Keep a small PostgreSQL integration suite for PostgreSQL-specific queries, extensions, row locking, JSON semantics, collations, isolation, and migration validation. SQLite is a useful fast target, not a drop-in PostgreSQL substitute. Keep a small PostgreSQL integration suite for PostgreSQL-specific queries, extensions, row locking, JSON semantics, collations, isolation, and migration validation.
## One Construction Path ## Shared Construction Primitives
Make the application factory accept a database URL or settings object, and keep engine and session-factory construction in one function. The only test-specific inputs should be the URL and, for request tests, the session dependency override. Make the application factory accept a database URL or settings object. Production, workers, and ordinary integration tests enter the same [`database_scope()`](engine.md#one-engine-context-manager). Tests enter the lower-level `engine_scope()` only when they need direct engine or connection ownership for schema setup, an outer transaction, or engine-specific assertions:
```python ```python
from sqlalchemy.ext.asyncio import AsyncEngine, async_sessionmaker, create_async_engine from collections.abc import AsyncGenerator
from sqlmodel.ext.asyncio.session import AsyncSession
import pytest_asyncio
from sqlalchemy.ext.asyncio import AsyncEngine
from .engine import engine_scope
def create_database( @pytest_asyncio.fixture
database_url: str, async def test_engine(database_url: str) -> AsyncGenerator[AsyncEngine]:
) -> tuple[AsyncEngine, async_sessionmaker[AsyncSession]]: async with engine_scope(database_url) as engine:
engine = create_async_engine(database_url) yield engine
session_factory = async_sessionmaker(
engine,
class_=AsyncSession,
expire_on_commit=False,
)
return engine, session_factory
``` ```
Production passes its `postgresql+asyncpg://...` URL to `create_database()`. A local SQLite run passes `sqlite+aiosqlite:///./app.db`. Tests pass a dedicated test URL to the same function. Do not create an engine during module import: that makes it easy for tests to accidentally retain the production URL before an override is applied. Production passes its `postgresql+asyncpg://...` URL to `database_scope()`. A local SQLite run passes `sqlite+aiosqlite:///./app.db`. Tests pass a dedicated test URL to `database_scope()` or `engine_scope()` and receive deterministic disposal when the context exits. Do not create an engine during module import: that makes it easy for tests to retain the production URL before an override is applied.
Use migrations to provision an integration database when migrations are part of the release contract. `metadata.create_all()` is appropriate for focused ORM tests only when it accurately represents the schema under test. Import all table models before creating metadata; [SQLModel documents that model-registration order matters](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/#import-table-models). Use migrations to provision an integration database when migrations are part of the release contract. `metadata.create_all()` is appropriate for focused ORM tests only when it accurately represents the schema under test. Import all table models before creating metadata; [SQLModel documents that model-registration order matters](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/#import-table-models).
## Transactional Async Fixture ## Transactional Async Fixture
For tests that exercise code which calls `commit()`, start an outer transaction on one test connection. Bind the test `AsyncSession` to that connection and use `join_transaction_mode="create_savepoint"`. SQLAlchemy documents this as its test-suite pattern: session commits resolve a SAVEPOINT while fixture teardown rolls back the outer transaction. For tests that exercise code which commits, start an outer transaction on one test connection. Bind a test `SessionFactory` to that connection with `join_transaction_mode="create_savepoint"`. SQLAlchemy documents this as its test-suite pattern: sessions created by the factory resolve their commits through SAVEPOINTs while fixture teardown rolls back the outer transaction.
```python ```python
from collections.abc import AsyncGeneratorr from collections.abc import AsyncGenerator
import pytest_asyncio import pytest_asyncio
from sqlalchemy.ext.asyncio import AsyncEngine from sqlalchemy.ext.asyncio import AsyncEngine
from sqlalchemy.ext.asyncio import async_sessionmaker
from sqlmodel.ext.asyncio.session import AsyncSession from sqlmodel.ext.asyncio.session import AsyncSession
from .session import SessionFactory
@pytest_asyncio.fixture @pytest_asyncio.fixture
async def session(test_engine: AsyncEngine) -> AsyncGenerator[AsyncSession]: async def session_factory(test_engine: AsyncEngine) -> AsyncGenerator[SessionFactory]:
async with test_engine.connect() as connection: async with test_engine.connect() as connection:
transaction = await connection.begin() transaction = await connection.begin()
test_session = AsyncSession( factory = async_sessionmaker(
bind=connection, bind=connection,
class_=AsyncSession,
expire_on_commit=False, expire_on_commit=False,
join_transaction_mode="create_savepoint", join_transaction_mode="create_savepoint",
) )
try: try:
yield test_session yield factory
finally: finally:
await test_session.close()
await transaction.rollback() await transaction.rollback()
``` ```
Use the test session through the normal FastAPI dependency seam, and always remove the override after the test. [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/) are an application-level dictionary, so leaving one installed leaks test state. Each factory call still creates a distinct `AsyncSession`, matching [session factory mechanics](session.md#session-factory-mechanics). The factory belongs to the fixture's engine and outer transaction and must not escape either scope.
For service tests that need to pass a caller-owned active session into `transaction_scope(session=...)`, derive that session from the same factory:
```python ```python
@pytest_asyncio.fixture
async def session(session_factory: SessionFactory) -> AsyncGenerator[AsyncSession]:
async with session_factory() as test_session:
await test_session.begin()
yield test_session
```
The explicit `begin()` satisfies the supplied-session contract from [session management](session.md#one-optional-ownership-helper). Session closure rolls back unfinished work; the outer connection transaction remains the final isolation boundary even if application code commits its SAVEPOINT.
For FastAPI request tests, override `get_session_factory`, not only `get_session`. Both `SessionDep` and `TransactionSessionDep` then retain their production ownership behavior while receiving the test-bound factory. Always remove the override after the test because [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/) are stored in a mutable application-level dictionary.
```python
from collections.abc import Generator
import pytest import pytest
from fastapi import FastAPI from fastapi import FastAPI
from sqlmodel.ext.asyncio.session import AsyncSession
from .fastapi import get_session_factory
from .session import SessionFactory
@pytest.fixture @pytest.fixture
def app_with_test_session( def app_with_test_database(app: FastAPI, session_factory: SessionFactory) -> Generator[FastAPI]:
app: FastAPI, def get_test_session_factory() -> SessionFactory:
session: AsyncSession, return session_factory
) -> FastAPI:
async def get_test_session() -> AsyncGenerator[AsyncSession]:
yield session
app.dependency_overrides[get_session] = get_test_session app.dependency_overrides[get_session_factory] = get_test_session_factory
try: try:
yield app yield app
finally: finally:
app.dependency_overrides.clear() app.dependency_overrides.pop(get_session_factory, None)
``` ```
This fixture is deliberately serial: one mutable `AsyncSession` must not serve concurrent tasks. A test that verifies concurrently active sessions should create independent sessions from a factory and use a database target that supports independent connections. Construct `app` with test settings before lifespan starts so startup cannot resolve the production URL. The override changes request session creation; it does not prevent lifespan from entering its configured `database_scope()`.
The connection-bound factory is deliberately serial even though it creates distinct sessions: those sessions still share one connection and outer transaction. A test that verifies concurrently active sessions must use independent connections and a database target that supports them.
## SQLite Targets ## SQLite Targets
@@ -99,19 +117,21 @@ Use `sqlite+aiosqlite://` for a fresh in-memory database when the test runs all
Create the schema and dispose the engine deterministically: Create the schema and dispose the engine deterministically:
```python ```python
from collections.abc import AsyncGenerator
import pytest_asyncio import pytest_asyncio
from sqlalchemy.ext.asyncio import AsyncEngine from sqlalchemy.ext.asyncio import AsyncEngine
from sqlmodel import SQLModel
from .engine import engine_scope
@pytest_asyncio.fixture @pytest_asyncio.fixture
async def test_engine() -> AsyncGenerator[AsyncEngine]: async def test_engine() -> AsyncGenerator[AsyncEngine]:
engine, _ = create_database("sqlite+aiosqlite://") async with engine_scope("sqlite+aiosqlite://") as engine:
async with engine.begin() as connection: async with engine.begin() as connection:
await connection.run_sync(SQLModel.metadata.create_all) await connection.run_sync(SQLModel.metadata.create_all)
try:
yield engine yield engine
finally:
await engine.dispose()
``` ```
### Concurrent in-memory tests ### Concurrent in-memory tests
@@ -140,8 +160,9 @@ For both SQLite forms, enable and test the constraints your application depends
## Completion Checks ## Completion Checks
- A test run cannot reach the production URL; production credentials are absent from the test environment. - A test run cannot reach the production URL; production credentials are absent from the test environment.
- Production PostgreSQL, local SQLite, and in-memory SQLite all use the same engine/session-factory construction path. - Production PostgreSQL, local SQLite, and in-memory SQLite all use `database_scope()` unless a test explicitly needs lower-level engine or connection ownership.
- Every test owns its override, connection, transaction, session, and engine cleanup. - Every test owns its override, session factory, connection, transaction, session, and engine cleanup.
- Request tests override `get_session_factory`, preserving both read-session and transactional-session dependency behavior.
- Test data is deterministic, minimal, and expresses the scenario under test. - Test data is deterministic, minimal, and expresses the scenario under test.
- PostgreSQL integration tests cover every PostgreSQL-specific contract and run against migrations where migrations are shipped. - PostgreSQL integration tests cover every PostgreSQL-specific contract and run against migrations where migrations are shipped.
@@ -151,4 +172,5 @@ For both SQLite forms, enable and test the constraints your application depends
- [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html) - [SQLAlchemy asyncio extension](https://docs.sqlalchemy.org/en/21/orm/extensions/asyncio.html)
- [SQLAlchemy SQLite dialect and async in-memory pooling](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#using-a-memory-database-with-multiple-coroutines) - [SQLAlchemy SQLite dialect and async in-memory pooling](https://docs.sqlalchemy.org/en/21/dialects/sqlite.html#using-a-memory-database-with-multiple-coroutines)
- [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/) - [FastAPI dependency overrides](https://fastapi.tiangolo.com/advanced/testing-dependencies/)
- [SQLModel testing with FastAPI](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/) - [SQLModel testing with FastAPI](https://sqlmodel.tiangolo.com/tutorial/fastapi/tests/)
- [pytest-asyncio fixtures](https://pytest-asyncio.readthedocs.io/en/stable/how-to-guides/index.html)
@@ -126,7 +126,7 @@ def get_settings() -> Settings:
The argument-free [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) provider is appropriate here because both the project entry point and Uvicorn's zero-argument factory need process-lifetime access. Each reload or worker process gets its own settings instance. Do not add override arguments to `get_settings()`; inject a `Settings` instance directly into `create_app()` in tests or alternate composition roots. See the [Pydantic settings implementation guide](../../pydantic-settings/SKILL.md) for source precedence, independent settings boundaries, cache clearing, and runtime reload guidance. The argument-free [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) provider is appropriate here because both the project entry point and Uvicorn's zero-argument factory need process-lifetime access. Each reload or worker process gets its own settings instance. Do not add override arguments to `get_settings()`; inject a `Settings` instance directly into `create_app()` in tests or alternate composition roots. See the [Pydantic settings implementation guide](../../pydantic-settings/SKILL.md) for source precedence, independent settings boundaries, cache clearing, and runtime reload guidance.
```python title="src/my_app/main.py" ```python title="src/my_app/main.py"
from collections.abc import AsyncGeneratorr from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
import uvicorn import uvicorn
+25 -24
View File
@@ -270,36 +270,32 @@ After settings validation, select an async SQLAlchemy driver URL. This is a pure
from functools import cache from functools import cache
from sqlalchemy import URL from sqlalchemy import URL
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine from sqlalchemy.ext.asyncio import AsyncEngine
from sqlalchemy.ext.asyncio import create_async_engine
def get_database_url( def get_database_url(settings: Settings) -> str:
settings: Settings, match settings.database:
) -> str: case SqliteSettings(path=path):
match settings.database: url = URL.create(
case SqliteSettings(path=path): drivername="sqlite+aiosqlite",
url = URL.create( database=path,
drivername="sqlite+aiosqlite", )
database=path, case PostgresSettings() as database:
) url = URL.create(
case PostgresSettings() as database: drivername="postgresql+asyncpg",
url = URL.create( host=database.host,
drivername="postgresql+asyncpg", port=database.port,
host=database.host, database=database.database,
port=database.port, username=database.user,
database=database.database, password=database.password.get_secret_value(),
user=database.user, )
password=database.password.get_secret_value(), return url.render_as_string(hide_password=False)
)
return url.render_as_string(hide_password=False)
@cache @cache
def get_engine(database_url: str) -> AsyncEngine: def get_engine(database_url: str) -> AsyncEngine:
return create_async_engine( return create_async_engine(database_url, pool_pre_ping=True)
database_url,
pool_pre_ping=True,
)
async def dispose_engine(database_url: str) -> None: async def dispose_engine(database_url: str) -> None:
@@ -308,6 +304,11 @@ async def dispose_engine(database_url: str) -> None:
await engine.dispose() await engine.dispose()
finally: finally:
get_engine.cache_clear() get_engine.cache_clear()
async def refresh_engine(database_url: str) -> AsyncEngine:
await dispose_engine(database_url)
return get_engine(database_url)
``` ```
At the composition boundary, resolve the URL once with `get_database_url(settings)` and use it to retrieve the cached engine. In FastAPI, expose that engine through lifespan and build one `async_sessionmaker` from it; each request or unit of work then creates its own `AsyncSession`. Do not call `aiosqlite.connect()` or `asyncpg.create_pool()` directly: `aiosqlite` and `asyncpg` are selected as SQLAlchemy drivers by the URL, while SQLAlchemy owns pooling, disposal, and session integration. At the composition boundary, resolve the URL once with `get_database_url(settings)` and use it to retrieve the cached engine. In FastAPI, expose that engine through lifespan and build one `async_sessionmaker` from it; each request or unit of work then creates its own `AsyncSession`. Do not call `aiosqlite.connect()` or `asyncpg.create_pool()` directly: `aiosqlite` and `asyncpg` are selected as SQLAlchemy drivers by the URL, while SQLAlchemy owns pooling, disposal, and session integration.