345 lines
13 KiB
Markdown
345 lines
13 KiB
Markdown
---
|
|
name: pydantic-settings
|
|
description: "Practical guide for implementing typed application configuration with pydantic-settings. Use when designing BaseSettings models, choosing nested or independent settings boundaries, managing settings lifecycles, configuring dotenv or secrets, and customizing source priority safely."
|
|
x-personal-mcp:
|
|
id: pydantic-settings
|
|
version: 1.1.0
|
|
tags:
|
|
- python
|
|
- pydantic
|
|
- pydantic-settings
|
|
- configuration
|
|
- env-vars
|
|
- secrets
|
|
- dotenv
|
|
- source-priority
|
|
- caching
|
|
- lifecycle
|
|
capabilities:
|
|
- resource://skills/pydantic-settings/document
|
|
---
|
|
|
|
# Pydantic Settings Implementation Guide
|
|
|
|
Use this skill to implement robust, typed application configuration with `pydantic-settings` in production Python services.
|
|
|
|
## When to Use
|
|
|
|
- You need a single typed configuration model for app settings.
|
|
- You are migrating from ad-hoc `os.getenv(...)` calls.
|
|
- You need predictable precedence across init args, env vars, dotenv files, and secrets.
|
|
- You need nested settings models and reliable parsing behavior.
|
|
- You need to choose between one nested application settings object and independently owned settings objects.
|
|
- You need a deliberate construction, caching, or reload lifecycle.
|
|
- You need to customize settings sources or source order safely.
|
|
|
|
## Procedure
|
|
|
|
### 1. Baseline Model
|
|
|
|
Create a single settings model for the service boundary:
|
|
|
|
```python
|
|
from pydantic import BaseModel, Field
|
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
|
|
|
|
|
class DatabaseSettings(BaseModel):
|
|
host: str = "localhost"
|
|
port: int = 5432
|
|
user: str
|
|
password: str
|
|
|
|
|
|
class Settings(BaseSettings):
|
|
model_config = SettingsConfigDict(
|
|
env_prefix="APP_",
|
|
env_file=".env",
|
|
env_file_encoding="utf-8",
|
|
extra="ignore",
|
|
frozen=True,
|
|
)
|
|
|
|
debug: bool = False
|
|
log_level: str = "info"
|
|
database: DatabaseSettings
|
|
api_key: str = Field(validation_alias="MY_API_KEY")
|
|
```
|
|
|
|
Quality gate:
|
|
|
|
1. Required fields fail fast when missing.
|
|
2. Defaults are intentional and safe.
|
|
|
|
### 2. Pick Env Naming Rules
|
|
|
|
1. Choose one prefix and apply it consistently.
|
|
2. Use aliases only for compatibility or external contracts.
|
|
3. Document whether env names are case-sensitive.
|
|
|
|
Quality gate:
|
|
|
|
1. Team can derive env variable names without guessing.
|
|
2. Legacy names are supported only where needed.
|
|
|
|
### 3. Decide Nested Parsing
|
|
|
|
For nested models via env vars, configure delimiters intentionally:
|
|
|
|
```python
|
|
model_config = SettingsConfigDict(
|
|
env_prefix="APP_",
|
|
env_nested_delimiter="__",
|
|
env_nested_max_split=1,
|
|
)
|
|
```
|
|
|
|
Typical vars:
|
|
|
|
1. `APP_DATABASE={"host": "db", "port": 5432, "user": "svc", "password": "pw"}`
|
|
2. `APP_DATABASE__HOST=db.internal`
|
|
|
|
Quality gate:
|
|
|
|
1. Nested overrides behave as expected.
|
|
2. Delimiter choice does not collide with field names.
|
|
|
|
### 4. Confirm Source Priority
|
|
|
|
Default priority (higher first):
|
|
|
|
1. CLI args (if enabled)
|
|
2. init kwargs
|
|
3. env vars
|
|
4. dotenv
|
|
5. secrets dir
|
|
6. defaults
|
|
|
|
Only customize when required:
|
|
|
|
```python
|
|
from pydantic_settings import PydanticBaseSettingsSource
|
|
|
|
|
|
@classmethod
|
|
def settings_customise_sources(
|
|
cls,
|
|
settings_cls: type[BaseSettings],
|
|
init_settings: PydanticBaseSettingsSource,
|
|
env_settings: PydanticBaseSettingsSource,
|
|
dotenv_settings: PydanticBaseSettingsSource,
|
|
file_secret_settings: PydanticBaseSettingsSource,
|
|
) -> tuple[PydanticBaseSettingsSource, ...]:
|
|
return (init_settings, env_settings, dotenv_settings, file_secret_settings)
|
|
```
|
|
|
|
Quality gate:
|
|
|
|
1. Priority order is explicit in code.
|
|
2. Tests verify conflict resolution.
|
|
|
|
### 5. Add Secrets Strategy
|
|
|
|
1. In local development, dotenv is acceptable for non-production values.
|
|
2. In deployed environments, prefer env vars or secret managers.
|
|
3. For file-mounted secrets, use `secrets_dir`.
|
|
|
|
Example:
|
|
|
|
```python
|
|
model_config = SettingsConfigDict(
|
|
env_prefix="APP_",
|
|
env_file=".env",
|
|
secrets_dir="/run/secrets",
|
|
)
|
|
```
|
|
|
|
Quality gate:
|
|
|
|
1. No secret literals in repository code.
|
|
2. Missing secrets behavior is understood per environment.
|
|
|
|
### 6. Choose Nested Or Independent Settings Boundaries
|
|
|
|
Prefer one root `BaseSettings` object with nested `BaseModel` sections when the configuration belongs to one application lifecycle:
|
|
|
|
```python
|
|
from pydantic import BaseModel, Field
|
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
|
|
|
|
|
class DatabaseSettings(BaseModel):
|
|
host: str = "localhost"
|
|
port: int = 5432
|
|
|
|
|
|
class ObservabilitySettings(BaseModel):
|
|
log_level: str = "INFO"
|
|
json_logs: bool = True
|
|
|
|
|
|
class Settings(BaseSettings):
|
|
model_config = SettingsConfigDict(
|
|
env_prefix="APP_",
|
|
env_nested_delimiter="__",
|
|
frozen=True,
|
|
)
|
|
|
|
database: DatabaseSettings = Field(default_factory=DatabaseSettings)
|
|
observability: ObservabilitySettings = Field(
|
|
default_factory=ObservabilitySettings
|
|
)
|
|
```
|
|
|
|
This produces names such as `APP_DATABASE__HOST` and gives the application one validated, atomic configuration snapshot. Nested sections should normally inherit from `BaseModel`, not `BaseSettings`; otherwise each nested settings model can collect sources independently and produce surprising results.
|
|
|
|
Use independent `BaseSettings` classes when the objects have genuinely independent ownership:
|
|
|
|
1. Different packages or deployable components own the schemas.
|
|
2. Each object needs its own env prefix or source policy.
|
|
3. A component is optional or loaded lazily.
|
|
4. Components need different reload lifecycles.
|
|
5. The same component must run outside the application.
|
|
|
|
Construct independent objects explicitly at the composition root and inject each dependency. Do not nest one `BaseSettings` class inside another merely to reuse its fields. Extract a shared `BaseModel` schema when models need common structure.
|
|
|
|
Quality gate:
|
|
|
|
1. Nested sections share one source policy and lifecycle.
|
|
2. Independent settings have distinct owners, prefixes, or lifecycles.
|
|
3. The application does not repeatedly scan the same sources through accidental nested `BaseSettings` construction.
|
|
|
|
### 7. Own The Settings Lifecycle
|
|
|
|
For most applications, construct settings once at the composition root and pass the validated object to services:
|
|
|
|
```python
|
|
def main() -> None:
|
|
settings = Settings()
|
|
application = Application(settings=settings)
|
|
application.run()
|
|
```
|
|
|
|
This makes ownership, startup failure, and test overrides explicit. Treat the object as a snapshot: environment variables and files changing later do not update an existing instance. Prefer `frozen=True` for shared settings so consumers cannot silently mutate process-wide configuration.
|
|
|
|
Use [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache) only when process-lifetime singleton access is intentional and explicit injection is awkward, such as a framework dependency provider:
|
|
|
|
```python
|
|
from functools import cache
|
|
|
|
|
|
@cache
|
|
def get_settings() -> Settings:
|
|
return Settings()
|
|
```
|
|
|
|
Keep the cached factory argument-free. Passing override kwargs creates one cached instance per argument combination, retains those values for the process lifetime, and obscures which configuration is active. In tests, instantiate `Settings(...)` directly or override the dependency; when a test must exercise the cached getter, isolate environment changes with `get_settings.cache_clear()` before and after the assertion.
|
|
|
|
`cache` is process-local. Every worker process gets its own instance, and concurrent first calls can construct more than one instance before the cache is populated. Settings construction must therefore be side-effect free; create engines, clients, and sessions in their own lifecycle-managed providers.
|
|
|
|
Quality gate:
|
|
|
|
1. Settings are created once per intended application or worker lifecycle.
|
|
2. Cached factories are argument-free and side-effect free.
|
|
3. Tests do not leak cached settings or environment changes.
|
|
4. Resource construction is separate from configuration parsing.
|
|
|
|
### 8. Reload Deliberately
|
|
|
|
Static service configuration should normally require a process restart. If runtime reload is a real requirement, construct a fresh settings instance and atomically replace the owned reference. Do not call `__init__()` on a shared instance: readers can observe mutation in progress, and resources derived from old values may remain alive.
|
|
|
|
Settings sources are synchronous. In an async application, construction or reload that reads dotenv, secrets, JSON, TOML, or YAML files should run in a worker thread:
|
|
|
|
```python
|
|
import asyncio
|
|
|
|
|
|
async def load_settings() -> Settings:
|
|
return await asyncio.to_thread(Settings)
|
|
```
|
|
|
|
Clearing `get_settings` is sufficient for controlled tests or single-threaded administration, but it is not an atomic live-reload protocol. Concurrent applications should own the current reference behind an application-specific lock or lifecycle manager, swap in a fully validated replacement, and then rebuild dependent resources.
|
|
|
|
Quality gate:
|
|
|
|
1. Reload creates and validates a replacement before publication.
|
|
2. Readers cannot observe a partially mutated object.
|
|
3. Dependent resources are recreated after the settings reference changes.
|
|
4. File-backed source reads do not block an async event loop.
|
|
|
|
### 9. Add Focused Lifecycle Tests
|
|
|
|
Do not add tests that re-validate baseline `pydantic-settings` functionality unless custom behavior is layered on top. Test the application-owned behavior instead:
|
|
|
|
1. Repeated cached getter calls return the same instance.
|
|
2. Cache clearing after an environment change returns a newly validated instance.
|
|
3. Explicitly injected settings bypass global cached state.
|
|
4. Reload swaps the settings snapshot and rebuilds dependent resources, when reload is supported.
|
|
|
|
Suggested invocation:
|
|
|
|
1. `uv run pytest -q`
|
|
|
|
## Completion Checks
|
|
|
|
1. Settings ownership matches the application or component lifecycle.
|
|
2. Source precedence is documented and tested.
|
|
3. Env naming conventions and aliases are explicit and stable.
|
|
4. Nested parsing behavior is tested when custom parsing behavior is added.
|
|
5. Secrets and dotenv usage are environment-appropriate and do not leak sensitive defaults.
|
|
6. Validation errors are actionable and fail fast for required values.
|
|
7. Cached factories are argument-free, process-local, and cleared deliberately in tests.
|
|
8. Nested models share one source policy; independent settings have an explicit ownership reason.
|
|
9. Runtime reload, if supported, replaces a validated snapshot and rebuilds dependent resources.
|
|
|
|
## Output Contract
|
|
|
|
When this skill is applied, return:
|
|
|
|
1. Which references were consulted.
|
|
2. The chosen source-precedence model and why.
|
|
3. The exact parsing and alias decisions made.
|
|
4. Any deferred choices and their risk.
|
|
5. The validation commands or tests run to confirm behavior.
|
|
|
|
Use these upstream docs when implementing or reviewing `pydantic-settings` behavior.
|
|
|
|
## Source Docs
|
|
|
|
### Primary
|
|
|
|
- [Settings Management](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/)
|
|
- [pydantic-settings package repository](https://github.com/pydantic/pydantic-settings)
|
|
|
|
### Core Concepts
|
|
|
|
- [Field aliases](https://pydantic.dev/docs/validation/latest/concepts/fields/#field-aliases)
|
|
- [Alias choices](https://pydantic.dev/docs/validation/latest/concepts/alias#aliaspath-and-aliaschoices)
|
|
- [Validation default behavior](https://pydantic.dev/docs/validation/latest/concepts/fields#validate-default-values)
|
|
- [ImportString type](https://pydantic.dev/docs/validation/latest/api/pydantic/types/#pydantic.types.ImportString)
|
|
|
|
### Priority And Sources
|
|
|
|
- [Field value priority](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#field-value-priority)
|
|
- [Customise settings sources](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#customise-settings-sources)
|
|
- [Other settings source types](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#other-settings-source)
|
|
|
|
### Environment And Parsing
|
|
|
|
- [Environment variable names and prefix behavior](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#environment-variable-names)
|
|
- [Case sensitivity behavior](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#case-sensitivity)
|
|
- [Parsing environment variable values](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#parsing-environment-variable-values)
|
|
- [Nested model default partial updates](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#nested-model-default-partial-updates)
|
|
|
|
### Lifecycle And Reloading
|
|
|
|
- [In-place reloading](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#in-place-reloading)
|
|
- [Async environments](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#async-environments)
|
|
- [`functools.cache`](https://docs.python.org/3/library/functools.html#functools.cache)
|
|
|
|
### Dotenv And Secrets
|
|
|
|
- [Dotenv support](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#dotenv-env-support)
|
|
- [Secrets](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#secrets)
|
|
- [Nested secrets](https://pydantic.dev/docs/validation/latest/concepts/pydantic_settings/#nested-secrets)
|