Feature Request: Enhanced Support for Lightweight Multi-Instance Deployments ("Sharding") via Dynamic YAML Backends

[BeSovereign]

Userstory

As user I want a single instance holding the identity of all services in my shard in order to login only once per session.
As admin I want a multi tenenant capable solution for the identity management inside the individual shards.

Is your feature request related to a problem? Please describe.

Instead of a multi-tenant architecture with a centralized database, there should be an individual, fully isolated instance ("shards") for each tenant. Every shard requires its own Identity and Access Management (IAM), for which Authelia is the perfect fit due to its small footprint and open-source nature.

To keep the resource footprint minimal per shard, we should rely entirely on the file-based backend (users_database.yml) and file-based configuration (configuration.yml) instead of SQL databases. However, managing these YAML files dynamically for thousands of separate instances currently lacks flexibility, especially when users need to be added or configuration updated on the fly without restarting the Authelia container.

Describe the solution you'd like**

We should implement a solution in order to provision shards during the process of provisioning with an authelia instance managing all the IAM related tasks inside the shard. Specifically, this entails:

• Robust Hot-Reloading: Ensure that any external changes to the users_database.yml and configuration.yml are instantly and reliably detected and applied by Authelia without requiring a container restart.
• Standardized Minimal Shard Configuration: A streamlined, minimal boilerplate configuration specifically designed for isolated single-tenant/small-group setups.

Note: In a future follow-up request, we plan to propose or contribute a lightweight administrative API or local GUI specifically tailored for shard administrators to manage this YAML user backend directly, but the immediate need is ensuring the underlying YAML mechanics are fully dynamic and robust

Describe alternatives you've considered

• Using a centralized SQLite/PostgreSQL per shard: This creates unnecessary overhead (RAM/CPU) for instances that typically only serve 1 to 5 local users.
• Writing custom bash/Python scripts to modify the YAML files and forcefully restarting the Authelia Docker container: This disrupts active sessions and creates a poor user experience.

[ClaydeCode]

Preliminary Plan

The repo has no existing Authelia integration — this is a greenfield feature. The project already uses Jinja2 for docker-compose templates and Traefik with file watching, which establishes the patterns to follow.

Approach:

1. Add Authelia as a service in the core docker-compose stack (not an optional app), rendered via the existing Jinja2 template system.
2. Create minimal configuration.yml and users_database.yml Jinja2 templates rendered at shard provisioning time, placed in a mounted volume.
3. Rely on Authelia's built-in inotify-based hot-reload (enabled via --config.experimental.file-filters), so YAML changes take effect without container restarts.
4. Configure Authelia to support both forward-auth (for apps that use Traefik middleware) and OIDC (for apps that need an identity provider), with sensible defaults in the minimal config template.
5. Add API endpoints under the existing protected API path (accessible via the shard UI) for CRUD operations on the YAML user database, writing changes directly to the file.
6. Wire Authelia as Traefik forward-auth middleware in the dynamic config generator.

Confirmed decisions from @max-tet:

• Authelia is part of the core stack, always present in every shard.
• Both forward-auth and OIDC are in scope — Authelia serves as the single auth provider for all apps.
• User management API is exposed through the existing protected API path (shard UI integration).

---

This looks like a large issue. React with 👍 to approve this preliminary plan and proceed to a thorough implementation plan.

[max-tet]

Answers:

• Authelia should be in the core stack, part of the existing docker-compose.
• Authelia will be the authentication provider for all apps. Some work with forward auth, some with OIDC. We want to be prepared for many authentication needs.
• The user will interact with Authelia via the shard UI, so effectively via the "protected" API path.

[ClaydeCode]

Plan updated. Updated plan to reflect @max-tet's answers: Authelia goes into the core stack (not an optional app), both forward-auth and OIDC must be supported from the start, and user management will be exposed via the existing protected API path (shard UI). The clarifying questions section has been replaced with these confirmed decisions.

[ClaydeCode]

Implementation Plan

Implementation Plan: Authelia Core IAM Service

Summary

Add Authelia as a permanent core service in every shard, providing both forward-auth (for apps protected by Traefik middleware) and OIDC (for apps that act as OAuth clients). A user-management API exposed under /protected/authelia/users allows the shard UI to perform CRUD operations on the Authelia file-based user database. Authelia's built-in inotify hot-reload means file changes take effect immediately — no container restarts needed.

---

Affected Files

File	Change
docker-compose.yml	Add authelia service with volume mounts for config and user DB
shard_core/settings.py	Add AutheliaSettings (OIDC client secrets, JWT secret, session secret)
config.toml	Add [services.authelia] section with sensible defaults
shard_core/app_factory.py	Render Authelia config files at startup; write initial users DB if absent
shard_core/service/traefik_dynamic_config.py	Add authelia-forwardauth middleware pointing at the Authelia service; add public router rule for auth.<shard-domain>
shard_core/data_model/traefik_dyn_config.py	Verify or extend ForwardAuthMiddleware to support trustForwardHeader
shard_core/web/protected/__init__.py	Register new /protected/authelia/users router
shard_core/web/protected/authelia_users.py	New file — CRUD endpoints (list, get, create, update, delete)
shard_core/templates/authelia/configuration.yml.j2	New — Jinja2 template for Authelia config (hot-reload enabled, OIDC + file backend, public subdomain auth.<shard-domain>)
shard_core/templates/authelia/users_database.yml.j2	New — Jinja2 template for initial empty user database

---

Key Design Decisions

Authelia config lives on the host volume, not inside the container image. The two YAML files are rendered from Jinja2 templates at shard startup (in app_factory.py lifespan) using the portal identity (domain, secrets). Authelia watches them with inotify (--config.experimental.file-filters).

User records are stored only in users_database.yml. We do not mirror them in TinyDB — the YAML file is the single source of truth. The API endpoints read/write the YAML file directly via PyYAML, protected by a file-level lock to prevent concurrent writes.

Passwords are hashed by the API layer using argon2id before being written, so plain-text passwords never touch the file. Argon2id is the current OWASP-recommended algorithm and Authelia's default, combining resistance to side-channel and GPU/ASIC brute-force attacks. No algorithm override is needed in the Authelia config.

Forward-auth middleware applies to all app entrypoints. The existing compile_config function in traefik_dynamic_config.py already attaches middleware per access level; the new Authelia middleware replaces or wraps the current auth middleware for private paths.

Authelia is exposed publicly at auth.<shard-domain>. A dedicated Traefik router rule is created for the subdomain auth.<shard-domain>, routing traffic to the Authelia container. This public subdomain is required so that Authelia can serve its login portal, session cookies (scoped to the shard domain), and OIDC endpoints to end users and OAuth clients. The default_redirection_url and session.domain fields in configuration.yml.j2 are rendered with the shard domain at startup.

OIDC secrets are generated once at first startup and stored in config.toml (or a secrets file). Settings are loaded via the existing Pydantic settings pattern.

---

Edge Cases and Risks

• First-run bootstrapping: If users_database.yml does not exist yet, the API and Authelia both need a consistent empty-but-valid state. The startup renderer must create the file before Authelia starts.
• Concurrent writes: Multiple API requests writing to the YAML file simultaneously could corrupt it. Use a threading lock or write to a temp file then os.replace().
• Hot-reload lag: Authelia's inotify reload is near-instant but not atomic. Writes should be complete before the API returns 200.
• OIDC client registration: Apps using OIDC need their client IDs/secrets pre-registered in configuration.yml. If apps are installed after startup, the config must be re-rendered and Authelia will pick it up via hot-reload.

---

Verification

1. Unit tests for the user CRUD service layer (read/write/parse YAML, password hashing).
2. Integration test: start the stack, call the API to create a user, verify the YAML file is updated and Authelia returns a valid session cookie without restart.
3. Check Traefik routes a request through Authelia forward-auth and correctly rejects unauthenticated access.
4. Verify OIDC discovery endpoint (/.well-known/openid-configuration) is reachable at https://auth.<shard-domain>.
5. Verify the Authelia login portal is reachable at https://auth.<shard-domain> from a browser.

---

Decisions

• Authelia subdomain: auth.<shard-domain> — a public subdomain of the shard is required for Authelia's login portal, session cookies, and OIDC endpoints. A Traefik router rule is generated for this subdomain at shard startup.
• Password hashing: Argon2id — current OWASP best practice, also Authelia's default.

---

React with 👍 to approve this plan and start implementation.

[max-tet]

Answers:

• No need for a dedicated subdomain for Authelia.
• Chose the hash algorithm yourself according to the current best practice.

[ClaydeCode]

Plan updated. Two open questions from the plan have been answered by @max-tet:

1. Authelia subdomain: No dedicated public subdomain is needed. Authelia will be reachable only internally (by Traefik and the shard backend), not exposed via a public Traefik router rule. The Authelia UI/API will not be accessible to end users directly.

2. Password hashing algorithm: Argon2id is chosen — it is the current OWASP-recommended algorithm for password hashing, combining resistance to both side-channel and GPU/ASIC brute-force attacks. Authelia defaults to argon2id as well, so no extra configuration is needed beyond omitting any override.

No structural changes to the plan were required; the Questions section has been replaced with a Decisions section recording these answers, and the relevant notes in Key Design Decisions and the Authelia config template description have been updated to reflect internal-only access and argon2id.

[BeSovereign]

Correction on the subdomain: Authelia needs a subdomain. This subdomain shall be auth and a subdomain of the current shard.

[ClaydeCode]

Plan updated. Updated the plan to reflect @BeSovereign's correction: Authelia requires a public subdomain auth.<shard-domain> rather than being strictly internal-only. This affects the Traefik dynamic config (a public router rule must be added for auth.<shard-domain>), the Authelia configuration template (the default_redirection_url and session.domain must reference the public subdomain), and the two decisions sections. All other sections are unchanged.