Skip to content

Comments

feat: Add household support (CRUD, associations, impact analysis)#80

Merged
anth-volk merged 19 commits intoapp-v2-migrationfrom
feat/household-crud-v2
Feb 16, 2026
Merged

feat: Add household support (CRUD, associations, impact analysis)#80
anth-volk merged 19 commits intoapp-v2-migrationfrom
feat/household-crud-v2

Conversation

@anth-volk
Copy link
Contributor

@anth-volk anth-volk commented Feb 6, 2026

Fixes #74
Fixes #75
Fixes #76
Fixes #78
Fixes #79


Important

The policyengine-app-v2 repo will also need changes to consume the modifications from this PR. Frontend work should be coordinated once this API is merged.

Important

Production database migration required. This PR switches from Supabase SQL migrations to Alembic. After merging, run alembic stamp a17ac554f4aa on the production database to mark existing schema as migrated. See .claude/skills/database-migrations.md for details.


Summary

This PR adds comprehensive household support to the API v2 alpha and migrates to Alembic for database migrations.

1. Stored Household CRUD (#74)

  • Household model with household_data JSONB storage
  • POST/GET/DELETE /households endpoints
  • Supports both UK and US household structures

2. User-Household Associations (#75)

  • UserHouseholdAssociation model linking users to households
  • POST/GET/PUT/DELETE /user-household-associations endpoints
  • Unique constraint on (user_id, household_id)

3. Household Impact Analysis (#76)

  • Extended Simulation model with SimulationType enum, household_id, household_result
  • Extended Report model with report_type field
  • POST/GET /analysis/household-impact endpoints
  • Supports single runs and baseline vs reform comparisons
  • Computes impact across all entity types (UK: person/benunit/household, US: person/tax_unit/spm_unit/family/marital_unit/household)

4. Alembic Migration System (#78)

  • Replaced SQLModel.metadata.create_all() with Alembic migrations
  • Initial migration creates all 21 tables with proper ordering
  • Auto-generates migrations from SQLModel model changes
  • Added comprehensive migration guidelines in .claude/skills/database-migrations.md
  • Archived old Supabase table migrations (storage migrations remain)

5. Fix Policy Reforms Not Applied in Simulations (#79)

  • Fixed bug where policy reforms were not being applied to economy-wide simulations
  • The policyengine.core.Simulation class creates Microsimulation without the reform, then tries to apply policy after construction (which doesn't work due to caching)
  • Added helper functions _pe_policy_to_reform_dict() and _merge_reform_dicts() to convert PEPolicy objects to reform dict format
  • Created _run_us_economy_simulation() and _run_uk_economy_simulation() helpers that pass reforms at Microsimulation construction time
  • Also fixed the same issue in household calculations (_calculate_household_us and _calculate_household_uk)

Database Migrations

Now managed by Alembic in alembic/versions/:

  1. d6e30d3b834d_initial_schema.py - Creates all 21 tables
  2. a17ac554f4aa_add_parameter_values_indexes.py - Adds performance indexes

Files Changed

New Files

  • src/policyengine_api/models/household.py
  • src/policyengine_api/models/user_household_association.py
  • src/policyengine_api/api/households.py
  • src/policyengine_api/api/user_household_associations.py
  • alembic/ - Alembic configuration and migrations
  • .claude/skills/database-migrations.md - Migration guidelines
  • scripts/seed_nevada.py - Nevada dataset seeding script
  • scripts/test_economy_simulation.py - Test script to verify policy reform bug
  • test_fixtures/fixtures_households.py
  • test_fixtures/fixtures_user_household_associations.py
  • test_fixtures/fixtures_analysis.py
  • tests/test_households.py
  • tests/test_user_household_associations.py
  • tests/test_analysis_household_impact.py

Modified Files

  • src/policyengine_api/models/simulation.py - Added SimulationType, household_id, household_result
  • src/policyengine_api/models/report.py - Added report_type
  • src/policyengine_api/models/__init__.py - Exports new models
  • src/policyengine_api/api/__init__.py - Registers new routers
  • src/policyengine_api/api/analysis.py - Added household-impact endpoints
  • src/policyengine_api/api/household.py - Fixed policy reform application in household calculations
  • src/policyengine_api/modal_app.py - Fixed policy reform application in economy-wide simulations
  • src/policyengine_api/config/settings.py - Fixed database URL port
  • scripts/init.py - Now uses Alembic instead of create_all()
  • CLAUDE.md - Added Alembic documentation reference

Test Plan

  • Run unit tests for household CRUD
  • Run unit tests for user-household associations
  • Run unit tests for household impact analysis
  • Manual testing of endpoints
  • Verify Alembic migrations apply correctly locally
  • Run alembic stamp on production after merge
  • Verify policy reforms are correctly applied in economy-wide simulations

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.5 noreply@anthropic.com

anth-volk and others added 11 commits February 7, 2026 01:24
- Replace synchronous inline calculation with async trigger pattern
- Add _trigger_household_impact() mirroring _trigger_economy_comparison()
- Add _run_local_household_impact() for local execution (blocking)
- Add _run_simulation_in_session() for running individual simulations
- Update POST endpoint to trigger and return immediately
- Add test script for manual end-to-end testing

Note: Local execution blocks the request (same as economic impact).
True async requires Modal functions (household_impact_uk/us).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…lue serialization

- Remove Enum/date serialization from seed_common.py since policyengine.py
  now pre-serializes default_value for JSON compatibility
- Change default_value type from `str | None` to `Any` in Variable model
  since it stores JSON values (bool, int, float, str)

Depends on policyengine.py feat/add-variable-default-value branch

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
anth-volk and others added 3 commits February 7, 2026 02:07
- Add tests for Variable with int, float, bool, and string default values
- Add test for null default_value handling
- Add test for Household model creation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
session.get(Report, report_id) expects a UUID, but report_id was passed
as a string. This caused 'str' object has no attribute 'hex' errors.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@anth-volk anth-volk marked this pull request as draft February 11, 2026 15:27
@anth-volk
Copy link
Contributor Author

Converting to draft, as this implementation is actually flawed and duplicative.

… reform bug is fixed

This commit removes the workaround code that was added to bypass
policyengine.py's Simulation class. The workaround was needed because
policyengine.py's US simulation applied reforms via p.update() after
Microsimulation construction, which didn't work due to the US package's
shared singleton TaxBenefitSystem.

That bug has now been fixed in policyengine.py (issue #232), so we can
use policyengine.py's Simulation class directly again.

Changes:
- Revert household.py to use policyengine.core.Simulation instead of
  manually building Microsimulation with reform dicts
- Revert modal_app.py to use PESimulation instead of custom helper
  functions (_pe_policy_to_reform_dict, _merge_reform_dicts,
  _run_us_economy_simulation, _run_uk_economy_simulation)
- Remove now-obsolete test files for the workaround functions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@anth-volk
Copy link
Contributor Author

Update: Workaround code removed

The latest commit removes the workaround code (_pe_policy_to_reform_dict, _merge_reform_dicts, _run_us_economy_simulation, _run_uk_economy_simulation) and reverts to using policyengine.py's Simulation class directly.

This is possible because the root cause bug has been fixed in policyengine.py - see:

This PR now depends on policyengine.py PR #231 being merged first.

The bug was that policyengine-us uses a shared singleton TaxBenefitSystem, so reforms applied via p.update() after Microsimulation construction had no effect. The fix in policyengine.py now passes the reform dict at Microsimulation construction time for US.

This commit removes ~1,800 lines of workaround code from this PR.

Add tests to verify that US policy reforms are applied correctly
to household calculations. These tests cover:

- Integration tests via API endpoints (TestUSPolicyReform, TestUKPolicyReform)
- Unit tests for the calculation functions directly (test_household_calculation.py)

The tests verify:
1. Baseline calculations work correctly
2. Reforms change household net income as expected
3. Running a reform doesn't pollute subsequent baseline calculations
   (regression test for the singleton pollution bug)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
anth-volk and others added 3 commits February 16, 2026 23:26
The PyPI release (3.1.15) has a bug where US reforms silently fail
due to the shared singleton TaxBenefitSystem (policyengine.py#232).
The fix exists on the app-v2-migration branch but hasn't been released.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hatchling rejects git URL dependencies unless explicitly opted in.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ions

Test scripts in scripts/ were ad-hoc debugging aids, not part of the
test suite. Nevada seed is no longer needed. Archived Supabase migrations
are superseded by Alembic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@anth-volk anth-volk marked this pull request as ready for review February 16, 2026 23:39
@anth-volk anth-volk merged commit 0d78f61 into app-v2-migration Feb 16, 2026
1 check passed
@anth-volk anth-volk deleted the feat/household-crud-v2 branch February 16, 2026 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant