Skip to content

migration routers#3336

Open
cody-littley wants to merge 50 commits into
mainfrom
cjl/migration-integration
Open

migration routers#3336
cody-littley wants to merge 50 commits into
mainfrom
cjl/migration-integration

Conversation

@cody-littley
Copy link
Copy Markdown
Contributor

@cody-littley cody-littley commented Apr 29, 2026

Describe your changes and provide context

Use the migration router primitives to construct the migration workflows we will need.

Use the following link to view the README in rendered form. Suggested read prior to reviewing code. https://github.com/sei-protocol/sei-chain/blob/cjl/migration-integration/sei-db/state_db/sc/migration/README.md

Testing performed to validate your change

unit tests


Note

High Risk
Adds and wires new state routing/migration modes (including migration-manager integration) that determine where reads/writes land across memiavl and flatkv, so misrouting could corrupt state during migrations. Large new test coverage reduces risk but the changes touch core storage/migration paths.

Overview
Introduces a centralized BuildRouter in router_builder.go that constructs per-WriteMode ModuleRouter layouts for all migration phases and steady states (V0 MemiavlOnly, V1 EVMMigrated, V2 AllMigratedButBank, V3 FlatKVOnly) and for in-flight migrations (MigrateEVM, MigrateAllButBank, MigrateBank) by wiring MigrationManager routes alongside direct backend routes.

Adds a test-only TestOnlyDualWriteRouter and TestOnlyDualWrite mode to mirror EVM writes to both backends while serving reads from the primary, and updates docs to clarify which routers are wrapped with NewThreadSafeRouter.

Adds extensive new unit/integration tests that simulate block workloads, verify key placement/counts, and exercise restart/resume behavior across each migration transition and steady-state router configuration.

Reviewed by Cursor Bugbot for commit 893eb75. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 29, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedMay 12, 2026, 2:02 PM

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 30, 2026

Codecov Report

❌ Patch coverage is 53.00353% with 133 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.07%. Comparing base (654d40b) to head (893eb75).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
sei-db/state_db/sc/migration/router_builder.go 46.55% 67 Missing and 65 partials ⚠️
sei-db/state_db/sc/migration/write_mode.go 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3336      +/-   ##
==========================================
- Coverage   59.25%   59.07%   -0.18%     
==========================================
  Files        2110     2106       -4     
  Lines      174181   173350     -831     
==========================================
- Hits       103210   102409     -801     
+ Misses      62044    61990      -54     
- Partials     8927     8951      +24     
Flag Coverage Δ
sei-chain-pr 79.59% <53.00%> (?)
sei-db 70.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-db/state_db/sc/migration/dual_write_router.go 100.00% <100.00%> (ø)
sei-db/state_db/sc/migration/write_mode.go 0.00% <0.00%> (ø)
sei-db/state_db/sc/migration/router_builder.go 50.33% <46.55%> (+50.33%) ⬆️

... and 71 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cody-littley cody-littley marked this pull request as ready for review May 11, 2026 16:09
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 009bf03. Configure here.

Comment thread sei-db/state_db/sc/migration/router_builder.go
Comment thread sei-db/state_db/sc/migration/dual_write_router.go Outdated
// Module names must be unique; NewRoute's validation rules apply. The
// returned Route may be passed to NewModuleRouter alongside other
// Routes to compose multi-database setups.
func (t *TestOnlyDualWriteRouter) BuildRoute(moduleNames ...string) (*Route, error) { // TODO LLM you need to write a unit test for this
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stale TODO comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

}

// Sends evm/ traffic to both memIAVL and flatKV.
memiavlEvmRoute, err := routeToMemIAVL(memIAVL)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

routeToMemIAVL(memIAVL) with no module args does not send evm traffic anywhere

Copy link
Copy Markdown
Contributor Author

@cody-littley cody-littley May 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Klunky API, but functionally correct. I added the following comment:

	// Note that a TestOnlyDualWriteRouter ignores module names; it's only job is to duplicate traffic.
	// The routes given to the dual write router do not specifiy modules for this reason.
	memiavlEvmRoute, err := routeToMemIAVL(memIAVL)

I did not want to modify the route API for this case, since this is a test-only workflow.

@@ -0,0 +1,1059 @@
package migration
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not blocking: 1000+ lines testing file is a little to big. consider splitting by mode (memiavl_only_test.go, migrate_evm_test.go, evm_migrated_test.go, ...).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Splitting by mode will result in a very large number of test files. I instead split into two test files: one for testing static routes, and another for testing non-static migration. This cuts the size of the file roughly in half.

Is this satisfactory from your perspective, or would you like additional splitting?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants