Skip to content

Add EVM execution microbenchmarks via go test -bench#2901

Draft
pdrobnjak wants to merge 1 commit intomainfrom
feature/microbenchmarks
Draft

Add EVM execution microbenchmarks via go test -bench#2901
pdrobnjak wants to merge 1 commit intomainfrom
feature/microbenchmarks

Conversation

@pdrobnjak
Copy link
Contributor

@pdrobnjak pdrobnjak commented Feb 17, 2026

Summary

  • Widen SetupWithSc, NewTestWrapperWithSc, NewGigaTestWrapper, and NewGigaTestWrapperWithRegularStore to accept testing.TB instead of *testing.T, enabling benchmarks to use the SC code path
  • Add table-driven EVM microbenchmarks in occ_tests/evm_benchmark_test.go covering non-conflicting, conflicting, and mixed transfer workloads at 100-5000 tx scales
  • Each benchmark has OCC and sequential sub-variants and reports txns/op, tps, gas/op, gas/sec, ns/tx

Initial Results (benchtime=3s)

Non-Conflicting EVM Transfers

Txs Mode ns/op TPS gas/sec
100 occ 14.7M 6,820 143M
100 sequential 39.9M 2,509 52.7M
500 occ 92.0M 5,433 114M
500 sequential 224.7M 2,225 46.7M
1000 occ 388.5M 2,574 54.1M
1000 sequential 525.1M 1,904 40.0M
5000 occ 6.72B 744 15.6M
5000 sequential 4.82B 1,037 21.8M

Conflicting EVM Transfers

Txs Mode ns/op TPS gas/sec
100 occ 91.2M 1,096 23.0M
100 sequential 41.1M 2,435 51.1M
1000 occ 3.46B 289 6.1M
1000 sequential 519.3M 1,926 40.4M

Mixed (50/50 conflicting + non-conflicting)

Txs Mode ns/op TPS gas/sec
1000 occ 3.96B 253 5.3M
1000 sequential 510.0M 1,961 41.2M

Key Observations

  • OCC provides ~2.5-2.7x speedup for non-conflicting txs at smaller batch sizes (100-500)
  • OCC degrades with conflicts due to retry overhead (4-7x slower than sequential)
  • At 5000 txs, even non-conflicting OCC is slower than sequential (memory/contention overhead)
  • Sequential is consistently ~1,900-2,500 TPS regardless of conflict patterns

Usage

# Run all EVM benchmarks (~30s-2min depending on scale)
go test -bench=BenchmarkEVM -benchtime=3s -count=1 ./occ_tests/

# Run specific scale
go test -bench=BenchmarkEVMTransfer/txs_1000 -benchtime=5s -count=3 ./occ_tests/

# Compare before/after with benchstat
go test -bench=BenchmarkEVM -benchtime=5s -count=5 ./occ_tests/ > old.txt
# ... make changes ...
go test -bench=BenchmarkEVM -benchtime=5s -count=5 ./occ_tests/ > new.txt
benchstat old.txt new.txt

Test plan

  • go test -bench=BenchmarkEVM -benchtime=1x -count=1 ./occ_tests/ — all benchmarks run without error
  • go test ./occ_tests/ -run TestParallelTransactionsEvmTransfer — existing OCC tests still pass
  • go vet ./occ_tests/ — no errors
  • gofmt -s -l app/test_helpers.go occ_tests/evm_benchmark_test.go — no output

🤖 Generated with Claude Code

Widen SetupWithSc and related test wrapper functions to accept
testing.TB instead of *testing.T so benchmarks can use the SC
code path. Add table-driven EVM benchmarks covering non-conflicting,
conflicting, and mixed transfer workloads at various scales
(100-5000 txs) with OCC/sequential variants. Each benchmark reports
txns/op, tps, gas/op, gas/sec, and ns/tx.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions
Copy link

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedFeb 17, 2026, 1:32 PM

@pdrobnjak pdrobnjak marked this pull request as draft February 17, 2026 13:37
@codecov
Copy link

codecov bot commented Feb 17, 2026

Codecov Report

❌ Patch coverage is 75.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.25%. Comparing base (7fb1494) to head (d54ca37).
⚠️ Report is 8 commits behind head on main.

Files with missing lines Patch % Lines
app/test_helpers.go 75.00% 2 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2901      +/-   ##
==========================================
+ Coverage   57.16%   57.25%   +0.09%     
==========================================
  Files        2091     2093       +2     
  Lines      171179   171729     +550     
==========================================
+ Hits        97853    98324     +471     
- Misses      64616    64640      +24     
- Partials     8710     8765      +55     
Flag Coverage Δ
sei-chain 52.72% <75.00%> (+0.09%) ⬆️
sei-cosmos 48.19% <ø> (+0.05%) ⬆️
sei-db 68.94% <ø> (+0.22%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
app/test_helpers.go 40.16% <75.00%> (-0.04%) ⬇️

... and 76 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments