Skip to content

Add roadrunner framework#716

Merged
MDA2AV merged 22 commits into
MDA2AV:mainfrom
williamthome:feat/add-roadrunner-framework
May 18, 2026
Merged

Add roadrunner framework#716
MDA2AV merged 22 commits into
MDA2AV:mainfrom
williamthome:feat/add-roadrunner-framework

Conversation

@williamthome
Copy link
Copy Markdown
Contributor

Description

roadrunner is a pure-Erlang HTTP/1.1, HTTP/2, and WebSocket server from the Arizona framework, pinned to hex 0.1.0 on OTP 29.

  • Tier: production
  • Profiles (19): baseline, pipelined, limited-conn, json, json-comp, json-tls, upload, async-db, api-4, api-16, static, static-h2, fortunes, crud, baseline-h2, baseline-h2c, json-h2c, echo-ws, echo-ws-pipeline
  • All local validation passed

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run all benchmark tests
/benchmark -f <framework> -t <test> Run a specific test
/benchmark -f <framework> --save Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@MDA2AV
Copy link
Copy Markdown
Owner

MDA2AV commented May 17, 2026

/benchmark -f roadrunner

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: roadrunner | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 839,857 6400.4% 281MiB NEW NEW
baseline 4096 794,245 5630.2% 260MiB NEW NEW
pipelined 512 1,381,531 6543.3% 186MiB NEW NEW
pipelined 4096 1,401,356 6350.7% 216MiB NEW NEW
limited-conn 512 412,602 3508.4% 232MiB NEW NEW
limited-conn 4096 407,074 3912.6% 220MiB NEW NEW
json 4096 193,030 6268.4% 513MiB NEW NEW
json-comp 512 96,396 6343.1% 806MiB NEW NEW
json-comp 4096 103,401 6322.6% 859MiB NEW NEW
json-comp 16384 86,572 6182.4% 713MiB NEW NEW
json-tls 4096 52,015 3071.4% 483MiB NEW NEW
upload 32 2,879 1923.4% 386MiB NEW NEW
upload 256 2,888 4363.2% 519MiB NEW NEW
api-4 256 20,270 401.8% 165MiB NEW NEW
api-16 1024 43,406 1545.4% 315MiB NEW NEW
static 1024 16,263 1088.6% 166MiB NEW NEW
static 4096 16,656 1109.3% 167MiB NEW NEW
static 6800 17,457 1158.9% 173MiB NEW NEW
async-db 1024 83,823 4915.4% 636MiB NEW NEW
crud 4096 200,693 5017.7% 340MiB NEW NEW
fortunes 1024 46,482 6238.8% 1021MiB NEW NEW
baseline-h2 256 484,255 6504.4% 529MiB NEW NEW
baseline-h2 1024 497,181 6357.1% 515MiB NEW NEW
static-h2 256 18,757 1374.7% 730MiB NEW NEW
static-h2 1024 18,646 1502.5% 736MiB NEW NEW
baseline-h2c 256 921,545 6533.4% 542MiB NEW NEW
baseline-h2c 1024 955,089 6355.9% 515MiB NEW NEW
baseline-h2c 4096 959,696 6199.6% 536MiB NEW NEW
json-h2c 1024 141,377 6293.7% 1.7GiB NEW NEW
json-h2c 4096 132,267 6282.8% 1.6GiB NEW NEW
echo-ws 512 1,023,868 5865.1% 302MiB NEW NEW
echo-ws 4096 470,713 3379.5% 223MiB NEW NEW
echo-ws 16384 438,090 2421.9% 199MiB NEW NEW
echo-ws-pipeline 512 2,103,252 6259.0% 188MiB NEW NEW
echo-ws-pipeline 4096 1,502,936 4720.4% 216MiB NEW NEW
echo-ws-pipeline 16384 775,031 2778.4% 198MiB NEW NEW
Full log
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.60ms   1.83ms   2.43ms   2.51ms   3.60ms

  7514679 frames sent in 5.00s, 7514680 frames received
  Throughput: 1.50M req/s
  Bandwidth:  10.03MB/s
  WS upgrades: 150
  WS frames:   7514680
  Latency samples: 7514680 / 7514680 responses (100.0%)
  Reconnects: 231308
[info] CPU 4720.4% | Mem 216MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.52ms   1.81ms   2.40ms   2.59ms   3.92ms

  7242002 frames sent in 5.00s, 7241715 frames received
  Throughput: 1.45M req/s
  Bandwidth:  9.67MB/s
  WS upgrades: 150
  WS frames:   7241715
  Latency samples: 7241715 / 7241715 responses (100.0%)
  Reconnects: 224723
[info] CPU 4429.0% | Mem 212MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.50ms   1.77ms   1.94ms   2.15ms   3.38ms

  6776875 frames sent in 5.00s, 6776507 frames received
  Throughput: 1.35M req/s
  Bandwidth:  9.04MB/s
  WS upgrades: 141
  WS frames:   6776507
  Latency samples: 6776507 / 6776507 responses (100.0%)
  Reconnects: 222695
[info] CPU 4593.0% | Mem 225MiB

=== Best: 1502936 req/s (CPU: 4720.4%, Mem: 216MiB) ===
[info] saved results/echo-ws-pipeline/4096/roadrunner.json
httparena-bench-roadrunner
httparena-bench-roadrunner

==============================================
=== roadrunner / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.13ms   5.08ms   5.58ms   6.03ms   9.59ms

  2333223 frames sent in 5.00s, 2331463 frames received
  Throughput: 466.13K req/s
  Bandwidth:  3.11MB/s
  WS upgrades: 150
  WS frames:   2331463
  Latency samples: 2331463 / 2331463 responses (100.0%)
  Reconnects: 268247
[info] CPU 1931.2% | Mem 187MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.92ms   4.02ms   4.61ms   4.93ms   9.38ms

  3877430 frames sent in 5.00s, 3875158 frames received
  Throughput: 774.74K req/s
  Bandwidth:  5.17MB/s
  WS upgrades: 142
  WS frames:   3875158
  Latency samples: 3875158 / 3875158 responses (100.0%)
  Reconnects: 259653
[info] CPU 2778.4% | Mem 198MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.83ms   4.60ms   5.18ms   5.35ms   8.18ms

  3124988 frames sent in 5.00s, 3122588 frames received
  Throughput: 624.29K req/s
  Bandwidth:  4.17MB/s
  WS upgrades: 150
  WS frames:   3122588
  Latency samples: 3122588 / 3122588 responses (100.0%)
  Reconnects: 279309
[info] CPU 2152.9% | Mem 205MiB

=== Best: 775031 req/s (CPU: 2778.4%, Mem: 198MiB) ===
[info] saved results/echo-ws-pipeline/16384/roadrunner.json
httparena-bench-roadrunner
httparena-bench-roadrunner
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/fortunes-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

@MDA2AV
Copy link
Copy Markdown
Owner

MDA2AV commented May 18, 2026

/benchmark -f roadrunner --save

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

⚠️ /benchmark --save cannot start: main has diverged and cannot be auto-merged into this branch. Please merge or rebase main manually, push, and re-run /benchmark --save.

The auto-buffering path can't handle 128 concurrent 20 MB POSTs
regardless of how the conn accumulates bytes (peak RSS >8 GiB, 0
req/s under the validator's load). All four listeners now declare
`body_buffering => manual`; the upload handler drains the body in
64 KB chunks via `roadrunner_req:read_body/2 #{length => 65536}`.

benchmark-lite roadrunner upload (128c, 5s × 3):

| stage | req/s | RSS |
|---|---:|---:|
| auto (pre-fix) | 749 | 4.1 GiB |
| auto (post roadrunner fixes) | 0 | 8.4 GiB |
| manual stream | 1566 | 399 MiB |

Roadrunner SHA bumped to a8596b786eff for the iodata body field.
Listener opts reshaped in v0.1.0: h2c => enabled becomes
protocols => [http2] on plain TCP; TLS h2 uses protocols =>
[http2, http1] with auto-derived alpn_preferred_protocols.

OTP 29 GA dropped the rebar3 TLS-1.2 workaround needed for RC3 +
Fastly, so retire config/rebar3_ssl.config + ERL_FLAGS and bump
the Dockerfile base to erlang:29.0-alpine. Add 8082 to EXPOSE.

README: move baseline-h2c, json-h2c, static-h2 from deferred to
covered (19/28) to match meta.json and the live listeners.
The file's own header marked it as a local snapshot to be removed
before opening the PR; clearing it out so the merged contribution
doesn't ship a stale benchmark capture.
The branch added the framework dir and meta.json but never wired
the site-level registry, so the entry wouldn't render in the UI
even after benchmark CI populates the per-profile result files.
@williamthome williamthome force-pushed the feat/add-roadrunner-framework branch from d9dfda3 to 3ea3a09 Compare May 18, 2026 11:02
@williamthome
Copy link
Copy Markdown
Contributor Author

@MDA2AV rebased onto main

@MDA2AV
Copy link
Copy Markdown
Owner

MDA2AV commented May 18, 2026

/benchmark -f roadrunner --save

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

⚠️ /benchmark --save cannot start: main has diverged and cannot be auto-merged into this branch. Please merge or rebase main manually, push, and re-run /benchmark --save.

@MDA2AV
Copy link
Copy Markdown
Owner

MDA2AV commented May 18, 2026

nevermind Ill just run it directly

@MDA2AV MDA2AV merged commit 02591a8 into MDA2AV:main May 18, 2026
4 checks passed
@williamthome williamthome deleted the feat/add-roadrunner-framework branch May 18, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants