PBS WebSocket PoC

Overview

The PBS WebSocket extension adds a persistent WS channel between Commit-Boost and Helix, running in parallel with the standard REST API. REST handles registration, header requests, block submission, and unblinding. WS allows the proposer to:

• easily communicate when they will conclude their auction
• provides lower latency for bid delivery (push instead of poll)
• increases redundancy for blinded block submission (REST + WS)
• allows the relay to track up-to-date networking information via pings and pongs (for more reliable bid delivery)

Key principle: REST is the primary path. WS is layered on top. If WS drops, REST continues unaffected.

---

TL;DR

Without WS: The validator must poll periodically during the auction window. Late polls miss the best bid. Early polls see stale data.

With WS: The relay pushes bids at their discretion. The cb-pbs client caches the latest bid in a watch channel. When the validator selects their bid, either the best cached value (zero-latency lookup) or the best bid from REST is selected.

---

Usage

git clone --recurse-submodules https://github.com/Commit-Boost/ws-workspace.git

just build-all

just simulate

Wire Protocol

All WebSocket messages use a binary SSZ wire format, where each frame is:

[tag: u8] [fork?: u8] [ssz_payload: ...]

Tag	Name	Fork byte?	SSZ payload
0x01	Subscribe	No	SubscriptionBatch
0x02	RegistrationAck	No	RegistrationAck { accepted: u32, rejected: u32 }
0x04	BidPush	Yes	SignedBuilderBid (fork-specific SSZ)
0x05	SubmitBlindedBlock	Yes	SignedBlindedBeaconBlock (fork-specific SSZ)
0x06	SubmitBlindedBlockAck	No	SubmitBlindedBlockAck { status: u8 }
0x07	Ping	No	Ping { nonce: u64 }
0x08	Pong	No	Pong { nonce: u64 }

Fork bytes: Electra=1, Fulu=2, Gloas=3. The fork byte determines which SSZ schema to decode the payload with. The ws-wire crate is shared between relays and Commit-Boost.

---

Message Flows

Connection & Subscribe

CB → Relay:  ws://relay/eth/v1/builder/ws
Relay → CB:  101 Switching Protocols
CB → Relay:  WS Subscribe { registrations[], auction_conclusion_ms }
Relay → CB:  WS RegistrationAck { accepted: N, rejected: M }

auction_conclusion_ms is carried in the Subscribe message. Each re-subscribe (epoch boundary) updates it without needing to restart the WS connection. The relay verifies BLS signatures on each registration exactly as it does in REST. While this technically can be replayed, we avoid adding new type for the validator to sign. The burden is on the relay to try and differentiate between the real proposer and imposter clients, i.e., via IP filtering.

Bid Push (Two-Phase Policy)

Note this is a tentative bid pushing strategy. The current Helix implementation does not respect bid cancellations.

Builder → Relay:     submit_block (bid)
Relay → Auctioneer:  rank bids
Auctioneer → Push:   new winning bid
---- Phase A — debounce + rate-limit ----
Push → CB:           WS BidPush (best bid)
CB → watch channel:  cache latest bid

... (approaching auction_conclusion_ms) ...
---- Phase B — schedule force-push at deadline - RTT - safety_margin ----
Push → CB:           WS BidPush (best bid)

Validator → CB:      get_header(slot, pubkey)
CB → watch channel:  read cached bid (instant)
CB → Validator:      header

Phase A (bid arrival window): Debounces identical bids by content hash, rate-limits non-identical bids (minimum interval), caches the latest.

Phase B (deadline force-push): Schedules a one-shot push at slot_start + auction_conclusion_ms - EWMA(RTT) - safety_margin. RTT is measured per connection via Ping/Pong with EWMA α=0.3.

Settlement: When SubmitBlindedBlock arrives, PerSlotState.received_submission = true. The push system checks this before every push — if set, the push is suppressed. This prevents overwhelming the proposer during the critical unblinding window.

Slot rollover: On a new slot, PerSlotState resets (received_submission = false, deadline_scheduled = false, cached_bid = None).

Stale bid guard (cb-pbs): The watch channel retains the last send_replace value across slots. The clear_if_stale/mark_bid_contributed mechanism tracks which slot last consumed a cached bid and clears it when the slot advances. A fresh WS push resets the guard.

Blinded Block Submission

Validator → CB:      submit_blinded_block (V2)
CB → Relay:          REST: POST /eth/v2/builder/blinded_blocks (primary)
CB → Relay:          WS: SubmitBlindedBlock { fork, body_ssz } (secondary, fire-and-forget)
CB → Relay:          WS: SubmitBlindedBlockAck { status } (ack, 500ms timeout, up to 2 retries)

Relay → Relay:       set received_submission = true (settle all connections for this pubkey)
Relay → Auctioneer:  unblind payload, publish to beacon chain

Dual-path idempotency : Both WS and REST call _get_payload on the relay. The first call transitions the auctioneer to Broadcasting and starts delivery. The second hits DeliveringPayload — this is handled as success (202 for V2, cached payload body for V1):

Caller	Error	Behavior
get_payload_v2 (REST V2)	DeliveringPayload	Return 202 ACCEPTED
get_payload (REST V1)	DeliveringPayload	Serve from delivered_payloads cache
route_submit_blinded_block (WS)	DeliveringPayload	Return SubmitStatus::Accepted

The delivered_payloads cache (DashMap<(Slot, B256), GetPayloadResponse> on ProposerApi) stores the unblinded payload after successful delivery. V1 callers need the actual body — V1 spec doesn't support bare 202. Cleaned on auctioneer slot advance.

Retry (cb-pbs): WS submit waits 500ms for SubmitBlindedBlockAck. On timeout, resends (max 2 retries). If exhausted, logs error — REST already submitted in parallel. Duplicate relay entries are deduplicated by URL before spawning REST tasks.

---

System Architecture

ws-wire/                         Shared wire format crate (SSZ types, framing, fork mapping)
│
helix/crates/relay/src/
├── api/proposer/websocket/
│   ├── mod.rs                   WS handler: connect, ping/pong, subscribe, submit routing
│   ├── push.rs                  Bid push engine: registry, PerSlotState, two-phase policy, RTT
│   └── wire.rs                  Fork adapters, registration conversion (helix ↔ ws-wire)
├── api/proposer/
│   ├── get_payload.rs           Unblind payload, deliver to proposer, idempotency
│   └── mod.rs                   ProposerApi, delivered_payloads cache
├── auctioneer/                  Slot auction state machine (Slot → Sorting → Broadcasting)
└── spine/                       Tile/Spine actor framework

commit-boost-client/crates/pbs/src/
├── mev_boost/
│   ├── ws_client.rs             HelixWsClient: connection state machine, reconnect, retry, bid cache, stale guard
│   ├── mod.rs                   CompoundGetHeaderResponse (Full/Light)
│   ├── get_header.rs            Multiplex REST + WS bids, auction winner selection
│   ├── submit_block.rs          Dual-path submission (WS fire-and-forget + REST parallel), relay URL dedup
│   └── wire.rs                  Fork byte adapters (delegates to ws-wire)
├── routes/                      axum HTTP handlers
└── state.rs                     PbsState, WsClientMap

HelixWsClient State Machine

Disconnected → Connecting (spawn)
Connecting → Connected (WS upgrade OK)
Connecting → Backoff (connection failed)
Backoff → Connecting (timer expires)
Connected → Backoff (connection lost / pong timeout)

Reconnect: 500ms initial backoff, doubles each attempt, caps at 30s.

---

Key Design Decisions

Decision	Summary
SSZ binary wire format	SSZ over JSON — deterministic, smaller, no need for backwards compatability
Two-phase bid push	Phase A debounce + rate-limit, Phase B deadline force-push with EWMA RTT
Tile/Spine actor framework	Decoupled relay subsystems via event bus
DeliveringPayload → 202 at handler boundaries	submitBlindedBlockV2 idempotency without auctioneer changes
V1 payload cache + stale bid clearing + slot rollover fix	Three-layer fix: V1 body from cache, stale bid guard in cb-pbs, PerSlotState order swap
Fork byte is wire concern	Fork mapping lives in ws-wire, not duplicated
Settlement on submission	received_submission suppresses pushes after blinded block
Subscribe as auth	Signed validator registrations over WS (same crypto as REST)

---

Tuning Parameters

Constant	Default	Location	Purpose
PING_INTERVAL	15s	both	Keep-alive + RTT measurement
PONG_TIMEOUT	5s	cb-pbs	Disconnect if no pong
SUBMIT_ACK_TIMEOUT	500ms	cb-pbs	Wait for WS submission ack
MAX_SUBMIT_RETRIES	2	cb-pbs	Max WS submission retries
MIN_PUSH_INTERVAL	configurable	relay push.rs	Phase A rate limit
EWMA_ALPHA	0.3	relay push.rs	RTT smoothing
INITIAL_BACKOFF_MS	500ms	cb-pbs	WS reconnect initial
MAX_BACKOFF_MS	30s	cb-pbs	WS reconnect cap
MAX_SENDERS_PER_PUBKEY	8	relay push.rs	Anti-abuse connection cap
auction_conclusion_ms range	100–11,000ms	relay mod.rs	Clamp on subscribe