Skip to content

edge: add integration tests, synthetic publisher, and devnet guide#3525

Draft
armcconnell wants to merge 1 commit intoedge/3-runner-manager-clifrom
edge/4-integration-tests
Draft

edge: add integration tests, synthetic publisher, and devnet guide#3525
armcconnell wants to merge 1 commit intoedge/3-runner-manager-clifrom
edge/4-integration-tests

Conversation

@armcconnell
Copy link
Copy Markdown
Contributor

Part 4 of 4 — edge feed parser stack

  1. edge: add Top-of-Book parser framework #3522 — edge: add Top-of-Book parser framework ← start here
  2. edge: add JSON/CSV/Unix-socket output sinks #3523 — edge: add JSON/CSV/Unix-socket output sinks
  3. edge: add feed runner, manager, HTTP API, and CLI #3524 — edge: add feed runner, manager, HTTP API, and CLI
  4. this PR — edge: add integration tests, synthetic publisher, and devnet guide

Draft until the prior three land.

Summary

  • Integration tests exercise parser + sink end-to-end against synthetic wire bytes: normal refdata-then-marketdata ordering, cold-start buffering where marketdata arrives first, the buffer cap + most-recent-wins overwrite semantics, CSV output, and JSON output.
  • testutil/publisher.go is a programmable synthetic Top-of-Book publisher that builds raw wire-format frames (header + messages) for deterministic tests, with no real feed needed.
  • testutil/cmd/topofbook-publisher/ wraps the publisher as a CLI tool that streams frames onto a multicast group — useful for local devnet testing where a real producer isn't available.
  • DEVNET_TEST.md walks through bringing the edge feature up end-to-end in the local devnet: containers, device setup, multicast group creation with subscriber allowlist, running the synthetic publisher on one client, enabling the edge feed on another, and verifying records flow through.

Testing Verification

  • Integration tests run in go test with no special infrastructure.
  • DEVNET_TEST.md was dogfooded during development — any developer can follow it end-to-end to reproduce the feature locally.

Rounds out the edge feed parser with the test and operator materials
that let someone run the feature themselves.

The integration tests exercise the full pipeline — parser + sink —
against synthetic wire-format bytes, covering: normal
refdata-then-marketdata ordering, the cold-start scenario where
marketdata arrives first and is buffered until refdata lands, the
buffer cap + most-recent-wins overwrite semantics, CSV output, and
JSON output.

The testutil package provides a programmable synthetic Top-of-Book
publisher. publisher.go builds raw frames in the wire format
(header + messages) and is used by tests to construct deterministic
scenarios without needing a real feed. A small topofbook-publisher
binary wraps it as a CLI tool that streams frames onto a multicast
group, useful for local devnet testing where a real producer isn't
available.

DEVNET_TEST.md walks through bringing the feature up end-to-end in
the local devnet: building containers, adding devices and clients,
creating a multicast group with subscriber allowlist, running the
synthetic publisher on one client, enabling the edge feed on
another, and verifying records flow through.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant