Define Devnet-3 metrics

[KatyaRyazantseva]

This issue is a proposal to define and implement an initial set of metrics for Devnet-3.
The following metrics and areas can be discussed:

• lean_is_aggregator (gauge)
• lean_committee_attestation_subnet (gauge)
• lean_committee_attestation_subnets_count (gauge) - constant from config
• lean_aggregated_committee_signatures_total (counter)
• lean_attestations_in_committee_signatures_total (counter)
• lean_gossip_signatures_count (gauge)
• lean_latest_new_aggregated_payloads_count (gauge)
• lean_latest_known_aggregated_payloads_count (gauge)
• lean_attestations_in_aggregated_payloads_total (counter)
• lean_extracted_attestations_total (counter)
• lean_aggregated_proofs_total (counter)

Please, share your thoughts and suggestions below.

[bomanaps]

Suggested Additions

A few metrics that would help with debugging the aggregation pipeline:

Metric	Type	Rationale
lean_gossip_attestation_received_total	counter	Track attestations before subnet filtering
lean_cross_subnet_attestations_filtered_total	counter	Track how many are filtered due to subnet mismatch
lean_signature_verification_failed_total	counter	Track verification failures
lean_tick_interval_aggregation_triggered_total	counter	Track interval-2 aggregation triggers

These would help diagnose issues like "why aren't attestations being aggregated?" by showing where in the pipeline they're being dropped.

Questions

1. lean_is_aggregator - Should this be a static gauge (set once based on config) or dynamic (updated per-slot based on committee assignment)?
2. lean_attestations_in_committee_signatures_total vs lean_aggregated_gossip_signatures - These seem to measure similar things (signatures collected for aggregation). Should we clarify the distinction?
3. Labeling - Should counters like lean_aggregated_proofs_total have labels for slot or attestation_data_root for more granular tracking?
   cc @unnawut

[KatyaRyazantseva]

1. lean_is_aggregator - Should this be a static gauge (set once based on config) or dynamic (updated per-slot based on committee assignment)?

Currently, is_aggregator is strictly defined when the node starts and remains static throughout its runtime.

2. lean_attestations_in_committee_signatures_total vs lean_aggregated_gossip_signatures - These seem to measure similar things (signatures collected for aggregation). Should we clarify the distinction?

This part is WIP, It may be refactored.

3. Labeling - Should counters like lean_aggregated_proofs_total have labels for slot or attestation_data_root for more granular tracking?

Slot values change every 4s. The standard Prometheus scrape interval is 15s and it's configurable. Adding slots as labels would result in continuous label growth, which is considered a bad practice in Prometheus as it leads to cardinality issues and degraded performance.

[KatyaRyazantseva]

lean_gossip_attestation_received_total (counter) Track attestations before subnet filtering

We already have lean_attestations_valid/invalid_total with gossip label.

lean_signature_verification_failed_total (counter) Track verification failures

Will add lean_pq_sig_individual_signatures_valid/invalid_total.

lean_tick_interval_aggregation_triggered_total (counter) Track interval-2 aggregation triggers

Triggering interval in the metric is not relevant imo, because of the scrape interval. See the comment above.

lean_cross_subnet_attestations_filtered_total (counter) Track how many are filtered due to subnet mismatch

Could you please clarify this one? The link from the specs would be nice to have.