Add flag evaluation metrics via OTel counter and OpenFeature Hook by typotter · Pull Request #11040 · DataDog/dd-trace-java

typotter · 2026-04-02T15:34:34Z

What Does This Do

Records a feature_flag.evaluations OTel counter metric on every flag evaluation via an OpenFeature finallyAfter hook. The hook captures all evaluation paths including type mismatches that occur above the provider level in the OpenFeature SDK pipeline.

Creates a dedicated SdkMeterProvider with an OtlpHttpMetricExporter that sends metrics directly to the DD Agent's OTLP endpoint (/v1/metrics). This avoids the agent's OTel class shading (io.opentelemetry.api.* → datadog.trace.bootstrap.otel.api.*) which prevents using GlobalOpenTelemetry from the published dd-openfeature jar.

Metric attributes:

Attribute	When present	Value
`feature_flag.key`	Always	Flag key
`feature_flag.result.variant`	Always	Variant key (empty string if null)
`feature_flag.result.reason`	Always	Reason lowercased
`error.type`	On error	ErrorCode lowercased
`feature_flag.result.allocation_key`	When present	Allocation key from flag metadata

New files: FlagEvalMetrics.java, FlagEvalHook.java, FlagEvalMetricsTest.java, FlagEvalHookTest.java
Modified files: Provider.java (adds getProviderHooks()), ProviderTest.java, build.gradle.kts

Motivation

Evaluation metrics allow tracking how many times flags are evaluated, with which results, across sessions. This is the Java implementation of the evaluation logging spec (FFL-1942), matching the existing Python (dd-trace-py#17029) and Go (dd-trace-go#4489) implementations.

System tests: 11/17 pass. The 6 remaining failures are pre-existing DDEvaluator gaps (reason mapping, parse error codes) addressed in separate PRs (#11036, #10971).

References:

Python implementation: feat(openfeature): add flag evaluation metrics dd-trace-py#17029
Go implementation: feat(openfeature): add flag evaluation tracking via OTel Metrics dd-trace-go#4489
Go cross-tracer consistency: fix(openfeature): improve FFE eval metrics cross-tracer consistency dd-trace-go#4590
System tests: DataDog/system-tests (branch sameerank/FFL-1942/add-flag-eval-metrics)

Additional Notes

OTel SDK dependencies (opentelemetry-sdk-metrics, opentelemetry-exporter-otlp) are compileOnly — applications must include them on the classpath for metrics to flow. Falls back to silent no-op when absent.
Export interval: 10s (matching Go SDK and EVALLOG.4 spec)
Endpoint resolution follows OTel spec: OTEL_EXPORTER_OTLP_METRICS_ENDPOINT → OTEL_EXPORTER_OTLP_ENDPOINT + /v1/metrics → http://localhost:4318/v1/metrics

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels
Avoid using close, fix, or any linking keywords when referencing an issue

Jira ticket: FFL-1942

Record a `feature_flag.evaluations` OTel counter on every flag evaluation using an OpenFeature `finallyAfter` hook. The hook captures all evaluation paths including type mismatches that occur above the provider level. Attributes: feature_flag.key, feature_flag.result.variant, feature_flag.result.reason, error.type (on error), feature_flag.result.allocation_key (when present). Counter is a no-op when DD_METRICS_OTEL_ENABLED is false or opentelemetry-api is absent from the classpath.

Replace GlobalOpenTelemetry.getMeterProvider() with a dedicated SdkMeterProvider + OtlpHttpMetricExporter that sends metrics directly to the DD Agent's OTLP endpoint (default :4318/v1/metrics). This avoids the agent's OTel class shading issue where the agent relocates io.opentelemetry.api.* to datadog.trace.bootstrap.otel.api.*, making GlobalOpenTelemetry calls from the dd-openfeature jar hit the unshaded no-op provider instead of the agent's shim. Requires opentelemetry-sdk-metrics and opentelemetry-exporter-otlp on the application classpath. Falls back to no-op if absent. System tests: 11/17 pass. 6 failures are pre-existing DDEvaluator gaps (reason mapping, parse errors, type mismatch strictness).

- Add explicit null guard for details in FlagEvalHook.finallyAfter() - Add OTEL_EXPORTER_OTLP_ENDPOINT generic env var fallback with /v1/metrics path appended (per OTel spec fallback chain) - Add comments clarifying signal-specific vs generic endpoint behavior

When the OTel SDK jars are not on the application classpath, loading FlagEvalMetrics fails because field types reference OTel SDK classes (SdkMeterProvider). This propagated as an uncaught NoClassDefFoundError from the Provider constructor, crashing provider initialization. Fix: - Change meterProvider field type from SdkMeterProvider to Closeable (always on classpath), use local SdkMeterProvider variable inside try block - Catch NoClassDefFoundError in Provider constructor when creating FlagEvalMetrics - Null-safe getProviderHooks() and shutdown() when metrics is null

pr-commenter · 2026-04-08T20:15:44Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	typo/evaluations-logging
git_commit_date	1775076273	1775686186
git_commit_sha	`755e6b7`	`4cb7bab`
release_version	1.61.0-SNAPSHOT~755e6b7855	1.61.0-SNAPSHOT~4cb7bab410

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1775687983	1775687983
ci_job_id	1579663783	1579663783
ci_pipeline_id	106786082	106786082
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-010esk4i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-010esk4i 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 61 metrics, 10 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~4cb7bab410, baseline=1.61.0-SNAPSHOT~755e6b7855

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1058415
Total [baseline] (11.139 s) : 0, 11139058
Agent [candidate] (1.065 s) : 0, 1064762
Total [candidate] (11.172 s) : 0, 11171890
section appsec
Agent [baseline] (1.26 s) : 0, 1259542
Total [baseline] (11.258 s) : 0, 11257715
Agent [candidate] (1.25 s) : 0, 1249787
Total [candidate] (11.135 s) : 0, 11134505
section iast
Agent [baseline] (1.228 s) : 0, 1228245
Total [baseline] (11.324 s) : 0, 11323698
Agent [candidate] (1.227 s) : 0, 1227457
Total [candidate] (11.316 s) : 0, 11316169
section profiling
Agent [baseline] (1.193 s) : 0, 1193024
Total [baseline] (11.096 s) : 0, 11095794
Agent [candidate] (1.187 s) : 0, 1186722
Total [candidate] (11.135 s) : 0, 11134932

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	appsec	1.26 s	201.127 ms (19.0%)
Agent	iast	1.228 s	169.83 ms (16.0%)
Agent	profiling	1.193 s	134.61 ms (12.7%)
Total	tracing	11.139 s	-
Total	appsec	11.258 s	118.657 ms (1.1%)
Total	iast	11.324 s	184.64 ms (1.7%)
Total	profiling	11.096 s	-43.264 ms (-0.4%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.065 s	-
Agent	appsec	1.25 s	185.025 ms (17.4%)
Agent	iast	1.227 s	162.695 ms (15.3%)
Agent	profiling	1.187 s	121.96 ms (11.5%)
Total	tracing	11.172 s	-
Total	appsec	11.135 s	-37.385 ms (-0.3%)
Total	iast	11.316 s	144.279 ms (1.3%)
Total	profiling	11.135 s	-36.957 ms (-0.3%)

gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~4cb7bab410, baseline=1.61.0-SNAPSHOT~755e6b7855

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.211 ms) : 0, 1211
crashtracking [candidate] (1.221 ms) : 0, 1221
BytebuddyAgent [baseline] (632.799 ms) : 0, 632799
BytebuddyAgent [candidate] (637.487 ms) : 0, 637487
AgentMeter [baseline] (29.574 ms) : 0, 29574
AgentMeter [candidate] (29.761 ms) : 0, 29761
GlobalTracer [baseline] (249.767 ms) : 0, 249767
GlobalTracer [candidate] (250.759 ms) : 0, 250759
AppSec [baseline] (32.015 ms) : 0, 32015
AppSec [candidate] (32.238 ms) : 0, 32238
Debugger [baseline] (59.957 ms) : 0, 59957
Debugger [candidate] (60.503 ms) : 0, 60503
Remote Config [baseline] (595.785 µs) : 0, 596
Remote Config [candidate] (603.603 µs) : 0, 604
Telemetry [baseline] (8.022 ms) : 0, 8022
Telemetry [candidate] (8.211 ms) : 0, 8211
Flare Poller [baseline] (8.318 ms) : 0, 8318
Flare Poller [candidate] (7.557 ms) : 0, 7557
section appsec
crashtracking [baseline] (1.208 ms) : 0, 1208
crashtracking [candidate] (1.191 ms) : 0, 1191
BytebuddyAgent [baseline] (667.95 ms) : 0, 667950
BytebuddyAgent [candidate] (663.902 ms) : 0, 663902
AgentMeter [baseline] (12.169 ms) : 0, 12169
AgentMeter [candidate] (12.064 ms) : 0, 12064
GlobalTracer [baseline] (251.182 ms) : 0, 251182
GlobalTracer [candidate] (249.163 ms) : 0, 249163
AppSec [baseline] (185.387 ms) : 0, 185387
AppSec [candidate] (183.752 ms) : 0, 183752
Debugger [baseline] (67.047 ms) : 0, 67047
Debugger [candidate] (65.992 ms) : 0, 65992
Remote Config [baseline] (617.9 µs) : 0, 618
Remote Config [candidate] (591.189 µs) : 0, 591
Telemetry [baseline] (8.863 ms) : 0, 8863
Telemetry [candidate] (8.632 ms) : 0, 8632
Flare Poller [baseline] (3.754 ms) : 0, 3754
Flare Poller [candidate] (3.531 ms) : 0, 3531
IAST [baseline] (24.843 ms) : 0, 24843
IAST [candidate] (24.537 ms) : 0, 24537
section iast
crashtracking [baseline] (1.206 ms) : 0, 1206
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (803.396 ms) : 0, 803396
BytebuddyAgent [candidate] (802.498 ms) : 0, 802498
AgentMeter [baseline] (11.482 ms) : 0, 11482
AgentMeter [candidate] (11.416 ms) : 0, 11416
GlobalTracer [baseline] (239.786 ms) : 0, 239786
GlobalTracer [candidate] (239.619 ms) : 0, 239619
AppSec [baseline] (27.794 ms) : 0, 27794
AppSec [candidate] (29.464 ms) : 0, 29464
Debugger [baseline] (62.178 ms) : 0, 62178
Debugger [candidate] (61.497 ms) : 0, 61497
Remote Config [baseline] (523.024 µs) : 0, 523
Remote Config [candidate] (513.988 µs) : 0, 514
Telemetry [baseline] (14.364 ms) : 0, 14364
Telemetry [candidate] (14.728 ms) : 0, 14728
Flare Poller [baseline] (4.277 ms) : 0, 4277
Flare Poller [candidate] (4.113 ms) : 0, 4113
IAST [baseline] (26.704 ms) : 0, 26704
IAST [candidate] (26.078 ms) : 0, 26078
section profiling
ProfilingAgent [baseline] (94.414 ms) : 0, 94414
ProfilingAgent [candidate] (93.97 ms) : 0, 93970
crashtracking [baseline] (1.178 ms) : 0, 1178
crashtracking [candidate] (1.185 ms) : 0, 1185
BytebuddyAgent [baseline] (697.671 ms) : 0, 697671
BytebuddyAgent [candidate] (693.384 ms) : 0, 693384
AgentMeter [baseline] (9.142 ms) : 0, 9142
AgentMeter [candidate] (9.104 ms) : 0, 9104
GlobalTracer [baseline] (208.187 ms) : 0, 208187
GlobalTracer [candidate] (207.336 ms) : 0, 207336
AppSec [baseline] (32.716 ms) : 0, 32716
AppSec [candidate] (32.557 ms) : 0, 32557
Debugger [baseline] (66.023 ms) : 0, 66023
Debugger [candidate] (65.821 ms) : 0, 65821
Remote Config [baseline] (569.498 µs) : 0, 569
Remote Config [candidate] (569.414 µs) : 0, 569
Telemetry [baseline] (7.82 ms) : 0, 7820
Telemetry [candidate] (7.818 ms) : 0, 7818
Flare Poller [baseline] (3.613 ms) : 0, 3613
Flare Poller [candidate] (3.591 ms) : 0, 3591
Profiling [baseline] (94.993 ms) : 0, 94993
Profiling [candidate] (94.539 ms) : 0, 94539

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~4cb7bab410, baseline=1.61.0-SNAPSHOT~755e6b7855

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.058 s) : 0, 1057839
Total [baseline] (8.856 s) : 0, 8855789
Agent [candidate] (1.056 s) : 0, 1056397
Total [candidate] (8.898 s) : 0, 8897978
section iast
Agent [baseline] (1.225 s) : 0, 1225124
Total [baseline] (9.584 s) : 0, 9584474
Agent [candidate] (1.234 s) : 0, 1234170
Total [candidate] (9.58 s) : 0, 9579915

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.058 s	-
Agent	iast	1.225 s	167.285 ms (15.8%)
Total	tracing	8.856 s	-
Total	iast	9.584 s	728.686 ms (8.2%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.056 s	-
Agent	iast	1.234 s	177.774 ms (16.8%)
Total	tracing	8.898 s	-
Total	iast	9.58 s	681.937 ms (7.7%)

gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~4cb7bab410, baseline=1.61.0-SNAPSHOT~755e6b7855

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.21 ms) : 0, 1210
crashtracking [candidate] (1.213 ms) : 0, 1213
BytebuddyAgent [baseline] (634.195 ms) : 0, 634195
BytebuddyAgent [candidate] (632.941 ms) : 0, 632941
AgentMeter [baseline] (29.509 ms) : 0, 29509
AgentMeter [candidate] (29.397 ms) : 0, 29397
GlobalTracer [baseline] (249.512 ms) : 0, 249512
GlobalTracer [candidate] (249.407 ms) : 0, 249407
AppSec [baseline] (31.999 ms) : 0, 31999
AppSec [candidate] (32.002 ms) : 0, 32002
Debugger [baseline] (59.079 ms) : 0, 59079
Debugger [candidate] (59.183 ms) : 0, 59183
Remote Config [baseline] (604.101 µs) : 0, 604
Remote Config [candidate] (588.965 µs) : 0, 589
Telemetry [baseline] (8.028 ms) : 0, 8028
Telemetry [candidate] (8.065 ms) : 0, 8065
Flare Poller [baseline] (7.381 ms) : 0, 7381
Flare Poller [candidate] (7.419 ms) : 0, 7419
section iast
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.201 ms) : 0, 1201
BytebuddyAgent [baseline] (801.367 ms) : 0, 801367
BytebuddyAgent [candidate] (808.151 ms) : 0, 808151
AgentMeter [baseline] (11.384 ms) : 0, 11384
AgentMeter [candidate] (11.526 ms) : 0, 11526
GlobalTracer [baseline] (239.222 ms) : 0, 239222
GlobalTracer [candidate] (240.83 ms) : 0, 240830
IAST [baseline] (26.065 ms) : 0, 26065
IAST [candidate] (26.148 ms) : 0, 26148
AppSec [baseline] (32.281 ms) : 0, 32281
AppSec [candidate] (30.56 ms) : 0, 30560
Debugger [baseline] (57.796 ms) : 0, 57796
Debugger [candidate] (59.813 ms) : 0, 59813
Remote Config [baseline] (525.359 µs) : 0, 525
Remote Config [candidate] (521.088 µs) : 0, 521
Telemetry [baseline] (14.854 ms) : 0, 14854
Telemetry [candidate] (14.828 ms) : 0, 14828
Flare Poller [baseline] (3.867 ms) : 0, 3867
Flare Poller [candidate] (3.83 ms) : 0, 3830

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	typo/evaluations-logging
git_commit_date	1775076273	1775686186
git_commit_sha	`755e6b7`	`4cb7bab`
release_version	1.61.0-SNAPSHOT~755e6b7855	1.61.0-SNAPSHOT~4cb7bab410

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1775688548	1775688548
ci_job_id	1579663784	1579663784
ci_pipeline_id	106786082	106786082
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-ml19q19m 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-ml19q19m 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 18 metrics, 17 unstable metrics.

scenario	Δ mean agg_http_req_duration_p50	Δ mean agg_http_req_duration_p95	Δ mean throughput	candidate mean agg_http_req_duration_p50	candidate mean agg_http_req_duration_p95	candidate mean throughput	baseline mean agg_http_req_duration_p50	baseline mean agg_http_req_duration_p95	baseline mean throughput
scenario:load:petclinic:no_agent:high_load	better [-2.265ms; -0.624ms] or [-11.922%; -3.282%]	unstable [-3.913ms; -0.559ms] or [-12.488%; -1.784%]	unstable [-10.932op/s; +46.932op/s] or [-4.520%; +19.406%]	17.557ms	29.099ms	259.844op/s	19.001ms	31.335ms	241.844op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~4cb7bab410, baseline=1.61.0-SNAPSHOT~755e6b7855
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.233 ms) : 1220, 1245
.   : milestone, 1233,
iast (3.307 ms) : 3257, 3358
.   : milestone, 3307,
iast_FULL (5.968 ms) : 5907, 6029
.   : milestone, 5968,
iast_GLOBAL (3.737 ms) : 3675, 3800
.   : milestone, 3737,
profiling (2.213 ms) : 2191, 2235
.   : milestone, 2213,
tracing (1.885 ms) : 1869, 1901
.   : milestone, 1885,
section candidate
no_agent (1.242 ms) : 1230, 1253
.   : milestone, 1242,
iast (3.236 ms) : 3192, 3280
.   : milestone, 3236,
iast_FULL (6.005 ms) : 5944, 6066
.   : milestone, 6005,
iast_GLOBAL (3.687 ms) : 3620, 3755
.   : milestone, 3687,
profiling (2.245 ms) : 2225, 2265
.   : milestone, 2245,
tracing (1.855 ms) : 1840, 1870
.   : milestone, 1855,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.233 ms [1.22 ms, 1.245 ms]	-
iast	3.307 ms [3.257 ms, 3.358 ms]	2.075 ms (168.3%)
iast_FULL	5.968 ms [5.907 ms, 6.029 ms]	4.736 ms (384.2%)
iast_GLOBAL	3.737 ms [3.675 ms, 3.8 ms]	2.505 ms (203.2%)
profiling	2.213 ms [2.191 ms, 2.235 ms]	980.736 µs (79.6%)
tracing	1.885 ms [1.869 ms, 1.901 ms]	652.55 µs (52.9%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.242 ms [1.23 ms, 1.253 ms]	-
iast	3.236 ms [3.192 ms, 3.28 ms]	1.994 ms (160.6%)
iast_FULL	6.005 ms [5.944 ms, 6.066 ms]	4.763 ms (383.6%)
iast_GLOBAL	3.687 ms [3.62 ms, 3.755 ms]	2.446 ms (197.0%)
profiling	2.245 ms [2.225 ms, 2.265 ms]	1.003 ms (80.8%)
tracing	1.855 ms [1.84 ms, 1.87 ms]	613.373 µs (49.4%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~4cb7bab410, baseline=1.61.0-SNAPSHOT~755e6b7855
    dateFormat X
    axisFormat %s
section baseline
no_agent (19.304 ms) : 19107, 19502
.   : milestone, 19304,
appsec (18.748 ms) : 18559, 18937
.   : milestone, 18748,
code_origins (17.749 ms) : 17575, 17924
.   : milestone, 17749,
iast (17.919 ms) : 17742, 18095
.   : milestone, 17919,
profiling (18.915 ms) : 18728, 19103
.   : milestone, 18915,
tracing (18.816 ms) : 18631, 19001
.   : milestone, 18816,
section candidate
no_agent (17.962 ms) : 17780, 18143
.   : milestone, 17962,
appsec (18.784 ms) : 18597, 18970
.   : milestone, 18784,
code_origins (17.854 ms) : 17679, 18029
.   : milestone, 17854,
iast (17.903 ms) : 17725, 18081
.   : milestone, 17903,
profiling (19.722 ms) : 19523, 19922
.   : milestone, 19722,
tracing (18.194 ms) : 18016, 18373
.   : milestone, 18194,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	19.304 ms [19.107 ms, 19.502 ms]	-
appsec	18.748 ms [18.559 ms, 18.937 ms]	-556.43 µs (-2.9%)
code_origins	17.749 ms [17.575 ms, 17.924 ms]	-1.555 ms (-8.1%)
iast	17.919 ms [17.742 ms, 18.095 ms]	-1.386 ms (-7.2%)
profiling	18.915 ms [18.728 ms, 19.103 ms]	-389.239 µs (-2.0%)
tracing	18.816 ms [18.631 ms, 19.001 ms]	-488.436 µs (-2.5%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	17.962 ms [17.78 ms, 18.143 ms]	-
appsec	18.784 ms [18.597 ms, 18.97 ms]	821.851 µs (4.6%)
code_origins	17.854 ms [17.679 ms, 18.029 ms]	-107.985 µs (-0.6%)
iast	17.903 ms [17.725 ms, 18.081 ms]	-58.526 µs (-0.3%)
profiling	19.722 ms [19.523 ms, 19.922 ms]	1.761 ms (9.8%)
tracing	18.194 ms [18.016 ms, 18.373 ms]	232.626 µs (1.3%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	typo/evaluations-logging
git_commit_date	1775076273	1775686186
git_commit_sha	`755e6b7`	`4cb7bab`
release_version	1.61.0-SNAPSHOT~755e6b7855	1.61.0-SNAPSHOT~4cb7bab410

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1775688151	1775688151
ci_job_id	1579663785	1579663785
ci_pipeline_id	106786082	106786082
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-0-5ob9owme 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-0-5ob9owme 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~4cb7bab410, baseline=1.61.0-SNAPSHOT~755e6b7855
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.503 s) : 15503000, 15503000
.   : milestone, 15503000,
appsec (14.907 s) : 14907000, 14907000
.   : milestone, 14907000,
iast (18.419 s) : 18419000, 18419000
.   : milestone, 18419000,
iast_GLOBAL (17.857 s) : 17857000, 17857000
.   : milestone, 17857000,
profiling (15.595 s) : 15595000, 15595000
.   : milestone, 15595000,
tracing (14.908 s) : 14908000, 14908000
.   : milestone, 14908000,
section candidate
no_agent (14.999 s) : 14999000, 14999000
.   : milestone, 14999000,
appsec (14.461 s) : 14461000, 14461000
.   : milestone, 14461000,
iast (18.98 s) : 18980000, 18980000
.   : milestone, 18980000,
iast_GLOBAL (17.975 s) : 17975000, 17975000
.   : milestone, 17975000,
profiling (15.687 s) : 15687000, 15687000
.   : milestone, 15687000,
tracing (14.808 s) : 14808000, 14808000
.   : milestone, 14808000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.503 s [15.503 s, 15.503 s]	-
appsec	14.907 s [14.907 s, 14.907 s]	-596.0 ms (-3.8%)
iast	18.419 s [18.419 s, 18.419 s]	2.916 s (18.8%)
iast_GLOBAL	17.857 s [17.857 s, 17.857 s]	2.354 s (15.2%)
profiling	15.595 s [15.595 s, 15.595 s]	92.0 ms (0.6%)
tracing	14.908 s [14.908 s, 14.908 s]	-595.0 ms (-3.8%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	14.999 s [14.999 s, 14.999 s]	-
appsec	14.461 s [14.461 s, 14.461 s]	-538.0 ms (-3.6%)
iast	18.98 s [18.98 s, 18.98 s]	3.981 s (26.5%)
iast_GLOBAL	17.975 s [17.975 s, 17.975 s]	2.976 s (19.8%)
profiling	15.687 s [15.687 s, 15.687 s]	688.0 ms (4.6%)
tracing	14.808 s [14.808 s, 14.808 s]	-191.0 ms (-1.3%)

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~4cb7bab410, baseline=1.61.0-SNAPSHOT~755e6b7855
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.483 ms) : 1472, 1495
.   : milestone, 1483,
appsec (2.527 ms) : 2472, 2581
.   : milestone, 2527,
iast (2.265 ms) : 2197, 2334
.   : milestone, 2265,
iast_GLOBAL (2.302 ms) : 2233, 2371
.   : milestone, 2302,
profiling (2.514 ms) : 2348, 2680
.   : milestone, 2514,
tracing (2.067 ms) : 2013, 2120
.   : milestone, 2067,
section candidate
no_agent (1.48 ms) : 1469, 1492
.   : milestone, 1480,
appsec (2.515 ms) : 2461, 2570
.   : milestone, 2515,
iast (2.265 ms) : 2196, 2333
.   : milestone, 2265,
iast_GLOBAL (2.304 ms) : 2235, 2373
.   : milestone, 2304,
profiling (2.113 ms) : 2057, 2169
.   : milestone, 2113,
tracing (2.07 ms) : 2017, 2123
.   : milestone, 2070,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.483 ms [1.472 ms, 1.495 ms]	-
appsec	2.527 ms [2.472 ms, 2.581 ms]	1.043 ms (70.3%)
iast	2.265 ms [2.197 ms, 2.334 ms]	781.999 µs (52.7%)
iast_GLOBAL	2.302 ms [2.233 ms, 2.371 ms]	818.155 µs (55.2%)
profiling	2.514 ms [2.348 ms, 2.68 ms]	1.031 ms (69.5%)
tracing	2.067 ms [2.013 ms, 2.12 ms]	583.052 µs (39.3%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.48 ms [1.469 ms, 1.492 ms]	-
appsec	2.515 ms [2.461 ms, 2.57 ms]	1.035 ms (69.9%)
iast	2.265 ms [2.196 ms, 2.333 ms]	784.513 µs (53.0%)
iast_GLOBAL	2.304 ms [2.235 ms, 2.373 ms]	823.858 µs (55.7%)
profiling	2.113 ms [2.057 ms, 2.169 ms]	632.629 µs (42.7%)
tracing	2.07 ms [2.017 ms, 2.123 ms]	589.843 µs (39.8%)

FlagEvalHook references FlagEvalMetrics in its field declaration. On JVMs that eagerly verify field types during class loading, constructing FlagEvalHook outside the try/catch could throw NoClassDefFoundError if OTel classes failed to load. Moving it inside the try block ensures both metrics and hook are null-safe when OTel is absent.

typotter added 3 commits April 1, 2026 20:29

typotter added type: feature request tag: ai generated Largely based on code generated by an AI or LLM comp: openfeature OpenFeature labels Apr 2, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add flag evaluation metrics via OTel counter and OpenFeature Hook#11040

Add flag evaluation metrics via OTel counter and OpenFeature Hook#11040
typotter wants to merge 5 commits intomasterfrom
typo/evaluations-logging

typotter commented Apr 2, 2026 •

edited by atlassian bot

Loading

Uh oh!

pr-commenter bot commented Apr 8, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

typotter commented Apr 2, 2026 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

pr-commenter bot commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

typotter commented Apr 2, 2026 •

edited by atlassian bot

Loading

pr-commenter bot commented Apr 8, 2026 •

edited

Loading