FDR correction

Test five metrics at α = 0.05 and the chance of at least one false positive is ~23%, not 5%. Multi-metric experiments need multiple-comparison control, or "significant" becomes noise.

Benjamini–Hochberg in LLMJury

LLMJury controls the false discovery rate (FDR) — the expected fraction of significant results that are false — with the Benjamini–Hochberg procedure, applied across all metric × treatment comparisons in an experiment run.

Every result carries both values:

• p_value — the raw, uncorrected p-value from the metric's test.
• p_value_fdr — the BH-adjusted value. Significance is judged on this one (< 0.05 by default), in the dashboard and in the API.

Why both are shown

The raw p-value tells you what the single test saw; the corrected one tells you what you can claim given how many questions you asked. Showing both keeps the analysis honest and auditable — the same philosophy as recording the test, window, and permutation count on every result.