Solution
The Step-Up Surprise
Show the problem again
Ten hypotheses are tested, with p-values 0.720, 0.049, 0.001, 0.205, 0.060, 0.008, 0.390, 0.045, 0.074, 0.021 (in no particular order). Apply the Benjamini-Hochberg procedure at false discovery rate level q = 0.10: sort the p-values as p(1) ≤ ... ≤ p(10), find the largest k with p(k) ≤ 0.10 k / 10, and reject the k hypotheses with the smallest p-values. How many hypotheses are rejected? Give an integer.
Worked solution
Sorted: 0.001, 0.008, 0.021, 0.045, 0.049, 0.060, 0.074, 0.205, 0.390, 0.720, with thresholds 0.01, 0.02, 0.03, ..., 0.10. Rank 4 fails (0.045 > 0.04), rank 5 passes (0.049 ≤ 0.05), rank 6 passes (0.060 ≤ 0.06), rank 7 fails (0.074 > 0.07), and ranks 8 to 10 fail. The largest passing rank is k = 6, so six hypotheses are rejected, including the fourth, which on its own missed its threshold. This is the step-up property: only the largest passing rank matters. Bonferroni at the same level (0.01 per test) would reject only two.
Source: Benjamini and Hochberg (1995) step-up procedure for false discovery rate control. Statement written for AxiomIQ.