Solution
A Second Look at the Data
Show the problem again
Under the null hypothesis, a z-test statistic is computed on a first batch of n observations, giving Z1. If |Z1| > 1.96 the experimenter stops and rejects. Otherwise a second batch of n observations is added, and the z-statistic Z2 of the pooled 2n observations is computed; the experimenter rejects if |Z2| > 1.96. The observations are i.i.d. standard normal with known variance. What is the overall probability of rejecting a true null? Give your answer rounded to 4 decimal places; 3 decimal places is also accepted.
Worked solution
Z1 and Z2 are jointly standard normal with correlation ρ = √(n/2n) = 1/√2, because Z2 = (Z1 + W)/√2 with W an independent standard normal. The probability of no rejection at either look is P(|Z1| ≤ 1.96, |Z2| ≤ 1.96), a bivariate normal rectangle probability, which evaluates to about 0.9169. The overall Type I error is therefore 1 - 0.9169 ≈ 0.0831, roughly 66% above the nominal 5%. (A Monte Carlo run with 4 million samples gives 0.0832.)
Source: Group-sequential testing; two-look error inflation is a standard calculation (cf. Armitage, McPherson and Rowe, 1969). Statement written for AxiomIQ.