Solution
The Sign Test That Cannot Reach Five Percent
Show the problem again
Seventeen subjects are measured before and after a treatment. Four of them show no change and are dropped, leaving 13 non-zero differences. Under $H_0$ (median difference zero) the number of positive differences is Binomial(13, 1/2). The two-sided sign test rejects when the count of positive differences lies in a symmetric region $\{0,\dots,c\}\cup\{13-c,\dots,13\}$, where $c$ is chosen as large as possible subject to the true size being at most 0.05. What is the exact true size (type I error probability) of this test? Give a reduced fraction.
Worked solution
Tail probabilities are $P(X\le c)=\sum_{i\le c}\binom{13}{i}/8192$. Cumulative counts: $c=2$ gives $1+13+78=92$; $c=3$ adds $\binom{13}3=286$ to give 378. Two-sided size for $c=3$ is $756/8192\approx 0.092>0.05$, too big. For $c=2$ it is $184/8192=23/1024\approx 0.0225$. So the largest admissible region is $\{0,1,2,11,12,13\}$ and its true size is 23/1024, well under the nominal 5%, which is the usual conservativeness of exact tests on small discrete samples.
Source: Original problem written for AxiomIQ.