Independent filtering increases detection power for high-throughput experiments

R Bourgon, R Gentleman… - Proceedings of the …, 2010 - National Acad Sciences
R Bourgon, R Gentleman, W Huber
Proceedings of the National Academy of Sciences, 2010National Acad Sciences
With high-dimensional data, variable-by-variable statistical testing is often used to select
variables whose behavior differs across conditions. Such an approach requires adjustment
for multiple testing, which can result in low statistical power. A two-stage approach that first
filters variables by a criterion independent of the test statistic, and then only tests variables
which pass the filter, can provide higher power. We show that use of some filter/test statistics
pairs presented in the literature may, however, lead to loss of type I error control. We …
With high-dimensional data, variable-by-variable statistical testing is often used to select variables whose behavior differs across conditions. Such an approach requires adjustment for multiple testing, which can result in low statistical power. A two-stage approach that first filters variables by a criterion independent of the test statistic, and then only tests variables which pass the filter, can provide higher power. We show that use of some filter/test statistics pairs presented in the literature may, however, lead to loss of type I error control. We describe other pairs which avoid this problem. In an application to microarray data, we found that gene-by-gene filtering by overall variance followed by a t-test increased the number of discoveries by 50%. We also show that this particular statistic pair induces a lower bound on fold-change among the set of discoveries. Independent filtering—using filter/test pairs that are independent under the null hypothesis but correlated under the alternative—is a general approach that can substantially increase the efficiency of experiments.
National Acad Sciences