Familywise Error Rate Website Links For
Error
 

Information About

Familywise Error Rate





CLASSIFICATION OF ''M'' HYPOTHESIS TESTS

The following table defines some random variables related to the m hypothesis tests.

The ''m'' specific hypotheses of interest are assumed to be known, but the number of true null hypotheses ''m''0 and of alternative hypotheses ''m''1, are unknown. ''V'' is the number of Type I Errors (hypotheses declared significant when they are actually from the null distribution). ''T'' is the number of Type II Errors (hypotheses declared not significant when they are actually from the alternative distribution). ''R'' is an observable Random Variable , while ''S'', ''T'' , ''U'', and ''V'' are unobservable random variables.

In terms of random variables,

: \mathrm{FWER} = \Pr(V \ge 1), \,

or equivalently,

: \mathrm{FWER} = 1 -\Pr(V = 0).

=What constitutes a family=

In confirmatory studies (i.e., where one specifies a finite number of a priori inferences), families of hypotheses are defined by which conclusions need to be jointly accurate or by which hypotheses are similar in content/purpose. As noted by Hochberg and Tamrane (1987), "If these inferences are unrelated in terms of their content or intended use (although they may be statistically dependent), then they should be treated separately and not jointly" (p. 6).

For example, one might conduct a randomized clinical trial for a new antidepressant drug using three groups: existing drug, new drug, and placebo. In such a design, one might be interested in whether depressive symptoms (measured, for example, by a Beck Depression Inventory score) decreased to a greater extent for those using the new drug compared to the old drug. Further, one might be interested in whether any side effects (e.g., hypersomnia, decreased sex drive, and dry mouth) were observed. In such a case, two families would likely be identified: 1) effect of drug on depressive symptoms, 2) occurrence of any side effects.

  • 3 side effects --- 3 pairwise comparisons per side effect = 0.45 (i.e., 45% chance of making a Type I error). Thus, a more appropriate control for side effect family-wise error might divide alpha by three (.05/3 = .0167) and allocate .0167 to each side effect multiple comparison procedure. In the case of Tukey's HSD (a strong control multiple comparison procedure), one would determine the critical value of Q, the studentized range statistic, based on the alpha of .0167.


=See also=


=References=

Hochberg, Y., & Tamhane, A. C. (1987). ''Multiple comparison procedures''. New York: Wiley.