What is power? Merriam Webster defines power as the “possession of control, authority, or influence over others.” The power I will talk about today, however, is statistical power. Statistical power measures the ability of a statistical test to determine whether the null hypothesis is false. For instance, in the U.S. judicial system, the null hypothesis is that the defendant is innocent. Trials that can more accurately determine when the defendant is in fact guilty have more power.
In statistics, there are two types of errors: Type I and Type II. The probability of a Type II error, a false negative, is represented by the symbol β. Thus, the probability of correctly rejecting the null (i.e., the power) is 1-β.
The larger the magnitude of the hypothesized effect, the higher the power. It is much easier to detect a large effect than a small effect. Also, as the size of the sample increases, so does a test’s statistical power.
The more variation that exists in the data, however, the lower the power. If there is a lot of variation in the data, it is difficult to determine if null hypothesis is false or if observing a phenomenon that contradicts the null is simply due to the excessive amount of variability in the data. On the hand, if the variability (i.e., standard deviation) is low, then one can generally conclude that that the null hypothesis is false, since the low variability indicates that the anomaly is not caused by normal variation in the data.