Econometrics P4P Quality

Finite Population Correction

Asymptotic theory has played a large role in the development of many recent econometric methods. For instance, the central limit theorem states that distribution of the mean drawn from any large samples is approximately normally distributed. Asymptotic theory, however, generally assumes that sampling occurs infinitely and with replacement. In the real world, populations are not infinite and sampling does not occur with replacement.

To take into account these real-world challenges, a finite population correction (fpc) factor is needed. One can express the fpc mathematically as:

  • fpc={(N-n)/(N-1)}1/2

where n is the sample size and N is the population size. For instance, one can calculate the standard error for the mean for finite populations as:

  • σX_bar=σ*n1/2 * {(N-n)/(N-1)}1/2
  • σp_bar={[p(1-p)/(n)}1/2 * {(N-n)/(N-1)}1/2

This website has some examples of how to apply fpc‘s in practice.

One potential application for fpc is physician ratings. Physicians who treat lots of patients eligible for a given quality measure certainly can have an accurate score. Some physicians, however, treat only a handful of patients eligible for any given quality metric. In this case, should the physician be punished if he happens to have one bad quality score among the very few observations? Can fpc correct for this problem?

A paper by Elliott, Zaslavsky and Cleary argues that fpc is not appropriate for adjusting physicians scores or confidence intervals to take into account the physicians small sample size. They cite work by Birnbaum who argues that profiling of hospitals is essentially an attempt to make inferences about future performance at the same facility if nothing changes, and involves “a theoretically infinite population.” The authors give the following example to explain why the fpc is not appropriate for rating physicians.

…suppose we had 50 responses out of 50 total patients at a small hospital, and 300 responses out of 1000 total patients at a larger hospital. Under the finite population model, there is no sampling variability at the smaller hospital (because we have information for all patients) but considerable sampling variability at the larger hospital, which is the appropriate inference if all we care about is the experience of those 50 and 1000 patients. However, to tell a new group of patients what their experiences are likely to be like at each hospital, we have much more information about the large hospital.

Our concern, then, is that FPSM-based approaches would under-represent the uncertainty in data for small facilities with high (possibly 100%) sampling rates, misleading users into thinking that such a facility would be likely to provide below-average (or above-average) care to them.

To correct for the small sample size problem for providers treating few patients eligible for quality scores each year, the authors believe using a moving average score over multiple years would be more appropriate. The benefit of this method is that the sample size increases, but the drawback is that provider quality improvements will not be fully reflected in the provider’s score for a number of years.

Leave a Reply

Your email address will not be published. Required fields are marked *