How useful are health rankings?

According to an HSR editorial by Stephan Arndt, the answer is not very.  Generally, county level health rankings are too variable to be of much use.  Further, while some metropolitan regions may have large sample sizes, the sample sizes in less densely populated rural counties will be far lower leading to less precise estimates of any quality measure of interest.

As an exercise to demonstrate the lack of reliability, the author did the following:

I calculated the gross overall mortality rate in Iowa as thetotal number of deaths (Bureau of Vital Statistics, Iowa Department of Public Health 2009) divided by Iowa’s 2010 U.S. Census population total. Using this constant mortality rate, I then generated a random Poisson value for each county in Iowa based on the expected number of deaths given the constant state rate and the county population totals. Then, I calculated the counties’ observed mortality rate. Note that the rate parameter was constant across all counties, so no county had any more or less “problem” than another.Nonetheless, there was a large difference among the observed rates, randomly affecting county rank. The lowest mortality rate (rank = 1) was in Monroe County with 66.5 deaths per 10,000, and the highest mortality rate (rank = 99) was in Adams County with 114.2 deaths per 10,000. Adams County had over 1.7 times the mortality rate as that of Monroe County. Of course, that is nonsense since these are random variations around a constant rate.

In summary, even when the true baseline mortality risk is identical across counties, we still observe significant variation in actual mortality across regions due to random noise.  This is not to say that all studies looking at regional variation in quality are not worthwhile, but rather that there should be some caution when interpreting these results, particularly when there are small sample sizes in certain areas.



1 Comment

Leave a Reply

Your email address will not be published. Required fields are marked *