yellowbrickstats.com

~ a loose collection of statistical and quantitative research material
for fun and enrichment ~

yellow brick road


Critique:   Occasional Commentary on
Other Researchers' Methods and Analyses

by roland b. stark

yellowbrickstats homepage || my statistical and research consulting




Readmission Rates: 58% of Variance Explained!?



“Fifty-eight percent of national variation in hospital readmission rates was explained by the county in which the hospital was located,” announce Jeph Herrin et al. in Community Factors and Hospital Readmission Rates, published in 2014 in Health Services Research. Sound odd to you? After all, for most readmission studies the percent explained is in the single digits. Being able to account for 4 or 5% of the variation translates to an ability to assess individual risk that can meaningfully aid in clinical decisions. Even Harlan Krumholz and his team of 17 researchers and statisticians, the ones whose predictive models form the basis for the national readmission penalty system imposed by Medicare, have usually only explained 3-8%. And those models have taken into account about 50 input variables.

It turns out that Herrin et al. took their data on 4,073 hospitals and broke it down by 2,254 counties. There were almost as many counties as hospitals themselves. And many counties contained only a single hospital.

Now, suppose the authors had divided the 4,073 into, say, 4 groups defined by region, and found that the 4 groups had sizeable differences in readmission rate. That would have been a meaningful way to summarize the data. Even if they had formed somewhat more groups—say, one for each of the 50 states—that might have been meaningful; the data would have been spread pretty thin for some states. But to “explain” differences using 2,254 groups? It’s not a far cry from simply listing the readmission rates of all 4,073 hospitals and claiming victoriously to have “explained” 100% of the variance in the hospital-to-hospital rate.

One reason why this matters a great deal is that, to the extent that some geographic factor is considered responsible for this outcome, hospital performance will no longer be considered responsible. So if county in fact explained 58% of the variance, then hospital performance, it might be argued, couldn't account for more than 42%. This is the incorrect conclusion that was unabashedly reported by news outlets such as Becker's Hospital Review.

The article by Herrin and colleagues makes contributions in other ways, of course, but the chief findings are very misleading. Watch for dialogue, in Health Services Research or elsewhere, on how to interpret the results. The upshot should be quite a bit more nuanced and moderated than what we've seen above. And if you're interested in the role of socioeconomic factors in hospital readmission, you'll find information at ReInforced Care, Inc.


yellowbrickstats.com homepage