~ a loose collection of statistical and quantitative research material
for fun and enrichment ~
Critique: Occasional Commentary on
Other Researchers' Methods and Analyses
by roland b. stark
Readmission Rates: 58% of Variance Explained!?
“Fifty-eight percent of national variation in hospital readmission rates was explained by the county in which the hospital
was located,” announce Jeph Herrin et al. in
Community Factors and Hospital Readmission Rates, published in 2014 in
Health Services Research.
Sound odd to you?
After all, for most readmission studies the percent explained is in the
single digits. Being able to account for 4 or 5% of the variation translates to an ability to
assess individual risk
that can meaningfully aid in clinical decisions.
Even Harlan Krumholz and his team of 17 researchers and statisticians, the ones
whose predictive models form the basis for the national readmission penalty system imposed by Medicare, have usually
only explained 3-8%. And those models have taken into account about 50 input variables.
It turns out that Herrin et al. took their data on 4,073 hospitals and broke it down by 2,254 counties.
There were almost
as many counties as hospitals themselves. And many counties contained only a single hospital.
Now, suppose the authors had divided the 4,073 into, say, 4 groups defined by region, and found that the 4 groups had
sizeable differences in readmission rate. That would have been a meaningful way to summarize the data.
Even if they had formed somewhat more groups—say, one for each of the 50 states—that might
have been meaningful; the data would have been spread pretty thin for some states.
But to “explain” differences using 2,254 groups? It’s not a far cry from simply listing the readmission rates of all 4,073
hospitals and claiming victoriously to have “explained” 100% of the variance in the hospital-to-hospital rate.
One reason why this matters a great deal is that, to the extent that some geographic factor is considered
responsible for this outcome, hospital performance will no longer be considered responsible. So if county
in fact explained 58% of the variance, then hospital performance, it might be argued, couldn't account for more than 42%. This is the
incorrect conclusion that was unabashedly reported by news outlets such as Becker's Hospital Review.
The article by Herrin and colleagues makes contributions in other ways, of course, but the chief findings are very
misleading. Watch for
dialogue, in Health Services Research or elsewhere, on how to interpret the results. The upshot should be quite
a bit more nuanced and moderated than what we've seen above. And if you're interested in the role
of socioeconomic factors in hospital readmission, you'll find information at
ReInforced Care, Inc.