Various sources publish annual or semi-annual rankings of top-performing hospitals, but those lists can actually be misleading, especially when they offer conflicting results, according to an article published in NEJM Catalyst.
That’s because those ratings, such as lists from the U.S. News & World Report or even CMS, have different standards.
“Each rating system had unique weaknesses that led to potential misclassification of hospital performance, ranging from inclusion of flawed measures, use of proprietary data that are not validated, and methodological decisions,” writes Karl Y. Bilimoria, MD, MS, of Northwestern Medicine, and colleagues.
However, U.S. News & World Report did receive the highest grade from researchers, who set out to rank the rankers. That publication, which recently released its 2019-2020 best hospital list, got a B grade in the NEJM article. CMS, by comparison, received a C. Among the lowest rankers, Leapfrog was given a C-, while Healthgrades received a D+.
The rankers received lower grades in part because of their flawed rating systems, which included limited data and measures, a lack of robust data audits, composite measure development, a lack of formal peer review and measuring diverse hospital types together, the authors note.
Still, the rankings from these publishers tend to inform stakeholders on the top-performing hospitals and can influence patients when selecting where to receive care. When the lists conflict, ranking one hospital high in one list and low on another, it can cause confusion for patients and stakeholders alike.
In addition, the hospitals that clinicians like to refer to are often not represented as top hospitals in these various lists. The researchers were the six evaluators to grade the raters under certain criteria, including the potential for misclassification, scientific acceptability, transparency, usability and more.
None of the hospital quality rating systems receive an A or A- from the group.
“We qualitatively agreed that the U.S. News rating system had the least chance of misclassifying hospital performance,” Bilimoria et al. write.
The researchers see opportunities for hospitals to be ranked differently, but there are risks in doing so. Above all, “the field definitely needs better data,” they concluded, noting that most systems rely on self-reported data, which is limited, and on Medicare claims data, which doesn’t represent the entire population.
Amy joined TriMed Media as a Senior Writer for HealthExec after covering home care for three years. When not writing about all things healthcare, she fulfills her lifelong dream of becoming a pirate by sailing in regattas and enjoying rum. Fun fact: she sailed 333 miles across Lake Michigan in the Chicago Yacht Club "Race to Mackinac."