Benchmarking Hospital Quality
Benchmarking Hospital Quality: Template Matching versus Conventional Regression Approaches
Team Members: Hallie Prescott, MD, MSc (PI) Tim Hofer, MD, MSc Jeremy Sussman, MD, MS Jack Iwashyna, MD, PhD Sanjay Saint, MD, MPH Wyndy Wiitala, PhD Jane Forman, ScD, MHS Kate Luginbill, MPH Brenda Vincent, MPH Jenny Burns, MHSA Daniel Molling, MS Funding: 3/1/2018 – 8/31/2020 Partners: VA Office of Reporting, Analytics, Performance Improvement, and Deployment (RAPID); VA Inpatient Evaluation Center (IPEC) |
Background:Identifying and remediating low-quality care is at the heart of systematic quality improvement, particularly in VA. However, comparing hospitals to identify under-performing hospitals is difficult because of inherent differences in patient case-mix and illness severity across hospitals (i.e. some hospitals take care of sicker patients, while other hospitals take care of older patients).
Clinicians consider the current approach to evaluating hospitals using regression methods, which adjust for differences in patients across hospitals, to be unfair, unclear, and unhelpful. Therefore, while substantial resources are devoted to hospital evaluations, the current return on this investment is limited because clinicians do not understand or trust the methods.
The National Academy of Medicine has recently called for investing in the science of hospital performance measurement and for increasing its transparency and validity. “Template matching” has been proposed as an alternative methodological approach for hospital evaluation that is fair, clear, and helpful. However, this new approach has never been tested outside of limited research settings.
Objectives: In this IIR, we are testing the usefulness of template matching for comparing quality of care across VA’s diverse acute care hospitals—in terms of feasibility, accuracy, and interpretability. Specifically, the project will: (Aim1) Feasibility: Develop and optimize two template matching approaches for comparing 30-day mortality across VA hospitals. (Aim2) Accuracy: Compare the ability of template matching versus conventional regression to correctly identify under-performing hospitals. (Aim3) Interpretability: Compare the interpretability and credibility of hospital performance data generated from template matching versus conventional regression models with clinical leaders. We expect that template matching will identify under-performing hospitals at least as well as current benchmarking with conventional regression, and that it will be more interpretable and credible to VA Chiefs of Medicine than current performance reports. We anticipate many future expansions to this work, including the use of template matching for VA to private sector comparisons.