Mortality Rate Is Not A Valid Indicator of Quality Differences Between Pediatric Cardiac Surgical Programs
Ann Thorac Surg 2010;89:139-46
Reviewed by Robert Moore, MD and Anshuman Sharma, MD
This retrospective cohort analysis of Congenital Cardiac Surgical procedures contained in the Nationwide Inpatient Sample (NIS) database tested the hypothesis that pediatric cardiac surgical procedures are performed too infrequently or have too low of a mortality rate to allow valid hospital quality comparisons based upon mortality.
Data were obtained from the NIS, a stratified cross-sectional database of hospital discharges that contains data pooled from approximately 1,054 hospitals in 37 states. ICD-9-CM procedure and diagnosis codes were used to identify patients who were under 18 years of age and underwent cardiac surgical procedures between 2000 and 2005. Collected data included: in-hospital- mortality based upon discharge disposition records; volume of cases in cases / year; and RACHS-1 category. For hospitals appearing for more than one year in the NIS, individual years were treated as separate entities. 21,709 operations occurring at 161 hospitals for a total of 252 hospital-year combinations were identified.
A series of single tailed t-tests were performed to determine if any hospital had worse outcomes than the mean mortality rate or the benchmark mortality. No hospital performed a sufficient number of cases to detect a doubling of mortality or a 5-percentage point increase in mortality vs. the mean. Similarly, following stratification of cases into rachs-1 categories, only a minority of institutions performed a sufficient number of cases to detect a difference. Minimum case volumes to detect such a difference ranged from 71 for RACHS 1 cases to 588 for RACHS 5 cases. It would take a hospital performing the median number of cases approximately 6 years to reach the category-1 volume and 120 years to reach the category-6 volume. Additionally, if all RACHS categories were aggregated a median volume institution would need a 15% mortality rate to detect such a difference. A series of two-tailed tests indicated that no hospital had sufficient volume to detect such a difference between institutions for overall cases performed or for any RACHS category.
This study is limited by possible sampling error due to the use of a representative database. These databases are intended to serve as administrative databases. There is risk of misidentification of subjects and the possibility of confounding variables due to the use of RACHS-1 categories to stratify patients. The use of the NIS and RACHS-1 data might not address the utility of a risk-adjusted mortality model. However, the NIS is a large well-powered database with stable mortality rates and mortality data with limited clinical adjustment has been successfully applied to post-CABG mortality (1). Based upon the NIS data, congenital cardiac surgery is performed too infrequently with too low of a mortality rate to allow for differentiation between hospitals based on a doubling of mortality or a 5 percentage point increase in mortality.
This study provides strong evidence to suggest that case volumes and mortality rates are too low to allow the valid use of mortality rate as a quality metric . It highlights the limitations of universally applied quality metrics. Alternative metrics need to be developed to avoid specious conclusions about the quality of care. Such metrics could be based upon more frequently occurring adverse events or novel metrics based upon aspects of the process of care delivery. The development of such measures is critical to future health policy and patient care decisions.
1. Med Care. 1992;30:892- 907.
