Explanation of Quality Rating, Composite Score, and Star Ratings

Early initiatives measured the quality of cardiac surgery only by how often patients died. Furthermore, because coronary artery bypass grafting surgery (CABG) is the most common, signature cardiac surgical procedure, CABG mortality served for many years as the sole metric of cardiac surgical performance. A major early contribution of the Database was the development of risk-adjustment techniques. To accurately measure cardiac surgery outcomes, it is essential to adjust the results for each surgeon and hospital by accounting for the severity of their patients’ illnesses, a process commonly known as risk adjustment. Some hospitals care for more seriously ill patients, and such patients are at greater risk of dying because of their underlying conditions, not necessarily because of the care they receive. If there was no way to account for such risks, some excellent surgeons and hospitals would have higher death rates and appear to be providing poor quality care, when in fact they simply have sicker patients. To level the playing field, statistical techniques have been developed to account for the condition of patients before surgery. This has resulted in one of the most commonly reported measures of cardiac surgery quality, the risk-adjusted or risk-standardized mortality rate. STS has developed risk models for CABG, valve, and combined valve and CABG operations that have all received national recognition and endorsement.

There is now an increasing recognition nationally that performance measurement must be more comprehensive than just single procedures and outcomes.  For example, consider two patients who survive CABG surgery. One has a perfectly uncomplicated course and receives all the appropriate treatments postoperatively, whereas the other patient develops kidney failure or a serious wound infection. Although both patients are survivors, their quality of care may have been very different. Because of such considerations, many organizations have recommended the use of multiple measures of quality for specific conditions and procedures, sometimes combining them into one number called a composite score. The composite score is a single number or rating that summarizes all available information about the quality of care delivered by an individual provider. It is this principle that led The Society of Thoracic Surgeons to develop what is known as the STS CABG composite score and rating, now one of the most sophisticated and widely regarded overall measures of quality in health care. The STS AVR composite score was subsequently developed due to the success of its CABG predecessor, and further composite measures for other procedures are currently being developed.

The STS CABG Composite Score

The STS CABG composite score is calculated using a combination of 11 measures of quality divided into four broad categories or domains. The first domain is risk-adjusted mortality, and the second domain is risk-adjusted major morbidity, which represents the percentage of patients who leave the hospital with none of the five most serious complications (often referred to as morbidities) of CABG—reoperation, stroke, kidney failure, infection of the chest wound, or prolonged need to be supported by a breathing machine, or ventilator. Overall, based on data from the STS Database, about 85 percent of patients are discharged with no such complications. The third domain measures the percentage of CABG procedures that include the use of at least one of the arteries from the underside of the chest wall--the internal mammary (or internal thoracic) artery— for bypass grafting. This artery has been shown to function much longer than vein grafts, which can become blocked over time. The final domain measures how often all of the four medications believed to improve a patient’s immediate and long-term outcomes were prescribed. Those drugs include beta-blocking drugs prescribed pre-operatively, as well as aspirin (or similar drugs to prevent graft clotting), and additional beta-blockers and cholesterol-lowering medicines prescribed at discharge. Importantly, the 11 individual measures and the overall composite measure methodology are all endorsed by the National Quality Forum and have undergone careful scrutiny by quality measure experts.

STS Public Reporting Online lists participants’ scores for each of the four domains and the STS CABG overall composite score. Each of these numerical scores can be compared with the average scores for all Participants in the Database. Participants also have a star rating. The star rating calculation begins by assuming all providers are average and then determines statistically if there is at least a 99 percent probability that the performance of any specific provider is lower than average (one star) or higher than average (three star). For the several years that the STS has been calculating these scores, about 10-15 percent of all Adult Cardiac Surgery Database participants have been one star, about 10-15 percent have been three -star, and the remainder have been two -star, or average programs.

Finally, it is important to understand that these scores compare the results of a hospital or surgical practice/group with those of an average hospital or practice participating in the Database and treating patients with the same mix of severity of illness. Even when the results are risk-adjusted, it is not necessarily appropriate to compare directly the scores of individual hospitals or surgical groups to each other, especially if they treat markedly different kinds of patients. In addition, a surgical group may practice at more than one hospital, and more than one surgical group may practice at a given hospital.

The STS AVR Composite Score

The STS AVR Composite measure uses the same basic statistical approach as in the CABG model with several modifications that reflect the differences between isolated CABG and isolated AVR populations. The AVR Composite includes two outcomes domains, risk-adjusted mortality and risk-adjusted morbidity, which includes the same complications as the CABG composite (i.e., reoperation, stroke, kidney failure, infection of the chest wound, or prolonged need to be supported by a breathing machine, or ventilator). However, as there is nothing comparable to the use of the internal mammary artery in valve surgery, and because appropriate medications are less well-defined for valve procedures, these two process domains are not included. In addition, due to the smaller number of AVR patients compared with CABG, a 95% Bayesian credible interval for star rating determination and a data collection period of 3 years (rolling) are used.

STS Public Reporting Online lists participants’ scores for each separate domain as well as the STS AVR overall composite score. As in the CABG composite, numerical scores and star ratings are provided. The star rating calculation again begins by assuming all providers are average and then determines statistically if there is at least a 97.5 percent probability that the performance of any specific provider is lower than average (one star) or higher than average (three star). Otherwise, the participant receives two stars.

|||