In the evolving environment of modern health care, the values of data and measurement seem to be constantly referenced and increasingly looked to as mechanisms to unlock secrets to improve outcomes. The idea of measuring the results of medical care is not new.¹ However, with electronic data platforms and large-scale data aggregation, the potential is perhaps greater than ever before.

The volume of available health care data carries promise, but as anyone who has attempted to use such raw data can attest to, realizing this potential requires more than data alone. Successful use of health care data for improvement involves first processing unstructured data in a manner that can provide a succinct understanding of an issue. This process of turning data into information is an important step, but is also only a part of the journey to improvement. Information needs an audience to have an impact. The ideal audience is in a place to use what they learn from the information presented to make changes that can affect results. Scorecards are a mechanism to facilitate this process where raw data can be distilled into information for presentation; it can be disseminated to stakeholders who can use the information to form new understanding and hopefully use it to change performance.

Within urology there are a growing number of examples and likely many more that remain unpublished where scorecard mechanisms are used to understand performance and drive quality improvement. These principles have been used to improve adoption of active surveillance for prostate cancer.^2,3 Quality interventions utilizing scorecards have significantly reduced opioid prescribing following prostatectomy and nephrectomy.⁴ Our group has used these mechanisms to provide surgeons with insight into their performance during transurethral resection of bladder tumors, and we have demonstrated an increase in detrusor muscle sampling and decreased disease recurrences when comparing a period when a scorecard was available to a prior era when it was not.⁵

Scorecard development can be a valuable mechanism for improvement, but it is also a labor-intensive process requiring resources and a team with diverse skills to achieve maximal impact. The first step of the process is to formulate a rationale for the question or issue the scorecard plans to address (eg postoperative outcomes, prescribing patterns, procedural costs). Ideally at the outset, stakeholders should be engaged to ensure sufficient buy-in to justify the effort. It is also essential to evaluate feasibility (ie is there sufficient access to the right type and quality data to answer the proposed question). If those prerequisites are met, then working to turn raw data into information is the next step. Data analytic expertise and a structured method for measurement are important. It is helpful to define specific measure parameters (eg what patients are to be evaluated, what is the definition of an event of interest and how will it be identified, what is the time period of measurement, what factors should exclude a patient from the measure etc). The National Quality Forum is considered the national leader in health care measure development and provides extensive resources related to this topic (https://www.qualityforum.org). After defining a measure, it is important to validate the measure’s performance through data audits often carried out by manual validation. If one is confident the measure is of sufficient quality, then moving toward scorecard integration is the next step. In this phase, it is important to consider the audience for the scorecard, the limitations of the data, and other messaging that will come along with the scorecard. Given the observational nature of most scorecard data, it is important to consider steps such as risk adjustment to account for factors that may drive warranted variation in measure performance. It is important to consider and explain to recipients the limitations of measures included in a scorecard. As an example, sample size and measurement biases have been well described limitations of attempts to report surgeon level performance in scorecard formats.⁶

Going forward, surgeon scorecards will likely continue to play a role in informing and, hopefully, driving improvement. With the current trend of hospital mergers and health system aggregation into large networks, the power of these tools may only increase. As health systems grow, so too do the data they capture. With unified patient records and the ability of a large health system to reach a greater number of surgeons with a scorecard rollout, there is ever increasing potential to increase the scale of these initiatives.

Recognizing the power of health care data to drive change, the American Urological Association has invested in the AUA Quality (AQUA) Registry. Excitingly, as the organization looks to further realize the potential of this tool, it has dedicated recent focus to invigorating the process of quality measurement. The newly formed Measure Evaluation Panel within the Quality Improvement and Patient Safety Committee has assembled a multidisciplinary team consisting of urological quality improvement leaders and nonphysician experts in measure development and utilization. This team will work to develop a battery of quality measures specific to the AQUA Registry. In time, this will hopefully further unlock the potential of data in the AQUA Registry to drive change. If successful, national scorecards where surgeons can understand their own performance benchmarked against peers may become more routine. Once deployed, this may carry enormous potential to improve the lives of patients seeking urological care on a national scale.