Since the launch of ChatGPT in November 2022, public awareness of the capabilities and potential uses of artificial intelligence (AI) and machine learning (ML) has increased exponentially. In tandem with this public explosion, a similarly meteoric rise in biomedical research involving AI or ML components has occurred. In the 5 years from 2017 to 2022, the number of PubMed articles including the terms “artificial intelligence” or “machine learning” increased from 10,156 to 51,995.¹ Within the medical field, AI can be used to interpret radiologic, endoscopic, and histologic images; analyze a patient’s disease, comorbidities, and other components of medical care such as previous treatments, number of visits, potential side effects, and costs; recommend management strategies; and predict outcomes.² Within the scope of urologic oncology, we have found some of the most impressive applications lie within the realm of kidney tumor imaging.

The first step in any fully automated process involving imaging of kidney tumors is to create an algorithm that can reliably and accurately differentiate a malignant tumor from the surrounding renal parenchyma and hilar structures. Segmentation, as this process is often referred to, is the bedrock of any future automated quantitative examinations of the radiologic features of kidney tumors. In order to mobilize international interest and effort toward this lofty goal, we launched segmentation challenges, including the 2019, 2021, and 2023 Kidney Tumor Segmentation Challenges (KiTS). In KiTS 2019, 106 international teams used a public training set of 210 cross-sectional CT images with kidney tumors and corresponding hand-drawn semantic segmentation masks (generated by human annotators) to develop automated systems predicting the segmentation masks of 90 test CT images (Figures 1 and 2). The winning model achieved a Sørensen-Dice coefficient of 0.974 for the segmentation of the kidney and 0.851 for the tumor, nearing the interannotator agreement for both the kidney (0.983) and the tumor (0.923).³ KiTS 2021 was a sequel to this, with an innovative challenge design and a larger dataset. The highest-ranked teams performed better than those of the KiTS 2019 challenge, with respective scores achieving even closer to the human-level performance.⁴ KiTS 2023 is ongoing and is the first KiTS to incorporate nonarterial contrast phases into both the training and test sets, hopefully resulting in broader applicability.

As AI-generated segmentation has garnered more interest and supporting evidence, we’ve found ourselves running into one of the biggest obstacles to the broader adoption of AI in the clinical setting: the so-called “black box” issue.⁵ Physicians are reluctant to trust AI in their practice as they lack an understanding of the processes underpinning the ML algorithms. Similarly, patients are skeptical of AI-based technologies, and while they may tolerate human errors, AI errors are often more difficult to accept. Because of these issues, increasing comprehension and trust in AI algorithms are just as important as the development of the algorithms themselves. To combat this, our team elected to adopt a stepwise approach to opening the AI black box in kidney tumor imaging. Small steps were taken to explain and familiarize the use of AI in kidney cancer to build trust and acceptance.

Our first goal in establishing trust in AI-based kidney tumor segmentation was to replicate known and trusted clinical tumor scoring systems using the publicly available KiTS 2019 data and the winning segmentation model. Thus, we automated the R.E.N.A.L. (for radius, exophytic/endophytic, nearness of tumor to collecting system, anterior/posterior, location relative to polar line) nephrometry score on 300 preoperative CT scans and created an AI-generated score. The fully automated AI R.E.N.A.L. score was able to predict meaningful patient-centered and oncologic outcomes with similar predictive utility to human-generated scores, including presence of malignancy, presence of necrosis, high-grade disease, and high-stage disease.⁶

Building on these results, our next goal was to demonstrate how AI/computer-aided systems might enhance, rather than simply replicate, existing models. One major limitation of the R.E.N.A.L. score is the categorical, unweighted nature of its components. In other words, the R.E.N.A.L. score categorizes its variables, which would otherwise be continuous in nature (radius, endophycity, nearness to the collecting system, and the longitudinal location) in order to simplify the calculation. Thus, we also used the KiTS data to examine a computer-aided transition to continuous instead of categorical variables.

In this study, we explored the ability of continuous versions of R.E.N.A.L. components generated from fully automated AI-based segmentation to predict oncologic outcomes of patients with renal mass. The oncologic predictions made by our AI+ score (AI segmentations, continuous variables) surpassed those of the AI-generated (AI segmentations, categorical variables) and the human expert–generated R.E.N.A.L. scores.⁷ Notably, calculations with the AI+ score can be performed without human intervention at any step, given the complete automation of the generation of segmentation masks, measurement of continuous R.E.N.A.L. components, and weighting and combination of components into a multivariate model, each of which is a complex and time-consuming step for humans.

Our hope is that our work provides reliable and relatable evidence for trusting fully automated versions of previously understood scoring systems, while at the same time promising benefits to nephrometry score utilization in both clinical practice and research settings. Our study is also an essential intermediary step in developing and implementing more complex ML-based radiomic scoring systems that cannot be realistically calculated by humans.

In more broadly looking to the future of AI within clinical care, we hope that AI can help to resolve inherent biases and noise within clinician decision-making. Although AI does not follow common sense and may be biased, it can be reprogrammed. AI is faster, more reproducible, continuously adapting, and with less noise. Of course, how AI tools are developed and implemented will remain crucial to their safety and reliability. It is up to urologists and urologic researchers to maintain an active involvement in the development and validation of AI algorithms to ensure their appropriate use within the realm of urology.⁸