The R.E.N.A.L. (radius, endophytic/exophytic, nearness to the collecting system, anterior/posterior, location relative to the polar lines) nephrometry score aims to quantify the surgical complexity of a renal mass.¹ Although prospectively validated as a predictor of perioperative, pathologic, and survival outcomes,^1-4 the score has seen limited clinical adoption due to its interobserver variability and considerable needed time for generation by clinicians.^5,6 Artificial intelligence (AI) technology could alleviate these 2 barriers, as we previously demonstrated the ability of AI-generated R.E.N.A.L. nephrometry scores to predict oncologic outcomes in patients with renal mass as reliably as human experts, which can be a valuable tool for clinicians.⁷ Furthermore, an AI scoring system can overcome another fundamental limitation of the R.E.N.A.L. score, being the ordinal nature of its components. Although categorizing continuous variables (eg, a renal mass diameter of 4-7 cm gets a score of 2) was meant to simplify, standardize, and reduce the time needed for the score generation, some valuable information might be lost on the way,⁸ thus potentially decreasing the score’s predictive accuracy by blurring the information within the same category. For example, a 6.9-cm 95% endophytic renal mass and a 4.1-cm 50% endophytic renal mass are equally scored by the current ordinal R.E.N.A.L. score, however, their prognosis varies considerably. Additionally, the R.E.N.A.L. score equally weighs each of its components and simply adds their values to simplify its generation. An automated AI-based scoring would not be bounded by these limitations and could combine continuous score components through complex multivariate models. Thus, we generated a modified R.E.N.A.L. score (AI+ score) using AI-generated segmentations and continuous rather than ordinal variables combined using multivariate logistic regression. We hypothesized that the AI+ score would have a better predictive ability of oncologic outcomes than that of scores using ordinal variables.

Table 1. Baseline Characteristics (N=300)

Table 2. Multivariable Logistic Regression Models Using Continuous Artificial Intelligence+ Score Components

This was a retrospective study of 300 consecutive patients with preoperative CT scans showing suspected renal cancer at a single institution from 2010-2018. We excluded patients with a tumor thrombus and used the largest tumor to generate the nephrometry score in case of multiple tumors. The inclusion criteria were based on the previously published KiTS19 (kidney and kidney tumor segmentation challenge) protocol.⁹ This cohort, with scans, segmentations, clinical details, and outcomes, is now publicly available at https://kits21.kits-challenge.org/. We used a deep neural network approach to generate kidney segmentation masks of parenchyma and tumor automatically. Geometric algorithms automatically estimated the score components as ordinal and continuous variables. Each tumor was assigned an AI-generated continuous score for each component of the R.E.N.A.L. score (AI+ score), an AI-generated ordinal R.E.N.A.L. score (AI-score), and a human-calculated traditional R.E.N.A.L. score (H-score). The H-score was tabulated by 3 trained medical personnel. The AI+ score was considered a combination of the automatically generated R.E.N.A.L. continuous components without performing the addition of the variables to generate a total score. Thus, its predictive capacity of outcomes was studied using multivariate logistic regression. We then compared the predictive ability of outcomes between the AI+, AI, and H-scores, and we analyzed the AI+ score components’ relative importance.

The baseline characteristics of the cohort are represented in Table 1. The median age was 60 years (IQR 51-68), and 40% were female. The median tumor size was 4.2 cm (2.6-6.12), and 92% of the tumors were malignant, including 27%, 37%, and 23% with high-stage, high-grade, and necrosis, respectively. As represented in Figure 1 and Figure 2, the AI+ score demonstrated a superior capacity over AI and H-scores for predicting malignancy (AUC 0.69 vs 0.67 vs 0.64, respectively), high-stage (AUC 0.82 vs 0.65 vs 0.71, respectively), high-grade (AUC 0.78 vs 0.65 vs 0.65, respectively), pathologic tumor necrosis (AUC 0.81 vs 0.72 vs 0.74, respectively), and a partial nephrectomy approach (AUC 0.88 vs 0.74 vs 0.79, respectively). Of AI+ score components, the maximal tumor diameter (“R”) was the most important outcome predictor. As shown in Table 2, using multivariable logistic regression modeling, the R component was a statistically significant predictor of high-stage disease (per cm, odds ratio [OR] 2.84 [1.34-4.45; P < .001]), high-grade disease (per cm, OR 1.80 [0.33-3.30]; P = .01), and the presence of tumor necrosis (per cm, OR 4.41 [2.60-6.10]; P < .001). Our study had some limitations, including previously reported segmentation-related barriers.⁷ Although the algorithm could not generate an AI score for 6 cases for an unclear reason, we believe a 2% failure rate is admissible at this point. Furthermore, despite our single-center experience for the surgeries, the CT scan images were acquired from around 70 medical facilities, thus laying a strong foundation for the generalizability and external validity of our work.

The AI+ score, generated with continuous variables and without human intervention at any step, provided meaningful predictions of oncologic outcomes superior to those generated in an ordinal fashion. These fully automated segmentations and nephrometry scores can offer substantial benefits on the clinical and research levels, despite being in their debut. Indeed, automatically generating R.E.N.A.L. nephrometry scores increases the uniformity, quality, and volume of renal mass complexity data by reducing interobserver variability in interpretation and generation and the needed manpower and time. Furthermore, in the era of new developments in the management of renal cell carcinomas, such as adjuvant immunotherapy use and active surveillance, there is a crucial need to enhance the accuracy of patients’ counseling on the possibility of adverse pathology and other oncological outcomes. Our data objectively reassured clinicians to rely on the fully automated version of the known R.E.N.A.L. nephrometry score to predict such important outcomes and can be considered a key intermediary step in the development of even more advanced machine-learning–based radiomic scoring systems.