In the past 2 decades, the use of robotic surgery has significantly increased given its minimally invasive nature, offering benefits like enhanced 3D vision, improved precision through the articulated instrument tips, ergonomic advantages for surgeons, and shorter learning curves when compared to classical laparoscopy. The importance of surgical skill in influencing patient outcomes¹ highlights the need for validated training curricula in robotic surgery. However, there is currently no universally accepted standard for structured training and assessment. Various manual and automated methods for skill assessment have been proposed.²

Manual qualitative assessment approaches involve global scores using Likert scales, exemplified by the Global Evaluative Assessment of Robotic Skills or Objective Structured Assessment of Technical Skills,³ and have widespread acceptance in the surgical community, although error-based approaches deploying binary scored metrics, such as proficiency-based progression (PBP),⁴ have proven superior to traditional training for several tasks. For robotic surgery, PBP has recently been demonstrated to have significantly better scoring reliability, sensitivity, and specificity when compared to the Global Evaluative Assessment of Robotic Skills.⁵ The time-intensive nature of manual expert assessments has led to increased interest in automated approaches using artificial intelligence (AI). Up to present, these methods often rely on kinematic data extracted from virtual reality simulators or robotic systems. Other recent evidence on AI robotic surgical skill assessment suggests that further AI research should most probably be focused on validating and implementing existing tools rather than developing completely new AI-driven assessment methods.⁶

One specific enabler for automated PBP scoring is the automated indexation of procedure-specific surgical phases and steps as defined in PBP curricula.⁷ Figure 1 shows the increased complexity in surgical steps and errors when comparing the scoring of a surgical suturing and knot-tying task to a fully characterized procedure such as radical prostatectomy or partial nephrectomy.

Consequently, automated surgical phase or step detection represents an enabling step towards realizing automated PBP surgical skill assessment. From a computational point of view, automated PBP skill assessment systems should most probably first identify the current phase and subsequent step before further specifying the errors intrinsic to that particular step.

Apart from PBP, automated surgical phase indexation enables consistent and standardized categorization across procedures, facilitating the processes of video review, analysis, and sharing. Advancements in video analysis, computer vision, AI, and data analytics, sometimes collectively referred to as “surgical data science,” are slowly integrating into surgical practice. With it, the significance of accurately and uniformly defining surgical phases equally increases.⁸ Previous research has highlighted the lack of uniform definitions for phase delineations.⁹ PBP can help in acquiring clinically relevant surgical phase definitions; surgical phase durations as defined from PBP curricula have recently shown to correlate to certain patient specific parameters for robot-assisted partial nephrectomy.¹⁰

Figure 2 provides an example of automated surgical phase recognition for robot-assisted radical prostatectomies. It depicts the manually annotated, ground truth surgical phases on the top line as reviewed by a urologist, compared to the AI-assisted prediction for 2 unseen procedures. The AI system (more precisely, a deep learning network) was trained on 50 robot-assisted radical prostatectomies, labeled according to PBP phases. The system was able to predict surgical phases in unseen prostatectomies with an 89.07% sensitivity, 90% accuracy, 90.4% precision, and an 89.32% F1-score, representing the number of video frames that were automatically assigned to the correct phase.

In conclusion, present evidence indicates significant potential for using binary-scored, error-based metrics such as PBP as objective and validated skill assessment tools. The time-intensive nature of these manual assessment methods is expected to be boosted through the use of AI video analysis, while PBP phase assessment in turn is also expected to generate more homogenous and evidence-based definitions for surgical phase definition for the whole surgical AI field.