Ahmed ME, Joshi V, Alamiri J et al: A survey of urology residency program directors assessing criteria to interview applicants during the COVID-19 pandemic. Urol Pract 2021; 8: 472

Urology residency represents one of the most competitive residency specialties in the United States. Given the high number of applicants, most residency programs use some objective parameters, such as a United States Medical Licensing Examination® (USMLE®) cutoff score and/or the Medical School Performance Evaluation (MSPE), to select applicants for interviews. Due to the COVID-19 pandemic, the selection process for residency interviews has become more challenging and necessitates a discussion on how this process has been impacted for the benefit of both applicants and programs. Therefore, we anonymously surveyed urology program directors (PDs) nationwide on their criteria of interest for virtually interviewing applicants amid the COVID-19 pandemic.¹ Almost 50% of urology PDs responded to our survey and shared their selection criteria as discussed below.

Most Important Criteria: In our survey, urology PDs highly emphasized the importance of letters of recommendation (LORs) based on nonvirtual rotations, urology clerkship grades, research experience, visa status and class rank when selecting applicants for interview, with average importance rates of 4.39, 4.14, 4.05, 3.92 and 3.75 (range 1–5; fig. 1), respectively. Furthermore, more than 50% of urology program directors graded LORs based on nonvirtual rotation and applicant’s visa status as very important (score 5 of 5). The high importance of urology LORs was not surprising. In a 2015 survey of urology program directors, Weissbart et al reported that urology LOR was the most important factor in selecting applicants to interview.² However, it was notable to see how important the applicant’s visa status was to the surveyed urology PDs in our study. More than 50% of program directors cited the applicant’s visa status as a very important factor (scale 5 of 5) in selecting applicants for interview (fig. 2), which is significantly higher compared with the reported importance of visa by the National Resident Matching Program (NRMP). The NRMP reports that applicant’s visa status is considered in the selection process for interview by only 28% of program directors in specialties other than urology.³ This difference in the reported numbers could be due to the unfamiliarity of urology PDs with the medical education and training of international medical graduates (IMGs). Unfortunately, this puts IMG urology applicants in a challenging position and reduces the opportunity of increasing the diversity in urology.

Figure 1. Mean degree of importance of each factor as cited by PDs.1 AOA, Alpha Omega Alpha Honor Medical Society.

Figure 2. Percentage of PDs who graded each factor as very important factor in selecting applicants for interview (score of 5).¹ AOA, Alpha Omega Alpha Honor Medical Society.

Furthermore, urology clerkship grades and urology research experience were among the most important criteria to urology PDs. The value of clerkship evaluation to urology PDs is not limited to urology clerkship only. In a previous study, we reported that honors in all core clerkships were a significant predictor of success during residency.⁴ In addition, MSPE was an important factor in selecting applicants. However, the lack of a standardized evaluation has made it difficult for PDs to differentiate applicants based on MSPE. Westerman et al reported that 97% of medical students received the highest grade in some clerkship in their MSPEs, emphasizing the need for a standardized approach in evaluating medical students that could help PDs find the correct fit for their programs.⁵

The Value of USMLE Steps and the Impact of Recent Changes: In comparison to the 2015 survey, USMLE step exams were not among the top-rated factors by urology PDs in our survey. Only 20% and 15% of urology PDs cited USMLE Step 1 and USMLE Step 2 scores, respectively, as very important factors (score 5 of 5). The slightly higher degree of importance given to USMLE Step 1 compared to USMLE Step 2 CK (Clinical Knowledge) might change in the future due to the recently announced change that USMLE Step 1 will become a pass/fail exam. In a survey study of urology PDs, Chisholm and Drolet reported that urology PDs disagreed with these modifications and believed it would make the selection process more challenging and give more weight to the USMLE Step 2 CK.⁶ Interestingly, 37% of urology PDs in our survey study cited passing USMLE Step 2 CS (Clinical Skills) as very important (score 5 of 5). This could be due to the temporary suspension of the exam at the time the survey was sent out. Of note, on January 26, 2021, USMLE announced the discontinuation of USMLE Step 2 CS and determined newer innovative ways to assess the clinical skills of medical students and applicants.⁷

Least Important Criteria: Being a graduate of the program’s medical school, a urology virtual subinternship, a urology LOR based on virtual rotation(s), participation in open-houses and social media activities were consistently rated as the least important factors by PDs when selecting applicants for interview. Less than 5% of the PDs in our survey rated these factors as very important (score 5 of 5). This could reflect the difficulty in differentiating applicants and selecting the right fit to the program based on this virtual evaluation.

Conclusion and Recommendations: Our study showed that urology PDs place strong value on urology LORs based on in-person rotations, the applicant’s visa status, urology clerkship grades, and research experience. Contrary to what is perceived, participation in social media activities and virtual rotations did not seem to influence the applicant’s chances of being selected for an interview.

Our study also highlights the need for an open discussion on the criteria to select urology applicants for interviews. There is a lack of diversity in urology, and a diverse working environment has been reported to improve the cultural competence of health care providers, especially when making health care decisions.⁸ Therefore, efforts to improve a diverse workforce would potentially enhance our training programs.