United States Medical Licensing Examination® Now and Then

The United States Medical Licensing Examination (USMLE®) Step score has become a career defining data point for U.S. medical students applying into residency and continues to be a topic of controversy in medical education. Despite the examination’s original purpose to measure competency for granting medical licenses to physicians,¹ Step 1 of the USMLE series is often used to guide applicant selection for residency. The examination yields a threshold passing score in addition to a numerical score that is reported to programs and expectedly holds more weight. However, this score is not a measure of trainee clinical performance potential and was not designed for residency programs to use in the comparison of applicants.^2,3 Doing so has many unintended consequences. Now, USMLE has decided to transition to a pass/fail scoring system for Step 1.¹

Impact of Shift to Pass/Fail System

Since the announcement of the scoring change to pass/fail, numerous studies have emerged examining the potential impact. Although there are limited studies quantifying the medical student perspective, proposals prior to the announcement suggest medical students are in favor of the scoring change whereas program directors are not.⁴ Only 15.3% of program directors among all specialties agree with this change in scoring,⁵ and only 19% of urology program directors agree.⁶ Many reservations regarding this change center around the impact on residency program selection.^5,6 Nevertheless, among program directors, medical school administrators, and medical students, there is agreement regarding the potential positive and negative impacts of a pass/fail scoring system (see figure).

Figure. Comparison of potential positive and negative impacts of pass-fail Step 1 scoring system.

What does a pass/fail Step 1 examination mean for the urology residency application cycle? Undoubtedly, this will change application requirements. Historically, despite the availability of multiple USMLE scores, programs often only consider the USMLE Step 1 score. Programs may now mandate a Step 2 Clinical Knowledge (CK) score, which remains on a numerical scale, a change nearly a quarter of urology program directors supported.⁶ In the context of the early match, the requirement of Step 2 CK scores will likely impose on the already tight timeline for applicants managing away rotations and residency interviews.

Alternatively, programs may migrate from USMLE altogether and develop a urological subspeciality examination. While USMLE may not prove urological knowledge, some program directors believe the USMLE score is a marker of dedication, self-drive and stamina,⁷ which are attractive qualities in a resident. Development of a urology-focused test may allow applicants to demonstrate these qualities while better assessing their knowledge of the field. However, the development of such a test may again emphasize medical student knowledge and standardized test taking skills as a basis for residency selection, disadvantage students without home urology programs or access to preparatory resources and contribute to already observed testing related disparities.⁸

Lastly, urology has historically lacked in diversity⁹ and the elimination of numeric Step 1 scoring has the potential to expand applicant recruitment. The removal of score thresholds may capture qualified applicants who defer applying due to a “noncompetitive” test score, while also ensuring qualified applicants with lower test scores undergo review by programs, therefore expanding diversity in the applicant pool and in turn within the field of urology. However, with a steadily increasing number of urology applicants, there is concern regarding the increased burden on program leadership.

Adapting to USMLE Scoring Change

We have recently seen various adaptions to the interview process with the COVID-19 pandemic and with the scoreless application season approaching, residency programs must again undergo process transformation. There is a call to transition to a holistic review of applications.¹ Although an all-inclusive evaluation is time consuming, it would be the best way for programs to assess an applicant’s fit. However, there remains concern for bias due to the lack of objective data to compare applicants. Program directors already express frustration regarding applicant similarity secondary to lack of medical school grades or distinctive letters of recommendations, and the elimination of test scoring further complicates this issue. In order to truly assess candidates’ alignment with their program, faculty will be tasked with creating other metrics for evaluation, such as secondary applications, additional essays, or video interviews, aimed at capturing attributes prioritized by programs.

Still, a barrier to holistic review or additional application components is the urology match timeline constraint. Programs are tasked with developing strategies to effectively and efficiently evaluate candidates without overwhelming program directors and selection committees. To offset this burden, urology could consider delaying the match or joining the regular match to extend the timeline for reviewing applications or contemplate transitioning to a tiered approach with multiple stages within the cycle.¹ There is also discussion around limiting program applications per student.

Regardless of the adaptations made, urology as a field will undoubtedly continue selecting top-notch candidates as future trainees. With the imminent changes to the residency match process, we must innovate and refine–abilities well honed by urologists–our practices to strengthen the capability to attract and choose capable, committed, and diverse candidates.