USMLE® Step 1 Pass/Fail:
The IMG Perspective
Dr. George Koshy Vilanilam, MD
Resident Physician
Department of Radiology
University of Arkansas for Medical Sciences
Introduction
 
On February 12th, 2020, the USMLE announced a major decision with regards to the residency application process and graduate medical education: The USMLE program will change score reporting for Step 1 from a three-digit numeric score to reporting only a pass/fail outcome. This is based on the Summary Report and Preliminary Recommendations from the Invitational Conference on United States Medical Licensing Examination® (USMLE®) Scoring (InCUS) conducted over the past 18 months.
 
This came as a shock to many, puzzling to most, leaving International Medical Graduates (IMGs), Caribbean graduates, and DO students with the most important question, “What happens to us now?” Before we discuss how this is going to impact residency application preparation as an IMG, we have to understand a little background of what happened with the survey that led to the decision.
 
As per the report, of 349 program directors who were surveyed, only less than a third of them wanted a shift from the current numerical scoring system (26%)- see below
​Secondly, of all the 3,555 IMGs surveyed (this includes both USIMG and non-USIMG), less than a third of them were favorable to changing the scoring system to pass/fail. This percentage is markedly different in AMGs. (Fig.2) 
This clearly suggests that, most IMGs and most program directors were against, and likely unprepared for the decision that was made. Additionally, only a very small proportion of respondents brought up the question - “What about IMGs?” 
What's the expected timeline?
 
“…And then for the policy that relates to Step 1 score reporting, that will be implemented no earlier than January of 2022” - Michael Barone, Vice President for Licensure at NBME.
 
 
What Does this Mean for IMGs and Caribbean graduates?
 
"The ECFMG supports the informed decision making of the NBME and FSMB Boards on these policy changes. We are looking forward to continually advocating for international graduates as well as participating in the important future conversations on residency selection and transition." – Dr. William W. Pinsky, President and CEO of ECFMG/FAIMER
 
The first thoughts that went to my mind was, “What about IMGs?” Are IMGs and Carribean graduates, (and DOs while we’re at it) the biggest losers from this move? The only “objective” metrics available for Program Selection committees were Step 1, Step 2CK, and Step 3 in some programs. The advantage of having this “objective” score was that it could aid in comparing candidates from different (read, less-known) medical schools around the world. 
 
Let’s take the residency application process apart and discuss the so-called “ripple effect” by 2022: 
 
USMLE Step 1: Numerical score has been done away with, and there is a single pass/fail outcome. So, spending 6+ months to prepare for Step 1, will not benefit your application. But it is cautioned that a prominent filter will now be to screen out the applicants with failed USMLE Step 1. Previously, students who did poorly on Step 1 had a “second chance” to show their test-taking ability by improving on Step 2 CK. Now, they only have one shot to prove themselves to residency programs.

​USMLE Step 2 CK: How eager the residency programs will be to adapt this score into their screening process (especially university programs), will depend on the AMGs. It requires for them to start preparing for Step 2 CK much earlier on, and take it by their late 3rd or early 4th year, so that they can decide what specialties they are eligible to apply to. What AMGs do (or be advised to do) will influence what big university/medical school-associated residency programs will do. For community programs, programs that considered Step 2CK, and those who have traditionally been “friendly” to IMGs, will continue to screen Step 2 CK, as it is the only remaining “objective” metric remaining. Also, Step 2 CK has been discussed as the more “clinically relevant” exam. But, IMGs have time and again shown grit and scored high on Step 2 CK, and continuing that performance despite a pass/fail Step 1 outcome will only be beneficial for IMGs. Remember that AMGs (and their advisors) are yet to observe trends on their performance on Step 2 CK, because that has never been too important; However, the average Step 2 CK scores of AMGs is greater than that of IMGs (NRMP Charting outcomes 2018), therefore, it is all the more imperative that IMGs focus on acing this exam.

USMLE Step 2 CS: No change except that it can only be done after passing Step 1, which makes sense. It is reasonable to expect some demonstration of success before doing a more advanced test and it saves the applicant money. But remember that a fail in CS will be much more of a handicap than before.

USMLE Step 3: The importance of USMLE Step 3 will gradually increase. Many east-coast programs already desire that IMGs come with a competitive Step 3 score. Now, with one metric down, Step 3 is going to be more important, and getting a more than average score will be extremely beneficial.

Year of Graduation: Increased importance. Most programs prefer YOG within 3 years. Others within 5 years.

Advanced degrees/home country residency: Always beneficial, especially if YOG is >5.

Medical School name: Your school name/brand will become much more important. Setting AMG schools aside, Caribbean candidates from well-known schools, and IMGs from well-known schools in UK, India, Pakistan, Philippines, etc. might have an increased chance of standing out. It is predicted to be an inevitable bias.

USCE: Now, here is where IMGs will have to work even harder. Focus on obtaining electives early on before graduation. Electives and hands-on experiences generally pave the way to a strong letter of recommendation - since they can vouch for your clinical skills. An observership may be fine if that’s all you can get, but the letter writer might be less inclined to describe your clinical proficiency, and you may end up with a generic letter.

Letters of recommendations: One of the problems with removing Step 1 scores was that students from different schools could earlier (=before 2022) be compared using that score, and an applicant from a new medical school/school without established track record could stand out from other well-known schools, by having a higher score. In fact, 18% of responses (InCUS) were concerned that this change may cause residency programs to rely more on medical school’s standing or prestige. In this scenario, having strong letters from top-tier programs will help to increase your chances of being noticed. Who you know, how well you know them, and how strongly they are willing to vouch for you, will matter a lot now (NRMP Program Director survey, 2018, see below) That being said, a strong letter from a community program is still much more valuable than a generic letter from a top-tier program.

Medical Student Performance Evaluation (MSPE): Program selection committees continue to discuss how important MSPEs are in the selection process. As IMGs, this is “foreign” (pun intended) to us, and we know how difficult it is to get even a generic MSPE from our home country medical schools. However, from an unpublished Twitter survey from faculty in residency programs, having key comparison statements like “Mr. Jones was in the top 5% of his class (of 200) in his Internal Medicine rotations/exam” and “This candidate is an excellent candidate for residency” in the MSPE help them understand and let you stand out, and also lets them know that your school likely evaluated you in a standard objective way.

Medical School Transcript: Likely no change, but whenever possible, wording like that of MSPE will be beneficial.

Research: There will be an increasing importance to research (depending on specialty, of course). However, the absolute number of research experiences/presentations/abstracts/publications for a successful IMG will only be known after a year or two of implementing these changes (NRMP reports). But considering that this is a quantifiable metric, the importance will likely increase, even for recent graduates.

Volunteer experiences: This will undoubtedly be of increased importance. Because, now the onus is on the IMG to stand out in any way possible, and build up a well-rounded application.

Honors and awards: Needless to say, these remain important. But highlighting the importance of the award will help it stand out; eg: “Awarded the Jones prize for the most innovative research proposal from a pool of 1,000 applicants, Iowa, United States, 01/2019.”

Personal statement: Likely be no change in this aspect. But an impactful personal statement can make programs curious enough to invite you. It is critical to get your PS reviewed multiple times, and attempt to start preparing for it at least a couple of months in advance. 
 
 
 
Your Burning questions
 
Q.1 What went into the process and who was involved?
This decision by the USMLE co-sponsors (FSMB and NBME) culminated a year-long process that included the Invitational Conference on USMLE Scoring (InCUS). In reaching this decision, the USMLE co-sponsors engaged educators, regulators, examinees and members of the public through surveys, presentations at national meetings, webinars, podcasts, etc. Various score reporting options were considered, including maintaining the status quo (i.e., no change to score reporting).
 
Q.2 Why was this decision made?
The USMLE co-sponsors believe that changing Step 1 score reporting to pass/fail can help reduce some of the current overemphasis on USMLE performance, while also retaining the ability of medical licensing authorities to use the exam for its primary purpose of medical licensure.
 
Q.3 How will this impact scores and transcripts?
The transition will occur no earlier than January 1, 2022. The USMLE program understands this change will have impacts across the continuum of medical education and licensure and will share information as early as possible. As further details become available, updates will be posted to USMLE social media.
 
Q.4 How will I differentiate myself?
The USMLE co-sponsors recognize that an ideal system for evaluating candidates for residency is holistic. This system should feature trusted quantitative and qualitative assessments of a candidate’s competencies and unique characteristics. In such a system, USMLE would serve as just one of these objective measures and be weighted appropriately.
 
Q.5 Will score reports be faster now that scores do not have to be calculated?
The same scoring and quality control measures will continue to be applied; USMLE will continue releasing scores weekly (Wednesdays), with Step 1 scores generally released within 3-4 weeks of the test date.
 
Q.6. As an IMG, can I take Step 2 CK before Step 1?
Many students have done this in the past, and in fact may prove beneficial. But remember that you can take Step 2 CS only after passing Step 1.
 
Q.7. Many electives ask for Step 1. How should we prepare for electives now?
Step 1 will continue to have a numerical score until January 2020 at least. Whether elective programs will be happy with a pass score on Step 1, or if they will require a Step 2 CK score (unlikely) is unknown at this point. Elective programs will soon update their criteria as USMLE continues to provide timely information.
 
Q.8. If I take my Step 1 next year and receive a numerical score, and apply for the 2023 match, when other applicants have a Pass/fail score, how will I be assessed?
 
As long as the score is valid, do your very best to score high on Step 1. There are two things that can happen:
a. Programs may still see your numerical score on your transcript, but choose to only look at the pass/fail section. 
b. USMLE creates a uniform transcript, transmitted via ERAS, that formats all Step 1 scores to Pass/Fail and not show the numerical score regardless of whether you have it or not.
 
Conclusion, and what’s next?
 
With this drastic change, residency programs will likely lean on Step 2 CK and Step 3 as a filtering mechanism (Step 2 CK> Step 3), triggering a cascade of changes in other areas of the residency application process. As we look at the timeline (below), further details will have to be ironed out and are anticipated later this year. 
Only time will tell whether these changes will improve or worsen the chances for IMGs for graduate medical education in the United States, but if history is any proof, despite all odds, IMGs will continue to excel.

Our team at USMLEsarthi is always available to answer any questions you have and to work with you on your Match journey. Get in touch!
 
​
References:
 
  1. InCUS (held in March 2019) was sponsored by the American Medical Association, the Association of American Medical Colleges, the Educational Commission for Foreign Medical Graduates, the Federation of State Medical Boards and the National Board of Medical Examiners. For more information, visit https://www.usmle.org/usmlescoring( Accessed 12 Feb 2020)
  2. https://www.usmle.org/usmlescoring/faq.html (Accessed 12 Feb 2020)