A program director’s take on whether you’ll pass boards…and what to do about it if you won’t

 

by:  David Kashmer, MD MBA FACS

 

I use certain things to predict your chances of passing boards, and what our program will need to do to be sure that you do pass boards.  The publisher of ABSITE Smackdown! asked me to do a little write up and share because so many seem to ask about this.

Let me tell you a little about what we do at our program when it comes to making sure our team passes boards.  Maybe it will help you understand your own chances and what you can do to improve them!

 

For us, it starts with resident selection

I’ve never been big into standardized tests.  That’s probably because my early education was a very British model, essay-heavy style with few if any standardized tests.

 So, although I’ve gotten used to dealing with them, I remember my struggles with them too.  Maybe it’s for that reason that I’m particularly focused on what they can (and can’t!) do.

 For our program, your previous standardized test scores are NOT the only factor for whether we rank you highly to attend…but they are a factor.  Why?

 Well, I need to know whether we can help coach you to becoming the best surgeon you can be.  To that end, I need to know how much you will need, and what you will need, to pass your boards.  So to that end our team reviews your previous standardized test scores with an eye toward balancing what you’ll need to be successful with what we can offer and what else you contribute as a member of our team.

Some things we can’t improve.  But many things we can!

 

We know the factors that predict whether you’ll pass. 

Here, there’s a lot of literature about what certain things can predict whether you pass.  Note that none of these things comment on you as a person, your overall intelligence, your talent, or your ability to be a great surgeon.  But, for better or worse, they’re what we have.

For example, if your ABSITE score is less than 35% percentile during any year of your residency, you’re more likely to fail either your qualifying exam (aka written exam) or your certifying (aka oral) exam on your first attempt.

A mandatory research year is associated with an increase in your chances of passing your boards on the first attempt.

A score of less than 200 on the USMLE step 1 is also predictive.  Again, these are statistical associations and do not speak to any causal reasons why you’re at an increased risk.  And just because you’re at risk does NOT mean you will fail.

You can read more about those factors here:

 

https://www.ncbi.nlm.nih.gov/pubmed/20855755

 

So what can you do?  If you’re a resident, you can’t change your USMLE step 1 score of course.  For one, you could select a program with a mandatory research year.  But it’s hard to say (when you drill down on it) whether it’s really the mandatory year that’s responsible for the association with passing boards or no.  It may be the improved sleep, decreased call requirements, and ability to study that help so much!

Much of our efforts focus on the ABSITE and associated performance factors.  We put out an entire review book, and video course, to make sure the correct content and knowledge base were readily available.  But we do more.

There are other known factors associated with ABSITE performance, including timing of call, amount of call, sleep time, vacation timing, etc. etc.  We use each of those, and we do so consciously.  Take a look at some of the research above to do a deeper dive into those performance factors.

 

An ABSITE less than 35th percentile increases your risk of failure but doesn’t mean you WILL fail

Ut-oh...math!

This may help you understand how a low ABSITE score affects your chances of passing written boards on the first try.

Pretend you felt things could go either way for you on written boards.  You thought there’s a 50% probability you will pass.  We’re going to use something called Bayes’ Theorem to show how a low ABSITE score affects that chance.

First, we convert your 50% probability estimate to odds.  Probability divided by one minus probability gives us odds.  That’s p/(1-p), or odds of 1 to 1.

Next we use the odds ratio on passing written boards if you scored less than 35% on ABSITE during residency.  That number, from the article above, is 0.23.

Next, we multiply odds on passing you estimated before the info about the ABSITE (1:1) by that odds ratio. That’s 1 times 0.23.  That gives us 0.23.

Next we convert that 0.23 odds back to probability.  The equation is odds divided by odds plus one.  Or Odds/(odds+1).  That’s 1/1.23 or a probability of about 19%.

So, scoring less than 35% on the ABSITE at any time during residency would adjust a resident from a 50% chance of passing written boards (first try) to about 19%.

In other words, scoring less than 35% on ABSITE doesn’t guarantee boards failure but it does adjust it down.  A lot.

  

The take-home message

Like many important things, there’s a lot of knowledge and research about this topic.  I was really surprised, when I first read about it, that I’d never heard more about it as a resident or later as a faculty attending. Use the link above to learn more about the research behind performance on boards.

In our residency program, we use this research to identify an influence the numbers as much as possible in order to give our colleagues in training the best chance possible of passing boards on the first attempt.

After all, if we know the risk factors for inability to pass boards, aren’t we obligated as good faculty coaches to help our junior colleagues?  Of course we are.

For that reason, before our residency program started, I created an ABSITE review book and video course that specifically targets things the ABSITE tests, but which also shares key factors about test performance.  Yes, that work is ABSITE Smackdown! and I’m grateful to the publishing team at #ProjectAbsiteSmackdown and The Healthcare Lab, Inc. for making it happen!  I’m really glad to share it and the feedback has been amazing.  The team will be using that feedback to improve the work on an ongoing basis.

My hope is you find it useful if you choose to use it as a resource, but, more importantly, I hope you can use this info to maximize your success on boards.