By:  David Kashmer MD MBA FACS

 

Welcome back to The Absite Smackdown! podcast. Jessica has the day off today and she asked me to go through a novel paper with you about surgical education. 

 

And as you know at the podcast we do this all the time--we look for novel ways to improve our online curriculum.  Our online Absite Smackdown! review course.  And we share those with you and we share with you better ways to study for the Absite. And we really have an interesting one here.

 

So, hope Jessica is having a great day off and to you all…let's get to it!

 

This paper comes to us from the University of Southern California, and it's all about their experience with developing and implementing an online education curriculum. Now we'll put down in the description somewhere the link so that you can go to the paper and check it out.

 

Ok, here's that link to the paper: 

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7470743/

 

This is interesting because it shows us the experience when a residency had to use one. And what they did is they created a curriculum--basically on an Excel sheet. They kept it very simple. They got it done quickly and utilized subspecialty themed faculty and resident-created lectures.

 

These were weekly and utilize Absite practice questions from a standard question bank and weekly subspecialty synchronized readings. And they looked at attendance. They sent out a survey after to see how people felt about it. And they adapted the curriculum as the clinical restructuring from COVID-19 ended. The results were really fascinating.  About 77% of residents attended the lectures before the curriculum was implemented and about 80% attendance was seen after.

 

They said there was no significant difference between pre and post attendance as the P was greater than 0.05.

 

Now I want to take a second here to comment on statistical testing. First, we typically use statistical testing when we have a sample of a larger population.  That wasn’t the case here.  Also, in this case when we were dealing with percentages and the typical test performed is the Chi squared proportions test. But if we're looking at the whole population of people, patients, people some condition and we do something and we see a difference after, in the number of in the population doing something…then really the utility of a statistical test is low.  Meaning, do you need even do one at all?

 

Because the change happened to the whole population, it's different and significant no matter the P value.

 

Let me explain:  if we have a hundred people and we make a change and we have those same hundred people after, and we go from a percentage before intervention of like 45%. And after intervention to 90%, and again, the number of people stays the same throughout, and this is the whole population we're looking at. Then it is a difference. It's a real difference. The whole population is different. So statistical tests are typically used when we have a sample of a population and we're representing the larger population because we can’t see or record the whole thing. And we want to know if the population as a whole is different because we can't look at it all.  But in this study they could! They knew the number of residents and the number of faculty. And as long as those didn't change, but the number in attendance did…well, the change is significant no matter what a test says because the population changed.

 

Here, I don't know that we even needed a statistical test. And when it comes back as not significant at 0.05. I question whether that means anything at all.  Because again, we know that the whole population changed and the whole point of the statistical test is to determine if there’s a meaningful difference in the sample of taken from a population when we aren’t able to represent the whole population.

 

I'm interested in your thoughts on it.  Minor point, but an interesting one to me, I do love math and stats.

 

And this one, just kind of interesting, more than any sort of more minor point I think is to talk about what they've found in their study and what their experience was.

 

And it seems to have been overall a positive one. They list a couple of things, including the timing of their online lectures, the particular question bank that they use. And I invite you to click the link and look at it.

 

On or about March 23rd, they made these changes. They changed the curriculum as they went, which is a great way to do it. And afterward they gave an IRB approved questionnaire, which they list there, and they basically run through their experience as a positive one.

 

The key elements they highlight are ones that we've tried to build into the Absite Smackdown! course.  I'm not sure whether they recorded their lectures for use later, or what exactly, but, they used a standard program question bank. They had, readings, listed, et cetera.

 

With Absite Smackdown that's a lot of what we did.  I'm interested to see how the publishing team makes changes to the platform based on this as it seems like a lot of the features described are already covered and already built into how Absite Smackdown! works.  Absite Smackdown, of course, is available anytime, anywhere on multiple platforms.

 

A lot of this is a similar experience, to USC, although their meetings seem to be at one certain time.  Of course, survey studies have a pretty low, miserable, return rate. And it always makes you wonder about the studies you didn't get back.  But here 94% of them did the study and the majority reported an increase in the weekly time devoted to their education from less than two hours a week to six to eight hours a week.  And 56.2% of clinical residents reported completing around 40 to 60% of their assigned readings.

 

About 12% reported a failure to complete any assigned readings.  As a resident, would you ever report that you failed to complete any surgical readings? 

 

The article goes through other points, including the weaknesses of the online educational platform. About 66.7% stated too much content was assigned per week. So it sort of made them bite off more than they could chew.  And 85.7% of those indicated that the readings were over-assigned.

 

When I helped create Absite Smackdown!, we really wanted the book and course to follow each other. So you had auditory input and visual input from the lectures online and you had the book in front of you for the people who like to, you know, read it and experience it that way.  So you have them both. You can listen to it as you drive, et cetera. You know, of course, texting and driving is so worrisome, but listening to things in the car, people do it all the time including audio books, and this was a way to get that done.

 

We hope our work doesn't suffer from an over assignment of readings. Another weakness that was described was that the residents were too busy with clinical duties to attend the in-person sessions and the talks fell on their day off or their post-call day causing them to miss them because of duty hours restrictions.

 

So of course that’s worked out to with the Absite Smackdown! because you can watch it anytime, anywhere on any device. So for example a day off at the beach or in the mountains, and that's where Jessica and the team came up with #SundayStudySmackDown. I think I always mess the hashtag up, but it's #SundayStudySmackDown, or #AbsiteAnywhere. And, that's where you all have been sending in these photos of yourself with the book in all kinds of places. We've gotten them from ski lodges and we've gotten them from beaches. We've gotten them we got one with someone dancing with the book.  We've had all sorts of fun things like that get sent in for people who are using the book in different ways at different times, along with the lectures, and that's been really great and fulfilling for me to see as the author.

 

Other weaknesses listed by the USC residents, when asked to score the pre COVID in-person didactic educational curriculum versus the online on a scale of one to 10, the online platform had a higher scores with an average of 7.7 and in-person had about 4.8. They seemed to like that ability to do things online, and a significant percentage, 87.1 expressed a preference for the online educational platform.

 

So the headline is that it seems like we're on the right track for what we've done with Absite Smackdown!  And we're always looking for your feedback, whether you loved it, whether you didn't love it as much, or whether you find it so useful that you use it every day.

 

And I'm really happy to say most of what we're seeing and hearing is great feedback with minor tweaks that we could improve like this minor thing or this minor thing. And from version one to version two, that's already happened.  Version three is in the works and, well, it looks great. And I can't wait to see the improvement. This article kind of jives with that. It kind of says that resident staff who have scarce time appreciate a platform where they can review an online curriculum anytime, anywhere, and Absite Smackdown is particularly Absite focused. It has the online lectures and again that associated book.

 

And Jessica would really tell me she really gave me an earful if I didn't mention, there are so many different ways you can get the content. And most of what the team does is absolutely free. And we feel strongly about that. One is the Instagram account with the @daily.absite.fact every day (except Sunday when it’s #SundayStudySmackdown) the team posts a funny or interesting fact straight from the review book.  So that's in your Instagram feed every day.

 

There's the YouTube video channel where we post the video podcasts. there's this podcast that you've already found on Stitcher or iTunes or Amazon or Soundcloud:  The Absite SmackDown podcast, there's a LinkedIn @AbsiteSmackDown and there's Facebook @Absite SmackDown.

 

This sort of availability any time, for the videos and those facts seems to be what the resident staff from USC and their survey data really liked. So that's the story.

 

The story is, it seems like COVID-19 has pushed us to do things better. And part of doing it better is to have education available for residents with busy schedules, anytime, anywhere.  I know that's what I would have wanted when I was a resident.

 

That's what pushed me to help develop this.  So we sure do hope you find it useful. And as Jessica would say, #Absite Smackdown!