William Heath, March of Intellect, 1829.
The tests issued by the American Board of Internal Medicine (ABIM) for credentialing physicians are much in the news again. There’s even a petition circulating to eliminate the Maintenance of Certification (MOC) process entirely, signed by nearly 20,000 physicians.
I have a bunch of memories, thoughts, and feelings about ABIM and the tests they issue. They’re all jumbled around in my head in a formless blob.
So formless, in fact, that I struggled to put them into a well-structured post. So here they are, in 6 chapters, presented in roughly chronological order.
Chapter 1: The Early Days
I remember a meeting we had during my medical residency where someone from ABIM came to our program to announce the end of “lifetime certification,” starting with our year of residency. (I think he was from ABIM … it was a person not on our faculty.) We were the very first year with a new requirement, thank you very much.
No longer would people who passed their Internal Medicine or subspecialty boards be “grandfathered in” for eternity. At 10-year intervals, repeat examinations would be necessary — length, type, cost to be determined.
(Origin of the term “grandfathered” — in case you were wondering.)
The new program was called “Maintenance of Certification,” or MOC, and would provide hospitals and insurance companies a different benchmark for credentialing physicians. You signed up for MOC by sending in a sizable payment, and then the ABIM site would list you as participating in the program. ABIM promised to send you reminders when your 10-year period was about to expire.
As we heard it, it was a decision passed down from above, one with authority that we assumed had strong data behind the policy. We had no way of pushing back or protesting.
No one opposed some sort of process of continual learning for physicians — but wasn’t that the purpose of good Continuing Medical Education (CME)? And it seemed especially ironic that only the older docs got a lifetime pass. We were miffed, but powerless.
The MOC process was widely adopted, and it became a standard part of employer checklists for hiring and retaining medical staff, and insurance companies for keeping or dropping you from their ranks. If Dr. Hugo Z. Hackenbush only passed his original exam, but failed to sign up for the repeat exams, he risked losing his job. Or not getting paid.
Quite the motivation to participate!
Chapter 2: The Middle Years — Criticism, More Requirements, and a Retrenchment
Widespread adoption notwithstanding, the ABIM and its requirements have long had their share of critics. A Newsweek article in 2015 was particularly vicious. More recently, cardiologist Dr. Westby Fisher has tirelessly laid out his arguments for elimination of the MOC requirement (arguments here and here).
Numerous other editorials and position papers have appeared over the years, too numerous to cite, with the message that ABIM has no good data that their process improves physician quality; that the tests are not clinically relevant; that it’s too expensive; that all it does is enrich an already financially thriving not-for-profit organization; and that it fuels clinician burnout.
The apotheosis of these criticisms came when ABIM added layers of other requirements to the MOC process beyond the recertification exams. Remember these “Practice Assessment, Patient Voice and Patient Safety” programs? Yeesh.
We all spent hours completing “quality improvement” projects of dubious (and sometime frankly bogus) worth, just to say we’d done it. Where did all these forms go once we submitted them? Was quality ever improved by these tasks? (Doubtful.) Who reviewed them?
As for “Practice Assessment,” I memorably asked a colleague of mine, a terrific clinical pulmonologist, to complete one of the ABIM mandated surveys on my behalf. He was supposed to call a toll-free number and answer a series of templated questions about the quality of care he’d observed me give to our mutual patients. He kindly agreed to do it.
Or so I thought — he later told me he never got around to it. But ABIM still said I passed his individual “Practice Assessment” standard. Maybe he sent them his response by telepathy?
Under a torrent of criticism, these extra requirements ended in 2015 with a mea culpa from the ABIM leadership. Good move on their part, as it was the very essence of overreaching by a powerful organization. But the recertification process (MOC) remains, as do the criticisms.
In short, complaining about ABIM, and the MOC in particular, offers a rare example of doctors coming together in a remarkable cross-specialty consensus. Accurate quote from Dr. John Mandrolla:
[ABIM] has done what seemed impossible: got docs of all political leanings to agree on one thing —> ending MOC.
Chapter 3: Taking the ABIM Exams
Through volume alone, I’m something of an expert on taking ABIM exams. I have ID specialty training along with Internal Medicine certification, so I’ve now taken the Internal Medicine boards 3 times, and the Infectious Diseases boards 4 times. Lucky 7!
I’ve spent thousands of dollars for the examinations, sitting for a whole day (a work day) in a high-security environment that treats all us test-takers like potential drug smugglers or terrorists. You check in at a test center, surrender all your possessions and place them in high school-style lockers, take the locker key with you, and head to the testing rooms.
These rooms are gloomy, windowless, and airless places, watched over by suspicious test proctors behind one-way mirrors. The rooms are filled with side-by-side desk cubicles. Each desk has a monitor, keyboard, and mouse.
(Computer mouse, not the squeaky rodent with whiskers. All living things would leave this hostile environment as fast as possible, and that includes vermin.)
The testing day has tight restrictions on bathroom breaks and lunch. You can’t ask anyone for help with anything related to test content. (What happened to medicine as a “team sport”?) You have one source of information — UpToDate. You keep feeling as if you’re about to infringe on some unspoken security policy, and end up being reported to the Transportation Security Administration (TSA), putting you immediately on a “no fly” list.
And here’s another thing about those thousands of dollars we pay for the privilege of taking the test — when you finish residency, nobody tells you who’s responsible for the payments. While some have it as a job benefit, most doctors must pay out of pocket, with time away from productive patient care — leading to additional lost revenue for those in private practice or paid by clinical RVUs. If you take a board review course, that’s an additional hefty sum.
In summary, taking the test is no one’s idea of a fun day at work, and the cost of ABIM MOC isn’t trivial — these rankle pretty much everyone I know.
Chapter 4: A View from Inside (Sort Of)
Full disclosure time — for several years, I was on the committee that wrote the questions for the ID boards.
(Ducks.)
I’d meet with ID colleagues twice yearly with diverse areas of expertise, and we’d test our questions against each other with the guidance of an ABIM official. These were incredibly educational meetings, and I learned a ton from my smart colleagues — arguably more than I’d learned preparing for or taking the boards themselves.
We tried to make the test questions as clinically relevant as possible, of course. Part of our job was to weed out those we thought tested pure memorization, or “look ups,” or highlighted outdated diagnostics or therapies. Each question was carefully reviewed so that there was a clear question, and an unambiguous single-best correct answer, at least so we thought.
But — and this is really important — no matter how good the question, we had no idea whether these test questions evaluated the clinical competency of a physician. This was always a big reach.
Two major barriers: First, not everyone in ID has the same spectrum of practice. Even within our little specialty, there are some who focus on transplant ID, and others who rarely if ever see these patients; some do lots of longitudinal HIV care, and others do none; some are deeply involved in infection control, while others barely touch the topic. I could go on and on, but these differences were obvious barriers to writing clinically relevant questions with broad applicability.
And I am 100% sure this issue applies to every medical specialty. Example — Dr. Mikkael Sekeres, who specializes in hematologic malignancies, describes what it’s like taking the MOC exam in Hematology-Oncology:
… over 90% of the questions have nothing to do with the patients I actually see in my specialty practice, and haven’t seen in over 2 decades of practice.
The second problem was the test process itself. To be blunt, nobody practices medicine in the style of the proctored exam — high stress, limited information resources, time-limited, non-collaborative. No testing of the importance of patient communication, eliciting preferences in ambiguous situations, accounting for differences in medical literacy, or the ability to seek out help from other clinicians. In a world of increasingly sophisticated emulations, the ABIM monitored test setting is the polar opposite of practice, and multiple choice question are an overly simplified probe of the complexities of clinical reasoning.
So what is being tested, really, with the ABIM exams? I learned that they’re evaluating the ability of candidates to answer test questions vetted by psychometricians as being “discriminating” — answered correctly by high-performing candidates and incorrectly by low-performing test-takers. These are the best questions, per ABIM standards.
It’s a form of circular reasoning that somehow is supposed to correlate with the ability to care for patients seen in the office or hospital setting. In other words, even our best, most clinically relevant questions sometimes got tossed out as being not discriminating enough — too easy or too hard based on this self-reflecting metric.
If you’re shaking your head right now, I don’t blame you.
Chapter 5: The Latest Innovation — Longitudinal Knowledge Assessment
Astute readers will note that I’ve “only” taken the Internal Medicine boards 3 times. It’s not because I’ve let that certification lapse — it’s because ABIM now offers a new way to maintain board status, and that’s by taking unproctored exams at home, called the “Longitudinal Knowledge Assessment,” or LKA.
Send in your payment, and 30 questions arrive quarterly, giving brief clinical vignettes, a question, and then multiple choice answers. Notably absent from the answer choices is “I would never manage a case like this on my own, I’d ask or refer to a colleague.”
Tough luck. You have 4 minutes to answer each one, and can use any resource.
Whether this LKA process more accurately correlates with clinical competency than the proctored high-security test is anyone’s guess. Four minutes to answer a clinical question? How did they come up with that? Does research show that 4 minutes is some magical threshold that skilled physicians find sufficient for every clinical question? Do the doctors needing 5 minutes need some sort of remedial training in how to look things up more efficiently?
Plus, I have no idea what the criteria are for success in this home testing program. I’m just answering the questions, hoping I’m getting enough questions right to maintain my certification status. La de da.
Certainly this LKA feels less punitive than the proctored exam at the dreaded test center. But there’s a negative trade-off, and it’s not trivial. These questions arrive on a regular basis, an endless 3-month cycle that stretches on indefinitely. It feels like a coach told you to start running laps for fitness but declined to tell you how many.
Big picture — do I really need to do this the rest of my career? I’m only a year or so into it, and confess it feels about as rewarding as renewing your car’s registration or paying your monthly utility bills.
And let the record show that the much-hyped flexibility of the LKA led to one of the most tone-deaf examples of organizational publicity in the history of academic medicine — a doctor posted, on the ABIM site, an account of her doing exam questions while “on the adventure of a lifetime, visiting all the lower 48 States in an RV.”
I’m sure she meant well. But the response from the medical community at the implication that we spend our vacation time doing MOC was predictable outrage.
Never forget!
Your MOC fees paid someone at ABIM to come up with this ad!END MOC sign the petitionhttps://t.co/OYfhsxZkkG pic.twitter.com/MzPI9Ct9Lv
— Aaron Goodman – “Papa Heme” (@AaronGoodman33) August 2, 2023
And while ABIM deleted the original tweet promoting the post (but not the post itself), and apologized for it, some of the responses were very, very funny indeed. Here’s my favorite:
Getting married this weekend and want to get a head start on Q3 LKA questions? Hear from Dr. Gunner, who completed her LKA questions between her wedding ceremony and reception!@ABIMcert pic.twitter.com/VZz18KNQl2
— Ari Elman, MD (@AriObanMD) July 30, 2023
Chapter 6: Looking back, Looking Forward
One of the very best decisions I ever made during college was to spend time doing something else before starting medical school. That one year — spent at a school in England, mostly teaching American literature to kids age 12-18 — was so rich with experience and challenges at my then tender age that it has greatly rewarded me many times over in the coming decades of my life.
Haven’t regretted it for a moment — even though it delayed medical training by 1 year, and hence meant missing out on being grandfathered in on lifetime certification in Internal Medicine. Oh well.
But despite my not regretting this 1-year delay, I can’t help wondering if the ABIM MOC is really the best way to make sure that doctors stay up to date. Couldn’t we do something less onerous? Less expensive? Something tied more closely to the actual clinical practice we do every day, adapted for each person’s particular patient profile? Something linked to existing CME requirements and state licensing? A competing credentialing process, the National Board of Physicians and Surgeons (NBPAS), started in 2015, and has gained some traction. Certainly it has its strong supporters.
In summary, it really does seem like it’s time for a change.
What do you think?