How Faculty Really Write ‘Board-Level’ Questions for Your Conferences

Most “board-level” questions you see in residency conferences would never survive 10 seconds in a real item-writing committee.

Let me tell you how it actually works behind the scenes, because I’ve watched attendings slap these together in call rooms, office hours, and Uber rides to the airport. You’re building your exam strategy on these questions; you deserve to know which ones are legit and which ones are educational cosplay.

---

The Myth vs. the Reality

Residents love to say, “Our program is great, they always give us board-style questions.” That phrase makes a lot of faculty quietly roll their eyes.

Because “board-style” in residency has at least four very different meanings:

1. Looks like the boards (same A–E layout, long stem), but is garbage psychometrically.
2. Is actually modified directly from a Q-bank.
3. Is essentially a pimp question stuffed into a multiple-choice format.
4. Rarely: is written with real item-writing principles in mind.

At most programs, you get a mix of all four, heavily skewed toward the first three.

Here’s the uncomfortable truth: unless your faculty have been formally trained as item writers for ABIM/ABFM/ABS/etc. or for serious in-training exams, they’re mostly guessing at what “board-style” means. They’re pattern-matching from what they remember seeing on their own exams years ago, filtered through nostalgia and ego.

And the way they actually produce those questions? Not pretty.

---

How Questions Really Get Written the Night Before

Let’s walk through the most common “creation pipelines” for conference questions. You’ll recognize these once you see them.

1. The “It’s 11 PM and I Just Remembered I’m on Conference Tomorrow” Special

This is the dominant method.

Faculty is on the couch, Netflix halfway through, remembers tomorrow’s 7 AM conference. Panic. Laptop opens.

Workflow goes like this:

• Pull up UpToDate topic on “Hyponatremia Management”
• Skim the summary and treatment sections
• Grab a classic scenario from memory:
  • “65-year-old man with small cell lung cancer comes in with confusion… sodium 112”
• Turn the management decision into the “question”
• Add five answer choices, at least two of which are obviously wrong
• Maybe tweak the sodium to 108 to make it look more dramatic
• Label it “Board-style question #1”

The main design principle here is not “How will this perform as a test item?” It’s “Will I have something on the slide so I’m not humiliated in front of the chief residents?”

Is this useless? No. You can still learn from it. But calling this “board-level” in the psychometric sense is generous bordering on fiction.

2. The Direct Q-Bank Cannibalization

This one you’ve seen but maybe not recognized.

Faculty opens UWorld/AMBOSS/TrueLearn on a side monitor, filters by topic, then:

• Finds a question that hits exactly the teaching point they want.
• Screenshots or paraphrases the stem.
• Changes a few details to pretend it’s “their” patient.
• Sometimes changes the answer choices… usually making the question worse.

The better faculty will ask permission or use an institutional subscription and acknowledge the source. Many do not. I’ve literally seen stems copy-pasted with the exact same distractors and same order.

The problem is not that the question started in a Q-bank. That’s actually better than the midnight special. The problem is what happens when they “customize” it:

• They remove two good distractors and add nonsense choices.
• They change one lab value but forget to adjust the interpretation.
• They flip the age/sex/setting in a way that no longer matches the epidemiology.

So you end up with a question that looks polished but is now subtly broken. And faculty are puzzled when half the room misses it.

3. The Pimp Question in Disguise

Every specialty has attendings who live to ask certain questions. Their signature moves.

Cardiologist: “What’s the only class I antiarrhythmic shown to reduce mortality in heart failure?”
Surgeon: “What’s the most common cause of early post-op fever?”
Neonatologist: “What’s the oxygen saturation target at 28 weeks?”

They’ve been asking these orally forever. At some point, someone told them, “You should make your talks more board-relevant.” So they convert their favorite oral pimp line into a MCQ by:

• Wrapping it in a vague 2–3 line stem.
• Keeping the question essentially a fact recall.
• Adding four obviously wrong alternatives.

Now it’s “board-style.”

Here’s the problem: modern board exams are not testing raw recall of single numbers or magic phrases nearly as much as you think. They’re testing:

• Next best step
• Risk stratification
• Most likely diagnosis given conflicting data
• Interpretation, not regurgitation

So these pimp-in-disguise questions feel hard because they hit your insecurity. But they’re often less representative of true board questions than a good clinical reasoning case.

4. The Rare Unicorn: True Item-Writer Style

These are the questions you want to pay attention to. You’ll feel the difference.

They usually come from:

• Faculty who have served on national exam committees (ABIM, ABR, ABP, etc.)
• People heavily involved with your in-training exam or with specialty board review courses
• Faculty who have actually attended item-writing workshops and took them seriously

Their process looks completely different:

• They start with a clearly-defined objective:
  “Test whether the learner can choose the appropriate next step in management for NSTEMI in a high-risk patient with hypotension.”

• They then build backwards:

  • Define the key decision.
  • Construct a stem that contains all necessary info but nothing extra.
  • Write a keyed correct answer that aligns with current guidelines.
  • Write plausible distractors that reflect real missteps residents make.

• They ruthlessly avoid:

  • Trivial pearls (“Which receptor subtype…” for stuff no one uses)
  • Cute zebras that never show up on the real exam
  • Overcomplicated multi-step math that eats 5 minutes

These questions may not be flashy. They might feel “too straightforward.” That’s exactly why they’re good. They’re precise.

---

What Real Item-Writing Rules Look Like (and How Faculty Bend Them)

Most board exams follow a shockingly strict internal rulebook for question writing. I’ve seen versions of these used at ABIM and in large Q-banks.

Here’s the kind of structure they aim for:

Real Board Practice	What Happens in Conferences
One clear best answer	Two reasonable answers, faculty argues afterward
Stem includes only relevant info	Random noise: dogs’ names, irrelevant CTs
No negative phrasing if avoidable	“Which of the following is NOT…” all over the place
Answer choices similar in length/style	One answer is a 3-line paragraph, others 2 words
Current guidelines strictly followed	Based on what was true when attending trained

Let me break down a few of the key principles boards actually use, and how your conferences drift away from them.

Single Best Answer, Not “Guess What I’m Thinking”

Real item writers obsess over this:

• There must be one indisputably best answer.
• Ambiguous items get thrown out during review.
• If two answers are reasonable based on the stem, the question is either revised or killed.

At noon conference, that gatekeeping rarely happens. Here’s the common scenario:

• Faculty puts up a vague question.
• Audience splits between two answers.
• Faculty defends the “key” by citing “what they’ve seen” or some older trial.
• Residents walk away more confused than before.

Your boards won’t work like that. When two answers seem good on a real exam, it’s almost always because the question writer buried a small but decisive clue—timeline, comorbidity, single lab, age, subtle phrasing—that breaks the tie.

Train yourself to hunt for that tiny discriminator. That’s board skills. Conference arguments? Not so much.

Relevance and Noise

Actual boards:

• Strip out every irrelevant adjective.
• Remove decoration that doesn’t affect the decision.
• Avoid things like race or occupation unless it changes probability.

Conference questions often do the opposite because people confuse “realistic” with “cluttered.”

So you get this:

A 38-year-old female software engineer and marathon runner with no significant past medical history, who lives with her two dogs and husband, presents to the ED…

Then the actual decision turns on a single lab value buried in that mess.

When the test is real, irrelevant info is minimized because it muddies psychometrics. When the test is a talk, faculty often pad to make the case feel “real” or to kill time.

You need to mentally filter: what details truly affect diagnosis/management probabilities, and what’s just cosmetic dressing?

---

The Hidden Agendas Behind “Board-Level” Questions

Not every question is written just to assess knowledge. There are politics and personalities behind some of what you see.

1. Curriculum Signaling

Some chiefs push faculty: “Make sure your questions hit topics that show up on the ITE.”

So attendings align with:

Resident Perception of What 'Board-Level' Means

Category	Value
Looks Like Q-bank	40
Actually From Q-bank	25
Real Item-writing Style	15
Just Pimp in MCQ Form	20

But others use questions to signal what they think matters:

• The nephrologist who insists on rare electrolyte disorders.
• The ID doc obsessed with one obscure tropical disease.
• The surgeon who still thinks old dogma is current practice.

They’ll frame their pet topic as “this is what the boards love.” Sometimes that’s true. Often it’s just what they love.

2. “My Generation Did It This Way” Inertia

If an attending took their boards in 2004 and never updated their mental model, their idea of “board-level” may be hopelessly dated.

So you get:

• Questions about drugs that have been replaced.
• Guidelines that shifted years ago.
• Workups that reflect pre-CT, pre-high-sensitivity assays, pre-novel agents.

Residents notice the mismatch when they go back to UWorld. Faculty don’t always like hearing that.

3. Subtle Evaluations of You

Sometimes the “board-style” questions in morning report are also being watched by fellowship directors, PDs, and chiefs as an informal assessment tool.

Not the raw score. The behavior:

• Do you commit to an answer or dance around?
• Can you explain your reasoning in a way that sounds like a future attending?
• Do you collapse when you’re wrong or can you course-correct in real time?

So some faculty intentionally ramp up the question difficulty not to mirror the boards, but to see how you think when pushed.

Do not assume that because a question feels insanely hard that it must be “what the boards are like now.” Sometimes it’s just what that attending uses to separate “really sharp” from “average.”

---

How You Should Actually Use Conference Questions

You’re not going to change how your faculty write these questions. You can change how you extract value from them.

Treat Them as Reasoning Drills, Not Score Predictors

The real win is not whether you got A or D. It’s:

• Did you immediately lock onto the key decision (diagnosis vs next step vs complication)?
• Did you identify 1–2 data points that matter more than the rest?
• Could you verbalize why each wrong answer was wrong in the way boards mean, not just “too aggressive” or “not indicated”?

When a question is poorly written, ignore the noise and practice this triage:

1. What’s the most likely exam objective here?
   Diagnosis? Management? Complication? Risk?

2. What single piece of info would boards never include unless they wanted me to use it?

3. Which answer best fits both the guidelines and the patient’s specific context?

The stem can be flawed and you can still strengthen your exam brain with that practice.

Cross-Check with Real Q-Banks

If your program pushes conference questions as “representative of the boards,” sanity-check them:

• After conference, search your Q-bank for the same topic.
• Compare stems: length, focus, degree of noise.
• Compare distractors: how many are truly plausible versus cartoonishly wrong.

You’ll start to see patterns:

• Some faculty regularly hit very board-like patterns.
• Others write questions that consistently over-test esoterica or raw facts.

Use that map to decide how much weight to give what you see in conference.

Learn the Patterns of People Who Know Boards

Every program has a few people whose questions consistently feel like legit exam items. Clues:

• They write clear, concise stems.
• They rarely argue with the room about the key; they anticipated most objections.
• Their explanations map directly onto guideline language.
• When you do UWorld later, you keep thinking, “This is exactly like Dr. X’s cases.”

Gravitate to those people. Ask them, explicitly, how they write questions. Some will even share internal templates or “commandments” they use.

---

Inside the Faculty Room: How They Talk About This

You want real insider? Here’s the sort of stuff I’ve actually heard in faculty spaces.

• “I threw those questions together this morning, they’ll survive.”
• “They keep asking us for ‘board-level’ but nobody has time to do psychometrics on Wednesday noon conference.”
• “I just lifted that one from MKSAP and changed the name, don’t tell ABIM.”
• “Residents always miss the same distractor; maybe the stem is garbage.”
• “I hate those trick questions, I write straightforward ones on purpose even if they’re not ‘clever’.”

There’s more variation and more self-awareness than you might think. A lot of faculty know they’re not writing true exam-level items. But the educational culture keeps using that phrase because it sells: “Come to this session, you’ll get BOARD-STYLED QUESTIONS.”

So you need to build a small filter in your head:

• “Board-looking” is not the same as “board-valid.”
• “Hard” is not the same as “representative.”
• “Tricky” is not the same as “high-yield.”

Once you internalize that, you can stop panicking every time you miss a question in front of your peers. You start asking a more useful question:

“Would this type of thinking help me on the real exam?”

If yes, hold onto it. If not, treat it as theater.

---

A Simple Mental Model: Three Buckets

One practical way to stay sane is to silently sort questions you see into three buckets:

Typical Mix of Conference Questions by Type

Category	Value
True Board-Like	15
Modified Q-bank	25
Pimp/Trivia Disguised	30
Sloppy but Benign	30

1. True Board-Like:
   Clean stem, clear objective, single best answer, plausible distractors.
   Use these as mini-exam practice.

2. Modified Q-bank:
   Started strong, maybe slightly corrupted.
   Good for content review. Verify later with your own Q-bank.

3. Pimp/Trivia/Sloppy:
   Fuzzy stem, argues with the room, tests random factoids.
   Use these to learn content or pearls, but don’t overinterpret what they mean about your “board readiness.”

Once you start ranking them this way, you’ll feel less whiplash and less impostor syndrome.

---

How Questions Should Be Built: A Quick Walkthrough

To appreciate the difference, here’s how a real item-writing workflow for a single question ideally goes, when someone knows what they’re doing.

Ideal Board-Style Question Development Flow

Step	Description
Step 1	Define Objective
Step 2	Choose Key Decision Point
Step 3	Draft Stem With Essential Data
Step 4	Write Correct Answer
Step 5	Write Plausible Distractors
Step 6	Screen For Bias And Irrelevant Info
Step 7	Peer Review And Revise
Step 8	Field Test Or Use In Conference

Most conference questions skip at least three of those steps and compress the rest into 5–10 minutes. You don’t need perfection. But understanding what a properly constructed item looks like trains your eye to spot the good stuff when it appears.

---

FAQs

1. If my program’s questions are bad, am I doomed for boards?

No. Your board performance lives and dies on what you do with real Q-banks and your own study plan. Conference questions are bonus reps. Even mediocre ones can be useful if you use them to practice reasoning and then anchor your understanding in legitimate resources (UWorld, AMBOSS, MKSAP, PREP, etc.).

2. Should I call out faculty when a question is clearly flawed?

Tread carefully. Publicly cornering an attending over item-writing principles is a fast way to be remembered for the wrong reasons. A better approach: ask a clarifying question that exposes the ambiguity—“If X were different would that change the answer?”—and then debrief later with peers or a trusted mentor. If you have genuinely board-savvy faculty, ask them how they’d rewrite it.

3. How can I tell if someone is actually an experienced item writer?

Look for concrete signs, not vague “I know the boards.” Things like: they’ve served on ABIM/ABFM/ABS/etc. committees, written for national board review courses, or helped construct your specialty’s in‑training exam. Their questions feel clean, arguments during review are brief and grounded in guidelines, and they almost never rely on obscure one-liners.

4. Is it worth writing my own “board-style” questions as a resident?

Yes—if you do it right. Writing even a small set of honest, single-best-answer questions forces you to confront your own fuzziness and tighten your understanding. Use the proper workflow: start with a clear learning objective, build a focused stem, and craft plausible distractors. Do not just rewrite UWorld. And do not kid yourself: your questions won’t be perfect. But the process will make you sharper for the real exam.

---

Two things to remember: most “board-level” questions you see in residency are really just educational theater dressed up in MCQ format, and your job is not to worship them—it’s to extract value from them. Focus on how you reason through each case and anchor everything against real, vetted board prep resources.