Stats and Study Design on Boards: A Resident’s Targeted Review Map

Biostatistics and study design do not tank board exams because they are hard. They tank them because residents review them in the wrong way.

Let me be direct. Skimming a few tables on sensitivity/specificity the night before your in‑training exam or boards is not a strategy. It is superstition. The test writers know exactly which blind spots residents have in stats and methods, and they hit them with surgical precision.

You need a map. Not a pile of flashcards. A mental map of:

1. which concepts actually show up,
2. how they are tested, and
3. what traps they set.

That is what I am going to give you.

---

1. What “Stats and Study Design” Really Means on Boards

Approximate Stats/Study Design Content By Exam

Category	Value
ABIM	9
Family Med	7
Peds	6
Surgery Quals	4

Most residents wildly misjudge the weight of this section. They either ignore it (“only a few questions”) or obsess over it like it is a full subspecialty. Both approaches are wrong.

Realistic range on major U.S. boards and in‑training exams:

• ABIM: roughly 8–12% of questions
• Family Medicine: 6–10%
• Pediatrics: 5–8%
• Surgery, EM, others: typically 3–7% but variable, especially for quality‑improvement and EBM

So no, it is not half the exam. But it is often the most point‑dense per hour of study. One tight week of targeted review can convert a weak area into almost guaranteed points.

The “stats and study design” bucket for boards usually includes:

• Core test characteristics: sensitivity, specificity, predictive values, likelihood ratios, ROC curves
• Risk, association, and effect sizes: risk difference, relative risk, odds ratio, hazard ratio
• Study designs: RCT, cohort, case‑control, cross‑sectional, case series, diagnostic accuracy, noninferiority
• Bias and confounding: selection bias, information bias, confounding, effect modification
• Error types and interpretation: Type I/II error, power, confidence intervals, p‑values
• EBM basics: NNT/NNH, absolute vs relative risk reduction, pretest/posttest probability
• Quality and QI: sometimes PDSA cycles, run charts, process vs outcome measures (more common on internal medicine, pediatrics, family medicine boards)

Board writers love disguising simple concepts in clinical clothing. You get a case of a 63‑year‑old with NSTEMI and then at the end a question that is purely about study design or test performance.

---

2. The High‑Yield Stats Toolkit (Formulas You Actually Need)

I am going to be ruthless here. There are dozens of possible formulas. You only need to own a small core set cold. The rest you can infer.

2.1 Test characteristics: your non‑negotiables

You must be able to reconstruct this 2×2 table instantly:

Disease +	Disease -
Test +	TP	FP
Test -	FN	TN

From that, everything flows.

Key formulas (these must be automatic, no mental struggle):

• Sensitivity = TP / (TP + FN)
  Probability test is positive when disease is present.

• Specificity = TN / (TN + FP)
  Probability test is negative when disease is absent.

• Positive predictive value (PPV) = TP / (TP + FP)
  Probability disease is present when test is positive.

• Negative predictive value (NPV) = TN / (TN + FN)
  Probability disease is absent when test is negative.

• Likelihood ratio positive (LR+) = Sensitivity / (1 − Specificity)

• Likelihood ratio negative (LR−) = (1 − Sensitivity) / Specificity

Board patterns:

• They love qualitative questions: “In a low prevalence population, which changes?” → PPV falls, NPV rises. Sensitivity and specificity do not depend on prevalence.
• They also like “improving test performance” questions: narrowing the inclusion criteria usually increases specificity (and often PPV) but can drop sensitivity.

You do not need to compute LR+ and LR− numbers often; more commonly, you need to interpret them:

• LR+ > 10 = strong evidence to rule in
• LR− < 0.1 = strong evidence to rule out

That level of interpretation is enough 90% of the time.

2.2 Risk, odds, and the trap of case‑control studies

You must know how to distinguish risk from odds, then remember that case‑control studies only give you odds ratios.

Definitions:

• Risk (incidence proportion) = events / total at risk
  Example: 10 MIs in 100 people followed over 5 years → five‑year risk = 0.10

• Risk difference (absolute risk reduction) = Risk_exposed − Risk_unexposed
  If statin group MI risk = 5%, control = 10% → RD = −5% (or 5% absolute risk reduction).

• Relative risk (risk ratio, RR) = Risk_exposed / Risk_unexposed
  Using that example: RR = 0.05 / 0.10 = 0.5

• Odds = events / non‑events
  10 MIs, 90 no MI → odds = 10/90 ≈ 0.11

• Odds ratio (OR) = odds_exposed / odds_unexposed

Major board trick:

You see a “case‑control study of people with lung cancer and controls from clinics”. If they ask about risk, that is a red flag. In case‑control designs, the proportion with disease is fixed by design, so you cannot derive risk or incidence. You estimate an odds ratio instead.

If the outcome is rare, OR approximates RR. Many board stems explicitly state “rare disease” to hint that OR ≈ RR.

2.3 NNT/NNH and why residents blow this question

You have seen this formula a hundred times and still freeze under pressure.

• Absolute risk reduction (ARR) = Risk_control − Risk_treatment
• Number needed to treat (NNT) = 1 / ARR
• Number needed to harm (NNH) = 1 / absolute risk increase

Two details board writers exploit:

1. Percent vs proportion:
   If control mortality is 10% and treatment is 7%, ARR = 3% = 0.03.
   NNT = 1 / 0.03 ≈ 33. Not 3.3.

2. Direction:
   If treatment risk is higher than control, that is absolute risk increase → NNH, not NNT.

---

3. Study Designs: Recognizing Them From One Line

Most residents can name every study type when they see a list. Boards do not test that. They describe a single sentence of methods and expect you to classify it, infer the limitations, and interpret effect measures.

3.1 The four core epidemiologic designs

You should be able to map each to what it measures:

Design	Direction & Key Feature
RCT	Forward in time, randomized exposure
Cohort	Forward, exposure defined first
Case-control	Backward, outcome defined first
Cross-sectional	Single time point, exposure + outcome

How boards phrase them:

• Randomized controlled trial (RCT)
  “Patients with CHF were randomly assigned to receive drug A or placebo and followed for 24 months to assess mortality.”

  Look for “randomly assigned,” “placebo,” “double‑blind”. Effect measure: usually RR, hazard ratio.

• Prospective cohort
  “A group of smokers and nonsmokers were followed for 10 years to compare incidence of lung cancer.”

  Exposure first, then disease develops. Measures incidence, RR.

• Retrospective cohort
  Same idea as prospective, but data obtained from records: “Investigators reviewed charts from 2001–2010 to identify patients with and without occupational benzene exposure; subsequent leukemia diagnoses were recorded.”

• Case‑control
  “Investigators identified 120 patients with pancreatic cancer and 240 controls without cancer, then asked about prior smoking.”

  Outcome first, then look back for exposure. Effect: odds ratio.

• Cross‑sectional
  “A survey was sent to 2,000 adults to assess current BMI and presence of hypertension.”

  Snapshot. Prevalence. Does not establish temporality.

Board traps:

• Labeling a retrospective cohort as “case‑control” because they see chart review. The key is direction relative to exposure and outcome, not whether data are old.
• Calling a simple case series a cohort. “We describe 12 patients with…” is not a cohort; there is no comparison group.

3.2 Diagnostic test studies and noninferiority trials

Two less obvious designs that get tested:

• Diagnostic accuracy study
  “All patients undergoing coronary angiography also had a CT angiogram; CT results were compared with angiography as the gold standard.”

  They will ask about sensitivity/specificity or selection bias (spectrum bias).

• Noninferiority trial
  Buzzwords: “noninferiority margin,” “not worse than by more than X%”.
  Example: comparing a new oral anticoagulant to warfarin with a prespecified margin in stroke risk.

Do not overcomplicate this. For boards, you only need to know that:

• Noninferiority trials aim to show a new treatment is not unacceptably worse than standard by more than a predefined margin.
• Intention‑to‑treat vs per‑protocol analysis has different implications here. ITT is conservative in superiority trials, but in noninferiority trials, per‑protocol can actually favor finding noninferiority.

---

4. Bias, Confounding, and the Favorite Trick Phrases

Bias questions are often one sentence long and very unforgiving. You either recognize the pattern or you do not.

4.1 Core types of bias you must name on sight

• Selection bias
  Distortion because the relationship between exposure and outcome is different among those selected vs entire population.

  Signals:

  • “Only patients who completed follow‑up were analyzed”
  • “Volunteers from an online survey”
  • “Hospitalized patients were included…” (Berkson bias)

• Information (measurement) bias
  Misclassification of exposure or outcome.

  • Recall bias: cases remember exposure differently than controls.
  • Observer bias: knowledge of exposure affects outcome assessment.
  • Misclassification: different diagnostic criteria for groups.

• Confounding
  Third variable associated with both exposure and outcome that distorts the true relationship.

  Example: Coffee drinking associated with lung cancer, but smoking is the confounder.

Key board clue phrases:

• “After adjusting for age and smoking status, the association weakened.” → Confounding.
• “Patients lost to follow‑up differed in disease severity.” → Selection bias / attrition bias.
• “Investigators knew which patients received the new drug when assessing outcome.” → Observer bias.

4.2 Effect modification vs confounding (commonly missed)

Board writers love this.

• Confounder: extra variable that makes the crude association different from the true association. After stratifying or adjusting, the association tends to move toward the null or otherwise changes consistently.

• Effect modifier: the effect of exposure genuinely differs by levels of another variable. The stratified results are different from each other, not just from the crude.

Example:

• Oral contraceptives and DVT risk by age. In younger women RR = 1.2, in women >35 RR = 4.0. Age is modifying the effect, not just confounding.

If they show a table of stratum‑specific RRs that are very different from each other → effect modification.
If they are similar to each other but differ from the crude → confounding.

---

5. P‑values, Confidence Intervals, and Power: How They Actually Ask It

You know the textbook definitions. Boards push on the edges.

5.1 P‑values: what they are not

Board stems often embed a line like: “The p‑value was 0.03.” You need to fight the reflex to say, “There is a 3% chance the null hypothesis is true.” That is wrong.

Correct interpretation they expect:

• If the null hypothesis were true, there is a 3% probability of observing a result this extreme or more just by random chance.

Common questions:

• Which of these conclusions is justified? → “The difference is statistically significant at alpha 0.05” is fine. “The therapy is clinically important” is not guaranteed.
• Multiple comparisons: If they tested 20 outcomes and one has p = 0.04, ask yourself about type I error inflation.

5.2 Confidence intervals: the workhorse on modern boards

Confidence intervals (CIs) are far more common than naked p‑values now.

You must know:

• For a difference (like mean difference or risk difference), if the 95% CI does not include 0, it is statistically significant at the 0.05 level.
• For a ratio (RR, OR, hazard ratio), if the 95% CI does not include 1, it is statistically significant.

Examples boards love:

• “The odds ratio of MI with drug vs placebo was 0.6 (95% CI 0.4–0.9).”
  CI does not cross 1 → statistically significant reduction.

• “Relative risk for stroke was 0.7 (95% CI 0.4–1.2).”
  CI crosses 1 → not statistically significant. They may ask which conclusion is most appropriate: essentially, “no statistically significant difference detected.”

They also like power questions tied to CIs. Wide CI → small sample / low precision. Narrow CI → higher precision.

5.3 Type I/II error and power in one page

Board‑level summary:

• Type I error (alpha): reject null when it is actually true (false positive). Usually set at 0.05.
• Type II error (beta): fail to reject null when it is actually false (false negative).
• Power = 1 − beta. Probability of detecting a true effect.

They test:

• How to increase power: increase sample size, increase effect size, increase alpha, decrease variability.
• Post‑hoc rationalizations: “The study may have been underpowered to detect a clinically important difference” when confidence intervals are wide crossing the null.

---

6. How Boards Hide Stats in Clinical Questions

How Stats Concepts Hide Inside Clinical Stems

Step	Description
Step 1	Clinical Vignette
Step 2	Sensitivity & specificity
Step 3	Study design & OR/RR
Step 4	NNT, CI, p value
Step 5	Bias or confounding
Step 6	Hazard ratio & censoring
Step 7	What is tested

Board writers rarely start with, “Which of the following best describes the sensitivity of this test?” They integrate stats into realistic clinical stems.

Common patterns:

1. New biomarker for disease X

   • Table with true disease status vs test result.
   • Question about sensitivity, specificity, PPV, NPV, or effect of changing prevalence or cutoff.

2. Observational study about a risk factor

   • “Investigators evaluated association between obesity and atrial fibrillation…”
   • Then: “Which of the following best describes the study design?” or “Which measure of association is most appropriate?”

3. Survival curves and hazard ratios

   • Kaplan–Meier plots with two groups.
   • Ask: “At 3 years, which statement is true?” or “Interpret hazard ratio of 0.7 (95% CI 0.5–1.2).”

4. QI project in the hospital

   • “Rates of central line infection before and after a checklist intervention…”
   • They then ask about process vs outcome measures, run chart interpretation, or interrupted time series.

If during practice questions you feel like you are “missing the stats question at the end,” good. That means you are starting to see their pattern.

---

7. A Practical, One‑Week Resident Review Plan

You do not need a three‑month epidemiology fellowship. You need 7–10 focused hours.

Here is how I would structure one week for a reasonably busy resident (on a lighter rotation or evenings on a heavy one):

Suggested One-Week Stats/Methods Study Time Allocation

Category	Value
Day 1	1.5
Day 2	1.5
Day 3	1
Day 4	1
Day 5	1
Day 6	0.5
Day 7	0.5

Day 1: Test characteristics and NNT (1.5–2 hours)

• Rebuild the 2×2 table from scratch until you can do it under 20 seconds.
• Drill sensitivity/specificity, PPV/NPV with 10–15 problems.
• Do 5–10 NNT/NNH problems focusing on ARR and percent vs proportion.

Day 2: Study designs and effect measures (1.5–2 hours)

• Make yourself identify—without labels—10 stems as RCT, cohort, case‑control, cross‑sectional.
• Solve a mix of RR, OR, and risk difference calculations.
• Do 10 questions where you must say: “this is odds ratio” vs “this is relative risk.”

Day 3: Bias, confounding, and effect modification (1–1.5 hours)

• Rapid‑fire classification: 20 vignettes, each one a single line describing a bias. Name the bias.
• Review a table of stratified RRs or ORs and practice distinguishing confounding vs effect modification.

Day 4: p‑values, confidence intervals, and power (1–1.5 hours)

• Work 10–15 questions that give CIs and ask about significance and interpretation.
• Answer conceptual questions on Type I/II errors and ways to increase power.

Day 5: Mixed board‑style questions (1–1.5 hours)

• Do 25–30 mixed board questions that integrate stats into clinical vignettes.
• After each one, not just why your answer is right, but: “What concept did they test?” Label it.

Day 6–7: Short spaced review (15–30 minutes each)

• Random 10‑question blocks; review missed concepts.
• Quick mental reconstructions of 2×2 table and the main formulas.

That is it. If you do that honestly, your performance on stats/methods questions will jump dramatically. I have watched this happen year after year with residents who thought they were “bad at math.”

---

8. Quick Reference Summary Table

Print something like this for your last‑week review:

Concept	Core Idea	Board Trigger Phrase
Sens/Spec	Test vs disease status	New diagnostic test, gold standard
PPV/NPV	Disease probability given test result	Change in prevalence
RR vs OR	Risk vs odds ratio	Cohort vs case-control wording
NNT/NNH	1 / absolute risk change	Control vs treatment event rates
RCT/Cohort/CC	Direction of time and grouping	Randomized vs exposure-defined vs outcome
Bias/Confound	Selection/info vs mixing of effects	Loss to follow-up, recall, stratification
CI & p-value	Interval around estimate, significance	95% CI crossing 1 or 0

---

9. How To Practice Without Wasting Time

One last piece: you do not have time to grind 500 stats questions. You should not. The goal is mastery of concepts, not brute force exposure.

Use stats questions deliberately:

• When you miss a question, write a 1‑line label for the concept (“classification: sensitivity vs specificity” or “bias: recall bias”). Build a short list.
• Notice your pattern. Most residents consistently miss the same 2–3 categories (commonly: case‑control vs cohort, NNT, or confounding vs effect modification). Target those.
• Revisit every concept three times across a few weeks. Short, spaced hits. That is where retention happens.

If your board prep resource has mixed blocks, consider doing one stats/methods‑only block per week until exam month, then two per week in the final month. It keeps the material active without dominating your schedule.

---

---

FAQ (Exactly 6)

1. I am terrible at math. Can I still master stats for boards?
Yes. Board‑level biostatistics is mostly ratios and proportions, not calculus. If you can divide 20 by 100 without a calculator, you have all the math you need. The problem is usually unfamiliar wording, not actual numeric difficulty. Focus on pattern recognition and the core formulas (2×2 table, RR/OR, NNT). Do 10–15 carefully reviewed questions per category rather than mindless volume.

2. How many stats questions should I do before my boards?
For most residents, about 150–250 good‑quality stats/methods questions is enough, provided you review them properly. That usually comes from one full commercial Q‑bank plus your specialty’s in‑training exams and maybe one EBM‑focused resource. If you are consistently scoring >75–80% on fresh stats questions, you are where you need to be.

3. What is the single most commonly missed concept you see?
In practice question reviews, the top offender is confounding vs effect modification, followed closely by odds ratio vs relative risk in case‑control vs cohort designs. If you can reliably identify those correctly under time pressure, you are already ahead of most test takers.

4. Should I memorize every formula (variance, SD, etc.)?
No. For boards, you rarely need to compute variance or standard deviation manually. You should conceptually know what they represent (spread/variability) and what affects them (outliers), but rote formula memorization is low yield. Prioritize: test characteristics, RR/OR, ARR/NNT, basic CI interpretation, and power.

5. How close to the exam should I review stats and study design?
Very close. Stats/methods are ideal last‑week material because they are compact, rules‑based, and less prone to interference than big clinical topics. Do not cram it all the night before, but deliberately schedule a 1–2 hour focused refresher on stats and EBM in the final 3–5 days before your exam.

6. Do different specialties emphasize different stats topics?
Somewhat, but the core overlaps are large. Internal medicine, pediatrics, and family medicine emphasize RCTs, cohort/case‑control, NNT, bias, and CI interpretation. Surgery and EM may lean more on diagnostic test performance and perioperative risk studies. Psychiatry boards sometimes add more cross‑sectional surveys and rating scales. If you have old in‑training exams for your specialty, skim the stats/methods items; they are an excellent predictor of what your board will care about.

---

Key points to walk away with:

1. You do not need to know all of biostatistics. You need a small, high‑yield core—2×2 tables, RR/OR, NNT, CIs, bias, and the major study designs—absolutely cold.
2. Stats and study design are one of the highest‑ROI sections for a busy resident: 7–10 focused hours can convert them from a liability into easy points.