Residency Advisor Logo Residency Advisor

How Do I Judge Program Quality Quickly During SOAP Week?

January 6, 2026
12 minute read

Resident evaluating residency program board in hospital hallway -  for How Do I Judge Program Quality Quickly During SOAP Wee

What do you do when you have 10 minutes to decide whether to apply to a program you’ve never heard of, in a city you’ve never been to, that just showed up on the SOAP list?

Here’s the blunt truth: during SOAP you are not choosing your dream program. You’re triaging risk. The game is: “Is this program good enough that I will be safe, trained, and not trapped?”

You can judge that surprisingly fast if you know exactly what to look for.


The 3-Question Filter You Should Use First

During SOAP, you do not have time for a deep dive on every program. So start with three hard filters:

  1. Will I be safe here?
  2. Will I be trained well enough to have real options after residency?
  3. Is this program at risk of collapsing or harming my career?

If the answer to any of those is “probably not,” move on. There are plenty of bad fits on SOAP lists. You are not obligated to rescue any program.

Let me walk you through exactly how to answer those three questions quickly.


Step 1: Do a 5-Minute Hard-Screen on Every Program

You’ve got a list of unfilled spots. You’re exhausted. You’re on a countdown. This is where people make terrible, avoidable decisions.

Here’s your 5-minute workflow per program.

Mermaid flowchart TD diagram
Fast SOAP Program Filter
StepDescription
Step 1See unfilled program
Step 2Check specialty and location dealbreakers
Step 3Skip program
Step 4Look up on FREIDA
Step 5Check board pass and fill rate
Step 6Google + Reviews scan
Step 7Check ACGME and hospital health
Step 8Add to apply list

1. Instant dealbreakers (30–60 seconds)

Skip immediately if:

  • Specialty or track doesn’t match what you’re willing to do (e.g., prelim-only when you need categorical and have no reapplication plan).
  • Geography is a true no-go for personal or visa reasons.
  • It’s a non-ACGME program (rare in SOAP but verify for fellowships/prelims).

This is not where you get picky about weather or nightlife. This is about true, non-negotiable constraints.

2. FREIDA check (1–2 minutes)

Pull the program up on AMA FREIDA. You’re looking for fast, high-yield data:

Key FREIDA Data to Scan Fast
ItemGood SignRed / Yellow Flag
Board pass rate≥90% recent years<80% or “N/A”
Program sizeModerate, stable numbersTiny with big year-to-year swings
Fill rate (if listed)Consistently full or near-fullChronic unfilled spots
Fellowship placementSome graduates to decent fellowshipsNo data / no graduates tracked
Visa sponsorshipClearly stated (if you need it)“No” or unclear

You’re not trying to find the best program. You’re trying to weed out obviously unsafe ones.

3. Rapid Google + Reddit/sdN review (2–3 minutes)

Type:
"[Program Name] residency review",
"[Hospital Name] internal medicine residency Reddit",
"[Program Name] ACGME warning".

What you care about:

  • Clear reports of abuse, chronic violations, or toxic leadership.
  • Words that keep repeating: “unsafe,” “malignant,” “never at home,” “no supervision,” “lost accreditation,” “constant turnover.”
  • Very recent (last 2–3 years) negative patterns.

One angry review doesn’t mean much. Ten reviews saying the same thing? That’s a pattern.


Step 2: Understand Why This Spot Is Unfilled

Almost no one asks this directly, and it’s a mistake not to.

Programs are in SOAP for a reason. For your own safety, you should assume there’s a story and try to figure out which story it is.

Most common reasons (some are fine, some are not):

pie chart: Geography less popular, New program, Rapid expansion, Reputation / culture issues, Visa / candidate mismatch

Common Reasons Programs Enter SOAP
CategoryValue
Geography less popular30
New program20
Rapid expansion20
Reputation / culture issues20
Visa / candidate mismatch10

Safer reasons

These are usually acceptable risks during SOAP:

  • Less-desirable location (rural, small city, or “flyover” state).
  • Newer program with still-developing reputation.
  • Sudden expansion in positions.
  • Competitive specialty in a less “sexy” region (e.g., surgery in a small Midwest town).

Concerning reasons

Be much more cautious if:

  • Program has lost residents recently (transfers, non-renewals, high attrition).
  • There’s recent leadership chaos (multiple PD changes in a short time).
  • The program or hospital has had ACGME citations, loss of accreditation, or bankruptcy/closure headlines.
  • Chronic inability to fill in the main Match across multiple years.

Politely but directly ask on any SOAP call:

“Can you share why there are unfilled positions this year?”

If the answer is vague, defensive, or they blame “students these days” instead of giving a concrete explanation, that’s not a great sign.


Step 3: What Actually Predicts Program Quality During SOAP

Everyone obsesses over “prestige” and “name.” During SOAP, that’s not your priority. The big rocks that matter:

  1. Accreditation stability
  2. Clinical volume and case mix
  3. Supervision and education culture
  4. Graduate outcomes (jobs, fellowships, boards)
  5. Work environment and burnout risk

Let’s translate that into fast, practical checks.

1. Accreditation and hospital health (5 minutes)

Check ACGME and hospital news:

  • Google: "[Program Name] ACGME complaint", "ACGME notice", [Hospital Name] closure, [Hospital Name] bankruptcy.
  • Look for: “program under warning,” “initial accreditation with warning,” “program closure.”

If the hospital itself is financially unstable or closing service lines, your training will suffer. Period.

2. Clinical volume and case mix

You need to graduate competent. That means volume.

Ask on calls or emails:

  • “What’s your average admission volume per intern on call?”
  • “How often are residents doing [key procedures] by PGY-2?”
  • “Are there any major services that have been closed or outsourced recently?”

Too low volume → you graduate undertrained.
Wildly high volume with no support → you burn out or drown.

3. Supervision and education culture

This is the difference between surviving and actually growing.

Ask residents (if you get them, even briefly):

  • “When you’re overwhelmed at 2 a.m., who physically shows up to help?”
  • “Are attendings approachable for questions or do people get shamed?”
  • “What happens when a resident struggles clinically? Who supports them?”

If residents hesitate, look at each other, or give overly polished, canned answers, pay attention.


Step 4: Use a 10-Point Quick-Score for Each Program

In SOAP you need a simple mental rubric. Here’s one I’ve used with applicants who had to move fast.

Rate each item 0–2 (0 = concern, 1 = mixed, 2 = reassuring). Add up to a 10-point score.

Quick SOAP Program Quality Score
Category0 points1 point2 points
Accreditation / stabilityWarning / chaosNew / unclearStable, no major concerns
Board pass & outcomesLow / unknownAverage, limited dataHigh boards or clear decent outcomes
Work environmentMultiple serious red flagsMixed reviewsMostly positive, no major toxicity signs
Volume / training strengthToo low or dangerously highUnclear or mixedReasonable volume, clear procedures/cases
Leadership transparencyEvasive, defensiveNeutralDirect, honest answers

You can do this in 5 minutes per program.

Interpretation (this is my bias, but it works):

  • 8–10: Reasonable to pursue strongly in SOAP.
  • 5–7: Consider, especially if options are limited, but ask harder questions.
  • 0–4: Only if you have essentially no alternatives and understand the trade-offs.

Step 5: How to Handle SOAP Calls and Micro-Interviews

You’ll get very short calls. Sometimes 5–10 minutes. You must extract maximum information fast.

Here’s a simple script that doesn’t sound confrontational.

Questions to ask program leadership

You won’t get to ask all of these. Pick 2–3 that matter most to you:

  • “How would you describe the culture of the program in one or two sentences?”
  • “Where do most of your graduates go after residency in the last few years?”
  • “What changes has the program made in the last 1–2 years based on resident feedback?”
  • “What does support look like for residents who are struggling academically or personally?”
  • “Why do you think positions were unfilled this year?”

Listen just as much to how they answer as what they say.

If everything is spun as “perfect,” “amazing,” “best place ever,” with zero specifics, that’s not reassuring. Healthy programs are honest about their weaknesses and what they’re addressing.

Questions to ask residents (if you can)

These are gold. Keep it simple:

  • “If you had to choose again, would you pick this program?”
  • “What is one thing you’d change here if you could?”
  • “On your hardest days, what makes you think about leaving?”
  • “Do you feel comfortable calling attendings at night?”

If a resident says, “Honestly, I’d still choose it” without hesitating—that’s a very good sign in SOAP.


Step 6: Managing Risk vs. Perfection

SOAP is not where you build the perfect career. It’s where you avoid long-term damage.

Big mistakes I’ve seen:

  • People rejecting solid community programs in less-popular cities because they’re still mentally in the “I deserve a top academic name brand” headspace.
  • People taking clearly unstable, malignant programs just because it’s in a big city they like.
  • People ignoring preliminary vs categorical differences and then panicking PGY-2 when they have no next step.

You should be ruthless about one thing: trajectory.

Ask yourself:

“From this program, can I realistically build towards the next thing I want—another specialty, fellowship, community job, reapplication—without being blocked?”

If the answer is yes, even if it’s not glamorous, that program is absolutely SOAP-worthy.


Step 7: Strategic Prioritization When Time Is Tight

You might have 45–60 minutes to submit a wave of applications. Prioritize fast with a simple tier approach.

bar chart: Initial screen per program, Top-tier short list review, Mid-tier programs, Low-tier quick glance

Time Allocation for SOAP Program Review
CategoryValue
Initial screen per program5
Top-tier short list review15
Mid-tier programs8
Low-tier quick glance2

Practical approach:

  • Create Tier 1: Programs with stable hospitals, reasonable cities for you, and no obvious red flags. These get your first and strongest attention.
  • Create Tier 2: Unknowns with neutral-but-not-bad vibes and limited data.
  • Create Tier 3: Programs with mild concerns but possibly acceptable if options drop.

Apply widely enough that you’re not relying on any single program, but not so chaotically that you’re applying to places you’d never actually attend. If you truly would not go there under any circumstance, do not apply. Matching there and scrambling to escape is worse than reapplying in many cases.


A Quick Visual Summary: What to Look At, In What Order

Mermaid timeline diagram
SOAP Program Evaluation Timeline
PeriodEvent
First Pass - 0-1 minDealbreaker check - specialty, visa, geography
First Pass - 1-3 minFREIDA snapshot - boards, size, stability
Deeper Screen - 3-6 minGoogle and reviews - red flags, hospital health
Deeper Screen - 6-10 minQuick-score 0-10 and assign tier
Calls - Call dayAsk 2-3 targeted questions to PD and residents
Calls - After callsFinal adjustment of tiers and preferences

FAQ: Fast Judging of Program Quality During SOAP

1. Should I ever take a clearly malignant program just to match?

If you have absolutely no other path to any residency in your desired field and you understand the mental and physical cost, maybe. But in most cases, a truly malignant, unstable program will hurt you more than a strategic reapplication year where you fix your weaknesses and aim for a decent environment. Do not sacrifice your safety and license risk for a line on your CV.

2. How much does program “prestige” matter in SOAP?

Much less than people think. For core fields like internal medicine, family, peds, psych—being at a stable, mid-tier community program with supportive faculty often beats being at a “big name” where you’re miserable, under-supported, or burned out. For ultra-competitive fellowships (ortho, derm, etc.), prestige matters more, but in SOAP you’re usually trading some prestige for survival. That’s acceptable.

3. Is a brand-new program too risky?

Not automatically. New programs can be great if they’re backed by a strong hospital system and led by experienced faculty from established places. But you must check: leadership experience, hospital stability, presence of other existing residencies, and whether they’ve already recruited decent faculty. If everything is “still being built,” be cautious.

4. What if board pass data is missing or incomplete?

Missing data is not ideal, but during SOAP, it’s not an automatic dealbreaker. For new programs, there simply won’t be board data. For older ones with missing numbers, ask directly: “How have your residents done on boards recently?” If they dodge, change the subject, or give generic lines like “we fully support them” without specifics, lower your score for that category.

5. What’s one thing I should do today to be ready for SOAP week?

Make a one-page personal rubric now. Write down: your true dealbreakers, your “must haves” (like visa sponsorship or location constraints), and your rapid 0–2 scoring system. Keep it on your desk or saved as a note. When SOAP hits and you’re running on adrenaline and very little sleep, you’ll make far better decisions if you’re not inventing criteria on the fly.


Open a blank note right now and draft your 10-point quick-score rubric plus your non-negotiables. When the SOAP list drops, you’ll be deciding with a system, not with panic.

overview

SmartPick - Residency Selection Made Smarter

Take the guesswork out of residency applications with data-driven precision.

Finding the right residency programs is challenging, but SmartPick makes it effortless. Our AI-driven algorithm analyzes your profile, scores, and preferences to curate the best programs for you. No more wasted applications—get a personalized, optimized list that maximizes your chances of matching. Make every choice count with SmartPick!

* 100% free to try. No credit card or account creation required.

Related Articles