You're three charts behind. It's 6:47 PM. The waiting room is finally empty, and you've got fourteen unfinished notes staring at you from the EMR. Someone told you that if you paste the raw visit data into ChatGPT and ask for a "concise summary," you can knock out each note in forty seconds instead of six minutes. It sounds like salvation. It's actually one of the fastest ways to lose your license, compromise patient safety, and hand plaintiff attorneys exactly what they need to destroy you in deposition.
This article is for educational purposes only. It is not financial advice, not legal advice, and not tax advice. Figures vary by individual circumstances, so consult a qualified professional before acting.
The Illusion of Efficiency: Why Your EMR Data Is Not Public Domain
The appeal is obvious. Your EMR's native summarization tool is clunky. The templates generate bloated, repetitive notes. A large language model can take your messy, dictated, fragmented encounter and produce something clean. Readable. Almost elegant.
But here's what's actually happening when you paste patient data into a consumer-facing chatbot: you're uploading Protected Health Information to a server environment that has no Business Associate Agreement with your practice. No HIPAA safeguards. No audit trail. No breach notification protocol. You are, in a very literal sense, voluntarily exfiltrating patient records to a third party that is under no obligation to protect them.
That's not a technicality. That's a reportable breach.
Public Models vs. Enterprise AI: Know the Difference
Not all AI tools carry the same risk profile. The distinction matters more than most physicians realize.
Consumer-facing LLMs (ChatGPT's free tier, Claude's public interface, Gemini's standard web app): These platforms may use input data to train future model iterations. Read the terms of service. Some have changed their policies, but the default assumption should be that anything you type could be stored, reviewed, or used for training. Even if a company claims it doesn't train on user data today, policy shifts happen. Subpoenas happen. Breaches happen.
Enterprise-grade, sandboxed AI environments (Microsoft Copilot for Healthcare, Epic's AI integrations, AWS HealthLake with Comprehend Medical): These operate under signed Business Associate Agreements. Data is encrypted, isolated, and not used for model training. Audit logs are maintained. There's a legal framework holding the vendor accountable.
The problem? Most physicians I've spoken with are using category one. They're pasting patient names, lab values, medication lists, and social histories directly into a public chat window because it's free, it's fast, and nobody told them it was a problem.
Before you automate your documentation, learn why some practices risk compliance with unapproved apps.
The Catastrophic Scenario
Here's how this plays out when it goes wrong. A patient discovers their confidential information was disclosed, maybe through a data breach at the AI company, maybe because another user somehow accessed a shared conversation, maybe because the physician mentioned the tool during an unrelated audit. The patient files a complaint with HHS Office for Civil Rights. An investigation reveals that the physician regularly input PHI into a non-compliant platform. The physician's malpractice insurer may decline coverage for a willful HIPAA violation. The state medical board opens an inquiry. The hospital system, if the physician is employed, may terminate the contract for cause.
Your medical license isn't protected by "I didn't know." Ignorance of HIPAA's requirements has never been a defense. The statute doesn't care that you were tired. It doesn't care that the tool was convenient. It cares that you disclosed protected information without safeguards.
Stop pasting patient data into public AI tools. Today. Not next month. Today.
Hallucinations in the Chart: The Clinical Risk of Synthesized Notes
Let's assume you've moved past the HIPAA issue. Maybe your institution has a compliant AI tool. Maybe you're using a vendor with a proper BAA. You're in the clear legally, at least on the privacy front.
If you are concerned about malpractice, it is worth understanding the legal reality of AI scribe usage.
Now meet the second trap. It's worse in some ways, because it's harder to detect and it directly harms patients.
The Confabulation Problem
Large language models don't "know" anything. They predict the next most likely token in a sequence based on patterns in their training data. When you ask an LLM to condense a clinical encounter, it's not extracting key facts like a human would. It's generating text that statistically resembles a coherent medical summary.
Most of the time, this works fine. The model produces a plausible, well-structured paragraph that captures the gist of the visit. But sometimes, unpredictably, without warning, the model invents details. It fills gaps with plausible-sounding information that was never in the original input. This is called confabulation, or hallucination, and it's not a bug you can patch. It's an inherent feature of how these models work.
The hallucination doesn't announce itself. It doesn't flag the invented information with a warning label. It sits in your note, looking exactly like every other sentence, waiting to be read by the next clinician, or by a plaintiff's attorney.
The Clinical Narrative Matters More Than You Think
When you delegate summarization to an AI, you're not just saving time. You're outsourcing the cognitive process of synthesizing a patient's story. That synthesis, the act of deciding what's relevant, what's changing, what's concerning, is part of clinical reasoning. It's how you catch the subtle deterioration that doesn't trigger an alert. It's how you notice that the patient's weight loss, fatigue, and new medication form a pattern that demands investigation.
An AI summary strips out the analytical work. It gives you the product without the process. And in medicine, the process is where the diagnostic insight lives.
I've seen notes where the AI "cleaned up" a patient's medication history by removing a discontinued drug, without flagging that the discontinuation was recent, the reason was adverse reaction, and the drug was in the same class as a newly prescribed medication. The summary looked tidy. The allergy risk was invisible.
A Case Study in Hidden Harm
Consider this scenario. A physician uses an AI summarization tool to condense a lengthy discharge summary for a patient being transferred to a skilled nursing facility. The original note contains this passage:
"Patient was started on warfarin 5 mg daily on day 2 of admission for new-onset atrial fibrillation. INR on discharge: 2.1. Patient educated on dietary interactions and bleeding precautions. Follow-up with cardiology in 5 days."
The AI summary produces:
"Patient started on warfarin for atrial fibrillation. INR therapeutic at discharge. Follow-up scheduled."
Clean. Concise. And dangerously incomplete. The specific dose is gone. The timeline for cardiology follow-up is vague. The phrase "INR therapeutic" is an interpretation, not a datum, 2.1 is at the low end of the therapeutic range, and depending on the patient's indication, the cardiology team might want closer monitoring. The nursing facility receiving this summary has less information than they need to safely manage the transition.
Now imagine a different error. The original note says: "Patient reports occasional chest pain, associated with exertion, relieved by rest. Denies radiation, diaphoresis, or syncope." The AI summary says: "Patient denies chest pain."
That's not a formatting issue. That's a fabricated clinical statement that directly contradicts the source material. If that summary drives the next clinician's assessment, the patient's cardiac symptoms are invisible. And when the patient shows up to the ED three weeks later with an MI, the chart says they never complained of chest pain.
The physician who signed the note owns that statement. Not the AI vendor. Not the EMR company. You.
The Liability Gap: Identifying Who Is Truly Responsible
Here's the question every physician using AI summarization needs to answer honestly: when the AI makes an error, who pays for it?
A Computer Cannot Be Sued
The legal framework is unambiguous. AI tools are not licensed practitioners. They hold no medical degree. They carry no malpractice insurance. They cannot be named as a defendant in a medical malpractice suit in any meaningful way.
You can't depose an algorithm. You can't ask ChatGPT why it decided to condense "normocytic anemia" as "normal hemoglobin." You can't cross-examine a language model about its differential diagnosis process.
The physician who signs the note is the responsible party. Full stop. Your signature attests that you have reviewed the content, verified its accuracy, and approved it as part of the medical record. If you didn't verify, that's on you. If you didn't have time to verify, that's still on you. The court won't accept "the AI summarized it and I was behind on charts" as a defense.
The Standard of Care Problem
Here's where it gets thornier. The standard of care isn't defined by what tools you use. It's defined by what a reasonable physician would do in similar circumstances. If a reasonable physician would review the raw encounter data and synthesize it themselves, or at minimum, verify any AI-generated summary against the source material, then signing an unverified AI summary falls below that standard.
And the standard is shifting. As AI tools become more common, plaintiff attorneys are becoming more sophisticated about how to question them in depositions. Expect questions like:
- "Did you review the AI-generated summary against the original clinical data?"
- "What tool did you use, and is it covered by a Business Associate Agreement?"
- "Were you aware that this tool has a documented hallucination rate?"
- "Did you receive any training on the limitations of this tool before using it in patient care?"
If you can't answer these questions confidently and honestly, you're exposed.
Actionable Steps to Protect Yourself
You don't need to abandon AI entirely. But you need to use it like a clinician, not like a consumer.
Audit your current workflow. Right now, this week, identify every tool you're using that touches patient data. Ask: Is this covered by a BAA? Does it store input data? Does it use input data for training? If you can't answer these questions, assume you're at risk and stop using the tool until you can.
Transition to sanctioned tools. If your institution offers an enterprise AI solution with a BAA and proper safeguards, use that. If they don't, advocate for one. Don't build a workaround with consumer tools in the meantime.
Never sign a note you haven't personally reviewed against source data. AI summaries are drafts, not finished products. Treat them like a medical student's note, useful starting point, but you verify every clinically relevant detail before you sign.
Document your verification process. If you use AI summarization, make it clear in your workflow that you've reviewed and verified the content. Some institutions are adding attestation language to AI-assisted notes. Ask your compliance officer what's expected.
Flag and report hallucinations. When you catch an AI error, and you will, report it to your IT team, your compliance office, and the vendor. These errors need to be tracked. If your institution doesn't have a reporting mechanism for AI-related errors, that's a systemic problem that needs to be raised.
Educate your colleagues. Most physicians using these tools dangerously don't know they're doing anything wrong. A five-minute conversation at your next faculty meeting could save someone's career.
Key Takeaways
Never input PHI into consumer-facing generative AI tools. You are essentially publishing your patient records to a training dataset. There is no safe way to do this, and the legal exposure is severe.
AI-generated summaries are probabilistic, not diagnostic. They are incapable of clinical reasoning. They are prone to "hallucinating" patient history, medication details, and lab results. Every generated summary is a draft that requires verification.
Your signature is your word. If an AI fabricates a lab result, invents a medication dose, or omits a critical symptom, you bear 100% of the legal and moral responsibility. The tool doesn't protect you. The vendor doesn't protect you. Your diligence is the only protection you have.
The pressure to see more patients, document faster, and finish charts on time is real and relentless. I'm not dismissing that. But the solution to a broken documentation system isn't to hand your clinical judgment to a probabilistic text generator with no accountability and no license. Protect your patients. Protect your license. Use AI tools where they're sanctioned, verified, and appropriate, and nowhere else. Your future self, sitting in a deposition room staring at a note you didn't actually write, will thank you for the discipline you showed today.
