“It’s just a note-taking tool.”
That’s the pitch. Clean. Frictionless. Supposedly low stakes. A vendor rep demos a slick transcript, a SOAP note appears in seconds, and suddenly everyone in the room is acting like the only real question is whether the UI looks nice.
That’s the myth.
Here’s what the data actually shows: the legal issue is not whether a product calls itself an “AI scribe,” “ambient documentation assistant,” or “clinical productivity platform.” The issue is much simpler and much less sexy. If the vendor creates, receives, maintains, or transmits protected health information on your behalf, HIPAA is in the room whether the sales team mentions it or not. And if that’s happening, a business associate agreement — a BAA — is not a luxury item or a procurement checkbox. It’s the backbone of lawful use.
I’ve seen this go sideways in the most predictable way possible. A physician group gets excited about reduced pajama time. Someone starts a pilot. A few attendings record visits on their phones or through a browser extension. Weeks later, compliance finally asks the basic question nobody asked upfront: where exactly is the audio going, who can access it, and is the vendor willing to sign a BAA? Awkward silence. Then scrambling. Then the pilot disappears.
That’s not a technology problem. It’s a diligence problem.
This article is for educational purposes only and is not legal advice. HIPAA obligations and contract terms vary by setting, state, workflow, and vendor structure, so review any AI scribe deployment with qualified legal, compliance, and security professionals before using it with clinical PHI.
Hook: The AI Scribe Pitch Sounds Simple — It Isn’t
The sales framing is designed to make you relax: “We don’t diagnose. We just listen.” As if “just listening” somehow keeps you outside privacy law. It doesn’t. Listening to a patient encounter, converting speech to text, drafting a note, routing that note into an EHR, storing snippets for QA, or keeping logs for troubleshooting can all involve PHI. Usually do.
And that matters because HIPAA doesn’t care about the vendor’s preferred branding. It cares about function. Data flow. Actual handling of PHI.
A lot of clinicians still imagine HIPAA as something that mainly governs hospitals and insurers while tech vendors float around in a lighter, fuzzier universe. Wrong. If an outside company is doing PHI-related work for a covered entity, it often lands squarely in business associate territory. That means the relationship needs legal structure before the tool goes live, not after someone falls in love with the transcript quality.
The contrarian point here is simple: AI scribes are not inherently a compliance problem. Sloppy procurement is. A good vendor can absolutely fit within a compliant workflow. A bad vendor will talk in vibes: “secure,” “enterprise-grade,” “de-identified,” “temporary,” “HIPAA-friendly.” Friendly is not a legal category. “Will you sign a BAA, and does your architecture support what that BAA promises?” That’s the category.
What HIPAA Actually Covers When an AI Scribe Touches PHI
Protected health information is not some exotic subset of data. It’s individually identifiable health information held or transmitted by a covered entity or its business associate. Names, dates of birth, diagnoses, medications, visit details, recordings of clinical conversations, even contextual details that make a person identifiable in care. In the AI scribe world, that’s basically the whole product.
If the tool captures a patient saying, “My chest pain started after chemo last Thursday,” you’re already in PHI land. If it drafts “58-year-old woman with metastatic breast cancer presenting for follow-up,” you’re in PHI land. If it logs metadata tied to a patient encounter or clinician account in a way that can be linked back, you may still be in PHI land. The idea that a vendor can avoid HIPAA because it “only drafts notes” is nonsense. Notes about identifiable patients are exactly the kind of thing HIPAA contemplates.
Now the key distinction. Covered entities are the usual HIPAA-regulated healthcare actors: providers, health plans, clearinghouses. Business associates are outside persons or entities that perform functions or services for a covered entity involving PHI. That can include cloud hosting companies, billing services, and yes, many AI scribe vendors.
That’s why the phrase “we’re just a software tool” should make you more skeptical, not less. Software can absolutely be a business associate functionally. If the company receives your encounter audio, stores transcripts, hosts drafts, routes documents, or allows support staff to access records for troubleshooting, it may be acting as a business associate in practice whether it likes the label or not.
And let’s kill another lazy myth: AI does not magically de-identify data. Temporary storage does not magically de-identify data. “We strip identifiers” is not the same as compliant de-identification. HIPAA de-identification has standards. It depends on what identifiers remain, what can be re-linked, and how the system is architected. If a product stores patient audio, timestamps, clinician names, and encounter context, don’t let anyone wave that away with “our model anonymizes everything.” Marketing language is cheap. Technical reality is what counts.
What a BAA Really Does — and What It Does Not Do
A BAA is necessary in many AI scribe arrangements. But let’s bust the next myth too: a signed BAA is not a magic shield. It does not sprinkle compliance dust over a reckless product.
What a BAA does is set the rules of engagement. It defines how the business associate may use and disclose PHI, what safeguards it must maintain, what happens if there’s a breach, whether subcontractors are allowed, and what has to happen to the data when the relationship ends. It allocates responsibility. It creates enforceable obligations. That matters. A lot.
But if the contract says one thing and the product does another, the paper loses its shine fast. I’ve reviewed vendor arrangements where the BAA was polished and reassuring, while the actual security review showed broad internal access, vague retention terms, unclear subprocessors, and soft answers about model training. That’s not a legal solution. That’s theater.
The minimum issues worth caring about are not mysterious. What uses and disclosures are permitted? Can the vendor use your data only to deliver the service, or also to improve models? Are subcontractors involved, especially cloud providers or speech-processing vendors? What’s the breach notification timeline? How long do they retain audio, transcripts, prompts, and logs? Can you force deletion, and does deletion include backups or derived artifacts? What security safeguards are promised? Do you have audit rights or at least meaningful documentation rights?
And no, “everybody signs our standard BAA” is not reassuring. Standard for whom? Written to cover what? Does it address AI model training? Fine-tuning? Human review for quality assurance? Cloud hosting regions? Third-party subprocessors? A BAA drafted for a basic storage vendor may be badly mismatched to an AI workflow with layered vendors under the hood.
This is where clinicians get burned. They hear “BAA available” and assume the hard work is done. It isn’t. The BAA has to match the real-world data lifecycle. If it doesn’t, you’ve got paperwork without protection.
The Real Risk Questions to Ask Before Saying Yes
Before you say yes to an AI scribe, stop admiring the demo and ask the ugly questions.
Where is the audio stored? Not the polished answer. The actual answer. Is it buffered only in memory, stored temporarily, retained for days, or archived by default? Where are transcripts stored? In the vendor environment? In your EHR? In both? Is encounter data used to train, fine-tune, evaluate, or benchmark models? If the answer is “sometimes” or “only in de-identified form,” ask them to define that in writing. Slippery language is the oldest trick in the tech-sales book.
Who can access the data? Clinicians only? Vendor support staff? Engineers? QA personnel? Is access role-based? Is it logged? Is multi-factor authentication enforced? Are there routine access reviews? If there’s a security incident at 2 a.m., does the company have a real incident response process or just a policy PDF nobody has read since onboarding?
Subcontractors matter too. A lot. Many AI products are really stacks of vendors: cloud hosting, speech-to-text, large language model providers, analytics tools, monitoring services. Every layer is a possible PHI touchpoint. If the primary vendor can’t name its subprocessors, where data is processed, and what contractual controls bind those subprocessors, that’s a red flag, not a minor omission.
Then ask about deletion. Real deletion. Can encounter audio be deleted on demand? What about transcripts? What about logs, caches, backups, and training corpora? “We delete from the user interface” is not the same thing as deletion from the system.
And let’s talk about the blind spots that keep showing up. Consumer-grade recording apps dressed up as workflow hacks. Browser extensions that capture more than people realize. Overseas processing no one disclosed in the initial demo. Practices assuming they’re protected because the tool “integrates with the EHR.” Integration proves almost nothing. A bad tool can integrate beautifully. APIs do not confer compliance. Neither does a logo in an app marketplace.
Security controls are as important as the contract. I’d argue more important, because they determine what really happens day to day. You want encryption in transit and at rest. You want role-based access. You want logging and monitoring. You want MFA. You want clear incident response. You want a retention schedule that makes sense. You want someone on the vendor side who can answer direct technical questions without hiding behind buzzwords.
If the answers are vague, that is the answer.
Bottom Line: The Safe Answer Is Not “AI or No AI” — It’s “Show Me the Controls”
HIPAA does not ban AI scribes. That part is easy. The law is not anti-technology. What it does demand is much less glamorous and much more important: know exactly how PHI is handled, know whether the vendor is functioning as a business associate, and get the right BAA in place before clinical use.
That’s the myth-busting takeaway. Compliance is about actual data flow, not marketing copy. A BAA matters, but only if it matches the vendor’s real security practices, retention rules, subcontractor chain, and limits on data use.
So here’s the decision rule I use: if the vendor can’t explain, clearly and specifically, where the data goes, who touches it, how it’s secured, whether it trains models, what subcontractors are involved, and how deletion works, the product is not ready for your patients’ PHI. Full stop.
“Just a note-taking tool” is how people talk right before a preventable mess. Don’t be that easy to impress. Show me the controls.
