Where does PHI actually live in a voice AI stack? Three architectures compared
Every voice AI platform will tell you it is HIPAA compliant. None of them will draw you the diagram showing which of the seven hops in a call path retains your patients' data, because on most of those hops the answer is “we do.” So here is the diagram.
The compliance question people ask is “is this tool approved?” The one that matters is “where does the audio go?”
There is a failure mode that recurs in medical and dental practices, and it has nothing to do with buying the wrong software. A staff member has a task — summarise this voicemail, draft this recall message — and pastes a patient's name and chart number into a consumer AI tool to get it done faster. A patient's name is one of the eighteen identifiers HHS defines as protected health information, so the data left the covered boundary the moment it was submitted. Nobody intended a breach. The organisation had a policy. What it did not have was a drawn boundary, so a person doing their job improvised one.
Voice AI reproduces that failure at machine scale, and it hides it better. A single inbound call touches seven or eight systems in under a second. Each one may hold the audio, the transcript, or both, for a retention period set by a default you did not choose. The architecture, not the marketing page, determines whether that is a compliant workflow or a reportable incident.
Nobody in the platform business has an incentive to publish this breakdown. Twilio's own HIPAA-eligible services document is one of the few honest artefacts in the category, and it is honest precisely because it is a scope limitation rather than a claim. What follows is the same treatment applied to a whole voice stack: every hop, what persists there, and the three ways to arrange it.
Who this is for. Practice owners evaluating a vendor, compliance officers reviewing an architecture, and engineers who have been handed “make the phone agent HIPAA compliant” as a ticket. There is a nine-point BAA checklist at the end you can take into a vendor call.
One call. Eight hops. Seven of them can keep a copy.
This is the generic path for a voice agent that answers a phone, understands the caller, and answers back. Swap vendor names as you like; the shape does not change.
-
The telephony carrier
The moment a call connects, the carrier holds the audio. If recording is on, it holds a stored copy too. This is the hop teams forget because it predates the AI: the phone number was set up years before anyone said the word LLM.Covered only if the carrier signed a BAA and recording is on its eligible-products list
-
Speech-to-text
Audio goes out to be transcribed. The transcript comes back. What most teams never check is the request log on the vendor side — many speech APIs retain request payloads for a debugging window by default, and that payload is the audio.Retention default is the risk, not the transcription itself
-
The language model
The prompt carries whatever context the agent needs to be useful: the caller's name, why they are calling, sometimes their appointment history. Prompts and completions are logged by default on most standard API tiers.Consumer and standard developer tiers are usually not BAA-covered
-
Text-to-speech
Easy to overlook, because it feels like output rather than data. But the string sent for synthesis is frequently the most identifying text in the whole call: “Hi Margaret, I have you down for a crown prep on Thursday at 2.”Synthesis input is PHI and needs the same coverage as the transcript
-
Your own application logs
Traces, error reports, request dumps, the LLM observability tool someone added during a debugging week. This hop is inside your boundary, which is exactly why it gets the least scrutiny — and why PHI ends up in a log aggregator nobody scoped.Inside your boundary, still a breach if it is unredacted and over-retained
-
Analytics and call QA
Whatever reads transcripts to score calls, spot intents, or build a dashboard. These tools are bought by operations, not engineering, so they are rarely in the architecture diagram at all.The most common uncovered sub-processor we find in a review
-
Backups and snapshots
Every store above has a backup, and backups outlive retention policies. A 30-day purge on the primary store means nothing if the nightly snapshot is kept for a year in a bucket with different rules.Retention is only real if it applies to backups too
Count what that means in practice. Of the seven places PHI can persist, exactly one — your own application logs — sits in code your team reviews. The other six are configuration: a retention setting, an account tier, a region flag, a sub-processor list. This is why voice AI compliance work is mostly not programming, and why a code review will not find the problem.
Managed platform under a BAA.
You buy a voice agent platform, it signs a BAA, and the entire call path runs in its cloud. This is what the market sells and, for a large share of practices, it is the correct answer — live in weeks, no infrastructure to own, per-minute pricing.
The trade is that your covered boundary is the vendor's boundary. You inherit its sub-processor list, its retention defaults, its region choices, and its incident timeline. When an auditor asks what happened to one specific call, you open a support ticket. That is an acceptable posture; it is just worth naming as a posture rather than assuming it away.
What to verify before signing: that the BAA names the specific products in the path rather than the vendor; that recording and transcription retention are configurable and that the setting reaches backups; that customer audio is excluded from model training on your tier, in writing; and that the sub-processor list comes with a notice period. The failure mode here is not the architecture, it is signing a BAA and assuming it did more work than a BAA does.
Where it stops fitting: multi-location groups with their own security review, anyone whose data-residency requirement is specific rather than general, and practices whose legal counsel wants audit evidence they can produce without a third-party dependency.
Self-hosted media plane in infrastructure you control.
The orchestration and media handling run in your own cloud account — a self-hosted LiveKit deployment in your VPC — and only the model calls leave it, each to a vendor holding a BAA for the specific product used. Recordings, transcripts, session state and the audit log stay in storage you own, under keys you hold.
This is the architecture we deploy for practices where the compliance review is adversarial, and it is the same stack described on the HIPAA-compliant AI receptionist page. The point of it is not that self-hosting is inherently safer — a badly configured VPC is worse than a well-run platform. The point is that the boundary is one you can define, instrument, and prove.
The configuration details that carry the compliance weight: zero-retention or minimum-retention mode enabled explicitly on every model vendor, not assumed from the tier; egress restricted at the security-group level so an uncovered endpoint cannot be reached even if a future code change points at one; audio written to object storage with immutability enabled and customer-managed keys, so the audit trail cannot be edited after an incident; PHI stripped from application traces at the logging layer rather than by convention; and the whole thing expressed in infrastructure-as-code in version control, because “here is the commit that created this bucket with these settings” is evidence and a screenshot is not.
What it costs: a self-hosted HIPAA voice stack is a $25,000–$50,000 build with an ongoing retainer, versus a per-minute subscription. It is justified by a specific requirement, not by preference. If you cannot name the requirement, Architecture A is the better commercial decision and we will say so on the call.
On-device and edge processing.
Wake-word detection, speech recognition and intent classification run on hardware in the practice, or on the handset. No audio crosses the network at all. Vendors in this space — Picovoice is the reference point — make a real architectural guarantee rather than a contractual one: there is no cloud hop to cover, so there is no BAA to negotiate for that hop.
The constraint is capability. On-device models are excellent at bounded tasks: recognising a fixed command set, routing by intent, capturing a callback number. They are not a substitute for a large model handling an open-ended conversation about a dental crown, a nervous new patient, and a schedule with three constraints on it. The moment you want that quality of conversation, you are back to a cloud model and back to Architecture A or B for that portion of the path.
Where it genuinely wins: as a front door. Edge processing handles identification and routing with no PHI egress, and only escalates to a cloud model for the calls that need one — which shrinks the volume of audio crossing the boundary rather than eliminating it. That hybrid is under-used, and it is worth pricing if your call mix is dominated by simple, repetitive requests.
The three architectures, side by side.
| A · Managed under BAA | B · Self-hosted media plane | C · On-device / edge | |
|---|---|---|---|
| PHI boundary | The vendor's cloud | Your VPC, with model calls as the only egress | The premises |
| Compliance rests on | Contract | Contract plus architecture | Architecture |
| Audit evidence | Request it from the vendor | Produce it yourself from IaC and immutable logs | Nothing left the building |
| Conversation quality ceiling | Frontier models, no ceiling | Frontier models, no ceiling | Bounded — routing and capture |
| Time to live | Days to weeks | 4–6 weeks per site | Weeks, plus hardware |
| Commercial shape | Per-minute subscription | $25,000–$50,000 build + retainer | Hardware plus licence |
| Fits | Single sites and small groups without a bespoke security review | Groups with adversarial compliance review or specific residency rules | Fixed-intent front doors, and hybrid front-ends to A or B |
There is no winner in that table. There is a match between a requirement you can articulate and the cheapest architecture that satisfies it. The mistake we see most often is a practice buying Architecture B for a reason that Architecture A already covered, and the second most common is buying A while believing it delivered B.
Nine questions to take into the BAA conversation.
A signed BAA assigns liability. It does not move data. These are the nine things that determine whether the architecture behind the signature holds up, in the order we ask them during a review. The useful signal is often not the answer — it is how long it takes to get one.
- Product scope. Which named products does the BAA cover? Vendor-level HIPAA claims are meaningless; coverage is per service. Twilio publishes a HIPAA-eligible services list, and anything you use outside it is uncovered even though your account holds a signed BAA.
- Account tier. Is the BAA active on the account you are actually calling from? Standard developer accounts are typically excluded until the agreement is explicitly executed, and sending PHI before that point is the violation — not a paperwork gap.
- Retention, per artefact. Get a number in hours or days for each of: audio, transcript, request logs, and derived embeddings. Then ask whether each is configurable, and whether the setting applies to backups.
- Training exclusion. Is customer data excluded from model training, in writing, on your tier? Enterprise tiers usually exclude it; the same vendor's default tier often does not.
- Sub-processors. Get the current list, and the notice period before it changes. A vendor that can add a sub-processor silently has handed you an audit finding you cannot see coming.
- Region pinning. Which regions can the data be processed in? “Primarily US” is not an answer. You want a configuration setting you can verify from your own console.
- Incident response. How many hours from vendor detection to your notification, and is that number in the contract or in a support page they can edit? Your own breach clock depends on it.
- Right to audit. Can you request evidence, or only a SOC 2 report? A SOC 2 is useful and is not the same as the right to ask what happened to one specific call.
- Deletion on exit. What happens to everything above when you terminate, on what timeline, and what proof do you get?
How to read the answers. A vendor that answers all nine from documentation in one call is a vendor that has done this before. A vendor that needs to check with legal on retention is normal. A vendor that treats the sub-processor question as unusual has told you where its architecture is weakest, and that is worth more than the rest of the call.
What Architecture B looks like once it is running.
Figures verified 26 July 2026 against the deployment. The full write-up is in the CleverAnswerAI case study, and the practice-facing version of this architecture is on the AI dental receptionist page, including which practice management systems accept a direct booking and which need a gateway.
The number in that set that took the most engineering is the third one. Answer rate is a capacity problem and cost per call is an optimisation problem; three years without a reportable incident is an architecture problem, and it is the one that is invisible in a demo.
Three things worth keeping.
One. Compliance is a property of a data flow, not of a vendor list. Draw the seven hops for your own stack before you evaluate anything, and the vendor conversation changes character entirely.
Two. BAAs are scoped to products and to account tiers, not to companies. Most of the uncovered PHI we find in reviews sits with a vendor that does hold a BAA, on a service that BAA does not name.
Three. Retention defaults are the quiet part. Nobody chooses a 30-day audio retention window on a speech API; it is simply what was there, and it applies to the backups too.
Questions we get from compliance officers.
Does a signed BAA make a voice AI platform HIPAA compliant?
Where does PHI persist in a voice AI call path?
Is self-hosting required for HIPAA compliance?
What should we ask a vendor about their BAA?
Can call transcripts be used to train a vendor's models?
How do you prove to an auditor that PHI stayed inside the boundary?
Who wrote this, and why it matters when the decision is a compliance decision.
This article was written by Viktor Andriichuk, founder and lead architect at DataFlux Software, who designs and operates the self-hosted architecture described in section B for medical and dental clients. It is written from deployment work rather than from a vendor datasheet, which is why it recommends a managed platform in the cases where a managed platform is the right answer.
DataFlux Software has shipped 50+ production systems since 2018, and works on AI voice and agent infrastructure exclusively since 2023. Contracting entity: Gazolin Production SRL (Romania, EU).
The nine questions above, on one page.
Same nine questions, same order, formatted so you can work down them while the vendor is on the line.
Bring us a stack and we will draw the boundary on it.
Twenty minutes. Tell us which telephony, speech and model vendors are in your path today, and we will tell you which hops are uncovered and which architecture fits the requirement you actually have. If that turns out to be a managed platform rather than a build, we will say so.