What your compliance team should be asking every AI vendor - and what Pegasi's architecture actually does.
When a health system evaluates an AI diagnostic platform, the compliance conversation often focuses on a narrow set of questions: Is there a Business Associate Agreement? Is data encrypted in transit and at rest? Has the vendor completed a HIPAA risk assessment? These are necessary starting points, but they are far from sufficient.
The more probing questions - the ones that actually differentiate compliant AI platforms from ones that create regulatory exposure - are about where patient data goes during the training and inference pipeline, how the vendor handles de-identification, what happens to PHI when a model is retrained, and what the audit trail looks like when a compliance officer needs to reconstruct the full data flow after an incident. Most AI vendors are not prepared to answer these questions in the detail that a sophisticated compliance team should require.
At Pegasi, we believe that HIPAA compliance for AI diagnostic platforms deserves a more specific and technical treatment than it typically receives. This article is an attempt to provide that treatment - both to explain how we have built our own architecture and to give health system compliance teams a framework for evaluating any AI vendor's claims.
The Health Insurance Portability and Accountability Act does not specifically address AI or machine learning - it predates the clinical AI era by two decades. However, its core provisions apply fully to AI platforms that process PHI as part of clinical workflows.
Under the HIPAA Privacy Rule, a covered entity (your health system) must have a Business Associate Agreement in place with any vendor that creates, receives, maintains, or transmits PHI on its behalf. An AI diagnostic platform that ingests patient data to generate diagnostic alerts is unambiguously a business associate. The BAA must specify the permitted uses and disclosures of PHI - including whether the vendor may use PHI to train or improve their models, which is a distinct use that requires explicit authorization.
The HIPAA Security Rule requires administrative, physical, and technical safeguards for all electronic PHI. For AI platforms, this includes encryption standards (AES-256 at rest, TLS 1.2+ in transit is the current baseline), access controls with minimum necessary access principles, audit controls that log all access and activity, and integrity controls that detect unauthorized PHI modification. SOC 2 Type II certification is the most common independent validation that these controls are in place and operating effectively, though it is a framework attestation rather than a HIPAA certification per se.
The HIPAA Breach Notification Rule requires covered entities and business associates to notify affected individuals, HHS, and in some cases the media following a breach of unsecured PHI. For AI vendors, this creates specific requirements around incident response timelines and notification obligations that should be spelled out in the BAA. A vendor that cannot describe its breach response protocol in detail is a vendor whose incident response you should not trust.
The most significant and underappreciated HIPAA compliance risk in clinical AI is in the model training pipeline. An AI platform that improves its diagnostic models by learning from your patients' outcomes is using PHI for a purpose beyond the original treatment, payment, and operations context - unless the BAA explicitly authorizes this use, or the data is properly de-identified before model training occurs.
There are three compliant approaches to model improvement using patient data, each with different tradeoffs. First, explicit patient authorization: patients consent to their data being used to improve AI models. This approach is rigorous but practically difficult at scale - it requires IRB oversight in most contexts and significantly limits the data available for training. Second, HIPAA-compliant de-identification: patient data is stripped of the 18 identifiers specified in the HIPAA Safe Harbor standard before being used for model training. This is the approach most AI vendors claim to use, but the implementation details matter enormously - re-identification risk in genomic datasets is non-trivial, and Safe Harbor de-identification is not sufficient for protecting genetic information in all contexts. Third, federated learning with no PHI transmission: model updates are computed locally within the institution's data environment and only mathematical gradient updates are transmitted to the central model. No PHI ever leaves the institution's network. This is Pegasi's approach, and it is the most structurally sound from a HIPAA perspective because it eliminates the PHI transmission risk rather than managing it.
HIPAA's audit control requirement is often implemented minimally - a log that records which user accounts accessed which records at what times. For AI platforms with complex data pipelines, this minimum implementation is inadequate for a real compliance investigation.
When an HHS Office for Civil Rights investigator arrives after a reported breach involving an AI diagnostic system, the questions they will ask include: which patient records were accessed by the model inference engine, when, and for what purpose? Which model version produced which diagnostic alerts? Were any PHI records transmitted outside the institution's network during the relevant period? Was the BAA in effect at the time, and did it authorize the data uses that occurred?
Pegasi maintains a structured audit trail that answers all of these questions. Every FHIR data pull is logged with the source record identifier, the timestamp, the data elements retrieved, and the model inference run ID that requested them. Every alert generation event is logged with the model version, the input data elements, and the confidence scores. This audit trail is available to the institution's compliance team on demand, in a format designed to support a regulatory investigation rather than just an internal review. We understand that compliance teams need to be able to reconstruct the full data flow, not just confirm that encryption was enabled.
Pegasi has completed three consecutive annual SOC 2 Type II audits with zero material findings. This is a meaningful credential, and we cite it prominently. But it is important to be clear about what SOC 2 Type II certification actually validates, because it is commonly misrepresented.
SOC 2 Type II is an attestation by an independent auditor that the vendor's security controls have been in place and operating effectively over the audit period (typically 6-12 months). It covers five Trust Service Criteria: Security (the foundational criterion), Availability, Processing Integrity, Confidentiality, and Privacy. Not all SOC 2 reports cover all five criteria - some vendors only attest to the Security criterion, which is a more limited scope than full SOC 2 coverage.
SOC 2 does not certify HIPAA compliance. It is a different standard. A vendor with a clean SOC 2 report may still have HIPAA exposure if their BAA terms, de-identification practices, or breach notification procedures do not meet HIPAA requirements. Conversely, some HIPAA-required controls may not be explicitly tested in a SOC 2 audit. Compliance teams should treat SOC 2 as evidence of security discipline, not as a substitute for HIPAA-specific due diligence.
For health systems that require additional assurance, Pegasi supports a formal HIPAA security assessment process as part of our enterprise onboarding, working directly with the institution's information security and compliance teams to provide the documentation they need for their own risk management programs.
Based on our experience working through compliance reviews with 12 health system partners - some with sophisticated security teams and some with more limited resources - these are the questions that consistently reveal the most about an AI vendor's actual compliance posture.
Ask for the specific BAA language covering model training and improvement. If the BAA permits PHI use for "product improvement" or "service enhancement" without more specificity, that is a red flag. Model training using patient data should be explicitly addressed with defined purposes and limitations.
Ask for a data flow diagram that shows every point at which PHI enters, moves within, and exits the vendor's system. If the vendor cannot produce this diagram within a reasonable timeframe, or if the diagram reveals data flows that were not previously disclosed, treat this as a serious compliance concern.
Ask about subprocessors - the third-party vendors the AI platform uses for its own infrastructure. AWS, Azure, and GCP all offer HIPAA-eligible services, but the vendor must have appropriate BAAs with these subprocessors, and the subprocessors must be operating PHI-handling services within the HIPAA-eligible configuration. Ask to see the list of subprocessors and the BAA coverage for each.
Ask about the last penetration test date, the scope, and whether the findings are available for review. Annual penetration testing is a standard best practice. If the last test was more than 18 months ago, or if the vendor is reluctant to share findings, that is a concerning indicator for organizations managing PHI at scale. As we discuss in detail in our article on patient data security in healthcare AI platforms, third-party security testing is a foundational requirement for any vendor processing protected health information.