Do AI scribes hallucinate?

Rarely, but not never. The rate is low enough that ongoing single-note review is sufficient after an initial audit period; it is not low enough that unreviewed signing is safe.

Should I keep the audio in case of a note dispute?

Only if the vendor's terms and your practice policy explicitly support this. Retained audio is a compliance risk of its own; the finished, clinician-signed note is the record of the visit.

How many visits should I audit before trusting a scribe?

10 to 20 visits with the templates you use most, spread across visit types. If your error rate is below 0.3 per note by the end of that window, standard review is sufficient.

What error rate should I expect?

On mature psychiatry-native templates, most clinicians report roughly 0.1 to 0.3 minor errors per note requiring edits. Severe errors (medication, risk paraphrasing, fabrication) should be rare.

Verified July 202610 min read

How to audit AI scribe note accuracy in psychiatry

AI scribe accuracy in psychiatry is not one number. It is a distribution across failure modes, and the failure modes with the highest clinical risk are medication errors, paraphrased risk-assessment language, and fabricated content. Every psychiatric note should be reviewed by the clinician before signing, and for the first 10 to 20 visits with any new scribe, that review should be a deliberate audit against the visit transcript — not a scan.

The failure modes that matter

Medication errors

Wrong drug name, wrong dose, wrong frequency, or missing medications. The single highest-risk failure mode in psychiatric documentation. Every medication in the plan must be verified against your prescribing record, not just the note.

Risk-assessment paraphrasing

Suicide and homicide risk assessment must preserve the clinician's exact language. Paraphrasing 'passive suicidal ideation without plan or intent' as 'endorses some suicidal thoughts' changes clinical meaning and creates medicolegal exposure.

Fabricated content ('hallucination')

Content in the note that was not in the visit — a symptom the patient did not mention, a family history detail, a plan step the clinician did not propose. Rare but not zero, and higher risk when the note is longer than the visit warranted.

Omissions

Content that was in the visit but not in the note. Often affects the interval history, side effects, and psychosocial context. Lower medicolegal risk than fabricated content but reduces the note's clinical usefulness.

Template drift

The scribe uses a template shape that no longer matches your template, silently. Usually appears after a vendor update. Compare structure against your saved template quarterly.

A practical audit protocol

For the first 10 to 20 visits with any new scribe: keep the audio available and read the finished note against your recollection of the visit and, when available, the transcript. Grade each note on the five failure modes above and record errors per note. After the audit period, spot-check one note per full clinic day for the next month. After that, standard clinician review before signing is the ongoing floor.

Scoring rubric

Zero errors: sign as-is after review. One or more low-severity errors (minor omissions, phrasing changes): edit and sign. Any medication error, any risk-assessment paraphrase that changes clinical meaning, any fabricated content: correct, sign, and open a support ticket with the vendor citing the specific example.

Track your error rate

Errors per note, by category and severity. A sustained rate above roughly 0.3 errors per note on a mature template is worth escalating with the vendor.

Why vendor accuracy claims are not comparable

Different vendors measure accuracy differently — some against a reference note written by a clinician, some against clinician-edited output, some against a synthetic gold standard. The methodologies are not standardized. Treat vendor accuracy numbers as directional, not comparable. Your own audit on your own visit mix is the only measure that matters.

Long-term monitoring

Ongoing clinician review before signing is non-negotiable. Beyond that, spot-check one note per full clinic day, watch for template drift after vendor updates, and re-run the initial audit protocol after any major product update that changes note structure.

Frequently asked

Do AI scribes hallucinate?: Rarely, but not never. The rate is low enough that ongoing single-note review is sufficient after an initial audit period; it is not low enough that unreviewed signing is safe.
Should I keep the audio in case of a note dispute?: Only if the vendor's terms and your practice policy explicitly support this. Retained audio is a compliance risk of its own; the finished, clinician-signed note is the record of the visit.
How many visits should I audit before trusting a scribe?: 10 to 20 visits with the templates you use most, spread across visit types. If your error rate is below 0.3 per note by the end of that window, standard review is sufficient.
What error rate should I expect?: On mature psychiatry-native templates, most clinicians report roughly 0.1 to 0.3 minor errors per note requiring edits. Severe errors (medication, risk paraphrasing, fabrication) should be rare.

Scribes referenced in this guide

Related guides

AI scribes for psychiatric medication-management visits
How AI scribes handle 20-minute psychiatric med checks — dose changes, side-effect review, adherence, DSM-5 reassessment, and structured medication lists.
HIPAA and AI scribes: what psychiatrists must verify
The specific HIPAA, BAA, encryption, audio-retention, and 42 CFR Part 2 checks a psychiatrist should run before using any AI scribe with real patients.
The best AI scribes for psychiatrists in 2026
Vendor-neutral 2026 ranking of AI scribes for psychiatry, scored on MSE structure, medication management, risk assessment, HIPAA, EHR fit, and pricing.