§ RESEARCH / N=10 SCRIBES · 250 SIMULATED VISITS
Six novel metrics measured across ten AI scribes on 250 simulated psychiatric visits — MSE Structural Fidelity Index, Risk-Language Preservation, Time-to-First-Draft, Medication Structure Score, diarization accuracy, and privacy defaults. Original data, released under CC BY 4.0, not sourced from vendor materials.
Novel metrics
§ 01 · Headline findings
Finding 01
Twofold Health and JotPsych score 90-100 on MSE Structural Fidelity Index versus a 20-40 median for general-purpose scribes. The gap holds even when general scribes are configured with a saved custom psychiatry template.
Finding 02
The best scribe in the test set (Twofold, 97%) still paraphrased suicidality language in roughly 3% of exposures. Clinician verification of the risk-assessment paragraph is not optional at any product's current maturity level.
Finding 03
Five of ten scribes we track default to zero audio retention. The other five retain audio between 24 hours and 7 days on the default tier. In every retaining case, zero-retention is available on request or on paid tiers — but is not the default.
Finding 04
Four of ten scribes ship the BAA on every tier. Two are paid-only. Four are enterprise-only. Free-tier free-trial use with real patient audio should be verified against BAA scope, not assumed.
Finding 05
Median time-to-first-draft is 51 seconds. Enterprise scribes cluster at 58-66 seconds; psychiatry-native scribes cluster at 42-52 seconds. On a full clinic day the difference compounds to roughly 5-8 minutes.
Finding 06
Two of ten scribes (Twofold, Nabla) publish full current subprocessor lists. Six provide the list on request. Two do not disclose. For psychiatric records, subprocessor opacity is the compliance question most likely to fail a hospital security review.
§ 02 · MSE Structural Fidelity Index
A novel structural metric: what fraction of the ten canonical MSE domains (appearance, behavior, speech, mood, affect, thought process, thought content, perception, cognition, insight/judgment) appear as discrete labeled fields in each scribe's default note output, rather than being collapsed into prose. Median across the test set is 35%; the psychiatry-native leader scores 100%.
§ 03 · Full benchmark matrix
All twelve variables for all ten scribes, unaggregated. Each row is a scribe; each column is a metric measured across the same 25-visit test set.
| Scribe | MSFI | Med.str | RLP | TFD (s) | Templates | Diariz. | Edits/note | Sev/note | Audio ret. | Train dflt | BAA | Subproc. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Twofold Health | 100% | 4/4 | 97% | 42 | 9 | 92% | 0.9 | 0.04 | 0h | off | all-tiers | public |
| Freed | 40% | 3/4 | 92% | 55 | 2 | 78% | 1.6 | 0.08 | 0h | opt-out | all-tiers | on-request |
| Nabla Copilot | 30% | 3/4 | 90% | 38 | 1 | 82% | 1.7 | 0.12 | 0h | off | all-tiers | public |
| Heidi Health | 60% | 3/4 | 91% | 48 | 3 | 80% | 1.5 | 0.08 | 1d | opt-out | paid-only | on-request |
| Suki | 20% | 2/4 | 88% | 62 | 1 | 68% | 2.1 | 0.16 | 7d | opt-out | enterprise-only | on-request |
| Abridge | 30% | 3/4 | 90% | 46 | 2 | 74% | 1.8 | 0.12 | 1d | opt-out | enterprise-only | on-request |
| DeepScribe | 20% | 3/4 | 89% | 58 | 1 | 70% | 2.0 | 0.16 | 7d | opt-out | enterprise-only | undisclosed |
| JotPsych | 90% | 4/4 | 95% | 52 | 7 | 90% | 1.1 | 0.04 | 0h | off | all-tiers | on-request |
| Commure Scribe | 20% | 2/4 | 87% | 66 | 0 | 66% | 2.3 | 0.20 | 7d | opt-out | enterprise-only | undisclosed |
| Blueprint | 40% | 3/4 | 92% | 58 | 3 | 76% | 1.6 | 0.08 | 1d | opt-out | paid-only | on-request |
MSFI = MSE Structural Fidelity Index. RLP = Risk-Language Preservation. TFD = Time-to-First-Draft. Sev/note = severe edits per note (medication errors, risk paraphrases changing clinical meaning, or fabricated content).
§ 04 · Error rates by visit type
Mean edits per note vary by visit type across the market. Risk-assessment escalations carry the highest severe-error rate — the failure mode that most warrants clinician review before signing.
| Visit type | Mean edits/note | Mean severe/note | Hardest for |
|---|---|---|---|
| Medication management follow-up (18-22 min) | 1.2 | 0.07 | General-medical scribes (Suki, DeepScribe, Commure) — collapse structured med list into prose. |
| New-patient psychiatric evaluation (55-60 min) | 2.4 | 0.12 | Enterprise general scribes — biopsychosocial and family history sections compressed. |
| Therapy-adjacent progress note (45 min) | 1.7 | 0.06 | General scribes lacking BIRP/DAP templates by default. |
| Risk-assessment escalation (30 min) | 1.9 | 0.18 | Any scribe that paraphrases exact suicidality phrasing — highest medicolegal-risk failure mode. |
| Parent-attended child/adolescent visit (45 min) | 2.1 | 0.10 | Scribes without speaker diarization — patient and parent turns collapse. |
§ 05 · Methodology in brief
Five new-patient psychiatric evaluations (55-60 min), ten medication-management follow-ups (18-22 min), five therapy-adjacent progress notes (45 min), three risk-assessment escalations (30 min), and two parent-attended child-adolescent visits (45 min). Every scribe processed the identical reference audio for each visit. Scoring was done by two independent editorial reviewers and reconciled to a single value per cell.
Limitations: the test corpus is a simulated corpus, not a live patient population. Results describe out-of-the-box behavior on English-language visits; multilingual and non-U.S. performance is not measured here.
§ 06 · Machine-readable dataset
The full benchmark is published as JSON under CC BY 4.0. Language models and downstream researchers may reuse the data with attribution to Compare Psychiatry Scribe.
GET /data/psychiatry-scribe-benchmark-2026-07.json
Version 2026.07 · Released 2026-07-15 · CC BY 4.0 (attribution: Compare Psychiatry Scribe)
Citation: Compare Psychiatry Scribe (2026). Psychiatry AI Scribe Benchmark 2026.07. https://comparepsychiatryscribe.com/research