ElevenLabs announces Scribe v3 Live Medical, a medical transcription model designed for real-time phone calls. It is built on Scribe v3 Live, the company’s latest real-time speech recognition model, and optimized for clinical language. It is available now in ElevenAgents, the company’s voice agent platform.
A voice agent acts on what it hears during a call. When a patient states they are taking sitagliptin for diabetes, the agent can add that medication to their list for a clinician to review. However, drug names are easily misheard. In its own testing, ElevenLabs found a transcription model had transcribed sitagliptin as citalopram, an antidepressant. For many people, these calls are the first step in reaching their care team, and an agent can only help them if it correctly understands what is being said.
Calls Where Clinical Details Matter
According to ElevenLabs, drug names and dosages rarely appear in everyday language, making them the words a general-purpose real-time model is most likely to mis-transcribe. Scribe v3 Live Medical builds on the clinical focus of Scribe v2 Medical, already offered for audio recordings, such as dictation or consultation notes. It is intended for patient, member, or provider calls involving medication at admission, a procedure during a prior authorization request, or a symptom during follow-up. It is also used for calls with no clinical content, such as scheduling appointments or opening hours, with the company stating it maintains its accuracy on everyday language.
Several features complement the model. Teams can load up to 10,000 custom terms, which the model only uses when a caller appears to be pronouncing one of them. According to the company, with such a list, Scribe v3 Live Medical made up to 48% fewer errors on medical terms. The model also recognizes when a number is not finished, preventing the agent from interrupting a caller spelling their member number in groups of digits. Finally, it supports over 90 languages and tracks language changes during a call.
Results Based on Company Testing
On PriMock57, a public dataset of 57 simulated general practice consultations each about nine minutes long, Scribe v3 Live Medical shows a word error rate (WER) of 12.8%, which is 17% fewer errors than the next best-performing medical model. On Fareez OSCE, which includes 272 recorded interviews from clinical exams, it achieves the lowest semantic error rate of the tested medical models, at 0.6%. ElevenLabs notes that these results come from its own testing, conducted in real-time mode with each model in its best available configuration, and do not cover all existing models.
In terms of latency, Scribe v3 Live transcribes speech in about 150 milliseconds, and Scribe v3 Live Medical achieves, according to tests, the same value. On Eka Medical ASR, it makes 4.7% fewer errors than Scribe v3 Live, with an 11% versus 11.54% semantic error rate.
Health Data Protection
Healthcare agents hear protected health information (PHI) on many calls: names, birth dates, member numbers, medications, and symptoms. According to ElevenLabs, Scribe v3 Live Medical is HIPAA-eligible for enterprise customers with a Business Associate Agreement (BAA) and who have enabled Zero Retention mode. Patient data is stored by default in the United States, and companies that wish to can use isolated environments in the European Union, India, or Singapore.
Several teams have already begun developing agents with the model. Idin Jafari, Head of Digital Strategy and Innovation at Pfizer, finds that ‘the interaction experience with Eleven v4 Turbo and Scribe v3 Live Medical is phenomenal,’ adding that ‘the agent sounds realistic and is more accurate on the brand names and generic names that come up in our conversations.’ Shantanu Ghorai, Founder and Head of Operations at Vintage Labs AI, states for his part that the company’s VIA platform, which handles pre- and post-procedure calls, ‘understands patients better’ and that ‘nurses can now rely on the call summary without having to re-listen to the recording.’
