ElevenLabs introduces Cue, a forced alignment model that places each character of an audio file on a precise timeline. According to the company, it is more accurate than all aligners tested on real recordings, across the 29 languages in the benchmark. It already powers the Scribe family of models, ElevenLabs’ transcription model.
Any viewer knows the effect of a subtitle arriving a second too late: the scene loses coherence. Videos and software using voice models encounter the same problem when a transcription highlights the wrong word. Forced alignment involves matching each word of a transcription to the exact moment it is spoken. It should not be confused with AI alignment, which aims to match system behavior to human values and preferences. Here, it is solely about aligning text and sound.
Danila Kozlov, a researcher at ElevenLabs working on alignment, explains the difficulty: “Humans are quite good at aligning media, at least at the word level. We often have an intuitive idea of where a word begins, but at the character level and with sub-millisecond precision, it’s much harder.”
Limits for Classic Aligners
Before Cue, the company used an aligner that powered all its products. The team surveyed customers and internal users about their frustrations, and the complaints matched the limits of the open-source tools tested. The first problem concerns duration: many aligners work well on clips of about 30 seconds, but longer files must be split. Each piece can add a small synchronization error, and joining the segments can add others.
On full recordings, Cue’s average error is 179 milliseconds, compared to 283 ms to over 6,000 ms for open-source tools. Real-world conditions complicate the task further. Background noise, overlapping voices, and ambient music cause many tools to lose their bearings, yet these are precisely the files many users want to transcribe. On English recordings with such flaws, Cue’s average error reaches 57 ms, compared to 65 ms to 920 ms for others.
Language coverage is also an issue. Most open-source aligners require a separate model for each language, so languages with little training data are poorly handled or not supported. MMS-FA is the only other tool covering the 29 languages in the benchmark, and Cue generally proved faster at determining word boundaries.
Speed also matters, as aligning large volumes of audio can quickly become costly. According to ElevenLabs, Cue aligns one hour of audio in about half a second on its production GPU, a graphics processor. The second fastest tool, Qwen3-ForcedAligner, takes nearly 2 seconds, and the slowest exceeded one minute.
Aligned Songs
Music is a difficult exercise, as singing stretches and blurs words. However, Cue processed a subset of songs from the JamendoLyrics MultiLang dataset, under a Creative Commons license, in English, French, Spanish, and German, in a single pass. It placed 92.7% of words within 0.3 seconds of the reference, compared to 82.0% for the next tool. Its average error on word starts is 0.82 seconds, about half that of the closest competitor. It outperformed all other tools on at least 19 of the 22 songs.
Cue was deployed quickly, and a client reported that Georgian and a few other low-resource languages performed worse than expected. ElevenLabs says it identified the issue, conducted research on possible improvements, trained a new version, and released it within 24 hours of the initial report.
Cue is already used for all of ElevenLabs’ Speech to Text transcriptions. To use it directly, the company points to its Forced Alignment API.
