Raven understands German. Measured, not claimed.
5.8%
Word error rate on German audio
Speech recognition runs on our own infrastructure, with no US service in the data path. How well it understands German is not something we simply claimed: we measure it with an open tool, and every number can be recomputed. Everything on this page is measured on German audio; it says nothing about English.
The rankings
Two numbers of our own. Each with its ranking.
German speech recognition (in German)
How accurately a model transcribes German audio, per dataset, with cost and speed next to it.
Speaker diarization (in German)
Who spoke when. Raven records one audio track per participant, so the reference is measured and not estimated.
Dialects (in German)
Swiss German and Bavarian: the same measure, only the audio is different.
What the AI assistants answer (in German)
Six AI assistants, one question about German speech recognition, every source they named read.
Speakers per GPU (in German)
How many speakers a model transcribes live at the same time before the text arrives too late, with one curve per model.
23 vendors, 59 models, the same German audio and the same metric.
- AssemblyAI
- AWS
- Microsoft Azure
- Cohere
- IBM
- Microsoft
- Moonshine
- NVIDIA
- OpenAI
- OpenMOSS
- primeline
- Qwen
- Mistral
- DashScope
- Deepgram
- ElevenLabs
- Gladia
- Groq
- nyra health
- Reson8
- Speechmatics
- xAI
Where initials stand in place of a logo, the vendor does not release its mark for comparison pages.
Why we measure ourselves
Vendors measure themselves. We measure again.
Vendor numbers turn out as good as you would expect. So we play every model the same German recordings, compute with the same open tool and publish every number per dataset. That way we know what runs in Raven, and you can check it.
Two of the measured models are part of Raven.
Raven sells meeting transcription, and two models in these rankings are part of it: the speech recognition of the standard plans and the built-in speaker diarization. In the tables their rows are called “Raven”. Whoever measures and sells at the same time has to say so next to the numbers. What limits that interest is the protocol in raven.eval, not our intention.
- Scoring rules, bootstrap seeds and every model and dataset version are pinned.
- If a request fails or a transcript is missing, the check fails instead of the corpus shrinking silently.
- If a model was trained on parts of a test corpus, that is stated next to the rows it affects.
- No model is declared the winner.
make verifyrecomputes every published number from the stored model outputs, for anyone, without a key and without a GPU.
The datasets
Every number per dataset.
An average over everything blurs what a model is good for. What a corpus contains is described on the ranking that measures on it.
How we measure
Four principles. All verifiable.
Every model hears the same recordings, and the result is compared word for word with the checked transcript. The average over a dataset is weighted by length.
- 01
Our own measurements
No vendor numbers. Every model tested by us, on identical audio.
- 02
Real German audio
Parliamentary speeches, audiobooks, read sentences and real meetings.
- 03
One procedure for all
Self-hosted and through an API, the same data, the same metric. Empty cells stay empty.
- 04
Open method
Tool, datasets and scorer are public. Every number leads back to its run.
- WER
- Share of words that were misunderstood, left out or invented. Lower is better.
- DER
- Share of speaking time assigned to the wrong speaker. Lower is better.
- Speed
- Minutes of audio per minute of runtime, as a multiple of real time. Self-hosted models: throughput over the whole dataset with several simultaneous requests, on the hardware named. Cloud services: per single request, network path included. Every value says which measurement it is.
And now your audio
Hear for yourself how Raven understands your meetings.
Raven records one audio track per participant and is hosted in the EU. The same measuring discipline, applied to your audio.
The app is in German for now.
Every number on these pages comes from a run of our own. The tool for it is public: whoever doubts a number can measure it again.
STT benchmark · as of November 9, 2026 · every number independently verifiable














