Skip to content
Own measurement · German audio · as of November 9, 2026

Raven understands German. Measured, not claimed.

5.8%

Word error rate on German audio

Out of a hundred spoken German words, Raven gets about six wrong, as of November 9, 2026. How the number comes about and where the other models stand is shown in the ranking German speech recognition (in German).

Speech recognition runs on our own infrastructure, with no US service in the data path. How well it understands German is not something we simply claimed: we measure it with an open tool, and every number can be recomputed. Everything on this page is measured on German audio; it says nothing about English.

The rankings

Two numbers of our own. Each with its ranking.

Word error rate5.8%Raven's value · 59 models compared

German speech recognition (in German)

How accurately a model transcribes German audio, per dataset, with cost and speed next to it.

Diarization error rate8.07%Raven's value · on real German meetings

Speaker diarization (in German)

Who spoke when. Raven records one audio track per participant, so the reference is measured and not estimated.

Stress test58 modelsmeasured on the corpus Swiss Parliament

Dialects (in German)

Swiss German and Bavarian: the same measure, only the audio is different.

Study2 of 24sources are an independent measurement

What the AI assistants answer (in German)

Six AI assistants, one question about German speech recognition, every source they named read.

Live streaming≥ 128 tracksbest model · on one machine

Speakers per GPU (in German)

How many speakers a model transcribes live at the same time before the text arrives too late, with one curve per model.

59 models, the same German audio, the same metric. Every tick is one model. To the ranking (in German)

23 vendors, 59 models, the same German audio and the same metric.

  • AssemblyAI
  • AWS
  • Microsoft Azure
  • Cohere
  • IBM
  • Microsoft
  • Moonshine
  • NVIDIA
  • OpenAI
  • OpenMOSS
  • primeline
  • Qwen
  • Mistral
  • DashScope
  • Deepgram
  • ElevenLabs
  • Google
  • Gladia
  • Groq
  • nyra health
  • Reson8
  • Speechmatics
  • xAI

Where initials stand in place of a logo, the vendor does not release its mark for comparison pages.

Why we measure ourselves

Vendors measure themselves. We measure again.

Vendor numbers turn out as good as you would expect. So we play every model the same German recordings, compute with the same open tool and publish every number per dataset. That way we know what runs in Raven, and you can check it.

Two of the measured models are part of Raven.

Raven sells meeting transcription, and two models in these rankings are part of it: the speech recognition of the standard plans and the built-in speaker diarization. In the tables their rows are called “Raven”. Whoever measures and sells at the same time has to say so next to the numbers. What limits that interest is the protocol in raven.eval, not our intention.

  • Scoring rules, bootstrap seeds and every model and dataset version are pinned.
  • If a request fails or a transcript is missing, the check fails instead of the corpus shrinking silently.
  • If a model was trained on parts of a test corpus, that is stated next to the rows it affects.
  • No model is declared the winner.
  • make verify recomputes every published number from the stored model outputs, for anyone, without a key and without a GPU.

The datasets

Every number per dataset.

An average over everything blurs what a model is good for. What a corpus contains is described on the ranking that measures on it.

Read GermanWord error rate · speech recognition ranking
GermanFLEURSGermanMLSGermanGerman MixedGermanVoxPopuliFour read-speech and parliament corpora (in German)
DialectsWord error rate · dialects ranking
Swiss GermanSPC · Swiss ParliamentSwiss GermanFHNW · dialect regionsBavarianBAR · BavarianGermanBAR-HD · control trackThree dialect corpora without a common scale (in German)
DiarizationError rate for “who speaks when” · speaker diarization ranking
GermanRaven meetingsEnglishVoxConverseGermanCALLHOME-deEnglishAMIFour corpora for “who speaks when” (in German)

How we measure

Four principles. All verifiable.

Every model hears the same recordings, and the result is compared word for word with the checked transcript. The average over a dataset is weighted by length.

  1. 01

    Our own measurements

    No vendor numbers. Every model tested by us, on identical audio.

  2. 02

    Real German audio

    Parliamentary speeches, audiobooks, read sentences and real meetings.

  3. 03

    One procedure for all

    Self-hosted and through an API, the same data, the same metric. Empty cells stay empty.

  4. 04

    Open method

    Tool, datasets and scorer are public. Every number leads back to its run.

WER
Share of words that were misunderstood, left out or invented. Lower is better.
DER
Share of speaking time assigned to the wrong speaker. Lower is better.
Speed
Minutes of audio per minute of runtime, as a multiple of real time. Self-hosted models: throughput over the whole dataset with several simultaneous requests, on the hardware named. Cloud services: per single request, network path included. Every value says which measurement it is.

And now your audio

Hear for yourself how Raven understands your meetings.

Raven records one audio track per participant and is hosted in the EU. The same measuring discipline, applied to your audio.

Try Raven for free

The app is in German for now.

Every number on these pages comes from a run of our own. The tool for it is public: whoever doubts a number can measure it again.

Context and recomputingraven.eval · measuring tool

STT benchmark · as of November 9, 2026 · every number independently verifiable