Word error rate (WER)

What it is

Word error rate compares a machine transcript with a human reference transcript of the same audio. Every word the machine got wrong (substitution), left out (deletion) or added (insertion) counts as one error; the total is divided by the number of words in the reference. A WER of 10 percent means roughly one word in ten is wrong. For languages without spaces between words, or where spelling varies, character error rate is used instead.

Two details change the number more than the engine does. Text normalisation (how "Dr" and "doctor", "5" and "five", punctuation and casing are treated) can move WER by several points. The test set matters even more: studio read speech, podcast audio and eight-kilohertz telephone calls in a regional accent produce very different figures from the same system.

Why it matters when buying

Vendors quote WER from benchmarks that favour them. A number without the test set, the normalisation and the date is marketing. For a voice agent, errors are also not equal: a wrong "the" is harmless and a wrong digit in a date of birth is a failed call. Entity-level accuracy (names, numbers, addresses) predicts agent success better than overall WER.

What to ask

Ask for WER on telephone audio in your languages and accents, with the test set described and dated. Ask for entity accuracy on numbers and names separately. Better, run a bake-off on your own golden recordings and compute it yourself; the methodology page explains the normalisation to use.