Word error rate (WER) calculator, with Arabic normalisation

How the score is computed

Both texts are split into words after normalisation. The tool then finds the cheapest way to turn the transcript into the reference, where a substitution, a deletion (a reference word missing from the transcript) and an insertion (an extra word) each cost one. The word error rate is that total divided by the number of reference words. The character error rate does the same over characters, counting the spaces between words, which makes it kinder to languages and spellings where one wrong letter would otherwise make a whole word wrong.

Reference:   the cat sat on the mat
Transcript:  the cat sit on mat

the  cat  sat->sit  on  the (deleted)  mat
1 substitution + 1 deletion = 2 edits over 6 words = 33.3%

When several alignments cost the same, the tool prefers a match or substitution, then a deletion, then an insertion. The total number of edits never depends on that choice; only the split between substitutions, deletions and insertions can.

The normalisation options

• English lowercases, turns hyphens into spaces and keeps only letters a to z, digits and the apostrophe. Anything else, including accented letters, becomes a space.
• Arabic drops diacritics and the tatweel, unifies the alef, yeh and teh marbuta forms and removes punctuation. Digits and letters of any script are kept.
• Other languages lowercases and removes punctuation and symbols in any script. This option is a convenience and is not part of our benchmark.
• None splits on spaces only, so case and punctuation count as errors.

The English and Arabic options are ports of the normalise() function in the script that produced the numbers on our measurements page, and our tests run the two against each other on about a thousand strings and require identical results. The Arabic function is explained, with its Python code, in normalizing Arabic text for WER.

What WER does not tell you

It treats every word equally, so a missed “not” costs the same as a missed “the”. It says nothing about punctuation, speaker labels or timestamps after normalisation. It is only as good as the reference: a reference typed by hand from a noisy recording has errors of its own. And a score on one clean clip does not predict the score on your phone call. For the plain-language version, read what word error rate is.

Subtitle files

If either box holds an SRT or VTT file, its cue numbers and timing lines are ignored and only the words are scored. To see the plain text yourself first, use SRT to text or remove timestamps.