How Accurate Is AI Transcription?

Every transcription product would like to answer this with a number, and every honest answer to it is "it depends on your audio". This article explains what it depends on, and how to find out for your own recordings in about ten minutes.

Why the advertised percentages are not useful

Accuracy is usually reported as word error rate: the proportion of words inserted, deleted or substituted compared with a correct transcript. A figure like 95% accuracy means a 5% word error rate.

The problem is that the number is meaningless without the recording it was measured on. A word error rate measured on clean, read speech from a studio microphone tells you nothing about a conference room with four people and an air conditioner. Marketing figures are, reasonably enough, measured on the favourable case.

What actually determines accuracy

Roughly in order of impact:

1. Overlapping speech. The single biggest factor. One person at a time is a different problem from two.
2. Microphone distance. Close beats far, by a wide margin, and it beats expensive equipment used badly.
3. Room acoustics. Echo smears words into each other.
4. Background noise. Traffic, cafés, fans and air conditioning.
5. Accent and speech patterns. Accuracy varies by accent, and drops with very fast speech, mumbling and trailing off.
6. Vocabulary. Everyday language is easy; names, jargon and technical terms are not.
7. Audio format. Almost irrelevant. A good 128 kbps MP3 transcribes about as well as the WAV it came from.

How to find out for your own recordings

Ten minutes, and far more informative than any published benchmark:

1. Pick your hardest typical recording, not your cleanest one.
2. Take a representative three-minute section — ideally one with crosstalk or specialist vocabulary in it.
3. Transcribe it.
4. Read along with the audio and count the errors that would actually matter to you.
5. Divide by the number of words to get a rate you can trust, because it is measured on your material.

Judge by consequential errors, not raw word error rate. A missing "um" costs nothing. A misheard drug name, price or surname costs a great deal. A transcript with a mediocre word error rate and correct proper nouns is more useful than the reverse.

The failure mode that matters

Automatic transcription does not degrade gracefully into obvious nonsense. When the audio is poor, it produces fluent, grammatical, confident sentences that are wrong. Nothing in the text signals that it was a guess.

The practical consequence is that a transcript cannot be proofread on its own. It has to be checked against the audio, because reading it will not reveal the errors.

How to get better accuracy

Before recording:

• Get the microphone closer to the speakers than feels necessary.
• Choose a room with soft furnishings over a bare meeting room.
• Ask people not to talk over each other. Say it out loud at the start.
• Record each speaker on a separate track if you can. This eliminates the crosstalk problem entirely.

At transcription time:

• Set the language explicitly rather than relying on detection.
• Supply names and specialist terms if the tool accepts vocabulary hints.
• Upload the original recording, not a re-compressed export.

When automatic is not enough

Court and deposition transcripts, medical records, research being submitted for publication, and anything where a misheard word has legal or clinical consequences. In those cases a human transcriptionist, or at minimum human verification of a machine draft, is the right answer and the cost is proportionate to the risk.