The Best Accuracy Measurement for Captions Yet: The NER Model
Learn how the NER Model works and why it’s considered the global standard for measuring caption accuracy.
Blog
The Best Accuracy Measurement for Captions Yet: The NER Model
The accuracy of live captions has been an area of rich debate (and intensive research) for some time.
Popular platforms like YouTube have received criticism from the deaf community about their inaccurate computer-generated captions, sparking movements like deaf activist Rikki Poynter’s #nomoreCRAPtions.
But there are distinctions to be made between the accuracy of computer-generated captions (not so good) and human-generated captions (usually very good).
And perhaps even more importantly when it comes to quality, we need to understand the systems used to measure the accuracy of captions.
Until recently, the most common model used to measure the accuracy of captions today has been the Word Error Rate, or WER, model. However, this model leaves a lot to be desired in terms of how well it actually measures the quality of captions, especially for people who rely on them, like deaf and hard-of-hearing audiences.
For example, the difference between the words ‘can’ and ‘can’t’ would be considered [...]