AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |
Back to Blog
Automated lip reading software12/29/2022 The problem was that the audio and video streams were sometimes out of sync by almost a second, which would have made it impossible for the AI to learn associations between the words said and the way the speaker moved their lips.īut by assuming that most of the video was correctly synced to its audio, a computer system was taught the correct links between sounds and mouth shapes. To make the BBC data set suitable for automatic lip reading in the study, video clips had to be prepared using machine learning. Yannis Assael, who is working on LipNet, says he is looking forward to using it. The DeepMind and Oxford group says it will release its BBC data set as a training resource. In addition, the grammar in the BBC data set comes from a wide diversity of real human speech, whereas the grammar in GRID’s 33,000 sentences follows the same pattern and so is far easier to predict. But where GRID only contains a vocabulary of 51 unique words, the BBC data set contains nearly 17,500 unique words, making it a much bigger challenge. Two weeks ago, a similar deep learning system called LipNet – also developed at the University of Oxford – outperformed humans on a lip-reading data set known as GRID. “Without that huge data set, it’s very difficult for us to verify new technologies like deep learning.” “It’s a big step for developing fully automatic lip-reading systems,” says Ziheng Zhou at the University of Oulu in Finland. With these results, the system also outperforms all other automatic lip-reading systems. And many of its mistakes were small slips, like missing an ‘s’ at the end of a word. But the AI annotated 46.8 per cent of all words in the March to September data set without any error. The professional annotated just 12.4 per cent of words without any error. The AI vastly outperformed a professional lip-reader who attempted to decipher 200 randomly selected clips from the data set. And here’s the same clip with subtitles provided by the AI system: AI shows the way
0 Comments
Read More
Leave a Reply. |