New virtual avatar "Star" to bring books to life through sign language

Huawei, the European Union for the Deaf, Penguin Random House and Aardman Animations team up to develop AI-powered app for deaf children


In collaboration, Huawei, the European Union for the Deaf, Penguin Random House and the Aardman Animations has developed an AI-powered app, StorySign, which has been designed to cater to the needs of hearing-impaired children.

The app leverages Huawei's AI technology tools, such as image recognition and optical character recognition capabilities, to translate books into sign language via a virtual character named "Star".

To use the app, the user is required to point the smartphone's camera at the book at hand and the app will then transform the text into sign language.

Star was created by Aardman Animations, creators of animated movies such as Chicken Run and Shaun the Sheep. Furthermore, in developing Star, the Chinese tech giant used motion capture technology to offer enhanced sign language annotation, "to bring books to life through sign language".

Visit Innovation Enterprise's Machine Learning Innovation Summit, part of the DATAx New York festival, on December 12–13, 2018

"Nothing like StorySign exists," said Huawei global senior product marketing manager Peter Gauden. "It's great see with the use of AI and innovative technology we can open the world of books to deaf children."

" 90% of deaf children are born to hearing parents, some of whom do not know sign language. All of this means many deaf children struggle to match printed words with the things they represent, which has a big impact on their lives," Huawei stated.

The new app is free to download via the Google Play Store and the Huawei AppGallery and is offered in 10 sign languages including English, French, German and Italian. However, while the app can be downloaded for any smartphone, it was reportedly optimized for the Huawei Mate 20 Pro.


Image courtesy: Huawei

Can siri be a teacher  nlp in educationsmall

Read next:

Can Siri be a teacher? NLP in education