Towards Realizing Sign Language-to-Speech Conversion by Combining Deep Learning and Statistical Parametric Speech Synthesis

• 作者: Hongwu Yang Xiaochun An Zhenye Gan
• 来源期刊: 国际计算机前沿大会会议论文集
• 学科: 社会科学
• 关键词: Deep learning Support vector machine Static SIGN language recognition Context-dependent LABEL Hidden Markov model Mandarin-Tibetan BILINGUAL SPEECH synthesis
• 年，卷（期）: 2016,（1）
• 页码范围: 176-178
• 页数: 3页
• 分类号: C5

摘要

This paper realizes a sign language-to-speech conversion system to solve the communication problem between healthy people and speech disorders. 30 kinds of different static sign languages are firstly recognized by combining the support vector machine (SVM) with a restricted Boltzmann machine (RBM) based regulation and a feedback fine-tuning of the deep model. The text of sign language is then obtained from the recognition results. A context-dependent label is generated from the recognized text of sign language by a text analyzer. Meanwhile,a hiddenMarkov model (HMM) basedMandarin-Tibetan bilingual speech synthesis system is developed by using speaker adaptive training.The Mandarin speech or Tibetan speech is then naturally synthesized by using context-dependent label generated from the recognized sign language. Tests show that the static sign language recognition rate of the designed system achieves 93.6%. Subjective evaluation demonstrates that synthesized speech can get 4.0 of the mean opinion score (MOS).