Using Highway Connections to Enable Deep Small-footprint LSTM-RNNs for Speech Recognition?

• 作者: CHENG Gaofeng LI Xin YAN Yonghong
• 来源期刊: 中国电子杂志（英文版）
• 年，卷（期）: 2019,（1）
• 页码范围: 107-112
• 页数: 6页
• 语种: 英文
• DOI: 10.1049/cje.2018.11.008

摘要

Long short-term memory RNNs (LSTM-RNNs) have shown great success in the Automatic speech recognition (ASR) field and have become the state-of-the-art acoustic model for time-sequence modeling tasks. However, it is still difficult to train deep LSTM-RNNs while keeping the parameter number small. We use the highway connections between memory cells in adjacent layers to train a small-footprint highway LSTM-RNNs (HLSTM-RNNs), which are deeper and thinner compared to conventional LSTM-RNNs. The experiments on the Switchboard (SWBD) indicate that we can train thinner and deeper HLSTM-RNNs with a smaller parameter number than the conventional 3-layer LSTM-RNNs and a lower Word error rate (WER) than the conventional one. Compared with the counterparts of small-footprint LSTM-RNNs, the small-footprint HLSTM-RNNs show greater reduction in WER.