Feature Extraction by Using Attention Mechanism in Text Classification

• 作者: Yaling Wang Yue Wang
• 来源期刊: 国际计算机前沿大会会议论文集
• 学科: 工学
• 关键词: LSTM Attention
  Feature extraction INTERPRETABILITY Text classification
• 年，卷（期）: 2020,（2）
• 页码范围: 77-89
• 页数: 13页
• 分类号: TP3

摘要

In recent years,machine learning technology has made great success in fields of computer vision,natural language processing,speech recognition and so on.And machine learning model has been widely used in face recognition,automatic driving,malware detection,intelligent medical analysis and other practical tasks.In this paper,attention mechanism is proposed to combine with LSTM model to extract features in text classification.The results show that,on the one hand,LSTM+Attention can improve classification performance;on the other hand,by sorting by word weights generated by the attention layer,we find some meaningful word features,however,its recognition performance is not good.Some possible reasons were analyzed and it was found that attention mechanism sometimes misjudges wrong word features,resulting from these wrong words often appearing at the same time with meaningful word features.