Normalization of Homophonic Words in Chinese Microblogs

• 作者: Guohong Fu Jiaying Song Xin Zhang Yu He
• 来源期刊: 国际计算机前沿大会会议论文集
• 学科: 社会科学
• 关键词: Microblog analysis TEXT NORMALIZATION homophonic WORDS N-GRAM
• 年，卷（期）: 2015,（1）
• 页码范围: 51-53
• 页数: 3页
• 分类号: C5

摘要

Homophonic words are very popular in Chinese microblog, posing a new challenge for Chinese microblog text analysis. However, to date, there has been very little research conducted on Chinese homophonic words normalization. In this paper, we take Chinese homophonic word normalization as a process of language decoding and propose an n-gram based approach. To this end, we first employ homophonic–original word or character mapping tables to generate normalization candidates for a given sentence with homophonic words, and thus exploit n-gram language models to decode the best normalization from the candidate set. Our experimental results show that using the homophonic-original character mapping table and n-grams trained from the microblog corpus help improve performance in homophonic word recognition and restoration.