Exploiting word positional information in ngram model for Chinese text input method

Abstract

This paper aims to improve the performance of the Pinyin-to-Character Conversion system which is the core of Chinese text input method. The ngram model is the current solution to the Pinyin-to- Character Conversion system. This paper enhances the traditional ngram model by relaxing its stationary hypothesis and exploiting the word positional information. The Non-stationary ngram (NS ngram) model is proposed. Several related issues are discussed in detail, including the formal definition, the model implement, the training algorithm and the space complexity of the NS ngram model. Evaluated on the Pinyin-to- Character Conversion task, the NS ngram model outperforms the traditional ngram model significantly with great error rate reductions. Meanwhile, the training algorithm presented in this paper can estimate the parameters in the NS ngram model effectively and efficiently.