Bigram model generalisation using singular value decomposition

Abstract

This thesis is about building good models for predicting sequences, in particular, the sequence of words contained in written English. It presents a technique for making a bigram model more general by using Singular Value Decomposition (SVD). The thesis describes a system that was implemented to evaluate the technique on bigram models that predict the sequences of words in English text. Experiments carried out with the system showed the technique to be effective at generalising bigram models that predict simple artificial sequences. However, experiments on real text suggest that the technique is not effective on bigram models that predict the sequence of words contained in real English text.