In this paper, word output probabilities are given random weights, when the number of states is increased (Technique 2). Otherwise, we use parts of speeches or categories using word dictionaries. For example, words belonging to the same category are given the same weight. This is a good way of trying to obtain better language models.