A python package for HMM (Hidden Markov Model) model with fast train and decoding implementation
test by using Python3
pip install FastHMMpip install git+https://github.com/312shan/FastHMM.gitfrom FastHMM.hmm import HMMModel
# test model training and predict
hmm_model = HMMModel()
hmm_model.train_one_line([("我", "r"), ("爱", "v"), ("北京", "ns"), ("天安门", "ns")])
hmm_model.train_one_line([("你", "r"), ("去", "v"), ("深圳", "ns")])
result = hmm_model.predict(["俺", "爱", "广州"])
print(result)
# test save and load model
hmm_model.save_model()
hmm_model = HMMModel().load_model()
result = hmm_model.predict(["我们", "爱", "深圳"])
print(result)Output:
[('俺', 'r'), ('爱', 'v'), ('广州', 'ns')]
[('我们', 'r'), ('爱', 'v'), ('深圳', 'ns')]test on dataset 人民日报
python .test/test_postagging.py
Output:
train size 18484 ,test_size 1000
finish training
eval result:
predict 57929 tags, 54228 correct, accuracy 0.9361114467710473
runtime : 370.1029086 seconds
Most of time the consuming is on the decoding stage, I tried many ways to implement viterbi algorithm, The implementation I currently use is the fastest If you have suggestions for improving this decoding algorithm, please let me know, thank you very much.