Code

Improve Kiwi Builder#188

Code

Merged

bab2min merged 7 commits intomainbab2min/Kiwi:mainfrom dev/kiwi_builderbab2min/Kiwi:dev/kiwi_builderCopy head branch name to clipboardOct 3, 2024

+249-60

Commits

Commits on Oct 3, 2024

• make KiwiBuilder::addCorpusTo generic

  bab2min

  committed
  43e421a

• Change vocab type of HSDataset to uint32_t & Add knlmVocabSize property

  bab2min

  committed
  d9d6625

• Add separateDefaultMorpheme arg to KiwiBuilder::makeHSDataset

  bab2min

  committed
  e3473e8

• Prepare senseId property for WSD

  bab2min

  committed
  ddadee0

• Fix a bug where UnigramSwTrainer rejects some suffices with surrogated chrs

  bab2min

  committed
  170f7c6

• Update evaluator & model_builder

  bab2min

  committed
  4ae899d

• Fixed test cases

  bab2min

  committed
  981497a