/GaussianIOHMM

Gaussian Input Output HMM

Primary LanguagePython

Mixture Gaussian Input Output Hidden Markov Model

TODO

Gaussian IOHMM

  • Implement placeholder for mixture gaussian case.
  • Unit Test for Mixture Gaussian Sequence Labeling.
  • grid search tunner:
    • unknown error: not running but the thread is not ready.
    • clean code
    • qbs generator and submitter
  • Implement NER part
  • Implement new train strategy: first training mu. Then update variance
    • it seems work. Need to investigate the best strategy.
  • Investigate the benefit of our graph model. (Like known several truth label)
  • Investigate how to express interpretability.
  • Implement IOHMM
  • Investigate the use of inverse wishart prior
    • inverse wishart prior can significantly reduce training error (keep positive define).
  • accelerate:
    • No update var: pre calculate inverse and store.
      • Not useful for multi gaussian, thus useless.
    • Update Var: implement pseudo inverse like Moore–Penrose inverse

IOHMM

  • log format
  • higher dim test, and other trick
  • add reg term. (avoid the plain priority of model)

Issue

The current issues that are under processing dictionary

  1. sequence labeling forward issue:

    Before fixed, we used forward=True in gaussian_multiply_integral function during backward.

  2. Loss increase during training.

    After fixed issue 1, the loss changed sharply and may increase during training.

    Fixed: Due to in and out var init is too small. Re-implement random init part.

  3. Current loss also shown increase during training. Current temporary fix is not update variance.

    Fixed: wishart prior