strands-project/strands_qsr_lib

[qsr_prob_rep] - XML memory issue with larger HMMs

Opened this issue · 0 comments

When I tried to dump a relatively large HMM file, I used over 32GB of RAM, at which point my computer killed it. The actual HMM generation finished, but was not able to save the file. We think this is a result of the function which dumps it to a XML files found here.

We got around the issue by calling it directly, rather than using the service call, which allowed us to use numpy to dump the HMMs to 3 seperate files using np.savetxt and hmm.asMatrices

  • file**.**trans
  • file**.**emi
  • file**.**start

Using these files and gh.HMMFromMatrices we were able to recreate the HMM. However, this is a bit of a hack.

Can someone please add the label: "Reminder" and "Not Urgent" and assign to @cdondrup