deepgram/kur

Not getting a prediction on speech example.

tshields44 opened this issue · 3 comments

Is this normal after all this time? I know it's the small dataset but it hasnt given one prediction at all since the beginning of just a blank space.

Epoch 88/inf, loss=6.077: 97%|█████████▋| 2352/2432 [25:49<00:45, 1.74samples/s]
[INFO 2017-04-01 23:52:55,254 kur.model.executor:369] Training loss: 6.077
Validating, loss=372.212: 94%|█████████▍| 256/271 [00:45<00:02, 6.13samples/s]
[INFO 2017-04-01 23:53:40,300 kur.model.executor:206] Validation loss: 372.212
Prediction: ""
Truth: "that's it on your account"
Total wall-clock time: 38h 20m 31s
Training wall-clock time: 37h 08m 23s
Validation wall-clock time: 01h 12m 08s
Batch wall-clock time: 37h 07m 42s

Thanks for all your help in advance.

ajsyp commented

No, it isn't. If your model only spits out spaces (or an empty string), you can bet something went wrong. I discovered a problem with Keras 2 that impacted the speech model, and pushed fixes yesterday (2017-03-31). If you try this update and still get spaces, let me know!

This worked thank you.

I'm still getting this issue with version 0.7.0. I'm using the default examples/speech.yml Kurfile and I noticed that the loss isn't changing much across epochs as well.

Any tips on debugging?

EDIT: Here is a screenshot from the output:
screenshot

Are you able to help @ajsyp?