Several examples doesn't work (Sashimi checkpoints / sampleRNN training)

Hello,

So I've cloned the repo and downloaded the sashimi checkpoints on hugging face in order to reproduce them and apply the method on my dataset. I've found three main problems:

any audio generated with sashimi is basically noise i.e. :

this command :
python -m generate experiment=audio/sashimi-youtubemix checkpoint_path=checkpoints/sashimi_youtubemix_v4.pt n_samples=10 l_sample=16000 load_data=false

leads to :
sashimi generation.zip

=> It does work with SampleRNN though.

=> With Wavenet it doesn't work either python -m generate experiment=audio/wavenet-sc09 checkpoint_path=checkpoints/wavenet_sc09.pt n_samples=1 l_sample=16000 load_data=false gives the error :

module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for SequenceLightningModule:
	Missing key(s) in state_dict: "encoder.0.weight", "decoder.0.output_transform.weight", "decoder.0.output_transform.bias". 
	Unexpected key(s) in state_dict: "encoder.weight", "decoder.output_transform.weight", "decoder.output_transform.bias".

Training with SampleRNN doesn't work on youtubemix and beethoven:

so first i've noticed there was a reorganisation probably so I had to modify some .yaml for the new architecture

+++ b/configs/experiment/audio/samplernn-youtubemix.yaml
@@ -6,7 +6,7 @@ defaults:
   - /task: multiclass_classification
   - /optimizer: adamw
   - /scheduler: plateau
-  - /model: samplernn
+  - /model: baseline/samplernn

but then it still don't work, with
python -m train experiment=audio/samplernn
i get

    assert self.val_sanity_progress_bar_id is not None
AssertionError

and if I deactivate RichProgressBar, then I can see that it's because val/loss is not logged and I absolutely can't understand why. I've tried hunting differences between youtubemix and sc09 onwhich the training starts, but to no success.

What happens ? I've seen the "kernel what changed" to V4: is this why examples on sashmi are not working ? Should I go back to v3 ? I use pykeops but not the extension/kernels, would that be a reason ?

Regards,

Edit : also python setup.py install is deprectaed, I'm not sure how i should install the new kernels

Hello,

So I've cloned the repo and downloaded the sashimi checkpoints on hugging face in order to reproduce them and apply the method on my dataset. I've found three main problems:

any audio generated with sashimi is basically noise i.e. :

this command : python -m generate experiment=audio/sashimi-youtubemix checkpoint_path=checkpoints/sashimi_youtubemix_v4.pt n_samples=10 l_sample=16000 load_data=false

leads to : sashimi generation.zip

I have the same problem that ever audio generate with sashimi_sc09_v4.pt and sashimi_youtubemix_v4.pt is only noise.
The pretrained networks were downloaded at https://huggingface.co/krandiash/sashimi-release

I also tried to train my own network -- audio/sashimi-sc09, but it is also only noise

Sorry for the inconvenience. If you've trained your own model from scratch and it's still noise, there is probably a newly introduced mistake in the stepping code.

Actually, I do remember this issue but I thought it was fixed. Check this line here:

s4/src/models/sequence/modules/s4block.py

Line 170 in a246043

y = self.activation(y)

There was missing a corresponding activation applied in the step function:

s4/src/models/sequence/modules/s4block.py

Line 199 in a246043

if self.gate is not None:

Hopefully fixing that will fix the generation issues.

Thank you for the fast reply. It is working now

Fixed by 35cc242