Issue with Training and testing

Question

Issue with Training and testing

avisin02 opened this issue 6 years ago · 12 comments

Hello Yong,
I am very much impressed with work. I am trying to use the code on my data. My setup is as follows:

Right now I am using Python 3.6 on windows 10.
I have implemented runme.sh in a python code.
For training I am using 1350 sentences and 8 types of noises of duration same as combined duration of sentences.
For testing I am using 150 sentences with some noise same as training noise and some others.
Now, the issue is that the training is less but not close to 0 and testing loss is way more than expected and enhanced wav files have no sign of speech in any noise condition.
What should be done in this case? I am also attaching the plot of training stat.

Regards,
Ajay

Answer 1 · 2018-05-08T15:42:29.000Z

Hi Ajay, Have you tried on Linux? If you found that your test loss was quite large (over-fitting), I suggest you can try some simple experiments first, e.g., clean-to-clean, or with only one type noise in the training and the same noise type in the testing. This can help you to test your code. BTW, have you test one utterance of your training set? Is the reconstructed wav audible or not? Best regards, yong

…

________________________________ -------------------------------------------------------- Dr. Yong XU https://sites.google.com/view/xuyong/home From: avisin02<mailto:notifications@github.com> Date: 2018-05-08 19:52 To: yongxuUSTC/sednn<mailto:sednn@noreply.github.com> CC: Subscribed<mailto:subscribed@noreply.github.com> Subject: [yongxuUSTC/sednn] Issue with Training and testing (#10) Hello Yong, I am very much impressed with work. I am trying to use the code on my data. My setup is as follows: 1. Right now I am using Python 3.6 on windows 10. 2. I have implemented runme.sh in a python code. 3. For training I am using 1350 sentences and 8 types of noises of duration same as combined duration of sentences. 4. For testing I am using 150 sentences with some noise same as training noise and some others. Now, the issue is that the training is less but not close to 0 and testing loss is way more than expected and enhanced wav files have no sign of speech in any noise condition. What should be done in this case? I am also attaching the plot of training stat. [plot]<https://user-images.githubusercontent.com/39087945/39755651-53ddafbc-52e4-11e8-98a8-fe59700eb922.png> Regards, Ajay — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#10>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFJj0twccZkM7fECdKmkHE8s-ff-G1jCks5twYbtgaJpZM4T2gSj>.

Answer 2 · 2018-05-09T12:12:05.000Z

Hi Yong, I haven't checked on Linux and there some minor issues between python version 2.x and 3.x. 1. For large data the samples are not over fitting but test as well as the training loss is large. Earlier the wav files were not audible. 2. With 1 sentence and 1 noise type for both training and testing training loss = 0.080497 wav file is audible but distorted. 3. Now, I have checked with 1 sentence with 2 noise types for training the training loss is 0.034137 The wav file are audible but distorted. Please suggest ways to improve this and I will also try it in Linux but right now I am working windows. Regards, Ajay

…

On Tue, May 8, 2018 at 9:12 PM, yong xu @ surrey ***@***.***> wrote: Hi Ajay, Have you tried on Linux? If you found that your test loss was quite large (over-fitting), I suggest you can try some simple experiments first, e.g., clean-to-clean, or with only one type noise in the training and the same noise type in the testing. This can help you to test your code. BTW, have you test one utterance of your training set? Is the reconstructed wav audible or not? Best regards, yong ________________________________ -------------------------------------------------------- Dr. Yong XU https://sites.google.com/view/xuyong/home From: ***@***.***> Date: 2018-05-08 19:52 To: ***@***.***> CC: ***@***.***> Subject: [yongxuUSTC/sednn] Issue with Training and testing (#10) Hello Yong, I am very much impressed with work. I am trying to use the code on my data. My setup is as follows: 1. Right now I am using Python 3.6 on windows 10. 2. I have implemented runme.sh in a python code. 3. For training I am using 1350 sentences and 8 types of noises of duration same as combined duration of sentences. 4. For testing I am using 150 sentences with some noise same as training noise and some others. Now, the issue is that the training is less but not close to 0 and testing loss is way more than expected and enhanced wav files have no sign of speech in any noise condition. What should be done in this case? I am also attaching the plot of training stat. [plot]<https://user-images.githubusercontent.com/ 39087945/39755651-53ddafbc-52e4-11e8-98a8-fe59700eb922.png> Regards, Ajay — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://github.com/ yongxuUSTC/sednn#10>, or mute the thread<https://github.com/ notifications/unsubscribe-auth/AFJj0twccZkM7fECdKmkHE8s- ff-G1jCks5twYbtgaJpZM4T2gSj>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AlRvSbc3kOOK7n5ypG5vfFZOyTpC9NYeks5twbzngaJpZM4T2gSj> .

Answer 3 · 2018-05-09T12:16:20.000Z

Hi Ajay, I suggest "1000 sentences without any noise" which means "clean to clean mapping". You can firstly test this one. To see whether you can recover your wav for training set or testing set. Best regards, yong

________________________________ -------------------------------------------------------- Dr. Yong XU https://sites.google.com/view/xuyong/home From: avisin02<mailto:notifications@github.com> Date: 2018-05-09 20:12 To: yongxuUSTC/sednn<mailto:sednn@noreply.github.com> CC: yong xu @ surrey<mailto:xuyong62@mail.ustc.edu.cn>; Comment<mailto:comment@noreply.github.com> Subject: Re: [yongxuUSTC/sednn] Issue with Training and testing (#10) Hi Yong, I haven't checked on Linux and there some minor issues between python version 2.x and 3.x. 1. For large data the samples are not over fitting but test as well as the training loss is large. Earlier the wav files were not audible. 2. With 1 sentence and 1 noise type for both training and testing training loss = 0.080497 wav file is audible but distorted. 3. Now, I have checked with 1 sentence with 2 noise types for training the training loss is 0.034137 The wav file are audible but distorted. Please suggest ways to improve this and I will also try it in Linux but right now I am working windows. Regards, Ajay

On Tue, May 8, 2018 at 9:12 PM, yong xu @ surrey ***@***.***> wrote: Hi Ajay, Have you tried on Linux? If you found that your test loss was quite large (over-fitting), I suggest you can try some simple experiments first, e.g., clean-to-clean, or with only one type noise in the training and the same noise type in the testing. This can help you to test your code. BTW, have you test one utterance of your training set? Is the reconstructed wav audible or not? Best regards, yong ________________________________ -------------------------------------------------------- Dr. Yong XU https://sites.google.com/view/xuyong/home From: ***@***.***> Date: 2018-05-08 19:52 To: ***@***.***> CC: ***@***.***> Subject: [yongxuUSTC/sednn] Issue with Training and testing (#10) Hello Yong, I am very much impressed with work. I am trying to use the code on my data. My setup is as follows: 1. Right now I am using Python 3.6 on windows 10. 2. I have implemented runme.sh in a python code. 3. For training I am using 1350 sentences and 8 types of noises of duration same as combined duration of sentences. 4. For testing I am using 150 sentences with some noise same as training noise and some others. Now, the issue is that the training is less but not close to 0 and testing loss is way more than expected and enhanced wav files have no sign of speech in any noise condition. What should be done in this case? I am also attaching the plot of training stat. [plot]<https://user-images.githubusercontent.com/ 39087945/39755651-53ddafbc-52e4-11e8-98a8-fe59700eb922.png> Regards, Ajay — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://github.com/ yongxuUSTC/sednn#10>, or mute the thread<https://github.com/ notifications/unsubscribe-auth/AFJj0twccZkM7fECdKmkHE8s- ff-G1jCks5twYbtgaJpZM4T2gSj>. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AlRvSbc3kOOK7n5ypG5vfFZOyTpC9NYeks5twbzngaJpZM4T2gSj> .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub<#10 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFJj0kqVPebN3vTDOqEUmIwJieCaN4zIks5twt0WgaJpZM4T2gSj>.

Answer 4 · 2018-12-27T07:21:26.000Z

do you solve your problem , I miss the same problem

Answer 5 · 2018-12-28T00:01:19.000Z

Hi Ajay, Did you check the sampling rate of the speech and noise? They should all be resampled to 16 kHz. Did you try with TIMIT dataset with the noise dataset mentioned in the code? If it works then it might be the problem of the dataset you use. If it still not working it might be some problem of the configuration of the code on the windows machine. Best wishes, Qiuqiang

…

________________________________ From: avisin02 <notifications@github.com> Sent: 08 May 2018 12:52:13 To: yongxuUSTC/sednn Cc: Subscribed Subject: [yongxuUSTC/sednn] Issue with Training and testing (#10) Hello Yong, I am very much impressed with work. I am trying to use the code on my data. My setup is as follows: 1. Right now I am using Python 3.6 on windows 10. 2. I have implemented runme.sh in a python code. 3. For training I am using 1350 sentences and 8 types of noises of duration same as combined duration of sentences. 4. For testing I am using 150 sentences with some noise same as training noise and some others. Now, the issue is that the training is less but not close to 0 and testing loss is way more than expected and enhanced wav files have no sign of speech in any noise condition. What should be done in this case? I am also attaching the plot of training stat. [plot]<https://user-images.githubusercontent.com/39087945/39755651-53ddafbc-52e4-11e8-98a8-fe59700eb922.png> Regards, Ajay — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#10>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMt5yQLwM81E3_xaTj1UmgXvnYD5K4MCks5twYbtgaJpZM4T2gSj>.

Answer 6 · 2018-12-28T01:57:12.000Z

thank you for your reply
I use the orign wav file in mini data。I use this in window7.

Answer 7 · 2018-12-28T02:03:47.000Z

when I run the code in mixture2clean .it occurs a probelm。I changed the code from n_pad = (n_concat - 1) / 2 to int（n_pad = (n_concat - 1) / 2） .I don't think this change will affect the code.

Answer 8 · 2018-12-29T00:11:00.000Z

Hi, If mini data is used, the loss curve may behave what you plot. Because there are too little data for training so will not generalize to the testing data. Best wishes, Qiuqiang

…

________________________________ From: bailiangze <notifications@github.com> Sent: 28 December 2018 02:03:47 To: yongxuUSTC/sednn Cc: Kong, Qiuqiang (PG/R - Elec Electronic Eng); Comment Subject: Re: [yongxuUSTC/sednn] Issue with Training and testing (#10) when I run the code in mixture2clean .it occurs a probelm。I changed the code from n_pad = (n_concat - 1) / 2 to int（n_pad = (n_concat - 1) / 2） .I don't think this change will affect the code. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#10 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMt5yVErgXTwTJ_5Cg5i4qz8J1w7sFx0ks5u9XwDgaJpZM4T2gSj>.

Answer 9 · 2018-12-29T03:13:17.000Z

thank you for your replying ,I have tried this code on linux using this mini data,but this problem still exists.
I have some problems as follw:
1.when I run this code ,it occur this problem Traceback (most recent call last):
File "prepare_data.py", line 464, in
pack_features(args)
File "prepare_data.py", line 303, in pack_features
mixed_x = pad_with_border(mixed_x, n_pad)
File "prepare_data.py", line 361, in pad_with_border
x_pad_list = [x[0:1]] * n_pad + [x] + [x[-1:]] * n_pad
TypeError: can't multiply sequence by non-int of type 'float' .I changed the code from n_pad = (n_concat - 1) / 2 to int（n_pad = (n_concat - 1) / 2） .I want to konw this change will affect the code??
2.though I use the mini data ,but I get the enhace wav which is not audible .it seem unnomal. if I increase the trianning data . will it become audible?

Answer 10 · 2018-12-29T03:21:43.000Z

Sorry, the above expression is not very clear. I get enhanced voice which can be played but confused. Does this problem have anything to do with the size of the training level?

Answer 11 · 2019-01-02T08:56:41.000Z

Hi, If mini data is used for training, you may get enhanced speech but they are not good. But at least you should be able to recognize the speech. Using more data will lead to a better result. Best wishes, Qiuqiang

…

________________________________ From: bailiangze <notifications@github.com> Sent: 29 December 2018 03:21:43 To: yongxuUSTC/sednn Cc: Kong, Qiuqiang (PG/R - Elec Electronic Eng); Comment Subject: Re: [yongxuUSTC/sednn] Issue with Training and testing (#10) Sorry, the above expression is not very clear. I get enhanced voice which can be played but confused. Does this problem have anything to do with the size of the training level? — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#10 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMt5yWCGiERkefux3Oggl6Blpso5z2OKks5u9t_HgaJpZM4T2gSj>.

Answer 12 · 2019-01-05T02:39:01.000Z

Thank you 。Is the RBM code include in this code？