when training and converting, I econtered the problem of "RuntimeWarning: divide by zero "
Closed this issue · 3 comments
The divide by zero problem occurs at the training begins and converting, I trained a sf1-sm1 model for two days using GTX1080, the converted result was not so good as you demo .
Most importantly, the divide by zero error should be handled.
(t18) hu@hu-D520MT-K:~/disk1/Voice_Converter_CycleGAN$ python convert.py --model_dir ./model/sf1_sm1 --model_name sf1_sm1.ckpt --data_dir ./data/evaluation/SF1 --conversion_direction A2B --output_dir ./converted_voices
WARNING:tensorflow:From /home/hu/anaconda3/envs/t18/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py:497: calling conv1d (from tensorflow.python.ops.nn_ops) with data_format=NHWC is deprecated and will be removed in a future version.
Instructions for updating:
NHWC for data_format is deprecated, use NWC instead
WARNING:tensorflow:From /home/hu/anaconda3/envs/t18/lib/python3.6/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.
2018-07-23 09:02:21.888616: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-07-23 09:02:22.022387: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-07-23 09:02:22.022923: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1344] Found device 0 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.835
pciBusID: 0000:01:00.0
totalMemory: 7.92GiB freeMemory: 7.37GiB
2018-07-23 09:02:22.022942: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1423] Adding visible gpu devices: 0
2018-07-23 09:02:22.828472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:911] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-07-23 09:02:22.828522: I tensorflow/core/common_runtime/gpu/gpu_device.cc:917] 0
2018-07-23 09:02:22.828537: I tensorflow/core/common_runtime/gpu/gpu_device.cc:930] 0: N
2018-07-23 09:02:22.829284: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7119 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0, compute capability: 6.1)
/home/hu/disk1/Voice_Converter_CycleGAN/preprocess.py:170: RuntimeWarning: divide by zero encountered in log
f0_converted = np.exp((np.log(f0) - mean_log_src) / std_log_src * std_log_target + mean_log_target)
Thanks for regrading my proposal.
Hi hujinsen,
I did not get a chance to formally train sf1-sm1, so I am not able to say if the performance is good or not.
Regarding your second question, please check #1 .
Thanks for your interest.
Lei
Later, I trained a sf1-tm2 model , same as your demo, the conversion result are as good as yours.I don't konw why sf1-sm1 model's conversion results are poor. May be the dataset problem?
I don't think so. Because the audio data content is the same for different people in the dataset, if the conversion is not good as what you have found, it must be due to that the model is not trained well as expected. You may upload some demos on your GitHub and try to troubleshoot by optimizing the hyperparameters used during the training. Please check the demos on the NTT group. If their conversion is doing well, there is no doubt that this model could be further improved.