Error when converting from .flac to .wav
Closed this issue · 5 comments
Hello,
I recently downloaded the Slakh dataset for use in the context of my thesis on musical source separation.
When trying to convert the data from .flac to .wav using the provided script (flac_converter.py in conversion/) that I call with
python conversion/flac_converter.py -i ../Data/slakh2100_flac/train/ -o ../Data/slakh2100_wav/train/ -c False
, at some point I keep getting the following error message repeatedly:
Input #0, flac, from '../../Data/slakh2100_flac/train/Track00061/stems/S00.flac':
Metadata:
ENCODER : Lavf57.83.100
Duration: 00:04:20.17, start: 0.000000, bitrate: 129 kb/s
Stream #0:0: Audio: flac, 44100 Hz, mono, s16
Stream mapping:
Stream #0:0 -> #0:0 (flac (native) -> pcm_s16le (native))
Error while opening decoder for input stream #0:0 : Resource temporarily unavailable
with different stems each time.
After this a few huge chunks of tracks are skipped, and the script goes on to convert a few more tracks before stopping completely. I also get the following somewhere near the end:
Input #0, flac, from '../../Data/slakh2100_flac/train/Track01142/stems/S06.flac':
Metadata:
ENCODER : Lavf57.83.100
Duration: 00:06:12.01, start: 0.000000, bitrate: 24 kb/s
Stream #0:0: Audio: flac, 44100 Hz, mono, s16
Input #0, flac, from '../../Data/slakh2100_flac/train/Track01142/mix.flac':
Metadata:
ENCODER : Lavf57.83.100
Duration: 00:06:12.01, start: 0.000000, bitrate: 329 kb/s
Stream #0:0: Audio: flac, 44100 Hz, mono, s16
Stream mapping:
Stream #0:0 -> #0:0 (flac (native) -> pcm_s16le (native))
Error while opening decoder for input stream #0:0 : Resource temporarily unavailable
Stream mapping: time=00:00:56.63 bitrate= 703.6kbits/s speed=1.05x
Stream #0:0 -> #0:0 (flac (native) -> pcm_s16le (native))
Error while opening decoder for input stream #0:0 : Resource temporarily unavailable
Traceback (most recent call last):
File "flac_converter.py", line 207, in <module>
args.end, args.num_threads, args.verbose)
File "flac_converter.py", line 111, in _apply_ffmpeg
pool.map(_apply_convert_dir, track_directories)
File "/home/mthevoz/miniconda3/envs/MSS/lib/python3.7/multiprocessing/pool.py", line 268, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/home/mthevoz/miniconda3/envs/MSS/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
File "/home/mthevoz/miniconda3/envs/MSS/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/home/mthevoz/miniconda3/envs/MSS/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "flac_converter.py", line 109, in _apply_convert_dir
ffmpeg_func, verbose=verbose)
File "flac_converter.py", line 90, in _convert_folder
ffmpeg_func(src_path, out_stems_dir, verbose=verbose)
File "flac_converter.py", line 45, in _flac_to_wav
ffmpeg.input(input_path).output(output_path).run_async(overwrite_output=not verbose)
File "/home/mthevoz/miniconda3/envs/MSS/lib/python3.7/site-packages/ffmpeg/_run.py", line 285, in run_async
args, stdin=stdin_stream, stdout=stdout_stream, stderr=stderr_stream
File "/home/mthevoz/miniconda3/envs/MSS/lib/python3.7/subprocess.py", line 775, in __init__
restore_signals, start_new_session)
File "/home/mthevoz/miniconda3/envs/MSS/lib/python3.7/subprocess.py", line 1453, in _execute_child
restore_signals, start_new_session, preexec_fn)
BlockingIOError: [Errno 11] Resource temporarily unavailable
In the end my terminal becomes unresponsive and I end up with a bunch of tracks that seem to be properly converted but with big chunks missing in the middle and in the end. Any idea what's going on here?
Hmmm... I can't say that I've seen this issue before, but my guess is that there might be a bug with the threading. Can you try explicitly setting the [--num-threads NUM_THREADS] option to 1 or 2 form the command line and see if that still causes the issue?
I tried doing that and unfortunately it doesn't solve the issue, although changing the number of threads also changes which files get converted and which files do not, 2 threads giving the most successes as far as I can see. The process is still a bit random and there are slight differences between the end results of each run I attempt even with the same number of threads.
Note that I have the 3.4.6 version of ffmpeg installed if that's relevant.
EDIT: I tried re-doing everything with ffmpeg 4.2.1 and still have the issue, although results are a bit chaotic and vary with each run, regardless of threads or version.
Hi, I wrote a small piece of code that does the conversion with sox instead of ffmpeg and that worked fine for me. It's a lot slower but it gets the job done. Hope it helps anyone that may run into the same issues:
import os
import argparse
import subprocess
from distutils.util import strtobool
def flac_to_wav(in_folder):
extensions = ('.flac')
for root, _, files in os.walk(in_folder):
for file in files:
if file.endswith(extensions):
filename_wav = file.replace('.flac', '.wav')
subprocess.run(['sox', root+'/'+file, '-r',
'44100', '-c', '2', '-b', '16', root+'/'+filename_wav])
os.remove(root+'/'+file)
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--input-dir', '-i', type=str, required=True,
help='Base path to input directory. (Required)')
parser.add_argument('--verbose', '-v', type=lambda x:bool(strtobool(x)), default=False, required=False,
help='Whether to print messages while processing. (Optional)')
args = parser.parse_args()
flac_to_wav(args.input_dir)
Note that this version does not take an output directory but rather does the conversion "in place" by converting to .wav then deleting the original .flac file. I ran this on a copy of the original dataset so as to obtain the same result.
Cheers.
Hey, I'm glad you were able to find a solution. I think I might also have a solution, but I haven't tested it yet. I'll test your and my solutions next week and merge them in. Thanks for checking it out.
Based off of this:
https://stackoverflow.com/questions/33341303/ffmpeg-resource-temporarily-unavailable
I was able to fix this issue by adding 'threads=1' to every call of ffmpeg.input() and ffmpeg.output(). The parameterized thread count still works, as it splits apart the work across all of the tracks, however, my change forces ffmpeg to use only one thread at a time when dealing with the conversion of one file. I am not sure if this is the right way to fix this, especially if @ethman experiences no problems with the code as it stands. These observations seem to suggest that ffmpeg does not use one thread by default, and this can cause issues for some systems.