Memory and Specific effect issue
Opened this issue ยท 8 comments
Thanks for the code, I'm trying to run this in Windows to include in my audio sequencer.
After installing with pip and download the pytorch checkpoints, I can run it:
..\python scripts\remfx_detect.py +exp=remfx_detect +audio_input=r:\reverb.wav
- Can I have a specific effect instead of "remfx_detect" ? If i put "reverb" there I got
Error executing job with overrides: ['+exp=reverb', '+audio_input=r:\\reverb.wav']
Traceback (most recent call last):
File "F:\python\RemFx\scripts\remfx_detect.py", line 16, in main
for effect in cfg.ckpts:
^^^^^^^^^
omegaconf.errors.ConfigAttributeError: Key 'ckpts' is not in struct
full_key: ckpts
object_type=dict
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
- Eventually it gets out of memory with my 32 GB and it's too slow if it doesn't. A way to make it a bit faster would help.
Best and thanks for all that.
Hey, thanks for trying our project!
- In general, we were aiming to tackle the problem where the effects present were unknown beforehand. However, if you want to run the code for a specific effect on an audio file, probably the simplest way is to hack the
remfx_detect.py
script. I haven't tested the following code, but you'll need to do something like below:
- In the
RemFXChainInference()
call, setclassifier = None
- Modify
batch = [audio, audio, None, [[0, 0, 0, 0, 0]]]
, set one of those zeros to 1 based on which effect you want. The effects are in the order of[[Reverb, Chorus, Delay, Distortion, Compressor]]
- If you are using CPU, it will run very slowly, but GPU will speed it up significantly. If you follow the steps in 1., it should speed up the inference a decent amount (and use less memory overall) as it is not performing the classification, and only running 1 effect-specific model
Thanks for your response, do I use the GPU by installing the GPU-enabled torch?
I tried installing torch from site with GPU support and yet when I install remfx it still wants to install torch again and it uses CPU... any hints?
python scripts\remfx_detect.py +exp=remfx_detect +audio_input=r:\reverb.wav accelerator=gpu
Btw the hint for the specific effect with Batch [1...] worked, thanks :)
I saw that you are checking for cuda in your code so the issue is at my end, will fix and notify you. Thanks for everything.
I found a probable bug
scripts\pip install -e ./remfx ./remfx/umx
scripts\pip uninstall -y torch torchvision torchaudio functorch
scripts\pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
scripts\pip install --no-deps hearbaseline
cd remfx
scripts\download_ckpts.sh
but you use functorch which is not compatible with the cuda version of torch.
I have to install it again with
pip install git+https://github.com/pytorch/functorch.git
Then the cuda version works fine. I will build all that in my sequencer and let you know.
Congrats for the job.
I still have the memory issue. I got 32 gb or ram, 8 GB of GPU rtx 4060 and yet I get memory errors even if only the reverb is being removed.
Thanks a lot.
@WindowsNT
Did you manage to make this work without requiring more than 32GB of memory?
I'm looking for a solution to mitigate echo and distortion, something like this https://crowdunmix.org/the-best-way-to-recover-audio-distortion-using-ai/
No, not yet, still trying.