AttributeError: Can't pickle local object 'make_env.<locals>._make_env'
Opened this issue · 4 comments
Ubuntu 18.04.5 LTS
Meet this problem after adding 'mp.set_start_method('spawn')'
#40 (comment)
Traceback (most recent call last):
File "train.py", line 122, in
main(args)
File "train.py", line 54, in main
num_workers=args.num_workers)
File "/home/dchen/pytorch-maml-rl/maml_rl/samplers/multi_task_sampler.py", line 107, in init
worker.start()
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/process.py", line 105, in start
self._popen = self._Popen(self)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 32, in init
super().init(process_obj)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init
self._launch(process_obj)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/home/dchen/anaconda3/envs/torch/lib/python3.6/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'make_env.._make_env'
I see the similar problem in #41
I am able to replicate this issue using mp.set_start_method('spawn'). Can you try the fix/mp-spawn branch, see if that fixes your issue?
I'm having trouble understanding why this doesn't work though. I know that objects that are passed as arguments to a process are pickled, and local functions generally don't work well with pickle, but here that function is not passed as an argument, but is instead called inside the process (and therefore shouldn't need to be pickled?).
Thanks a lot. After adding 'mp.set_start_method('spawn')', I fix this problem using this branch https://github.com/tristandeleu/pytorch-maml-rl/tree/fix/mp-spawn
Thanks a lot. After adding 'mp.set_start_method('spawn')', I fix this problem using this branch https://github.com/tristandeleu/pytorch-maml-rl/tree/fix/mp-spawn
Hello, I have same problem with you，and I just wonder where 'mp.set_start_method('spawn')' should be placed in that position in the branch https://github.com/tristandeleu/pytorch-maml-rl/tree/fix/mp-spawn ?
Thank you very much