Problem with registration importing the basic modified environment

[amitfishy]

Hello,

Thank you for this code-base. I'm facing an issue with running the main experiment. Do you have any suggestions for getting it to work? I've tested out the openai gym and mujoco frameworks independently and they both seem to work fine. Not sure why something wrong is getting passed to one of internal checks in registration.py.

Thank you!

$ python main.py --env-name HalfCheetahDir-v1 --num-workers 8 --fast-lr 0.1 --max-kl 0.01 --fast-batch-size 20 --meta-batch-size 40 --num-layers 2 --hidden-size 100 --num-batches 1000 --gamma 0.99 --tau 1.0 --cg-damping 1e-5 --ls-max-steps 15 --output-folder maml-halfcheetah-dir --device cuda

Output:

Traceback (most recent call last):
  File "main.py", line 141, in <module>
    main(args)
  File "main.py", line 34, in main
    num_workers=args.num_workers)
  File "/home/fishy2/anaconda3/envs/comp767_maml_project/code/pytorch-maml-rl/maml_rl/sampler.py", line 21, in __init__
    queue=self.queue)
  File "/home/fishy2/anaconda3/envs/comp767_maml_project/code/pytorch-maml-rl/maml_rl/envs/subproc_vec_env.py", line 67, in __init__
    for (remote, env_fn) in zip(self.work_remotes, env_factory)]
  File "/home/fishy2/anaconda3/envs/comp767_maml_project/code/pytorch-maml-rl/maml_rl/envs/subproc_vec_env.py", line 67, in <listcomp>
    for (remote, env_fn) in zip(self.work_remotes, env_factory)]
  File "/home/fishy2/anaconda3/envs/comp767_maml_project/code/pytorch-maml-rl/maml_rl/envs/subproc_vec_env.py", line 15, in __init__
    self.env = env_fn()
  File "/home/fishy2/anaconda3/envs/comp767_maml_project/code/pytorch-maml-rl/maml_rl/sampler.py", line 10, in _make_env
    return gym.make(env_name)
  File "/home/fishy2/anaconda3/envs/comp767_maml_project/libraries/gym/gym/envs/registration.py", line 183, in make
    return registry.make(id, **kwargs)
  File "/home/fishy2/anaconda3/envs/comp767_maml_project/libraries/gym/gym/envs/registration.py", line 132, in make
    if (env.spec.timestep_limit is not None) and not spec.tags.get('vnc'):
AttributeError: 'NoneType' object has no attribute 'timestep_limit'

[amitfishy]

To boil this done into a simpler issue, I try to import the 'HalfCheetahDir-v1' in the python shell using the following and get the same error:

Python 3.7.2 (default, Dec 29 2018, 06:19:36) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gym
>>> import maml_rl.envs
>>> gym.make('HalfCheetahDir-v1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/fishy2/anaconda3/envs/comp767_maml_project/libraries/gym/gym/envs/registration.py", line 183, in make
    return registry.make(id, **kwargs)
  File "/home/fishy2/anaconda3/envs/comp767_maml_project/libraries/gym/gym/envs/registration.py", line 132, in make
    if (env.spec.timestep_limit is not None) and not spec.tags.get('vnc'):
AttributeError: 'NoneType' object has no attribute 'timestep_limit'

[amitfishy]

I was able to fix this by changing:
env.spec
to
env.unwrapped.spec
in the instances of the function where it occurs.

That is, I replaced it in 3 places:

if (env.spec.timestep_limit is not None) and not spec.tags.get('vnc'):
            env = TimeLimit(env, max_episode_steps=env.spec.max_episode_steps, max_episode_seconds=env.spec.max_episode_seconds)

It feels a bit choppy to do it this way, but it seems to fix the problem (it also does not mess with the existing environments).