TypeError: conv1d(): argument 'padding' (position 5) must be tuple of ints, not str

[mingfeisun]

Hi,

When I tried to run branch DM/torch_gpu with command python3 torch_runner.py --train --file configs/ppo_smac_cnn.yaml, I got the following complaint:

Traceback (most recent call last):
  File "torch_runner.py", line 141, in <module>
    runner.run(args)
  File "torch_runner.py", line 111, in run
    self.run_train()
  File "torch_runner.py", line 95, in run_train
    agent = self.algo_factory.create(self.algo_name, base_name='run', observation_space=obs_space, action_space=action_space, config=self.config)  
  File "/pymarl/common/object_factory.py", line 12, in create
    return builder(**kwargs)
  File "torch_runner.py", line 25, in <lambda>
    self.algo_factory.register_builder('a2c_discrete', lambda **kwargs : a2c_discrete.DiscreteA2CAgent(**kwargs)) 
  File "/pymarl/algos_torch/a2c_discrete.py", line 18, in __init__
    self.model = self.network.build(config)
  File "/pymarl/algos_torch/models.py", line 25, in build
(pid=67) Game has started.
    return ModelA2C.Network(self.network_builder.build('a2c', **config))
  File "/pymarl/algos_torch/network_builder.py", line 297, in build
    net = A2CBuilder.Network(self.params, **kwargs)
(pid=67) Sending ResponseJoinGame
  File "/pymarl/algos_torch/network_builder.py", line 157, in __init__
    'input_size' : self._calc_input_size(input_shape, self.actor_cnn), 
  File "/pymarl/algos_torch/network_builder.py", line 58, in _calc_input_size
    return nn.Sequential(*cnn_layers)(torch.rand(1, *(input_shape))).flatten(1).data.size(1)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/container.py", line 100, in forward
    input = module(input)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py", line 208, in forward
    self.padding, self.dilation, self.groups)
TypeError: conv1d(): argument 'padding' (position 5) must be tuple of ints, not str

Would you like to help me solve it? Or give me any guidline of how to run the ppo to get reported performance?

Really thanks

[Denys88]

hi @mingfeisun
I have already pushed it to the master.
You can use master branch.
I have a lot of old tensorflow configs. The only difference is that in TF you have padding 'same' and 'valid'. For our usecase we simply can replace 'same' with 1 and 'valid' with 0.
could you try to run this config:
python3 torch_runner.py --train --file configs/smac/3m_cnn_torch.yaml
It should be 3-4 minutes to 80% wr
I'd suggest to increase learning rate from 1e-4 to 5e-4

[mingfeisun]

Cool. Really thanks for the explanation. I will try the master branch. @Denys88

We also implemented one version of ppo, but we could not get a policy with good performance. We want to check your implementation to find out where we could be wrong.

Thanks again.

[Denys88]

@mingfeisun I just pushed config with running obs normalization to the master and it works even faster.
near 100 seconds to 98% WR on 3m. It would be nice if you can confirm the same result.
link to image

[schroederdewitt]

I cannot confirm the 98% WR on 3m. Attached: 3m_cnn_torch.yaml (blue), 3m_torch.yaml (red). Afaik 3m_torch.yaml has input normalisation switched on, but 3m_cnn_torch.yaml has not.

@Denys88 can you please confirm i am running the correct configs?

[schroederdewitt]

ok training just improved fast - @Denys88 can you confirm that number of env steps is as expected (time please ignore as running on my laptop, and both experiments in parallel)

[schroederdewitt]

drastic difference

[Denys88]

@schroederdewitt looks good.
convolution version for 3m is overkill so it is expected to converge significantly slower.
I see 0.9+ wr after 200k steps.
Simple mlp should perform better on simple and small tasks but doesn't converge at all on harder.
But conv1d works everywhere but worse on simple envs/

[Denys88]

@schroederdewitt I just checked if change these two params in 3m_cnn_torch.yaml
entropy_coef: 0.005
normalize_input: True
you will be able to achieve near 100% wr with conv1d for 3m too but slower.

[Denys88]