Question about the `noise-clip` parameter in DDPG.

cleanrl/cleanrl/ddpg_continuous_action.py

Lines 65 to 66 in 7e24ae2

    
           parser.add_argument("--noise-clip", type=float, default=0.5, 
        
               help="noise clip parameter of the Target Policy Smoothing Regularization")

It doesn't appear to be used in the code and there doesn't appear to be any mention of it in the documentation.

How should I use this parameter?

cleanrl/cleanrl/ddpg_continuous_action.py

Line 196 in 7e24ae2

actions += torch.normal(0, actor.action_scale * args.exploration_noise)

Can I just add torch.clip(or torch.clamp) to the above code as shown below?

actions += torch.clip(torch.normal(0, actor.action_scale * args.exploration_noise), -args.noise_clip, args.noise_clip)

This is a parameter introduced in TD3, an extension of DDPG (see http://proceedings.mlr.press/v80/fujimoto18a/fujimoto18a.pdf). Reading the CleanRL docs, its clear the DDPG implementation was heavily inspired by the TD3 authors implementation of TD3 and DDPG. My guess is this parameter got accidentally added to the parser because its required for TD3 and can safely be removed from this DDPG implementation.

@glass1720 Thank you

	parser.add_argument("--noise-clip", type=float, default=0.5,
	help="noise clip parameter of the Target Policy Smoothing Regularization")