Bug: The custom model is not used at the time of training
ID-Akash opened this issue · 2 comments
The custom model is not used at the time of training the RL Agent. Instead, the default built-in model is used. This can be verified by simply changing the output dimension of Sequential layer from 512 to any random number and still the training works.
Please suggest the changes required.
Just making the following changes would work:
config = ( PPOConfig() .environment(env=env_name, clip_actions=True) .rollouts(num_rollout_workers=4, rollout_fragment_length=128) .training( train_batch_size=512, lr=2e-5, gamma=0.99, lambda_=0.9, use_gae=True, clip_param=0.4, grad_clip=None, entropy_coeff=0.1, vf_loss_coeff=0.25, sgd_minibatch_size=64, num_sgd_iter=10, model={"custom_model": "CNNModelV2"}, ) .debugging(log_level="ERROR") .framework(framework="torch") .resources(num_gpus=int(os.environ.get("RLLIB_NUM_GPUS", "0"))) )
Would you like to make a PR for this?