Reward

[Yiquan-lol]

Hi,
Where is the reward function defined?
Is it in the https://github.com/praveen-palanisamy/macad-gym/blob/master/src/macad_gym/carla/reward.py file？

[praveen-palanisamy]

Hi @Yiquan-lol ,

The Reward class in reward.py has methods for the different types (currently 4) of reward functions that can be configured to be used in a MACAD-Gym environment.

For example, if you are using the following environment (as in the README example):

import gym
import macad_gym
env = gym.make("HomoNcomIndePOIntrxMASS3CTWN3-v0")
# Your agent code here

the HomoNcomIndePOIntrxMASS3CTWN3-v0 environment's config uses the following reward function:

macad-gym/src/macad_gym/envs/homo/ncom/inde/po/intrx/ma/stop_sign_3c_town03.py

Line 56 in 38884ac

"reward_function": "corl2017",

which is defined here:

macad-gym/src/macad_gym/carla/reward.py

Lines 43 to 67 in 38884ac

	def compute_reward_corl2017(self):
	self.reward = 0.0
	cur_dist = self.curr["distance_to_goal"]
	prev_dist = self.prev["distance_to_goal"]
	# Distance travelled toward the goal in m
	self.reward += np.clip(prev_dist - cur_dist, -10.0, 10.0)
	# Change in speed (km/h)
	self.reward += 0.05 * (
	self.curr["forward_speed"] - self.prev["forward_speed"])
	# New collision damage
	self.reward -= .00002 * (
	self.curr["collision_vehicles"] +
	self.curr["collision_pedestrians"] + self.curr["collision_other"] -
	self.prev["collision_vehicles"] -
	self.prev["collision_pedestrians"] - self.prev["collision_other"])
	# New sidewalk intersection
	self.reward -= 2 * (self.curr["intersection_offroad"] -
	self.prev["intersection_offroad"])
	# New opposite lane intersection
	self.reward -= 2 * (self.curr["intersection_otherlane"] -
	self.prev["intersection_otherlane"])
	return self.reward

[praveen-palanisamy]

Converting this to a discussion.