Other agents' observation information in the reward

[rsuwa]

I want to include other agents' observation information in the reward.
Is there a way to do that?

[praveen-palanisamy]

Hi @rsuwa ,

If you would like to include more information about the environment/other-agent's observation, you can subclass the base environment and create new environments to suit your needs.
Currently, to compute the reward for an agent, a set of information about the agent including it's interaction (collision info etc) with other agents are available. Although the reward computation only makes use of a subset of this information, the complete set of available information is shown below:

macad-gym/src/macad_gym/carla/multi_env.py

Lines 1235 to 1260 in 38884ac

	"episode_id": self._episode_id_dict[actor_id],
	"step": self._num_steps[actor_id],
	"x": self._actors[actor_id].get_location().x,
	"y": self._actors[actor_id].get_location().y,
	"pitch": self._actors[actor_id].get_transform().rotation.pitch,
	"yaw": self._actors[actor_id].get_transform().rotation.yaw,
	"roll": self._actors[actor_id].get_transform().rotation.roll,
	"forward_speed": self._actors[actor_id].get_velocity().x,
	"distance_to_goal": distance_to_goal,
	"distance_to_goal_euclidean": distance_to_goal_euclidean,
	"collision_vehicles": collision_vehicles,
	"collision_pedestrians": collision_pedestrians,
	"collision_other": collision_other,
	"intersection_offroad": intersection_offroad,
	"intersection_otherlane": intersection_otherlane,
	"weather": self._weather,
	"map": self._server_map,
	"start_coord": self._start_coord[actor_id],
	"end_coord": self._end_coord[actor_id],
	"current_scenario": self._scenario_map,
	"x_res": self._x_res,
	"y_res": self._y_res,
	"max_steps": self._scenario_map["max_steps"],
	"next_command": next_command,
	"previous_action": self._previous_actions.get(actor_id, None),
	"previous_reward": self._previous_rewards.get(actor_id, None),

If you want to extend the reward calculation using the above available information, you can follow this Wiki page to create a custom environment

[rsuwa]

@praveen-palanisamy

Thank you for your advice!

I have one more question. For example, if I set the send_measurements flag to True, I assume I can use other agents' information as observations. Is there a sample code for that somewhere?

https://github.com/praveen-palanisamy/macad-gym/blob/master/src/macad_gym/carla/multi_env.py#L914-L923