Open discussion: Object disappear after Agent passes by

[sherryxia2016]

Hi,

I'm working on a project that the Object is some kind of "dessert" that can be eaten by the Agent. Therefore, the Object should disappear after the agent trajectory passes by. Is it possible to realize this function in the current code? I know it requires to change the environment because the object is associated with it.

Thanks a lot!!

[TomGeorge1234]

Nice idea! This answer is sort-of-yes and sort-of-no.

sort-of-yes because you're definitely free to move objects around by resetting their positions or removing them entirely by accessing the Environment.objects dictionary. sort-of-no because your issue will be that this is not officially supported, particular by plotting functions in RiaB which only access the last state of the Environment. This would mean that the objects won't move or disappear in the animations (even though underlying data, e.g. if you're using ObjectVectorCells to encode objects, may correctly reflect the removed objects).

Supporting "Dynamic environments" of this kind is one of the primary goals for RiaB 2.0 as discussed in #84 (feel free to chip into the discussion) so we are moving in this direction.

Also check out this TaskEnv wrapper which a user build for RiaB. It contains a bunch of classes for creating spatial "goals" which disappear once they have been reached. In many ways this is exactly what you're looking for and, since it contains its own plotting/rendering pipelines, animations work too BUT it is a little abstracted away from the core RiaB API so for now I would most only recommend going down this pathway if you're really interested in reinforcement learning which this was built for.

[TomGeorge1234]

@mehulrastogi, do you have any comments here? I know you've been working with Environments in which object move around or disappear.

[mehulrastogi]

Yeah so implementing this has been kind of tricky and @TomGeorge1234 is right on how one can explicitly access these object. Right now the way to achieve this would be to access the object in the environment and manually change them according to your personal use case. This becomes a bit tricky(and messy) if one also needs OVCs to work in conjunction with these objects. These kinds of dynamic environments will be one of the main focus moving forward and introducing the Vector Cell class was an important step towards that (See AgentVector Cells, where we detect a moving agent).

Now if your use case is just a dessert that the agent eats when it passes by you can use SpatialGoals from the Task Environment and choose to send a reward (eating dessert) when it's near this goal. And I think it will be fairly easy to make these goals move/disappear as you wish.