Is the state part of the environment?

[zsunberg]

Hi guys, excited to see a reinforcement learning interface for Julia! I have a question about the interface. Is the state part of the environment?

If the state is part of the environment, then what is the reason for having it as an additional argument to step!? If the state is not part of the environment, then why would the environment be mutated in a call to step!?

The only RL package that I am really familiar with is OpenAI Gym, where the state is part of the environment. Perhaps it would make sense to follow their example since people are familiar with it, it has been successful, and it would allow simple interaction between environments and solvers written in julia and python.

[tbreloff]

Hi @zsunberg and thanks for the interest. You should probably review #2 and join the gitter chat to get more background on decisions that have been made. Really the design is to allow for maximum flexibility, and easier use than the gym. For example, you can do iteration like:

env = ...
policy = ...
for sars in Episode(env,policy)
    # optionally do something?
end

In practice this is much simpler and elegant than the gym api (IMO of course). You can look at the code of OpenAIGym.jl to get a feel of how they relate.

[zsunberg]

(continuing on gitter)