State node doesn't contain the state?
Closed this issue · 7 comments
Does anyone else find it inconvenient that the state node type has no field for the state?
Hmmm... yeah, that does seem annoying - it would probably be fine to just add it even though it's not strictly needed for the algorithm.
Actually, performance would probably be significantly improved by upgrading to a vector-based tree structure like in DPW:
Line 157 in 4c5e9d9
If you want to make a PR to add the state or upgrade the whole tree structure, I'm happy to review it :)
I'm not a pro-grade programmer, but my first take is that it would seem more logical to have StateNode contain the state and StateActionNode contain only the action - it now also contains the new state. That would be more memory efficient, but perhaps slower?
Performance in Julia is a little bit different than in other languages. Actually what is there right now is pretty slow because in Julia (afaik, at least in v0.6), every mutable object has to be allocated on the heap. So allocating all these state nodes is what is actually bad. I have re-implemented all of our other versions of MCTS (BasicPOMCP, MCTS-DPW, POMCPOW, etc.) with a vector-based tree structure (see link above) and they are much faster. That is what should be done.
Sounds good.
I started working on replacing the tree in the vector_tree branch
Cool, looking forward to it