Size of Q(s, a) for table-based RL algorithm
colm-connaughton opened this issue · 1 comments
colm-connaughton commented
You say after Eq.(5) that the size of the Q(s,a) table grows exponentially for high dimensional problems. Is that true? I thought that table grows as number of actions x number of states (still big) whereas it is the number of paths or strategies (over time) that grows exponentially?
ACabrejas commented
Now I am still sleepy and not certain that my brain is working at full capcacity so I may be wrong, but if we assume a constant discretisation of every dimension in 3 levels (low, mid, high).
Starting with 0 dimensions, as we add "x" dimensions, our state grows as 3^x.
So Q(s,a) would grow at a rate of A*S = A(3^x ) as we add extra dimensions?