Observations affected by variable frame cycles

[stangerm2]

The observation images returned by step vary depending on the game action performed.

Currently run_action_on_emulator() is hard coded to do 24 tick()'s as a 'turn' (where a turn would be defined as one action via step and one completed/settled image via for observation). As an reference here are some tick cycles it takes for various game states to settle, ie. no longer be animating (note these are also approximations and they also can vary depending on CPU load):

• Walk into wall (D-Pad L/R/U/D): 4 Frames
• Walk to new X,Y pos (D-Pad L/R/U/D): ~22 Frames
• Move to new map: ~63 Frames
• Chat box interaction: 40-67+ Frames
• Entering Battle: ~120 Frames

You can verify this simply by printing the screen after run_action_on_emulator(). Switch to interactive mode and try various scenes. You will notice that the animation or text for cycles above longer than 24 will still be mid animation.

I had been working on a patch to address this by identifying these different states by RAM flags and adjusting the frame cycle time accordingly. However, in the middle of this endeavor I learned that SubprocVecEnv via step is a blocking process; without the ability to buffer or normalize thread timings. This causes rendering problems and slows everything down as faster cycles, like running into a wall (4 frames) are held up by slower cycles (entering battle). In headless mode the program will often crash when threads time delta are too large.

The main reason for addressing this issue was to have clean consistent observations and for cycle/frame efficiency gains. However, due to the synchronous nature of the PPO model the above approach won't work.

I'm not an AI expert but I don't think there is a simple or reasonable fix to this currently, however I wanted to document this defect so others will be aware of the observation noise condition and maybe someone more knowledgeable in AI or stable baselines might be able to address it one day.

Until then I hope we can just leave this around as a known issue.

Written as of 67f407af1f97d484f1c27c13427eff5279224202