[learning_to_simulate] Clarification about training: simulator() vs. get_predicted_and_target_normalized_accelerations()

[gaoalexander]

Hello,

Thank you for sharing this great project and code. I have a question regarding the call to both the get_predicted_and_target_normalized_accelerations() and simulator() functions within the one_step_estimator_fn(), which defines the main training loop logic.

Is the former function simulator() strictly included in the training loop for evaluation/metrics purposes? i.e. its position prediction and corresponding gradients do not influence the optimization results at all -- is this understanding correct?

On a related note, since I am not highly familiar with TF-v1 -- when simulator() is called, can you confirm whether or not this specific operation is included in the computation graph? I would assume it should not be, since its gradients shouldn't be required for optimization, but am unsure of whether this happens by virtue of having no connection with the optimizer.

Referencing code here.

Thank you very much for your help.