TUM-AAS/ml-casadi

Performance Discrepancy Between Naive and Approximated Methods in ml-casadi

Closed this issue · 3 comments

merdan-9 commented

Hi,

I've been working with the MLP network examples provided in the repository, specifically the "Naive" and "Approximated" methods. Upon evaluation, I noticed a significant discrepancy in their performance rates—the naive method executed at 2116Hz, while the approximated method performed at a slower 162Hz.

Could you shed some light on the processes each method uses to derive the MLP network weights within the c_generated_code? Additionally, any insights into the substantial difference in their execution speeds would be greatly appreciated.

Thank you for your time and any information you can provide.

Warm regards,
Merdan

Tim-Salzmann commented

Hi Merdan,

Thanks for reaching out!

Differences are expected here. Generally, the naive approach will be faster for smaller models as no context switch between C and Python/PyTorch occurs. For larger models and especially multiple parallel calls as in MPC which can be batched together by PyTorch the approximated approach will be faster.

Naive will rebuild the PyTorch Graph in CasADi, while Approximated will only build the Taylor function in CasADi and updates the parameters.

I would encourage you to read our paper [1] and especially pay attention to Figure 3 where "Ours" is similar to "Approximated".

Let me know if this helps!

Best
Tim

merdan-9 commented

Hi Tim,

Thank you for your insightful response!

Your explanation clarified the distinctions between the naive and approximated approaches, and the emphasis on context switching was particularly helpful. I've gained a preliminary understanding of the differences and have referred to Figure 3 in your paper for a clearer comparison.

However, while reviewing the code, I noticed the absence of details on how the network weights are utilized within the c_generated_code. Could you provide a brief overview of how mlcasadi integrates the model into CasADi? Furthermore, I'm curious about the use of the model in c_generated_code.

Additionally, I observed the following function in the C code generated by the approximated method:

for (int ii = 0; ii <= N; ii++)
{
    wr_acados_update_params(acados_ocp_capsule, ii, p, NP);
}

I'm having difficulty understanding how this function interacts with the model in Python. An explanation of the principles behind this interaction would be greatly appreciated.

Thank you once again for your assistance, and I look forward to your guidance on these queries.

Best regards,
Merdan

Tim-Salzmann commented

Hi Merdan,

All functionality of MLCasADi has been moved to L4CasADi:

Thus MLCasADi is deprecated at this point. I would therefore ask you to re-open the issue in the L4CasADi repository so that other people looking into this can see it there too. I will be happy to try and answer your questions there.

Best
Tim