Error encountered when running ***_test.py with "--use_planning"

[WGBmushi]

Hi, thank you for your impressive work.
When we run closed_loop_test.py with "--use_planning", we get the following error.

python closed_loop_test.py --name closed_loop --test_file ./DIPP/replay_scenes/299/raw/training_20s.tfrecord-00299-of-01000 --model_path ./DIPP/training_log/DIPP/model_20_0.7207.pth --use_planning --render --save --device cpu

---

Traceback (most recent call last):
File "closed_loop_test.py", line 146, in
closed_loop_test()
File "closed_loop_test.py", line 82, in closed_loop_test
final_values, info = planner.layer.forward(planner_inputs, optimizer_kwargs={'track_best_solution': True})
File "/home/Academic/Toolboxs/theseus/theseus/theseus_layer.py", line 91, in forward
vars, info = _forward(
File "/home/Academic/Toolboxs/theseus/theseus/theseus_layer.py", line 151, in _forward
info = optimizer.optimize(**optimizer_kwargs)
File "/home/Academic/Toolboxs/theseus/theseus/optimizer/optimizer.py", line 43, in optimize
return self._optimize_impl(**kwargs)
File "/home/Academic/Toolboxs/theseus/theseus/optimizer/nonlinear/nonlinear_optimizer.py", line 388, in _optimize_impl
self._optimize_loop(
File "/home/Academic/Toolboxs/theseus/theseus/optimizer/nonlinear/nonlinear_optimizer.py", line 287, in _optimize_loop
self.linear_solver.linearization.linearize()
File "/home/Academic/Toolboxs/theseus/theseus/optimizer/linearization.py", line 54, in linearize
self._linearize_hessian_impl()
File "/home/Academic/Toolboxs/theseus/theseus/optimizer/sparse_linearization.py", line 137, in _linearize_hessian_impl
self._linearize_jacobian_impl()
File "/home/Academic/Toolboxs/theseus/theseus/optimizer/sparse_linearization.py", line 105, in _linearize_jacobian_impl
jacobians, error = cost_function.weighted_jacobians_error()
File "/home/Academic/Toolboxs/theseus/theseus/core/cost_function.py", line 68, in weighted_jacobians_error
jacobian, err = self.jacobians()
File "/home/Academic/Toolboxs/theseus/theseus/core/cost_function.py", line 278, in jacobians
err, optim_vars, aux_vars = self._compute_error()
File "/home/Academic/Toolboxs/theseus/theseus/core/cost_function.py", line 199, in _compute_error
err = self._err_fn(optim_vars=optim_vars, aux_vars=aux_vars)
File "/home/Academic/Planning/DIPP/model/planner.py", line 135, in lane_theta
lane_error = theta[:, 1::2] - ref_points[:, 1::2, 2]
RuntimeError: maybe_layer.has_value() INTERNAL ASSERT FAILED at "aten/src/ATen/VmapGeneratedPlumbing.h":10757, please report a bug to PyTorch.

---

It works if we run it without "--use_planning", but it doesn't work very well.
We have tried to change the linear_solver_cls from th.CholeskyDenseSolver to th.CholmodSparseSolver in the MotionPlanner class in planner.py, but it is invalid.
Has that ever happened to you? Or do you have any suggestions? We are looking forward to your reply.
Best wishes.

[MCZhi]

Hi, @WGBmushi, thank you for your interest in our work. This issue is because Theseus has recently got updated and I have fixed the problem in my code to adapt to the change. Please try again.

[WGBmushi]

Thank you very much for your timely reply. In terms of your guidance, we finally solved the issue.

Our previous Theseus library was installed from the source, and the error remained even though we updated planner.py and retrained. But once we reinstalled Theseus with the pip command, the closed-loop test code finally worked today.

The accuracy of the closed-loop test is not satisfactory at the moment. It may be because the dataset we used for training before is small, and training with a larger dataset may be an alternative.

We hope our experience will inspire others and thank the authors again for their work.