The CARLA simulator experiences a crash when the autonomous vehicle remains stationary for an extended period
AvinsWang opened this issue · 1 comments
Your work has set a benchmark in closed-loop evaluation for end-to-end autonomous driving methods, offering researchers a unified and fair evaluation framework that facilitates easier model assessment and performance comparison.
Despite the impressive contributions, there seem to be some issues. While using the code to closed-loop evaluate my own model, I encountered a situation where erroneous planning instructions from the model caused the ego vehicle to remain stationary for an extended period. After a certain duration, the Carla simulator crashed and threw a Timeout exception. I believe this behavior is reasonable, otherwise, the simulator would continue running. Curiously, this exception wasn't caught within the thread, leading to a crash in the closed-loop evaluation and an empty final result. I would like to understand the following:
- Under what conditions does the Carla simulator crash?
- How does the Carla simulator detect that the ego vehicle has remained stationary for an extended period?
- How can the above issue be resolved?
@AvinsWang You should read the code carefully in https://github.com/Thinklab-SJTU/Bench2Drive/blob/main/leaderboard/leaderboard/leaderboard_evaluator.py to understand the logic of closed-loop evaluation.