Live Grader not Reporting Time
Ryan-Amaral opened this issue · 1 comments
When submitting to the competition through the live grader (not docker), the mean simulation time comes out as 0. An example can be seen here https://www.aicrowd.com/challenges/neurips-2019-learn-to-move-walk-around/submissions/10540, and on other live grader submissions that I saw.
Edit: Though I now realize that timing will be unreliable since it is ran on the user's hardware, so I guess this is not so relevant.
@Ryan-Amaral Thanks for your note and update. Yes, it was a decision made by AIcrowd not to include simulation time for some reasons including the one you pointed out. Please let us know if there is a specific concern related to not showing simulation times; otherwise, we will probably go with the current set up.