non-deterministic runs with larger scenarios, seemingly related to dispatch
Closed this issue · 1 comments
robfitzgerald commented
running denver_demo.yaml twice in a row leads to the same result, which is great. however, doing the same with manhattan.yaml leads to different outcomes. an example comparing two runs:
0 1 diff
mean_final_soc 0.377002 0.384293 1.93%
requests_served_percent 0.715324 0.701735 -1.90%
total_vkt 91603.631193 89623.883075 -2.16%
total_kwh_expended 13042.389051 12820.078359 -1.70%
total_kwh_dispensed 8812.4137 8663.013321 -1.70%
station_revenue_dollars 705.011011 688.722572 -2.31%
fleet_revenue_dollars 98022.274954 96068.039515 -1.99%
final_vehicle_count 200 200 0.00%
looking into the vehicle state logs of both runs, they seem to diverge after 6 minutes of simulation time at the first dispatch trip assignment. below is the result of a file diff of two state logs (command: diff manhattan_20k_2023-06-14_09-46-38/state.log manhattan_20k_2023-06-14_10-08-36/state.log | head -n 20):
(base) rfitzger-36698s:hive_output rfitzger$ diff manhattan_20k_2023-06-14_09-46-38/state.log manhattan_20k_2023-06-14_10-08-36/state.log | head -n 20
2421c2421
< {"vehicle_id": "v31", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.7224945384467943, "energy_ELECTRIC": 39.83826229669757, "link_link_id": "5796726888-42437967", "link_geoid": "8f2a1072dd9a98c", "sim_time": "2014-12-21T00:06:00", "report_type": "VEHICLE_STATE"}
---
> {"vehicle_id": "v31", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.698421687711827, "energy_ELECTRIC": 39.841429975831964, "link_link_id": "42433537-588455742", "link_geoid": "8f2a1072c273935", "sim_time": "2014-12-21T00:06:00", "report_type": "VEHICLE_STATE"}
2458c2458
< {"vehicle_id": "v65", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.6944177178949575, "energy_ELECTRIC": 39.841956847027376, "link_link_id": "42435470-42432174", "link_geoid": "8f2a1072ca9ec8b", "sim_time": "2014-12-21T00:06:00", "report_type": "VEHICLE_STATE"}
---
> {"vehicle_id": "v65", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.7032594698222582, "energy_ELECTRIC": 39.840793385607704, "link_link_id": "42432282-42452056", "link_geoid": "8f2a1072c3246ec", "sim_time": "2014-12-21T00:06:00", "report_type": "VEHICLE_STATE"}
2773c2773
< {"vehicle_id": "v150", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.6804809216468265, "energy_ELECTRIC": 39.83040375990679, "link_link_id": "42441283-42441299", "link_geoid": "8f2a10088888473", "sim_time": "2014-12-21T00:07:00", "report_type": "VEHICLE_STATE"}
---
> {"vehicle_id": "v150", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.7057494136104798, "energy_ELECTRIC": 39.82703894349014, "link_link_id": "595314104-42442565", "link_geoid": "8f2a1008881dc08", "sim_time": "2014-12-21T00:07:00", "report_type": "VEHICLE_STATE"}
2780,2781c2780,2781
< {"vehicle_id": "v157", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.6970566296855898, "energy_ELECTRIC": 39.82835871400005, "link_link_id": "7732571915-6867580414", "link_geoid": "8f2a1008c023cdd", "sim_time": "2014-12-21T00:07:00", "report_type": "VEHICLE_STATE"}
< {"vehicle_id": "v158", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.6856632602848542, "energy_ELECTRIC": 39.82967123498042, "link_link_id": "3099327952-3099327937", "link_geoid": "8f2a1008d56035e", "sim_time": "2014-12-21T00:07:00", "report_type": "VEHICLE_STATE"}
---
> {"vehicle_id": "v157", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.6982881667633787, "energy_ELECTRIC": 39.82800486039621, "link_link_id": "42448605-42436734", "link_geoid": "8f2a1008ca8dc85", "sim_time": "2014-12-21T00:07:00", "report_type": "VEHICLE_STATE"}
> {"vehicle_id": "v158", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.6807339328767987, "energy_ELECTRIC": 39.830275450556606, "link_link_id": "42439852-42432900", "link_geoid": "8f2a1008dc88736", "sim_time": "2014-12-21T00:07:00", "report_type": "VEHICLE_STATE"}
2800c2800
< {"vehicle_id": "v175", "memberships": "", "vehicle_state": "DispatchTrip", "balance": 0.0, "distance_traveled_km": 0.7028028754733416, "energy_ELECTRIC": 39.82751830316738, "link_link_id": "42439851-42439849", "link_geoid": "8f2a1008d525396", "sim_time": "2014-12-21T00:07:00", "report_type": "VEHICLE_STATE"}
robfitzgerald commented
we are no longer observing non-deterministic runs