compare against FD with hff and gbfs
tomsilver opened this issue · 4 comments
tomsilver commented
currently the node results are favoring pyperplan, which is weird. see if pyperplan and FD match when the algorithm is the same.
tomsilver commented
some results:
AGGREGATED DATA OVER SEEDS:
env approach experiment_id solve_time nodes_created nodes_expanded success num_seeds
0 pyperplan-airport pure-planning airport-fd-hff-gbfs 0.20 (0.00) 386.10 (0.00) 305.70 (0.00) 1.00 (0.00) 1
1 pyperplan-airport pure-planning airport-fd-only 0.23 (0.00) 1715.30 (0.00) 753.80 (0.00) 1.00 (0.00) 1
2 pyperplan-airport pure-planning airport-pyperplan-only 0.71 (0.00) 238.10 (0.00) 196.80 (0.00) 1.00 (0.00) 1
3 pyperplan-blocks pure-planning blocks-fd-hff-gbfs 0.10 (0.00) 333.30 (0.00) 199.20 (0.00) 1.00 (0.00) 1
4 pyperplan-blocks pure-planning blocks-fd-only 0.10 (0.00) 203.10 (0.00) 51.50 (0.00) 1.00 (0.00) 1
5 pyperplan-blocks pure-planning blocks-pyperplan-only 0.07 (0.00) 201.60 (0.00) 107.00 (0.00) 1.00 (0.00) 1
6 pyperplan-elevators pure-planning elevators-fd-hff-gbfs 0.13 (0.00) 908.60 (0.00) 649.70 (0.00) 1.00 (0.00) 1
7 pyperplan-elevators pure-planning elevators-fd-only 0.14 (0.00) 1997.90 (0.00) 106.60 (0.00) 1.00 (0.00) 1
8 pyperplan-elevators pure-planning elevators-pyperplan-only 0.81 (0.00) 522.00 (0.00) 34.00 (0.00) 1.00 (0.00) 1
9 pyperplan-gripper pure-planning gripper-fd-hff-gbfs 0.13 (0.00) 12190.50 (0.00) 6718.50 (0.00) 1.00 (0.00) 1
10 pyperplan-gripper pure-planning gripper-fd-only 0.12 (0.00) 1443.50 (0.00) 102.50 (0.00) 1.00 (0.00) 1
11 pyperplan-gripper pure-planning gripper-pyperplan-only 0.41 (0.00) 1038.70 (0.00) 77.00 (0.00) 1.00 (0.00) 1
12 pyperplan-logistics pure-planning logistics-fd-hff-gbfs 0.10 (0.00) 382.50 (0.00) 321.80 (0.00) 1.00 (0.00) 1
13 pyperplan-logistics pure-planning logistics-fd-only 0.10 (0.00) 892.70 (0.00) 79.60 (0.00) 1.00 (0.00) 1
14 pyperplan-logistics pure-planning logistics-pyperplan-only 0.08 (0.00) 316.60 (0.00) 35.60 (0.00) 1.00 (0.00) 1
15 pyperplan-miconic pure-planning miconic-fd-hff-gbfs 0.11 (0.00) 1178.10 (0.00) 487.90 (0.00) 1.00 (0.00) 1
16 pyperplan-miconic pure-planning miconic-fd-only 0.12 (0.00) 3381.70 (0.00) 144.00 (0.00) 1.00 (0.00) 1
17 pyperplan-miconic pure-planning miconic-pyperplan-only 0.43 (0.00) 485.70 (0.00) 43.00 (0.00) 1.00 (0.00) 1
18 pyperplan-movie pure-planning movie-fd-hff-gbfs 0.09 (0.00) 596.00 (0.00) 29.00 (0.00) 1.00 (0.00) 1
19 pyperplan-movie pure-planning movie-fd-only 0.09 (0.00) 596.00 (0.00) 9.00 (0.00) 1.00 (0.00) 1
20 pyperplan-movie pure-planning movie-pyperplan-only 0.03 (0.00) 29.10 (0.00) 8.50 (0.00) 1.00 (0.00) 1
21 pyperplan-openstacks pure-planning openstacks-fd-hff-gbfs 0.14 (0.00) 882.30 (0.00) 743.30 (0.00) 1.00 (0.00) 1
22 pyperplan-openstacks pure-planning openstacks-fd-only 0.14 (0.00) 253.10 (0.00) 52.10 (0.00) 1.00 (0.00) 1
23 pyperplan-openstacks pure-planning openstacks-pyperplan-only 0.20 (0.00) 334.50 (0.00) 51.70 (0.00) 1.00 (0.00) 1
24 pyperplan-parcprinter pure-planning parcprinter-fd-hff-gbfs 0.14 (0.00) 285.10 (0.00) 243.70 (0.00) 1.00 (0.00) 1
25 pyperplan-parcprinter pure-planning parcprinter-fd-only 0.14 (0.00) 329.40 (0.00) 51.00 (0.00) 1.00 (0.00) 1
26 pyperplan-parcprinter pure-planning parcprinter-pyperplan-only 0.16 (0.00) 256.10 (0.00) 44.60 (0.00) 1.00 (0.00) 1
27 pyperplan-pegsol pure-planning pegsol-fd-hff-gbfs 0.13 (0.00) 1822.60 (0.00) 1303.20 (0.00) 1.00 (0.00) 1
28 pyperplan-pegsol pure-planning pegsol-fd-only 0.12 (0.00) 785.70 (0.00) 464.50 (0.00) 1.00 (0.00) 1
29 pyperplan-pegsol pure-planning pegsol-pyperplan-only 1.06 (0.00) 1737.40 (0.00) 1365.30 (0.00) 1.00 (0.00) 1
30 pyperplan-psr-small pure-planning psr-small-fd-hff-gbfs 0.12 (0.00) 164.40 (0.00) 112.20 (0.00) 1.00 (0.00) 1
31 pyperplan-psr-small pure-planning psr-small-fd-only 0.12 (0.00) 142.60 (0.00) 50.30 (0.00) 1.00 (0.00) 1
32 pyperplan-psr-small pure-planning psr-small-pyperplan-only 0.07 (0.00) 102.40 (0.00) 37.80 (0.00) 1.00 (0.00) 1
33 pyperplan-zenotravel pure-planning zenotravel-fd-hff-gbfs 0.30 (0.00) 5884.70 (0.00) 3944.30 (0.00) 1.00 (0.00) 1
34 pyperplan-zenotravel pure-planning zenotravel-fd-only 0.24 (0.00) 5814.00 (0.00) 157.90 (0.00) 1.00 (0.00) 1
35 pyperplan-zenotravel pure-planning zenotravel-pyperplan-only 9.75 (0.00) 1706.00 (0.00) 50.80 (0.00) 1.00 (0.00) 1
so the mystery remains...
tomsilver commented
opened an issue on pyperplan to ask for help aibasel/pyperplan#17
tomsilver commented
maybe also check against the latest version of fast downward
tomsilver commented
whoops, use "expanded" instead of "evaluated" in fast downward parsing!