Stockfish benchmark code

[guidopetri]

It would be great to have some stockfish benchmark code in order to determine what's the best depth for --local-stockfish to search at. Maybe with some graphs?

[guidopetri]

I was thinking of something like this:

for n in range(10, 26):
    sf = stockfish.Stockfish('./stockfish_10_x64.exe',
                             depth=n)
    print(n)
    %timeit sf.set_fen_position(fen); sf.get_best_move()

using IPython's %timeit magic. Here are some preliminary stats from depth 10-26:

pc = [29.4 * 1e-3,
38.5 * 1e-3,
57.5 * 1e-3,
97.6 * 1e-3,
221 * 1e-3,
353 * 1e-3,
786 * 1e-3,
1.1,
1.83,
3.46,
3.58,
4.15,
5.6,
8.23,
13.3,
15.2]

vps = [94.5 * 1e-3,
110 * 1e-3,
140 * 1e-3,
215 * 1e-3,
402 * 1e-3,
779 * 1e-3,
1.26,
1.94,
3.12,
4.14,
5.05,
10.5,
13.5,
17.2,
18.2,
27.2]

rpi = [1.78,
2.12,
2.85,
5,
8.81,
17,
29,
44.8,
1 * 60 + 12,
1 * 60 + 33,
1 * 60 + 58,
3 * 60 + 56,
4 * 60 + 29,
...,
...,
...]

the time being in seconds. Plotting code:

plt.plot(range(10, 26), pc, 'r', label='PC')
plt.plot(range(10, 26), vps, 'g', label='VPS')
plt.plot(range(10, 26), rpi, 'b', label='RPi')
plt.yscale('log')
plt.ylabel('Time in s')
plt.xlabel('Depth')
plt.title('How long does it take Stockfish to analyze a position at a certain depth?')
plt.legend(loc='best')

[guidopetri]

Last three entries on rpi:

5 * 60 + 59,
6 * 60 + 53,
10 * 60 + 17

[guidopetri]

Basics done with d54fcfc . Now to make a graph.

[guidopetri]

Graphed in 1724591 . Merging into dev/master.

[guidopetri]

VPS

[0.25551111879758537, 0.28169199894182384, 0.6356437890790403, 0.9658400469925255, 2.2811675728298724, 4.421990415081382, 5.283789199078456, 7.260874294908717, 12.383977574063465, 17.36204526317306, 22.26615801989101, 29.727813542820513, 42.0052341602277, 94.84192839683965, 116.49611252802424, 148.22141617909074]

RoyVPS

[0.19935076200636104, 0.230359834007686, 0.5645083490235265, 0.9161799979920033, 2.362423962011235, 4.72492092999164, 5.580920461012283, 7.250173951993929, 13.583324579987675, 18.682603237015428, 24.305201015988132, 32.43767024000408, 45.89748174400302, 105.57565680201515, 127.44099019299028, 168.7018144989852]

RPi

[5.221668511629105, 6.371294246986508, 16.39128729328513, 27.567460985854268, 69.34917690046132, 143.04488733038306, 171.47142096608877, 223.78672445565462, 403.3104937747121, 568.8896902631968, 739.9428261965513, 992.293460469693, 1417.9758676178753, 3123.685179369524, 3778.4266493804753, 4886.729606661946]

There's a small typo in the title - "positionat" is obviously "position at" - but this is fixed now.