JaGeo/LobsterPy

[CI] `test_cli` failed on MacOS due to different `matplotlib` rounding precision

DanielYang59 opened this issue · 4 comments

DanielYang59 commented

This last line:

LobsterPy/tests/cli/test_cli.py

Line 92 in 2ec4d19

assert plot_attributes[key] == pytest.approx(ref_value)

of

LobsterPy/tests/cli/test_cli.py

Lines 72 to 92 in 2ec4d19

@pytest.mark.parametrize("args", test_cases)
def test_cli_results(self, args, capsys, inject_mocks, clean_plot):
test = get_parser().parse_args(args)
run(test)
captured = capsys.readouterr()
assert captured.out == self.ref_results[" ".join(args)]["stdout"]
plot_attributes = self.get_plot_attributes(plt.gcf())
ref_plot_attributes = self.ref_results[" ".join(args)]["plot"]
if ref_plot_attributes is None:
assert plot_attributes is None
else:
for key, ref_value in ref_plot_attributes.items():
if key == "xydata":
for line, ref_line in zip(plot_attributes[key], ref_value):
assert np.array(np.array(line)) == pytest.approx(np.array(ref_line))
else:
assert plot_attributes[key] == pytest.approx(ref_value)

Would fail only on MacOS (passed on Ubuntu):

>                   assert plot_attributes[key] == pytest.approx(ref_value)
E                   assert [20.0, 12.36] == approx([20.0 ...96 ± 1.2e-05])
E                     
E                     comparison failed. Mismatched elements: 1 / 2:
E                     Max absolute difference: 0.0006797749978968426
E                     Max relative difference: 5.499797717611995e-05
E                     Index | Obtained | Expected                    
E                     1     | 12.36    | 12.360679774997896 ± 1.2e-05
JaGeo commented

@DanielYang59 You are invited to make a fix. We don't test for MAC at the moment. We can also add additional tests for mac if you think this is useful.

DanielYang59 commented

With pleasure :) I need a closer look at the reason though, because I would expect MacOS to be very similar to Ubuntu. Would do this very soon.

  • 👍1
JaGeo commented

Thanks! We simply only use Linux and therefore never checked the mac things!

DanielYang59 commented

I just had another look at this issue, and it seems to come from the difference in rounding precision from matplotlib.

The following test case:

LobsterPy/tests/cli/test_cli.py

Line 38 in 55d8d2e

["plot", "1", "--width", "20"],

The generated figure height is 12.360679774997896, completely agreeing with the reference on Ubuntu.

While on MacOS, only two decimal places are kept as 12.36, which leads to this test failure as the default relative tolerance is 1E-6.

To recreate this:

import matplotlib.pyplot as plt
import matplotlib as mpl
from math import sqrt

width = 20
ratio = (sqrt(5) + 1) / 2  # the golden ratio in _user_figsize func

height = width / ratio
mpl.style.use({"figure.figsize": (height * ratio, height)})

print(mpl.rcParams['figure.figsize'])  # >>> [20.0, 12.360679774997896]

print(plt.gcf().get_size_inches().tolist())  # >>> [20.0, 12.36] on MacOS / [20.0, 12.360679774997896] on Ubuntu

However I don't know exactly what caused the figsize to be rounded differently, might need to look into the matplotlib source code.

  • 👍1