/tmp no longer accessible for running regression tests
dirkwhoffmann opened this issue · 2 comments
In macOS Sonoma, it seems like Apple restricts access to the /tmp folder in a way that breaks vAmiga's regression testing system. E.g., it's no longer possible to copy the vAmiga executable to /tmp and run it directly from there within a shell.
TODO: Make the regression testing framework more flexible. I.e., make it work to run inside a local folder.
Update: Everything seems to work fine if the vAmiga executable is loaded from somewhere else than tmp. However, all regression tests now fail because tiffcmp reports a difference:
Scanline 38, pixel 164, sample 0: f0 ef
I don't see a difference:
To make a long story short: I had trouble with comparing TIFFs multiple times in the past, for whatever reason. Comparing TIFFs basically breaks from each OS release to the next.
TODO: Let the regression tester compare the raw image data. Save TIFFs only for visual control of the result.
The regression testing framework has been updated. Reference images are now stored and compared in raw format. Hopefully, this solution will prove more stable over time than the existing one.