Structuring tests layout

[mkaze]

In PR #215 we encountered a problem: there are two common.py files, one for matplotlib tests and another for Altair tests. Since we are not treating test directories as packages, i.e. there is no __init__.py file in directories, the import process only uses one of them and therefore it causes problems.

Now to resolve this, either we should:

• change the way for managing/loading helper functions in the tests, or
• use a different layout for the tests (add __init__.py files, and potentially put the source code of package in src), or
• set the --import-mode flag of pytest to importlib.

Here is the relevant section of the pytest documentation which mentions the last two solutions (I don't know the best way to manage common helper functions in pytest, actually!).

[koaning]

That's a good question. My first reaction moves to option number 2. I've never put the source code of packages in src though but I have use the __init__.py trick in a bunch of places. With regards to src I'm less aware of what some of the pros/cons might be.

[mkaze]

I also prefer the second option. The src is used to make sure that the tests are run against the installed version of the package, not the repository code:

... But now this introduces a subtle problem: in order to load the test modules from the tests directory, pytest prepends the root of the repository to sys.path, which adds the side-effect that now mypkg is also importable.
This is problematic if you are using a tool like tox to test your package in a virtual environment, because you want to test the installed version of your package, not the local code from the repository.
In this situation, it is strongly suggested to use a src layout where application root package resides in a sub-directory of your root:

I think this depends on local development workflow, though; if we install the package with -e flag using pip, then the installed version is actually the repository code itself and therefore I don't think this issue would arise there.