Add support for test subdirectories

[rawrgulmuffins]

Request

Add some sort of structure that incorporates data files in a subdirectory that aren't explicit name matches for a test file.

Issue Description

Often when projects get large you want to start using subdirectories in your project. You don't really want to share data between all of your test modules but you do want a subset of your test files to share the same data.

Example Test Structure

I want to share data files for all files in performance_stats.

test/
├── data
├── dict_types_unit_test.py
├── __init__.py
├── performance_stats
│   ├── 20181018_runlog.txt # Test Data
│   ├── 20181019_runlog.txt # Test Data
│   ├── fixtures.pyc
│   ├── __init__.py
│   ├── integration_tests.py
│   ├── test_cli.py
│   └── test_parsing.py
├── setz_unit_test.py
├── test-report.xml
└── yaml_config_base_unit_test.py

Current Work Around

use shared_datadir and accept a "global" namespace.

Unimportant note

I appreciate the library a lot. I've been writing my own fixtures to do this for years. =D

[nicoddemus]

Hi @rawrgulmuffins,

Thanks for writing.

It is not clear how the user would select which files would be copied. Is it a new fixture, a configuration option? Could you please elaborate on that? Thanks!

[igortg]

I'd just add that the layout @rawrgulmuffins presented is a somewhat unusual (IMHO). It's more common to have a test folder containing only test_*.py files and all data files in subfolders or in a folder in the project root.

[rawrgulmuffins]

@igortg This isn't a super unusual test structure at some of the companies I've worked for (Dell, EMC, small start-ups). Especially for projects that start to get into the hundreds of test_*.py files.

I've worked on one repo (an operating system) that had a fork of FreeBSD and about 3 million lines of python on top of that which had hundreds of thousands of test files.

¯\(ツ)/¯

[rawrgulmuffins]

@nicoddemus I think I opened this bug potentially prematurely. Originally I thought that the way that shared_datadir worked given the documentation was that you had this directory structure:

test/
├──data
│   ├── 20181018_runlog.txt
│   └── 20181019_runlog.txt
├── dict_types_unit_test.py
├── performance_stats

│   ├── integration_tests.py
│   ├── test_cli.py
│   └── test_parsing.py

And then all of your test files could access the datafiles with shared_datadir that were either on the same file system level or any recursive child level. But that didn't turn out to be the case.

In the end I figured out that this structure worked

test/
├── dict_types_unit_test.py
├── performance_stats
│   ├── data
│   │   ├── 20181018_runlog.txt
│   │   └── 20181019_runlog.txt
│   ├── integration_tests.py
│   ├── test_cli.py
│   └── test_parsing.py
├── setz_unit_test.py
├── test-report.xml
└── yaml_config_base_unit_test.py

So I believe that shared_datadir has the behavior that I was asking for in my original description. I don't have a current use case for why you would want to share data recursively with some (or all) children test files at this time.

Given that I believe this issue is either a documentation problem or a comprehension problem (on my part).

[nicoddemus]

@rawrgulmuffins thanks for the follow up then!

Feel free to open a PR with a suggestion to the docs. 👍 😁

[weblab-misha]

Is it possible to have a shared datadir across multiple packages (not just modules)?

E.g. here:

tests/feature_x/data/*
tests/feature_x/feature_x_helper_a/
tests/feature_x/feature_x_helper_a/test_a_1.py
tests/feature_x/feature_x_helper_a/test_a_2.py
tests/feature_x/feature_x_helper_b/
tests/feature_x/feature_x_helper_b/test_b_1.py
tests/feature_x/feature_x_helper_b/test_b_2.py

I don't have data/ subfolder in tests/feature_x/feature_x_helper_{a,b}/, and I want to access tests/feature_x/data/ from tests/feature_x/feature_x_helper_{a,b}/test_*.py