Create container with dynamic scope

[LostInDarkMath]

Hi there!

I wonder if there is any way to create containers with dynamic scopes. I would like to do something like this:

my_container = container(
    image='{my_image.id}',
    ports={'3306/tcp': None},
    network='{my_net.name}',
    tmpfs={'/var/lib/my_dirl': ''},
    environment={
        'PASSWORD': 'pw',
    },
    scope=SCOPE  # instead of 'session'
)

My problem is, that I read my_scope from the pytest command line interface via:

def pytest_addoption(parser):
    """Add command line option to pytest. This is just a flag"""
    parser.addoption(CLI_ARG_SCOPE_FUNCTION, action="store_true")  # store_true implies default=False


def get_scope(fixture_name: str, config: Config) -> str:
    """https://docs.pytest.org/en/stable/fixture.html#dynamic-scope
    Command "pytest maddoxtests" will start tests with scope session
    Command "pytest maddoxtests --function-scope" will start tests with function scope
    """

    scope = Scope.SESSION

    if config.getoption(name=CLI_ARG_SCOPE_FUNCTION, default=None):
        scope = Scope.FUNCTION

    print(f'Started pytest with scope "{scope}"')
    global SCOPE
    SCOPE = scope
    return scope

So the global variable SCOPE is not set when the containers are created which leads to an error (session cannot be None or something like that).

So I tried to use a fixture do create my containers:

@pytest.fixture(scope=get_scope, name='con')
def my_container() -> Container:
    assert SCOPE in [Scope.SESSION, Scope.FUNCTION]
    yield container(
        image='{my_image.id}',
        ports={'8086/tcp': None},
        network='{my_net.name}',
        tmpfs={'/var/lib/my_dir': ''},
        environment={
            'PASSWORD': 'pw',
        },
        scope=SCOPE
    )

@pytest.fixture(scope=get_scope, autouse=True)
def test_env_scope_session(con):
   ip, port = con.get_addr('8086/tcp')  # error

This leads to an error: AttributeError: 'function' object has no attribute 'get_addr'I expectedconto be anContainer` like before, but it is a Function.

Even ip, port = con().get_addr('8086/tcp') does not work either. This produces the error Fixture "container" called directly. Fixtures are not meant to be called directly, but are created automatically when test functions request them as parameters.

Do you have any idea have I can do this?
Best, Willi

[Jc2k]

container creates fixtures. It's a factory for fixtures that themselves return Containers iirc, rather than a factory for Containers. So I don't think the return value is something you can yield from another fixture, though its been a while since i worked on pytest stuff. 'con' is probably not actually a Container and instead you yielded a fixture functionn that pytest won't know how to call. I think that explains all the errors you are getting.

I don't actually know how to do this in pytest full stop. If you can demonstrate it working with a plain pytest.fixture (without pytest_docker_tools) and pytest_addoption it might help.

I have something similar where the image param is different between my dev environment and my CI environment. Again i couldn't figure out how to use pytest_addoption so i just used an environment variable instead.

[LostInDarkMath]

The addoption works fine for me and the global variable SCOPE is set correctly. But it is set too late because it will be set if pytest calls test_env_scope_session. So what I need is to find way to call container() later. Now it is called when conftest.py starts.

[Jc2k]

But container is a factory for creatinng methods to create containers that are wrapped in a pytest.fixture decorator. Calling container() isn't creating a container, its creating a function that pytest calls to create a container. So I think it has to run when it does, otherwise it isn't registered as a fixture and pytest won't run it. This is why in your example a function is passed to your test instead of a container object.

Which is why i asked to see an example of this without involving pytest_docker_tools at all. Basically, if you can make this dummy fixture work:

@pytest.fixture(scope=SCOPE)
def myfixture():
    print("Dummy fixture")
    yield {"dummy": True}
    print("Freeing resources")

def test_myfixture(myfixture):
    print(myfixture)

def pytest_addoption(parser):
    """Add command line option to pytest. This is just a flag"""
    parser.addoption(CLI_ARG_SCOPE_FUNCTION, action="store_true")  # store_true implies default=False

Then i might be able to make pytest_docker_tools support it too. Possibly.

[LostInDarkMath]

Yes, I can make this work:

from typing import Literal

import pytest
from _pytest.config import Config


class Scope:
    # type annotations are necessary here to avoid PyCharm type checker warnings
    FUNCTION: Literal['function'] = 'function'
    CLASS: Literal['class'] = 'class'
    MODULE: Literal['module'] = 'module'
    PACKAGE: Literal['package'] = 'package'
    SESSION: Literal['session'] = 'session'


SCOPE = None
CLI_ARG_SCOPE_FUNCTION = '--function-scope'


def pytest_addoption(parser):
    """Add command line option to pytest. This is just a flag"""
    parser.addoption(CLI_ARG_SCOPE_FUNCTION, action="store_true")  # store_true implies default=False


def get_scope(fixture_name: str, config: Config) -> str:
    """https://docs.pytest.org/en/stable/fixture.html#dynamic-scope
    Command "pytest conftest.py" will start tests with scope session
    Command "pytest conftest.py --function-scope" will start tests with function scope
    """

    scope = Scope.SESSION

    if config.getoption(name=CLI_ARG_SCOPE_FUNCTION, default=None):
        scope = Scope.FUNCTION

    print(f'Started pytest with scope "{scope}"')
    global SCOPE
    SCOPE = scope
    return scope


@pytest.fixture(scope=get_scope)
def myfixture():
    print("Dummy fixture: " + SCOPE)
    yield {"dummy": True}
    print("Freeing resources")


def test_myfixture(myfixture):
    print(myfixture)

Then pytest conftest.py --function-scope gives me Started pytest with scope "function" and pytest conftest.py gives me Started pytest with scope "session" as desired.

So how can I make this work with containers?

[alpex8]

Its possible to get this working by not using get_scope() but simply evaluating sys.argv

[Jc2k]

@LostInDarkMath sorry I didn't see the notification for this, and i also missed the link in your doc string.

The thing that you have to understand is that when you call container() under the hood a new fixture is generated.

So if you write:

mycontainer = container(
    image='redis:latest',
    scope=SCOPE,
)

It's (broadly) like you wrote this:

@pytest.fixture(scope=SCOPE)
def mycontainer():
    return Container(
        name = "mycontainer",
        image = "redis",
    )

(I'm simplifying quite a bit).

So i'd expect it's actually much closer to the example in the docs you referenced:

def determine_scope(fixture_name, config):
    if config.getoption("--keep-containers", None):
        return "session"
    return "function"


my_container = container(
    image='{my_image.id}',
    ports={'3306/tcp': None},
    network='{my_net.name}',
    tmpfs={'/var/lib/my_dirl': ''},
    environment={
        'PASSWORD': 'pw',
    },
    scope=detemine_scope
)

Whatever you pass to the scope argument of pytest.fixture you should be able to pass to the scope argument of container().

[Jc2k]

I've added a version of this to the README (which are also run as tests themselves, via pytest-markdown).

I'm going to close this now as i think we've met the initial requirements.