Google: Support expected failures

[fontealpina]

Hi, I need to treat also xfail for google test, for example I write 'xfail' in the gtest name and I would that pytest cpp mark the test as xfail, so pytest can run it as a real pytest test marked xfail.
Is it possible?
Thanks

[nicoddemus]

Hi!

Is adding "xfail" to a test name a builtin support for Google Test (IOW, the default runner supports that as well), or a specific customization that you would like to see in pytest-cpp?

[fontealpina]

It's only my customization, because I need to treat xfail also for Google
Test with pytest.
In other words, I don't know a way to run Google test directly as expected
failure, and I'm thinking to a my customization to do this.
I would to know if actually pytest-cpp allow in someway to treat xfail for
google test?
Many thanks

2015-02-24 14:18 GMT+01:00 Bruno Oliveira notifications@github.com:

Hi!

Is adding "xfail" to a test name a builtin support for Google Test (IOW,
the default runner supports that as well), or a specific customization that
you would like to see in pytest-cpp?

—
Reply to this email directly or view it on GitHub
https://github.com/nicoddemus/pytest-cpp/issues/14#issuecomment-75754446
.

[nicoddemus]

I see, thanks for the clarification. 😄

I'm reluctant in adding specific behavior to pytest-cpp that is not supported by the underlying testing framework.

But nonetheless it is very easy to add that feature for your own test suite. Just create a conftest.py file in your project with this content:

import pytest

def pytest_runtest_setup(item):
    if 'xfail' in item.name:
        pytest.xfail("test marked as xfail")

Now every test that contains "xfail" in its name will automatically be skipped (both normal python and C++ tests). If you want that behavior for C++ tests only, that's easy to add too because item will be a CppItem instance (see plugin.py).

Hope that helps! 😄

[fontealpina]

Thanks for the solution, but if I need also to run the xfail-marked test
(e.g. to get xfail or XPASS), I need to use

pytest.mark.xfail(reason="test marked as xfail", run=True)

in place of

pytest.xfail("test marked as xfail")

I've tried to simply substitute it, but it raises errors.
Is there an easy way to do use this solution?
Thanks

2015-02-24 16:00 GMT+01:00 Bruno Oliveira notifications@github.com:

I see, thanks for the clarification. [image: 😄]

I'm reluctant in adding specific behavior to pytest-cpp that is not
supported by the underlying testing framework.

But nonetheless it is very easy to add that feature for your own test
suite. Just create a conftest.py file in your project with this content:

import pytest
def pytest_runtest_setup(item):
if 'xfail' in item.name:
pytest.xfail("test marked as xfail")

Now every test that contains "xfail" in its name will automatically be
skipped (both normal python and C++ tests). If you want that behavior for
C++ tests only, that's easy to add too because item will be a CppItem
instance (see plugin.py
https://github.com/nicoddemus/pytest-cpp/blob/master/pytest_cpp/plugin.py
).

—
Reply to this email directly or view it on GitHub
https://github.com/nicoddemus/pytest-cpp/issues/14#issuecomment-75771884
.

[nicoddemus]

I didn't realize that, sorry:

import pytest

def pytest_runtest_setup(item):
    if 'xfail' in item.name:
        m = pytest.mark.xfail(reason="test marked as xfail", run=True)
        item.add_marker(m)

😄

Cheers,

[fontealpina]

I'm sorry, I have tried this solution but it doesn't run for CppItem
instance, only for Function instance.
When I try it on CppItem instance, are raised errors into skipping.py in
the MarkEvaluator instance.
Maybe CppItem are not treated as Function.

2015-02-25 23:37 GMT+01:00 Bruno Oliveira notifications@github.com:

I didn't realize that, sorry:

import pytest
def pytest_runtest_setup(item):
if 'xfail' in item.name:
m = pytest.mark.xfail(reason="test marked as xfail", run=True)
item.add_marker(m)

[image: 😄]

Cheers,

—
Reply to this email directly or view it on GitHub
https://github.com/nicoddemus/pytest-cpp/issues/14#issuecomment-76076401
.

[nicoddemus]

Can you post the errors you are getting?

[nicoddemus]

Please try this, it seems to work for me:

# contents of conftest.py
from pytest_cpp.plugin import CppItem

def pytest_runtest_makereport(__multicall__, item, call):
    if isinstance(item, CppItem) and 'xfail' in item.name and call.when == 'call':
        rep = __multicall__.execute()
        rep.wasxfail = "reason: c++ test marked as xfail"
        if call.excinfo:
            rep.outcome = "failed"
        else:
            rep.outcome = "skipped"
        return rep

[nicoddemus]

Any updates @fontealpina? Did you get a chance of trying the code I posted?

[fontealpina]

Yes, I tried the code and it was working, I'm sorry for the missing feedback.
Many thanks.

[nicoddemus]

No problems!

Would you care to contribute to the Wiki with a "how to", explaining the motivation and solution? No need for anything fancy, but it would help others with a requirement similar to yours! 😄