Use pybind11.get_include()?

[wjakob]

So pybind11 1.5 now ships with a pybind11.get_include() method. I am wondering if/how to call it in pbtest's setup.py file.

If the pybind package does not yet exist, setuptools.py will install it. However, I don't think it's safe to assume that it exists already in https://github.com/pybind/pbtest/blob/master/setup.py#L14.

Thoughts?

Thanks,
Wenzel

[SylvainCorlay]

My undertanding is that at the time when the build_ext command will run, the dependencies should be installed, so we can include numpy in the build_extensions method, however I am not sure how to add things to the ext_modules ...

Scipy is using numpy.get_include(), but their setup seems really overkill.

[benmaier]

I solved this problem by passing a string-like object to the Extension like this

from pip import locations

class get_pybind_include(object):

    def __str__(self):
        pybind_include = os.path.dirname(locations.distutils_scheme('pybind11')['headers'])
        return pybind_include

ext_modules = [
    Extension(
        'pbtest',
        ['py/main.cpp'],
        include_dirs=[
            # Path to pybind11 headers
            get_pybind_include()
        ],
        language='c++',
    ),
]

This evaluates the location as soon as get_pybind_include() is used as a string (i.e. after pybind11 is installed). However, I still have the problem that if I install pybind or pbtest with the flag --user, calling locations.distutils_scheme('pybind11')['headers'] does not yield the user directory but the standard include directory.

[wjakob]

Thank you for your feedback, Ben!

Does the following give the right answer on your machine?

from pip import locations
locations.distutils_scheme('pybind11', '--user')['headers']

I'm no setuptools expert and wonder if there is a way to get the command line parameters like --user to pass them to distutils_scheme as a second parameter?

[wjakob]

The __str__ trick is neat, by the way!

[benmaier]

Yes, this works, thanks! Actually, one can pass a boolean for the user. What about doing the following? it's a bit hacky though.

is_user_install = "--user" in sys.argv[1:]

class get_pybind_include(object):

    def __str__(self):
        pybind_include = os.path.dirname(locations.distutils_scheme('pybind11',is_user_install)['headers'])
        return pybind_include

[wjakob]

I just added this commit: pybind/pybind11@75b1b7d

So starting with the next version, this could look as follows:

class get_pybind_include(object):
    def __str__(self):
        import pybind11
        return pybind11.get_include(*[arg for arg in sys.argv if arg == '--user'])

Does that look reasonable?

[benmaier]

I think you should actually pass a boolean because that's what distutils_scheme wants:

distutils_scheme(dist_name, user=False, home=None, root=None, isolated=False, prefix=None)
    Return a distutils install scheme

Technically, passing "--user" still works as python treats non-empty strings as True, but say someone calls pybind11.get_include("--not-user") they will still get the user directory. Besides that, your solution seems fine!

[wjakob]

Sounds good -- so the proposed change is:

class get_pybind_include(object):
    def __str__(self):
        import pybind11
        return pybind11.get_include("--user" in sys.argv[1:])

and pybind11.get_include doesn't have to change.

[benmaier]

cool! btw, thanks a bunch for writing the package and also for providing this pbtest example, Sylvain. All of this has saved me a tremendous amount of work so far.

[benmaier]

I just ran into the following issue: I installed pybind with the flag --user, but the actual package I wrote is installed as root, hence it's not found during the install. I therefore propose to simply add both possibilities to the include list as:

class get_pybind_include(object):

    def __init__(self,user=False):
        self.user = user

    def __str__(self):
        import pybind11
        return pybind11.get_include(self.user)

ext_modules = [
    Extension(
        'pbtest',
        ['py/main.cpp'],
        include_dirs=[
            # Path to pybind11 headers
            get_pybind_include(),
            get_pybind_include(user=True)
        ],
        language='c++',
    ),

[wjakob]

Sounds reasonable to me.