Missing .gitattributes file

[SimonBoothroyd]

When installing versioneer a .gitattributes file is automatically created which allows versioneer to partially populate the _version.py file (see these lines in particular which get their placeholders filled) upon running git archive. This allows versioneer to attempt to provide a version when installing from a github archive tarball directly, but where the archive has been unzipped into a directory whose name does not contain version info (see e.g. conda-smithy installations.)

It seems that this file has been omitted from the cookiecutter template. Was this intentional or an accidental omission?

I came across this when trying to build a package using conda-smithy, and found that the build system was unable to properly detect a version when building from a github archive tarball.

If this is a bug I'd be happy to contribute a PR fix.

[Lnaden]

This appears to be an edge case which has not come up until now. We haven't included the .gitattributes file in the cookiecutter simply because its never been an issue. This also seems to be an interesting use case so before I can make an assessment, can you let me know your exact pipeline for building a package through conda-smithy and also let me me know your use case for conda-smithy over say pip/conda-build/etc.?

[SimonBoothroyd]

can you let me know your exact pipeline for building a package through conda-smithy

Sure - the feedstock can be found here and the build proceeds directly via Azure. Ultimately to release I (will in future) edit meta.yaml with the updated information, re-render and then push.

let me me know your use case for conda-smithy over say pip/conda-build/etc.

This is part of an exploration of intermediate steps between moving away from omnia and migrating to conda-forge (when the issue of how to handle openmm is resolved). Essentially for now it is just a tracked, automated way of releasing new versions of my packages.

If I understand the issue correctly (and please correct me if I'm wrong) - this will probably also be an issue when releasing packages created using the cookiecutter on conda-forge, given the same smithy build system.

[mattwthompson]

I can reproduce this by pip installing from a tarball attached to a GH release (say https://github.com/openforcefield/cmiles/archive/v0.1.5.tar.gz):

>>> import cmiles
>>> cmiles.__file__
'/Users/mwt/scratch/cmiles-0.1.5/cmiles/__init__.py'
>>> cmiles.__version__
'0+unknown'

This behavior already exists in at least one of our packages on conda-forge (one that was not developed off of the cookiecutter) but I'm pretty sure it, in its simplest forms, is a versioneer issue (i.e. should persist whether it's build with conda-build, conda-smithy, pip, maybe others) and will exist for cases like ours in which something is built with the cookiecutter and shipped off to conda-forge without modifying some set of files.

For what it's worth, I'm pretty sure this doesn't happen if the recipe in the conda-forge feedstock pulls a tarball from PyPI (as most/all of the QCarchive packages). Or, at least, there are some QCArchive packages that seem to use a boilerplate versioneer setup, no .gitattributes file, and works as it should. (I can offer a few educated guesses as to why, but I'm not familiar enough with pip's tooling to put my neck out like that.) Testing just one:

>>> import qcportal
>>> qcportal.__file__
'/Users/mwt/scratch/qcportal-0.13.1/qcportal/__init__.py'
>>> qcportal.__version__
'v0.13.1'

Tarball from here: https://pypi.org/project/qcportal/#files
No .gitattributes fiile: https://github.com/MolSSI/QCPortal
Feedstock pulling from PyPI: https://github.com/conda-forge/qcportal-feedstock/blob/72a84f0eae489dd5dadb476a3ad7ed66c952e526/recipe/meta.yaml#L9-L11

I'd expect this to be an issue even for projects that build both through pip and conda-forge if the recipe builds from a GitHub tarball. This is probably testable or demonstrable by finding a project that does this, but I'm already a bit off track here.

I would argue that a tarball of just the project's source code - from GitHub, without being packaged by pip - should be able to be downloaded, extracted, and installed and have the version displayed correctly.

[SimonBoothroyd]

@Lnaden any more thoughts on this?

[jaimergp]

Maybe worth checking setuptools-scm too. Example transition: psf/black#1008

[dgasmith]

Quick note that last I checked conda and setuptools-scm interacted pretty poorly with each other. I would need to hunt around, but in the conda-forge feedstock this came up in, we ended up making patches that just stamped the __init__.py with the correct version and stripped out setuptools-scm in a diff. If this has changed it would be great, but since setuptools-scm only records its version in somewhat questionable python metadata the non-python focused versioners seems a bit untenable.

It seems a general issue that core PSF has been forging ahead with some breaking changes (black refusing to read anything beyond pyproject.toml for example) at a dramatic pace now that there is less steady hands on the wheel of the ship so to speak. I'm not quite sure it is a good or bad thing, but certainly dynamic 😏 .

[radifar]

I faced the same issue here with my project. I think the .gitattributes is necessary for git/GitHub to generate the tar.gz archive, which will be used for the next step in the packaging process.

So what happen is the export-subst keyword in .gitattributes will direct the git archiver to extract the information from git version control and replace few keywords in _version.py (eg. tag/version). This is important because once the repository is turned into source distribution it will lost all git related information, and thus versioneer can not obtain the version information.

Also, note that setup.py rely on versioneer to assign the version.

[mattwthompson]

Bumping this - this is a one-line/one-file change that we need to make to each of our projects using this cookiecutter, unfortunately one that's easy to miss at the origin since it doesn't obviously produce an issue until releases and conda packages are made.

I grant that there are many moving parts and tools to consider, but I expect this could be resolved with a much smaller change to the cookiecutter than completely migrating the tool used to manage versioning.

[Lnaden]

You're absolutely right. Looking at this pipeline in more detail, it does appear to be an edge case we should fix. Could you gander at the PR in #147 and see if that fix would be sufficient to resolve this issue?

[mattwthompson]

That change looks perfect - thanks!