google-research/football

process-dependency-links flag not found

cyfra opened this issue · 9 comments

cyfra commented

process-dependency-links flag is missing from the new pip release.
(we're currently using it to fetch the newest version of openai baselines code).

If you hit this error - the current workaround is:
replace:

'baselines>=0.1.6',

with
'baselines @ git+git://github.com/openai/baselines.git@master#egg=baselines',

and remove the dependency_links section.

Sorry for the hassle - we're working on the better long term solution.

It can also be solved by downgrading pip to 18.1.

cyfra commented

Here's the longer description of the issue: maybe someone from the community could help us solve it better (as I'm far from the pip/setup.py expert :-)

  • Our code depends on openai/baselines library.
  • That library is available in pypi, but is very old (> 1 year old) - version 0.1.5
  • We depend on the features that were added more recently.

Therefore, in order to use the football library, the user must have installed the openAI baselines FROM github and not from pypi.

Now, our problem is: how to do it in the best way:

  • just depending on the "baselines" in setup.py doesn't work (pip will download the old 0.1.5 version from pypi)
  • we've added the dependency_links section (which seems to work in pip < 18), but this forced us to introduce the "fake" 0.1.6 dependency - so that we're sure that system takes the contents from the dependency_link part (rather than from pypi)
  • for pip > 18 - seems that you can specify the "baselines @ git+git:XXX" which should download it from github. Unfortunately the earlier versions of pip don't understand that notion (and they would happily install the old 0.1.5 from pypi).

Any ideas for a better solution that could work across multiple pypi versions ?

one good (step towards a?) solution would be to ask openai to update their version on pypi.

Newer version of pip did not support dependency links.
You can install baseline 0.1.6 manually.

git clone https://github.com/openai/baselines.git
cd baselines
pip install -e .
cyfra commented

Good news - openAI has increased their version in github repo to 0.1.6.

So people who install from github, should see the correct version now.

We just pushed a new release, process-dependency-links is not used anymore.

This issue is due to Python version, between Python 3.5 and Python 3.6 there was a change in pkgutil API which we use:
https://docs.python.org/3.6/library/pkgutil.html#pkgutil.iter_modules
https://docs.python.org/3.5/library/pkgutil.html#pkgutil.iter_modules

You can work around it by changing

return [m.name for m in pkgutil.iter_modules([path])]

to:

return [m[2] for m in pkgutil.iter_modules([path])]

In gfootball/env/scenario_builder.py

I have just pushed a new version with the workaround to the problem, so alternatively you can just sync to head.