cms-sw/cmsdist

Is py-xrootdpyfs still needed?

iarspider opened this issue · 10 comments

iarspider commented
  • Quick search through cmssw repository does not return any direct usage. Maybe it's needed for ipython/jupyter @davidlange6 ?
  • Requires fs >= 0.5.4,<2.0 (published in November 2015), which requires old setuptools (<58.0.0 ) for 2to3 support.
  • Requires xrootd >= 4.8.4, <5.0.0. 4.x series are no longer supported, current version is 5.4.0.
  • Issue about migrating to fs 2.x is open since 2017, but no actions since then
  • The code seems to be in maintenance mode - there are small updates for py3 compatibility, but no visible changes to support newer xrootd or py-fs.

I propose to drop it for now.

cmsbuild commented

A new Issue was created by @iarspider .

@Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

davidlange6 commented

its not needed for Jupyter, no.
Unless the way people interact with xroot in python has changed recently, this is a needed tool, yes

iarspider commented

@davidlange6 thanks. Just to be sure: is it the xrootdpyfs that is needed (pseudo-fs based on XRootD), or just xrootd (thin python wrapper around xrootd client)?

iarspider commented

And do you know if 4.x and 5.x versions of xrootd are compatible - i.e. will 4.x client work with 5.x server?

davidlange6 commented

you can see from searching out GitHub that xrootdpyfs is what has been brought up several times. I don't know much more.

i.e. will 4.x client work with 5.x server?

I guess lots of things fail if this is not workable..but I don't know anything on this topic.

iarspider commented

Then we will keep it.

davidlange6 commented

of course it may block too many other things to keep around...

iarspider commented

Reopening the ticket because py3-xrootdpyfs is not working in CMSSW:

$ ls /cvmfs/cms.cern.ch/slc7_amd64_gcc10/external/py3-xrootdpyfs/ -ltr
# 0.2.1-128a843981e195258e694a59f76468e9 is the latest installed version of py3-xroodpyfs for slc7_amd64_gcc10
$ source /cvmfs/cms.cern.ch/slc7_amd64_gcc10/external/py3-xrootdpyfs/0.2.1-128a843981e195258e694a59f76468e9/etc/profile.d/init.sh
$ python3 -c "from xrootdpyfs import XRootDPyFS"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/external/py3-xrootdpyfs/0.2.1-128a843981e195258e694a59f76468e9/lib/python3.9/site-packages/xrootdpyfs/__init__.py", line 186, in <module>
    from .fs import XRootDPyFS
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc10/external/py3-xrootdpyfs/0.2.1-128a843981e195258e694a59f76468e9/lib/python3.9/site-packages/xrootdpyfs/fs.py", line 28, in <module>
    from fs.errors import DestinationExistsError, DirectoryNotEmptyError, \
ImportError: cannot import name 'DestinationExistsError' from 'fs.errors' (/cvmfs/cms.cern.ch/slc7_amd64_gcc10/external/py3-fs/2.4.13-d2879515d40a13e06bc31e28ce70961f/lib/python3.9/site-packages/fs/errors.py)

It was broken almost 3 years ago by #5256 . Python bindings for XRootD are not broken (or, at least, the simple import test works):

$ python3 -c "import XRootD"
$ echo $?
0

Since no one has complained since then, I believe it is safe to drop this package (together with py3-fs etc). @smuzaffar @davidlange6 @kpedro88 objections?

iarspider commented

Notice that this package is not tested in our IBs, although it was tested at some point in the past (see, e.g., cms-sw/cmssw#20629)

smuzaffar commented

@iarspider , looks like xrootdpyfs requires

I would suggest to drop xrootdpyfs along with fs package from cmssw disto

  • 👍1