lb-run DaVinci/v45r1
Closed this issue · 9 comments
Hi everyone,
This is a really great resource, thanks for creating it! I made an issue here as I can't seem to access LHCb Mattermost and am not quite sure where else to go for help.
I've been following through as a new user on this section and have environment issues running:
lb-run DaVinci/v45r1 ipython -i first.py 00070793_00000001_7.AllStreams.dstwhich cannot run due to the following exception
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007f6d6da3f740 (most recent call first):
<no Python frame>
Some details:
- tried zsh, switched default shell to bash, no luck
- i'm in the z5 computing group on lxplus
I thought I'd raise it here first before making a topic elsewhere, as you may know better than me where to get help on this. I realise it might not be an issue with the StarterKit but possibly indicative of larger issues with lb-run.
Maybe there's a newer version of the environment/DaVinci that I should be using? Many thanks in advance.
Looks like a python2/python3 issue to me. Could you try with ipython3?
Hi @olantwin,
I tried
[roneil@lxplus722 starterkit]$ lb-run DaVinci/v45r1 ipython3 -i first.py 00070793_00000001_7.AllStreams.dst
with no luck, same error. I'll paste the full output below if that is helpful:
[roneil@lxplus722 starterkit]$ lb-run DaVinci/v45r1 ipython3 -i first.py 00070793_00000001_7.AllStreams.dst
WARNING:lb-run:Decided best platform to use is x86_64-centos7-gcc8-opt
Python path configuration:
PYTHONHOME = '/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/Python/2.7.16/x86_64-centos7-gcc8-opt'
PYTHONPATH = '/cvmfs/lhcb.cern.ch/lib/lhcb/DAVINCI/DAVINCI_v45r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip:/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/ROOT/6.18.00/x86_64-centos7-gcc8-opt/lib:/cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/TurboStreamProd/v4r2p11/python:/cvmfs/lhcb.cern.ch/lib/lhcb/ANALYSIS/ANALYSIS_v21r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip:/cvmfs/lhcb.cern.ch/lib/lhcb/STRIPPING/STRIPPING_v14r4/InstallArea/x86_64-centos7-gcc8-opt/python.zip:/cvmfs/lhcb.cern.ch/lib/lhcb/PHYS/PHYS_v26r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip:/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/pyqt5/5.12/x86_64-centos7-gcc8-opt/lib/python2.7/site-packages:/cvmfs/lhcb.cern.ch/lib/lhcb/REC/REC_v24r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip:/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/COOL/3_2_1/x86_64-centos7-gcc8-opt/python:/cvmfs/lhcb.cern.ch/lib/lhcb/LBCOM/LBCOM_v23r0p1/InstallArea/x86_64-centos7-gcc8-opt/python.zip:/cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v45r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip:/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/CORAL/3_2_1/x86_64-centos7-gcc8-opt/python:/cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/RawEventFormat/v1r9/python:/cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/PRConfig/v1r45/python:/cvmfs/lhcb.cern.ch/lib/lhcb/GAUDI/GAUDI_v32r1/InstallArea/x86_64-centos7-gcc8-opt/python/lib-dynload:/cvmfs/lhcb.cern.ch/lib/lhcb/GAUDI/GAUDI_v32r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip'
program name = '/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/979/stable/linux-64/bin/python'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/979/stable/linux-64/bin/python'
sys.base_prefix = '/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/Python/2.7.16/x86_64-centos7-gcc8-opt'
sys.base_exec_prefix = '/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/Python/2.7.16/x86_64-centos7-gcc8-opt'
sys.executable = '/cvmfs/lhcb.cern.ch/lib/var/lib/LbEnv/979/stable/linux-64/bin/python'
sys.prefix = '/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/Python/2.7.16/x86_64-centos7-gcc8-opt'
sys.exec_prefix = '/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/Python/2.7.16/x86_64-centos7-gcc8-opt'
sys.path = [
'/cvmfs/lhcb.cern.ch/lib/lhcb/DAVINCI/DAVINCI_v45r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip',
'/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/ROOT/6.18.00/x86_64-centos7-gcc8-opt/lib',
'/cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/TurboStreamProd/v4r2p11/python',
'/cvmfs/lhcb.cern.ch/lib/lhcb/ANALYSIS/ANALYSIS_v21r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip',
'/cvmfs/lhcb.cern.ch/lib/lhcb/STRIPPING/STRIPPING_v14r4/InstallArea/x86_64-centos7-gcc8-opt/python.zip',
'/cvmfs/lhcb.cern.ch/lib/lhcb/PHYS/PHYS_v26r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip',
'/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/pyqt5/5.12/x86_64-centos7-gcc8-opt/lib/python2.7/site-packages',
'/cvmfs/lhcb.cern.ch/lib/lhcb/REC/REC_v24r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip',
'/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/COOL/3_2_1/x86_64-centos7-gcc8-opt/python',
'/cvmfs/lhcb.cern.ch/lib/lhcb/LBCOM/LBCOM_v23r0p1/InstallArea/x86_64-centos7-gcc8-opt/python.zip',
'/cvmfs/lhcb.cern.ch/lib/lhcb/LHCB/LHCB_v45r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip',
'/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/CORAL/3_2_1/x86_64-centos7-gcc8-opt/python',
'/cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/RawEventFormat/v1r9/python',
'/cvmfs/lhcb.cern.ch/lib/lhcb/DBASE/PRConfig/v1r45/python',
'/cvmfs/lhcb.cern.ch/lib/lhcb/GAUDI/GAUDI_v32r1/InstallArea/x86_64-centos7-gcc8-opt/python/lib-dynload',
'/cvmfs/lhcb.cern.ch/lib/lhcb/GAUDI/GAUDI_v32r1/InstallArea/x86_64-centos7-gcc8-opt/python.zip',
'/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/Python/2.7.16/x86_64-centos7-gcc8-opt/lib/python38.zip',
'/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/Python/2.7.16/x86_64-centos7-gcc8-opt/lib/python3.8',
'/cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/Python/2.7.16/x86_64-centos7-gcc8-opt/lib/python3.8/lib-dynload',
]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'
Current thread 0x00007facca8f4740 (most recent call first):
<no Python frame>
It looks like it is setting up python3.8 libraries and such from LCG_96 at /cvmfs/lhcb.cern.ch/lib/lcg/releases/LCG_96/Python/2.7.16/ which might mean that somewhere the wrong paths are being mixed together (also, the full paths don't exist)
I'm clueless.
You didn't touch PYTHONHOME and PYTHONPATH yourself?
Nope, PYTHONHOME and PYTHONPATH are set by lb-run in this case. I've not touched anything in my .bashrc either.
So in my view, there is definitely something wrong with lb-run DaVinci/v45r1. It doesn't seem to be setting up python properly.
Should I just try setting up a later version of DaVinci to read these DST files so I can get on with the tutorial, or will that not work?
I've had this problem for a few days, you can get round it by rolling back to python 2 but it's not a permanent solution.
https://lhcb-core-doc.web.cern.ch/lhcb-core-doc/LHCbEnvironment.html#rolling-back-to-python-2
Thanks @djdthompson, I'm glad I'm not the only one with the issue. I'll try that. I also raised an issue on the lhcb-core-soft mailing list, maybe they'll find a more permanent fix.
Hi, the problem is that DaVinci/v45r1 doesn't provide ipython in the environment, so you pick up the one provided by LbEnv, which now uses Python 3 so you cannot mix it with the DaVinci environment.
The two options you have at the moment are:
- use the Python 2 version of LbEnv
- temporary solution that kind of works by chance
- use
pythoninstead ofipython- less nice, but more reliable, as
ipythonwas never officially part of the runtime environment of LHCb Physics Applications, whilepythonis
- less nice, but more reliable, as
- add
--ext=ipythonto thelb-runcall:lb-run --ext=ipython DaVinci/v45r1 ...- you get a compatible ipython, but it's not guaranteed it works, as we had several issues with the ipython deployment (never officially supported)