fggManageSamples.py review crashes on splitting step if files removed
Closed this issue · 3 comments
sethzenz commented
Now that the catalogue splitting is in from #804 , if I run this command:
fggManageSamples.py -C RunIISummer16-2_4_1-25ns_Moriond17 review
And then remove the EGM0 datasets, I get a crash:
Traceback (most recent call last):
File "/afs/cern.ch/work/s/sethzenz/fromscratch142/CMSSW_8_0_26_patch1/bin/slc6_amd64_gcc530/fggManageSamples.py", line 7, in <module>
run()
File "/afs/cern.ch/work/s/sethzenz/fromscratch142/CMSSW_8_0_26_patch1/python/flashgg/MetaData/samples_utils.py", line 963, in __call__
method()
File "/afs/cern.ch/work/s/sethzenz/fromscratch142/CMSSW_8_0_26_patch1/python/flashgg/MetaData/samples_utils.py", line 1053, in run_review
self.mn.reviewCatalog(pattern)
File "/afs/cern.ch/work/s/sethzenz/fromscratch142/CMSSW_8_0_26_patch1/python/flashgg/MetaData/samples_utils.py", line 462, in reviewCatalog
self.writeCatalog(catalog)
File "/afs/cern.ch/work/s/sethzenz/fromscratch142/CMSSW_8_0_26_patch1/python/flashgg/MetaData/samples_utils.py", line 777, in writeCatalog
self.sortDatasetFiles(content,rebuild)
File "/afs/cern.ch/work/s/sethzenz/fromscratch142/CMSSW_8_0_26_patch1/python/flashgg/MetaData/samples_utils.py", line 728, in sortDatasetFiles
entriesPerFile[ifile] = sum( map(entriesPerDataset.get,idatasets) )
TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'
I tried printing out map(entriesPerDataset.get,idatasets) and got:
[1006]
[20, 10, 10, 9, 19, 5, 3, 15, 8, 816, 2, 3, 7, 9, 8, 9, 5, 8, 11, 17, 3]
[38, 11, 8, 12, 145, 8, 45, 5, 5, 5, None, 11, 3, 57, 17, 15, 24, 441, 5, 6, 5, 7]
@musella Do you know how to fix this?
sethzenz commented
The following change appears to fix it, but will it do the right thing?
- entriesPerFile[ifile] = sum( map(entriesPerDataset.get,idatasets) )
+ theMap = map(entriesPerDataset.get,idatasets)
+ theMap = [0 if x==None else x for x in theMap]
+ entriesPerFile[ifile] = sum( theMap )
musella commented
Hi,
the fix is fine. I would need to investigate why the number of files in a
dataset is None, but maybe it is because of the deletion, which I hadn't
tested.
Cheers,
P.
…On 10 February 2017 at 19:58, Seth Zenz ***@***.***> wrote:
The following change appears to fix it, but will it do the right thing?
- entriesPerFile[ifile] = sum( map(entriesPerDataset.get,idatasets) )
+ theMap = map(entriesPerDataset.get,idatasets)
+ theMap = [0 if x==None else x for x in theMap]
+ entriesPerFile[ifile] = sum( theMap )
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#846 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAm_KQbiIZ8mIKtfApU4Cf-S1wzNBf7yks5rbLNZgaJpZM4L9qpa>
.
--
Pasquale Musella - ETHZ
http://musella.web.cern.ch/musella