scikit-hep/root_numpy

array2tree error

ascarff opened this issue · 18 comments

ascarff commented

Hi,

I'm using a combination of tree2array, doing some stuff to the array and then converting back with array2tree. The tree2array side works fine but I can't convert back again I get the error: "TypeError: object of type 'NoneType' has no len()".

The section of code I'm using is below, with 'det' being my TTree and 'HVvolts' being one of the branches. The array modHV is calculated correctly and gives me what I need. I just can't convert it back to a branch of my TTree.

import numpy
from root_numpy import tree2array, array2tree

HV = tree2array(det,branches = "HVvolts")
t_c = 0.83
modHV = (numpy.mod(-HV,t_c))
print "length of modHV:",len(modHV)
print "modHV type:",type(modHV)
array2tree(modHV,tree=det)

This prints out:

length of modHV: 113385
modHV type: <type 'numpy.ndarray'>

Then the error:

TypeErrorTraceback (most recent call last)
<ipython-input-6-b65205e494d9> in <module>()
      7 print "length of modHV:",len(modHV)
      8 print "modHV type:",type(modHV)
----> 9 array2tree(modHV,tree=zip4)

/usr/local/lib/python2.7/site-packages/root_numpy/_tree.pyc in array2tree(arr, name, tree)
    569     else:
    570         incobj = None
--> 571     cobj = _librootnumpy.array2tree_toCObj(arr, name=name, tree=incobj)
    572     return ROOT.BindObject(cobj, 'TTree')
    573 

root_numpy/src/tree.pyx in _librootnumpy.array2tree_toCObj (root_numpy/src/_librootnumpy.cpp:716)()
root_numpy/src/tree.pyx in _librootnumpy.array2tree (root_numpy/src/_librootnumpy.cpp:701)()
root_numpy/src/tree.pyx in _librootnumpy.array2tree (root_numpy/src/_librootnumpy.cpp:673)()

TypeError: object of type 'NoneType' has no len()

So the object has both a type (numpy.ndarray) and a length (113385). Does anyone know what I'm doing wrong here?

Thanks!

kratsg commented

What does modHV.dtype give you? And did you check that none of the values are None?

ascarff commented

modHV.dtype gives dtype('float64'). The original branch were doubles so that's as expected. None of the values are None.

kratsg commented

This is a dumb sanity check, but can you try

array2tree(modHV)

instead and see if that works? I guess I'm confused because of this line

array2tree(modHV,tree=zip4)

and I'm not sure where zip4 comes from.

ascarff commented

Ah sorry, that should be array2tree(modHV,tree=det), I tried with a couple of different trees and must've missed one when changing some bits at the end.

I get the same error when I do array2tree(modHV).

kratsg commented

Which version of root_numpy are you using?

kratsg commented

Also, can you run

modHV.dtype.names and modHV.dtype.fields? I suspect you might not have any field names, which might cause this error...

ascarff commented

I'm on root_numpy-4.7.3.

Both modHV.dtype.names and modHV.dtype.fields give no result!

kratsg commented

Both modHV.dtype.names and modHV.dtype.fields give no result!

There's your problem. You need to specify column names here. What you have is basically a list of values that you're asking to be converted to a TTree.. but we don't know what to set the branch names to. Specifically, the code is failing on this line (https://github.com/scikit-hep/root_numpy/blob/master/root_numpy/src/tree.pyx#L673) which does need some protection. Try something like..

modHV.dtype.names = ['HVvolts_modified']

before passing to the array2tree call.

ascarff commented

Then is says: ValueError: there are no fields defined. Do I need to put something for modHV.dtype.names too?

kratsg commented

Looks like you need to force it with something like

modHV.dtype = [('HVvolts_modified', 'float64')]

(Note this is now a numpy issue rather than a root_numpy problem.)

ascarff commented

Ok, so now I'm using:

import numpy
from root_numpy import tree2array, array2tree

HV = tree2array(zip4,branches = "HVvolts")
t_c = 0.83
modHV = (numpy.mod(-HV,t_c))
modHV.dtype = [('HVvolts_modified', 'float64')]
modHV.dtype.names = ['HVvolts_modified']
array2tree(modHV,tree=zip4)

And I get:

<ROOT.TTree object ("zip4") at 0x7fbc27fb1e50>
Error in <TBranch::TBranch::WriteBasketImpl>: basket's WriteBuffer failed.
Error in <TBranch::TBranch::Fill>: Failed to write out basket.

With the errors repeated multiple times

kratsg commented

Yes, because I don't think you can write out to the same tree that already exists if you open in READ only mode... You can do this:

newTree = array2tree(modHV)

and it should work as expected with newTree. If you want to add a new branch to an existing tree, you need to make sure you're opening the file in RECREATE/UPDATE mode to allow you to write out to the existing tree in memory. See #224 for an example of how to do it. Specifically:

import numpy, root_numpy
from rootpy.io import root_open
root_numpy.array2root(numpy.array(numpy.random.random([1000, 2]), dtype=[('a', 'float64'), ('b', 'float64')] ),
                      'test.root', mode='recreate')
with root_open('test.root', mode='a') as myfile:
    new_column = numpy.array(numpy.ones([1000, 1]) , dtype=[('new', 'f8')])
    root_numpy.array2tree(new_column, tree=myfile.tree)
    myfile.write()
ascarff commented

Hmm, it actually seems to be working adding it to a new tree. It just throws up these errors. They are still there when I tried newTree = array2tree(modHV). I'm not sure what the error is actually saying.

kratsg commented

I suspect because it's just creating the tree in memory instead of being able to write to an open, writeable file. Can you check whether you open the file for reading or reading+writing?

ascarff commented

Ah ok, yeah, that might be it. My current code is:
file = ROOT.TFile.Open("datafile.root")

kratsg commented

Try

file = ROOT.TFile.Open("datafile.root", "UPDATE")
ascarff commented

Ok, that has done it! Thank you very much for all the help and such quick replies! 😃

ndawe commented

Thanks @kratsg !