AttributeError: 'tuple' object has no attribute 'append'

[IanSudbery]

I'm getting this error on running one of my trackers:

Traceback (most recent call last):
  File "/ifs/devel/Ian/python/bin/cgatreport-test", line 9, in <module>
    load_entry_point('CGATReport', 'console_scripts', 'cgatreport-test')()
  File "/ifs/apps/apps/python-2.7.11-gcc-5.2.1/lib/python2.7/site-packages/CGATReport/test.py", line 508, in main
    result = dispatcher(**kwargs)
  File "/ifs/apps/apps/python-2.7.11-gcc-5.2.1/lib/python2.7/site-packages/CGATReport/Dispatcher.py", line 677, in __call__
    self.data = DataTree.asDataFrame(self.tree)
  File "/ifs/apps/apps/python-2.7.11-gcc-5.2.1/lib/python2.7/site-packages/CGATReport/DataTree.py", line 468, in asDataFrame
    df = concatDataFrames(dataframes, index_tuples)
  File "/ifs/apps/apps/python-2.7.11-gcc-5.2.1/lib/python2.7/site-packages/CGATReport/DataTree.py", line 239, in concatDataFrames
    df = pandas.concat(dataframes, keys=index_tuples)
  File "/ifs/devel/Ian/python/lib/python2.7/site-packages/pandas/tools/merge.py", line 1325, in concat
    copy=copy)
  File "/ifs/devel/Ian/python/lib/python2.7/site-packages/pandas/tools/merge.py", line 1464, in __init__
    self.new_axes = self._get_new_axes()
  File "/ifs/devel/Ian/python/lib/python2.7/site-packages/pandas/tools/merge.py", line 1552, in _get_new_axes
    new_axes[self.axis] = self._get_concat_axis()
  File "/ifs/devel/Ian/python/lib/python2.7/site-packages/pandas/tools/merge.py", line 1609, in _get_concat_axis
    self.levels, self.names)
  File "/ifs/devel/Ian/python/lib/python2.7/site-packages/pandas/tools/merge.py", line 1675, in _make_concat_multiindex
    levels.append(categories)
AttributeError: 'tuple' object has no attribute 'append'

To replicate run

cgatreport-test --path=../src/pipeline_docs/pipeline_proj028/trackers/ -t FirstLastExonCount 

in

/ifs/projects/proj028/project_pipeline_iCLIP5

code of the tracker is here:

/ifs/devel/projects/proj028/pipeline_docs/pipeline_proj028/trackers/NormalisedProfiles.py

[AndreasHeger]

This is an issue with pandas 0.19 I think.

Previously (in pandas 0.18.1), the following worked:

pandas.concat(dataframes, keys=[("a", "b"), ("a", "c")])

Now (pandas 0.19.0), an AttributeError is raised. I have not found a way around. For single level indices, the following works:

pandas.concat(dataframes, keys=["a", "b"])

[AndreasHeger]

I have released a new version that excludes pandas 0.19.0 from requirements (and fixes some py2/py3 issues)

[IanSudbery]

I andreas, I think something more than that is going on, because this:

df1 = pandas.DataFrame({"one":[1,2], "two":[3,4]})
df2 = df1.copy()
pandas.concat([df1,df2], keys=[("a","b"), ("a","c")])

works for me on pandas 0.19.0.

Your fix seems to check for length 1 tuples, but this:

df1 = pandas.DataFrame({"one":[1,2], "two":[3,4]})
df2 = df1.copy()
pandas.concat([df1,df2], keys=[("a",), ("a",)])

Also works for me.

However, 821851a, does fix the immediate problem

Plus, i'm pretty sure you have a requirement in their somewhere that requires the lastest ggplot, and the latest ggplot requires the latest pandas.

[IanSudbery]

A further clue:

So 821851a fixes the immediate problem, but I think it might reveal the underlying cause:

Now when I run the tracker, I should have two tracks 'FLAG' and 'GFP'. But what I'm getting is 'F/L/A/G' and 'G/F/P' - it looks like at some point the string track names are being split into their constituent characters - presumably the 'tuples' of the error message.

[AndreasHeger]

Thanks. 0.19.0 is the issue, see release notes:

http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#whatsnew-0191
'''
Bug in pd.concat where names of the keys were not propagated to the resulting MultiIndex (GH14252)
'''

0.19.1 does indeed work.

[AndreasHeger]

The label thing is another issue. I ran 2to3 on CGATReport and this has messed up a few things.