Assignment 3: "numpy.ufunc has the wrong size, try recompiling. Expected 192, got 216" numpy version issues
swyxio opened this issue · 2 comments
swyxio commented
probably too late for us but a few of us are getting this cryptic error:
Traceback (most recent call last):
File "run_experiment.py", line 234, in <module>
timings,
File "run_experiment.py", line 41, in run_experiment
exp.perform_cluster(dim)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/experiments/RF.py", line 85, in perform_cluster
hdf_path = self.dump_for_clustering(lambda x: filtr.fit_transform(x.get_details().ds.training_x,
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/experiments/base.py", line 105, in dump_for_clustering
ds_2.to_hdf(hdf_path, self._details.ds_name, complib='blosc', complevel=9)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/pandas/core/generic.py", line 1996, in to_hdf
return pytables.to_hdf(path_or_buf, key, self, **kwargs)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/pandas/io/pytables.py", line 278, in to_hdf
complib=complib) as store:
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/pandas/io/pytables.py", line 469, in __init__
import tables # noqa
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/tables/__init__.py", line 93, in <module>
from .utilsextension import (
File "__init__.pxd", line 872, in init tables.utilsextension
ValueError: numpy.ufunc has the wrong size, try recompiling. Expected 192, got 216
/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
this sounds like a situation where tables dont work? or is there a way to resolve this?
swyxio commented
after uninstalling and reinstalling numpy, i now get this different table related error:
/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/scipy/linalg/basic.py:1321: RuntimeWarning: internal gelsd driver lwork query error, required iwork dimension not returned. This is likely the result of LAPACK bug 0038, fixed in LAPACK 3.2.2 (released July 21, 2010). Falling back to 'gelss' driver.
x, resids, rank, s = lstsq(a, b, cond=cond, check_finite=False)
Traceback (most recent call last):
File "run_experiment.py", line 207, in <module>
timings,
File "run_experiment.py", line 41, in run_experiment
exp.perform_cluster(dim)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/experiments/ICA.py", line 80, in perform_cluster
preprocess=False)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/data/loader.py", line 226, in reload_from_hdf
df = pd.read_hdf(hdf_path, hdf_ds_name)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/pandas/io/pytables.py", line 394, in read_hdf
return store.select(key, auto_close=auto_close, **kwargs)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/pandas/io/pytables.py", line 741, in select
return it.get_result()
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/pandas/io/pytables.py", line 1483, in get_result
results = self.func(self.start, self.stop, where)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/pandas/io/pytables.py", line 734, in func
columns=columns)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/pandas/io/pytables.py", line 2928, in read
ax = self.read_index('axis%d' % i, start=_start, stop=_stop)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/pandas/io/pytables.py", line 2523, in read_index
_, index = self.read_index_node(getattr(self.group, key), **kwargs)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/pandas/io/pytables.py", line 2621, in read_index_node
data = node[start:stop]
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/tables/vlarray.py", line 685, in __getitem__
return self.read(start, stop, step)
File "/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/tables/vlarray.py", line 821, in read
listarr = self._read_array(start, stop, step)
File "tables/hdf5extension.pyx", line 2155, in tables.hdf5extension.VLArray._read_array
ValueError: cannot set WRITEABLE flag to True of this array
/Users/swyx/Desktop/OMSCS/ML/ASSIGNMENT3/swyx-a3/py3/lib/python3.7/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
swyxio commented
ok so it seems the version numbers conflict a lot especiallywhen it comes to numpy. you have 1.15 but a lot of things assume 1.16 (the current version at time of writing).
so you have to uninstall and reinstall everything again: https://stackoverflow.com/questions/11248073/what-is-the-easiest-way-to-remove-all-packages-installed-by-pip
my requirements from pip freeze:
cycler==0.10.0
kiwisolver==1.0.1
kneed==0.2.4
matplotlib==3.0.3
mock==2.0.0
numexpr==2.6.9
numpy==1.16.2
pandas==0.24.2
pbr==5.1.3
pyparsing==2.3.1
python-dateutil==2.8.0
pytz==2018.9
scikit-learn==0.20.3
scikit-optimize==0.5.2
scipy==1.2.1
seaborn==0.9.0
six==1.12.0
tables==3.5.1
xlrd==1.2.0