Segmentation fault when fitting `GroupLasso`
PABannier opened this issue · 6 comments
I just updated to Celer 0.7dev to benefit from the weights
argument in GroupLasso
.
After updating, I now ran into a segmentation fault error when fitting GroupLasso
.
Here is a minimal reproducible example:
import numpy as np
from celer import GroupLasso
X = np.random.normal(0, 1, (30, 60))
y = np.random.normal(0, 1, (30,))
clf = GroupLasso(3, 0.001, tol=1e-10, fit_intercept=False)
clf.fit(X, y)
@Badr-MOUFAD Are you able to reproduce this error on your machine?
Note that Lasso
is working perfectly, and that the error occurs only with GroupLasso
with or without the weights
argument.
@PABannier, The example works well for me.
Can you share the error logs that you get when running the code snippets?
[1] 92335 segmentation fault ipython
The error is not very verbose. It seems to be a common behavior when C programs try to access memory beyond reach (https://stackoverflow.com/questions/10035541/what-causes-a-python-segmentation-fault) . I tried a bunch of configurations and was able to make it work with 300 features, 100 samples and a group size of 100 (group size of 3 fails for instance).
From my experiments, when too much memory is allocated for large datasets, a segmentation fault occurs.
It's not likely to be a memory overflow, rather an issue with one BLAS routine reading outside of the program's memory.
I can't reproduce. @PABannier did you recompile the cython code after pulling ? You need to execute pip install -e .
I did, I tried to run pip install -e .
again. This time, I am told that weights
is not supported for GroupLasso
. I checked celer.__version__
, I do have 0.7dev
, and in the source I do see a weights
argument in GroupLasso
...
After switching everywhere to @cython.checkbounds(True)
, I have a more detailed traceback:
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
IndexError: Out of bounds on buffer access (axis 0)
Exception ignored in: 'celer.group_fast.dnorm_grp'
Traceback (most recent call last):
File "/Users/pierre-antoine/Desktop/celer/celer/homotopy.py", line 291, in celer_path
sol = celer_grp(
IndexError: Out of bounds on buffer access (axis 0)
Indeed, I have a buffer overflow when running celer_grp
.
I tried on another machine (Ubuntu with a clean installation of GCC), everything works like a charm.
I suspect I have conflicting installations, on my OS X System. I had similar seg faults in the past, so I'm closing the issue since it is not related to Celer.