mathurinm/celer

Segmentation fault when fitting `GroupLasso`

PABannier opened this issue · 6 comments

I just updated to Celer 0.7dev to benefit from the weights argument in GroupLasso.
After updating, I now ran into a segmentation fault error when fitting GroupLasso.

Here is a minimal reproducible example:

import numpy as np
from celer import GroupLasso

X = np.random.normal(0, 1, (30, 60))
y = np.random.normal(0, 1, (30,))

clf = GroupLasso(3, 0.001, tol=1e-10, fit_intercept=False)
clf.fit(X, y)

@Badr-MOUFAD Are you able to reproduce this error on your machine?

Note that Lasso is working perfectly, and that the error occurs only with GroupLasso with or without the weights argument.

@PABannier, The example works well for me.
Can you share the error logs that you get when running the code snippets?

[1]    92335 segmentation fault  ipython

The error is not very verbose. It seems to be a common behavior when C programs try to access memory beyond reach (https://stackoverflow.com/questions/10035541/what-causes-a-python-segmentation-fault) . I tried a bunch of configurations and was able to make it work with 300 features, 100 samples and a group size of 100 (group size of 3 fails for instance).

From my experiments, when too much memory is allocated for large datasets, a segmentation fault occurs.

It's not likely to be a memory overflow, rather an issue with one BLAS routine reading outside of the program's memory.

I can't reproduce. @PABannier did you recompile the cython code after pulling ? You need to execute pip install -e .

I did, I tried to run pip install -e . again. This time, I am told that weights is not supported for GroupLasso. I checked celer.__version__, I do have 0.7dev, and in the source I do see a weights argument in GroupLasso...

After switching everywhere to @cython.checkbounds(True), I have a more detailed traceback:

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
IndexError: Out of bounds on buffer access (axis 0)
Exception ignored in: 'celer.group_fast.dnorm_grp'
Traceback (most recent call last):
  File "/Users/pierre-antoine/Desktop/celer/celer/homotopy.py", line 291, in celer_path
    sol = celer_grp(
IndexError: Out of bounds on buffer access (axis 0)

Indeed, I have a buffer overflow when running celer_grp.

I tried on another machine (Ubuntu with a clean installation of GCC), everything works like a charm.
I suspect I have conflicting installations, on my OS X System. I had similar seg faults in the past, so I'm closing the issue since it is not related to Celer.