Solver 'SCS' failed when there is solver SCS in the cvxpy package.
yonghyeokrhee opened this issue · 11 comments
machine: windows 10
python evn ; anaconda 5.4/ python 3.6
pip install fancyimpute
conda install cvxpy
It keeps saying that SolverError: Solver 'SCS' failed. Try another solver or solve with verbose=True for more information. Try recentering the problem data around 0 and rescaling to reduce the dynamic range.
INSTALLED_SOLVERS
['ECOS', 'ECOS_BB', 'SCS', 'OSQP']
mat_restored = NuclearNormMinimization().fit_transform(mat_destroyed)
data = np.array([[3, 4, 1, 1, 5, 2, 3, 4, np.nan, 5],
[1, 5, 4, 2, 1, 3, 1, np.nan, 2, 1],
[1, 2, 4, 3, 2, np.nan, 2, 4, 4, 5],
[5, 4, 5, 5, 2, 2, 4, np.nan, 1, 3],
[1, np.nan, 2, 4, 2, np.nan, 1, 3, 5, 2],
[3,4,np.nan,1,np.nan,1,5,4,2,3],
[1,3,5,np.nan,3,np.nan,2,4,1,5],
[1,2,np.nan,4,np.nan,3,2,4,4,1],
[2,5,1,2,4,np.nan,2,1,np.nan,np.nan],
[4,1,3,2,5,3,np.nan,2,3,1]])
---------------------------------------------------------------------------
SolverError Traceback (most recent call last)
<ipython-input-25-123fc995ba03> in <module>
----> 1 complete_image(a)
<ipython-input-19-074d2c1581c1> in complete_image(mat_destroyed)
1 def complete_image(mat_destroyed):
----> 2 mat_restored = NuclearNormMinimization().fit_transform(mat_destroyed)
3 return mat_restored
~\AppData\Roaming\Python\Python36\site-packages\fancyimpute\solver.py in fit_transform(self, X, y)
187 type(X_filled)))
188
--> 189 X_result = self.solve(X_filled, missing_mask)
190 if not isinstance(X_result, np.ndarray):
191 raise TypeError(
~\AppData\Roaming\Python\Python36\site-packages\fancyimpute\nuclear_norm_minimization.py in solve(self, X, missing_mask)
127 max_iters=self.max_iters,
128 # use_indirect, see: https://github.com/cvxgrp/cvxpy/issues/547
--> 129 use_indirect=False)
130 return S.value
~\AppData\Roaming\Python\Python36\site-packages\cvxpy\problems\problem.py in solve(self, *args, **kwargs)
245 else:
246 solve_func = Problem._solve
--> 247 return solve_func(self, *args, **kwargs)
248
249 @classmethod
~\AppData\Roaming\Python\Python36\site-packages\cvxpy\problems\problem.py in _solve(self, solver, ignore_dcp, warm_start, verbose, parallel, **kwargs)
360 solution = self._solving_chain.solve_via_data(self, data, warm_start, verbose,
361 kwargs)
--> 362 self.unpack_results(solution, self._solving_chain, inverse_data)
363 return self.value
364
~\AppData\Roaming\Python\Python36\site-packages\cvxpy\problems\problem.py in unpack_results(self, solution, chain, inverse_data)
470 "Try another solver or solve with verbose=True for more information. " +
471 "Try recentering the problem data around 0 and rescaling " +
--> 472 "to reduce the dynamic range."
473 )
474 self._status = solution.status
SolverError: Solver 'SCS' failed. Try another solver or solve with verbose=True for more information. Try recentering the problem data around 0 and rescaling to reduce the dynamic range.
Do all the fancyimpute tests pass locally on your machine?
Same issue happened for me... It occurs with the test code as well of the nuclear_norm_minimization.
Can you paste the error trace?
System: Windows 10
Python: 3.6
Inside a conda environment
~\Anaconda3\envs\pytorch\python.exe ~/Documents/GitHub/fancyimpute/test/test_nuclear_norm_minimization.py
Using TensorFlow backend.
Traceback (most recent call last):
SCS v2.1.2 - Splitting Conic Solver
File
~/Documents/GitHub/fancyimpute/test/test_nuclear_norm_minimization.py", line 57, in
(c) Brendan O'Donoghue, Stanford University, 2012
test_rank1_convex_solver()
Lin-sys: sparse-direct, nnz in A = 228
File /Documents/GitHub/fancyimpute/test/test_nuclear_norm_minimization.py", line 34, in test_rank1_convex_solver\Documents\GitHub\fancyimpute\fancyimpute\solver.py", line 186, in fit_transform
eps = 1.00e-04, alpha = 1.50, max_iters = 50000, normalize = 1, scale = 1.00
XY_completed_rank1 = solver.fit_transform(XY_missing_rank1)
acceleration_lookback = 0, rho_x = 1.00e-03
Variables n = 105, constraints m = 155
File "
Cones: primal zero / dual free vars: 25
linear vars: 75
X_result = self.solve(X_filled, missing_mask)
sd vars: 55, sd blks: 1
File "\Documents\GitHub\fancyimpute\fancyimpute\nuclear_norm_minimization.py", line 127, in solve\Anaconda3\envs\pytorch\lib\site-packages\cvxpy\problems\problem.py", line 290, in solve
FATAL: Cannot solve SDPs with > 2x2 matrices without linked blas+lapack libraries
use_indirect=False)
Install blas+lapack and re-compile SCS with blas+lapack library locations
ERROR: init_cone failure
File "
Setup time: 2.09e-04s
return solve_func(self, *args, **kwargs)
Failure:could not initialize work
File "\Anaconda3\envs\pytorch\lib\site-packages\cvxpy\problems\problem.py", line 575, in _solve\Anaconda3\envs\pytorch\lib\site-packages\cvxpy\problems\problem.py", line 718, in unpack_results
self.unpack_results(solution, full_chain, inverse_data)
File "
"Try another solver, or solve with verbose=True for more "
cvxpy.error.SolverError: Solver 'SCS' failed. Try another solver, or solve with verbose=True for more information.
Weird. I guess something went wrong with CVXPY. We're not really supporting this package anymore unfortunately. You feel like mucking about with other solvers to see if the tests pass?
I'll see what I can do.
PS: sorry for the messed up traceback, I did not realize it is doing markdown formatting.
I finally managed to make it run on windows.
The way I managed to fix it was mainly by installing cvxpy with its dependencies from the unofficial Windows binaries except for SCS that I used the pip install method.
It is still not performing as well as the one I installed on Linux. Apparently cvxpy overflows some shape variables when data is big while on Linux it just converts to a bigger variable size (or maybe it starts with a long in from the beginning. Hopefully the updated Windows binaries could handle this better.
Currently, I am only able to solve very small data sets.
Well that sucks, sorry. Doesn't really seem like there's much we can do on our end except: (a) remove this algorithm entirely or (b) see if other solvers work better (not SCS).
That's fine, when I actually tested this method on Linux on my simulated data, it takes too long while not outperforming other algorithms that I even removed it from my codes. Thanks a lot though for the discussion.
Sure, thanks for your investigation! I'll reopen this ticket so others can see in case they have issues.