embotech/ecos

ECOS solver returns unsual results when used to train a soft-margin SVM

Closed this issue · 1 comments

CVXPY version: v1.0

Just thought of seeking some feedback on a bit of an unusual observation that I encountered while training a soft-margin SVM using the ECOS solver via Cvxpy framework. The implementation of the aforementioned soft-margin SVM goes below. The dataset I've used is a real-world dataset slightly modified to be able to train a binary SVM on it.

n = 50
C = Parameter(sign="positive")
C.value = 0.75
beta = cp.Variable(n, name="x")  # x in our terminology
v = cp.Variable(name="v")  # b in our terminology
Epsilon = cp.Variable(len(X_train), 1, name="Epsilon")  # punishing factor

objective = Minimize(sum_entries(square(beta)) + C * norm(Epsilon, 1))
g = C * cp.norm(Epsilon, 1)
for i in range(n):
  g = g + 0.5 * cp.square(beta[i])

constraints = [Epsilon >= 0]
for j in range(len(X_train)):
   constraints = constraints + [y_train[j] * (cp.sum_entries(cp.mul_elemwise(np.asmatrix(X_train[j]).T, beta)) + v) >= 1 - Epsilon[j]]
prob = Problem(Minimize(50*g), constraints)
try:
	prob.solve(solver=ECOS)
except SolverError as e:
	print(e)

I understand that different solvers use different ways of solving a given problem thereby returning different results. However, the results returned by the aforementioned implementation ranges around 98% for the given dataset, which is unusually high compared to the results returned by other implementations such as libsvm, etc, which were about 50%. Therefore, wondering if I might be missing something obvious.

Appreciate any input on this.

Turns out that the ECOS solver maybe solving a different problem than what I am looking at. Therefore, closing this ticket.