Doing gradient descent directly on input features