[ad_1]

Mathematically, optimizing an SVM is a convex optimization problem, usually with a unique minimizer. This means that there is only one solution to this mathematical optimization problem.

The differences in results come from several aspects: `SVC`

and `LinearSVC`

are supposed to optimize the same problem, but in fact all `liblinear`

estimators penalize the intercept, whereas `libsvm`

ones don’t (IIRC). This leads to a different mathematical optimization problem and thus different results. There may also be other subtle differences such as scaling and default loss function (edit: make sure you set `loss="hinge"`

in `LinearSVC`

). Next, in multiclass classification, `liblinear`

does one-vs-rest by default whereas `libsvm`

does one-vs-one.

`SGDClassifier(loss="hinge")`

is different from the other two in the sense that it uses stochastic gradient descent and not exact gradient descent and may not converge to the same solution. However the obtained solution may generalize better.

Between `SVC`

and `LinearSVC`

, one important decision criterion is that `LinearSVC`

tends to be faster to converge the larger the number of samples is. This is due to the fact that the linear kernel is a special case, which is optimized for in Liblinear, but not in Libsvm.

[ad_2]