0

I had some trouble finding a good transductive svm (semi-supervised support vector machine or s3vm) implementation for python. Finally I found the implementation of Fabian Gieseke of Oldenburg University, Germany (code is here: https://www.ci.uni-oldenburg.de/60506.html, paper title: Fast and Simple Gradient-Based Optimization for Semi-Supervised Support Vector Machines).

I now try to integrate the learned model into my scikit-learn code.

1) This works already: I've got a binary classification problem. I defined a new method inside the S3VM-code returning the self.__c-coeficients (these are needed for the decision function of the classifier). I then assign these (in my own scikit-code where clf stands for a svm.SVC-classifier) to clf.dual_coefs_ and properly change clf.support_ too (which holds the indices of the support vectors). It takes a while because sometimes you need numpy-arrays and sometimes lists etc. But it works quite well.

2) This doesnt work at all: I want to adapt this now to a multi-class-classification problem (again with an svm.SVC-classifier). I see the scheme for multi-class dual_coef_ in the docs at http://scikit-learn.org/stable/modules/svm.html I tried some things already but seem to mess it up all the time. My strategy is as follows:

for all pairs in classes:
   calculate the coefficients with qns3vm for the properly binarized labeled training set (filling 0s into spaces in the coef-vector where instances have been in the labeled training set that are not in the current class-pair) --> get a 1x(l+u)-np.array of coefficients
horizontally stack these to get a (n_class*(n_class-1)/2)x(l+u) matrix | I do not have a clue why the specs say that this should be of shape [n_class-1, n_SV(=l+u)]?
replace clf.dual_coef_ with this matrix

Does anybody know the right way to replace dual_coef_ in the multi-class-setting? Or is there a neat piece of example code someone can recommend? Or at least a better explanation for the shape of dual_coef_ in the one-vs-one-multiclass-setting?

Thanks!

Damian

Damian
  • 139
  • 3
  • 13
  • Maybe [this question and answer](http://stackoverflow.com/questions/22816646/the-dimension-of-dual-coef-in-sklearn-svc/) can help? Also, it appears that it can be difficult to create an estimator with just this array, so you may want to consider [calling libsvm directly for predictions](http://stackoverflow.com/questions/22815536/set-the-weights-of-decision-functions-through-stdin-in-sklearn/) – eickenberg May 27 '14 at 21:44
  • I now build a dual_coef_ - matrix as you state in your post ( is there a typo in "... # for problems 0v1, 1v2, ..., 1v(n-1) so n-1 columns for each of the svc.n_support_[1] support vectors..."? )shouldn't 'columns' be replaced by 'rows'? And I altered clf.support_, but now python crashes when trying to evaluate the classifier. | Additionally: When I look at dual_coef_ of another classifier I see a lot of 1s, 0s and -1s. Do these have any special meaning? – Damian May 28 '14 at 08:26
  • I'm pretty sure that in `sklearn.svm.SVC` the `dual_coef_` have `n - 1` columns, where `n` is the number of classes, and where, after chopping the rows into blocks corresponding to each class (indicated by `n_support_`) each column encodes the separating hyperplane of that class with respect to the `n-1` others. Each row corresponds to one support vector and the coefficients indicate to what extent it is implicated in the hyperplanes. If the coefficient is `0.`, then it isn't, in all other cases it is. – eickenberg May 28 '14 at 22:00
  • It is very difficult to create an `SVC` estimator for prediction without using `fit`, i.e. by just setting attributes. See my second link for a - somewhat dirty and unstable - workaround which is to call the libsvm predict directly. Note that if you are using a linear kernel, then there is no need for all that. – eickenberg May 28 '14 at 22:02

0 Answers0