8

Documentation here.

I'm wondering how important the coef0 parameter is for SVCs under the polynomial and sigmoid kernels. As I understand it, it is the intercept term, just a constant as in linear regression to offset the function from zero. However to my knowledge, the SVM (scikit uses libsvm) should find this value.

What's a good general range to test over (is there one?). For example, generally with C, a safe choice is 10^-5 ... 10^5, going up in exponential steps.

But for coef0, the value seems highly data dependent and I'm not sure how to automate choosing good ranges for each grid search on each dataset. Any pointers?

lollercoaster
  • 15,969
  • 35
  • 115
  • 173

1 Answers1

4

First, sigmoid function is rarely the kernel. In fact, for almost none values of parameters it is known to induce the valid kernel (in the Mercer's sense).

Second, coef0 is not an intercept term, it is a parameter of the kernel projection, which can be used to overcome one of the important issues with the polynomial kernel. In general, just using coef0=0 should be just fine, but polynomial kernel has one issue, with p->inf, it more and more separates pairs of points, for which <x,y> is smaller than 1 and <a,b> with bigger value. it is because powers of values smaller than one gets closer and closer to 0, while the same power of value bigger than one grows to infinity. You can use coef0 to "scale" your data so there is no such distinction - you can add 1-min <x,y>, so no values are smaller than 1 . If you really feel the need for tuning this parameter, I would suggest search in the range of [min(1-min , 0),max(<x,y>)], where max is computed through all the training set.

lejlot
  • 64,777
  • 8
  • 131
  • 164
  • I don't fully understand your first sentence - are you saying for sigmoid kernel that this is a useless parameter to optimize over? And from the rest of your comment, would you say it is fair to just use 1 always for polynomial kernels to avoid values < 1? – lollercoaster Jan 27 '14 at 22:59
  • 1
    First statement states, that sigmoid **is not a kernel** for most of the parameters. It is not about the tuning, it is generally **wrong function to use with kernel machines**. It was introduced to make neural network community more familiar with svms, but that was not a good idea. For poly, I would say it is save to check coef0 values 0 and 1. As both can have some good properties, but I would leave checking other values. – lejlot Jan 28 '14 at 06:43
  • 1
    I disagree with the general argument that sigmoid is the wrong kernel of choice. Everything starts from data. If your data maps well with sigmoid, then sigmoid is your choice. I have had cases where sigmoid proved to be the right function to model my data. – Milad M Apr 14 '17 at 00:04
  • @Milad There is nothing to disagree here, sigmoid **is not a kernel** from mathematical perspective, this is a fact, not an opinion. Of course things can work from time to time even if they are incorrect, but it does not make them valid. – lejlot Apr 19 '17 at 22:16
  • It is correct that sigmoid does not fit the definition of a kernel function. That is, sigmoid is not a positive definite function. – Milad M Apr 28 '17 at 22:40