How to predict probabilities in xgboost using R?

Question

The below predict function is giving -ve values as well so it cannot be probabilities.

param <- list(max.depth = 5, eta = 0.01,  objective="binary:logistic",subsample=0.9)
bst <- xgboost(param, data = x_mat, label = y_mat,nround = 3000)

pred_s <- predict(bst, x_mat_s2)

I google & tried pred_s <- predict(bst, x_mat_s2,type="response") but it didn't work.

Question

How to predict probabilities instead?

Answer 1

Just use predict_proba instead of predict. You can leave the objective as binary:logistic.

Answer 2

Know I'm a bit late, but to get probabilities from xgboost you should specify multi:softmax objective like this:

xgboost(param, data = x_mat, label = y_mat,nround = 3000, objective='multi:softprob')

From the ?xgb.train:

multi:softprob same as softmax, but output a vector of ndata * nclass, which can be further reshaped to ndata, nclass matrix. The result contains predicted probabilities of each data point belonging to each class.

Answer 3

After the prediction:

pred_s <- predict(bst, x_mat_s2)

You can get the probability by:

pred_s$data

If this is a binary classification then pred_s$data includes prob.0, prob.1, response.

So you can get prob.1 by:

pred_s$data$prob.1

Answer 4

Many years late, but noticed this; and this is how I do it:

pred_s <- predict(bst, x_mat_s2, type="prob")

Typically then I then work with probability of my upper class "1":

pred_s[,2]