Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [ensemble]

37 questions
6
votes
2 answers

Gridsearch XGBoost for ensemble. Do I include first-level prediction matrix of base learners in train set?

I'm not quite sure how I should go about tuning xgboost before I use it as a meta-learner in ensemble learning. Should I include the prediction matrix (ie. df containing columns of prediction results from the various base learners) or should I just…
doyz
  • 161
  • 3
6
votes
3 answers

Is it possible to build ensemble models without a decision tree?

Is it possible to build ensemble models without a decision tree? I know that the description of ensembles itself suggests otherwise. However, I am really new to machine learning and all the ensemble models I came across so far use or are described…
Raveen Diaz
  • 61
  • 2
5
votes
2 answers

Bagging vs pasting in ensemble learning

This is a citation from "Hands-on machine learning with Scikit-Learn, Keras and TensorFlow" by Aurelien Geron: "Bootstrapping introduces a bit more diversity in the subsets that each predictor is trained on, so bagging ends up with a slightly higher…
chekhovana
  • 51
  • 2
5
votes
1 answer

Methods for ensembling ranked lists?

I was wondering if there's a good way to use ensembling when I have two or more algoritims producing ranked lists. That is, suppose I have the following datasets consisting of ordered lists (higher to the top means more relevant): Method1_Rankings …
user3490622
  • 151
  • 2
4
votes
1 answer

Visualizing F-score differences in information extraction

I have several corpora and NLP systems (including a few merge ensembles of output of these systems combined in unions and intersections) with which I have extracted the annotation span sets {(begin, end)} for each corpus across all documents within…
horcle_buzz
  • 201
  • 1
  • 6
4
votes
1 answer

Geometric and harmonic means in ensembling methods

When using ensembling methods for regression, a common approach is to average (using the arithmetic mean) the outputs of the weak learners in order to obtain the output of the ensemble. Is there a reason why the average that people take is the…
David Masip
  • 6,136
  • 2
  • 28
  • 62
4
votes
3 answers

How to create an ensemble that gives precedence to a specific classifier

Suppose that in a binary classification task, I have separate classifiers A, B, and C. If I use A alone, I will get a high precision, but low recall. In other words, the number of true positives are very high, but it also incorrectly tags the rest…
Clement Attlee
  • 141
  • 1
3
votes
1 answer

Physical modelling with neural networks - single output + stack ensemble vs multi-output

We are trying to replace an existing physical model (8 inputs/7 outputs) with artificial neural networks. The physics behind the existing model is mainly thermodynamics of humid air for air conditioning, with some turbomachinery involved, which…
fernikalo
  • 31
  • 2
3
votes
2 answers

grid search - optimal weighting of classifiers

I am using three different of the shelf classifiers. It's a three class classification task. I want to calculate the optimal weights (c1weight, c2weight, c3weight) for each classifier (real task more classifiers and also weights for each…
bartman99
  • 51
  • 5
3
votes
3 answers

Combining Classifiers with different Precision and Recall values

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is high and the recall is low, whereas the precision…
user8115948
  • 51
  • 4
3
votes
2 answers

xgboost cannot identify perfectly fitting regression line

For a dataset I want to use xgboost for the optimal ensembling of $n$ forecasts instead of just using their arithmetic mean for combination. I found that xgboost generates forecasts that are worse than many of the $n$ individual forecasts the moedl…
HOSS_JFL
  • 149
  • 3
2
votes
1 answer

How do I combine predictions from classifiers for two different problem?

I am working on a classification problem for predicting whether the shipment is going to be late or not. I would say the classifier is mediocre at predicting the positive class at the moment. But the ambition is to improve it. However, after doing…
Jas999
  • 21
  • 3
2
votes
2 answers

Can i use other regression types that arent based in decision trees to use it like a weak learners in gradient boosting?

I was thinking if i can use polynomial regression like a weak learners in gradient boosting but i read that decision trees are used for that and i cannot find anything that show me the possibility of other weak learners could be used.
Nyut Nyuka
  • 29
  • 2
2
votes
1 answer

What is the difference between horizontal and vertical ensemble?

I am looking at different ways to do model ensembling and I came across the terms horizontal and vertical blending/ensembling but it is not well defined. My questions will be: What is the difference between horizontal and vertical blending? In…
Carlos Mougan
  • 6,430
  • 2
  • 20
  • 51
2
votes
1 answer

When and how to use bagging?

Can all types of ML methods benefit from bagging? Decision Tree Classification seems always be the go-to example of bagging, what about other classifiers or regressions? When it's suitable to do bagging, how to pick the size and number of…
Indominus
  • 155
  • 6
1
2 3