Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [bagging]

21 questions
6
votes
1 answer

Boosting with highly correlated features

I have a conceptual question. My understanding is, that Random Forest can be applied even when features are (highly) correlated. This is because with bagging, the influence of few highly correlated features is moderated, since each feature only…
Peter
  • 7,896
  • 5
  • 23
  • 50
5
votes
1 answer

Can Boosting and Bagging be applied to heterogeneous algorithms?

Stacking can be achieved with heterogeneous algorithms such as RF, SVM and KNN. However, can such heterogeneously be achieved in Bagging or Boosting? For example, in Boosting, instead of using RF in all the iterations, could we use different…
Ahmad Bilal
  • 177
  • 6
5
votes
2 answers

Bagging vs pasting in ensemble learning

This is a citation from "Hands-on machine learning with Scikit-Learn, Keras and TensorFlow" by Aurelien Geron: "Bootstrapping introduces a bit more diversity in the subsets that each predictor is trained on, so bagging ends up with a slightly higher…
chekhovana
  • 51
  • 2
3
votes
0 answers

Difference Bagging and Bootstrap aggregating

Bootstrap belongs to Efron. Tibshirani wrote a book about that in reference to Efron. Bootstrap process for estimating the standard error of statistic s(x). B bootstrap sample are generatied from original data. Finally the standard deviation of the…
martin
  • 329
  • 3
  • 14
2
votes
1 answer

Why can't we sample without replacement for each tree in a random forest if the subsample size is large enough?

Usually if we have $n$ observations, for each tree with form a bootstrapped subsample of size $n$ with replacement. On googling it one common explanation I've seen is that with replacement sampling is necessary for independence of individual…
user9343456
  • 187
  • 10
2
votes
1 answer

bagging vs. pasting in ensemble learning

I am bit confused about two concepts. From my understanding Bagging is when each data is replaced after each choice. so for example for each subset of data you pick one from population, replace it then pick one again, etc... and this is repeated for…
haneulkim
  • 479
  • 5
  • 17
2
votes
1 answer

Counting the number of trainable parameters in a gradient boosted tree

I recently ran the gradient boosted tree regressor using scikit-learn via: GradientBoostingRegressor() This model depends on the following hyperparameters: Estimators ($N_1$) Min Samples Leaf ($N_2$) Max Depth ($N_3$) which in-turn determine the…
AB_IM
  • 123
  • 5
2
votes
1 answer

Can I do bagging method as improvement technique to decision tree in research?

Bagging use decision tree as base classifier. I want to use bagging with decision tree(c4.5) as base as the method that improve decision tree(c4.5) in my research that solve problem overfitting. Is that possible because some lecturers said not right…
Farrah 1234
  • 73
  • 4
2
votes
1 answer

How does bagging help reduce the variance

I learned that bagging helps reduce variance by averaging but I couldn't understand this. Can someone explain this intuitively?
Bhuwan Bhatt
  • 121
  • 3
2
votes
1 answer

Random Forest Stacking Experiment for Imbalanced Data-set Problem

In order to solve a Imbalanced Dataset Problem, I experimented with Random Forest in the given manner (Somewhat inspired by Deep-Learning) Trained a Random Forest which will take in the input data and the predict probability of the label of the…
Aman Raparia
  • 257
  • 2
  • 8
2
votes
1 answer

Random LightGBM Forest

I'm not completly sure about the bias/variance of boosted decision trees (LightGBM especially), thus I wonder if we generally would expect a performance boost by creating an ensemble of multiple LightGBM models, just like with Random Forest?
CutePoison
  • 520
  • 3
  • 10
1
vote
1 answer

Base model in ensemble learning

I've been doing some research on ensemble learning and read that for base models, model with high variance are often recommended (can't remember which book I read this from exactly). But, it seems counter-intuitive because wouldn't having base…
haneulkim
  • 479
  • 5
  • 17
1
vote
1 answer

Why the accuracy of my bagging model heavily affected by random state?

The accuracy of my bagging decision tree model reach up to 97% when I set the random seed=5 but the accuracy reduce to only 92% when I set random seed=0. Can someone explain why the huge gap and should I just use the accuracy with highest value in…
Farrah 1234
  • 73
  • 4
1
vote
0 answers

Can bagging ensemble consist of heterogeneous base models?

Bagging or bootstrap aggregation seems to make sense for time series forecasting using an ensemble because bagging randomizes subsets of the data with replacement. However, I've only seen bagging used for homogeneous base learners when constructing…
develarist
  • 214
  • 1
  • 11
1
vote
1 answer

Average training instances sampled with bagging

The book Hands-On Machine Learning has a section on Out-of-Bag Evaluation related to Decision Trees, where it's stated that, By default a BaggingClassifier samples m training instances with replacement (bootstrap=True), where m is the size of the…
Sahil Gupta
  • 77
  • 4
1
2