When and how to use bagging?

Question

Can all types of ML methods benefit from bagging? Decision Tree Classification seems always be the go-to example of bagging, what about other classifiers or regressions?

When it's suitable to do bagging, how to pick the size and number of bags?

Update:

I am looking for something mathematically more rigorous, such as, for each model (single learner) we can break its total estimation error into:

$Error^2 = Bias^2 + Variance^2 + Irreducible^2$

If we can have a rough estimation of $Variance$ and the correlation between the predictions from all the single learners, then we can know about how low we can push variance to through ensemble.

Answer 1

Bagging main goal is to minimize variance of your model. Basically, if you have a model that is on average pretty accurate but inconsistent (meaning, it does well for a given data set, poorly generalizations) then bagging may be a way to produce a more consistent estimators. Decision trees are the common example of this because they are the canonical high variance machine learning algorithm.

As for your last question, the size of each new training set needs to be the same size as the original training set. The way you achieve this is by random sampling of the original dataset with replacement (meaning the new dataset may have duplicates). The number of new training sets is dependent on the problem. Sometimes 100 is fine other times you need 1000 or so. There isn't a way to just know how many sets you need. It is a parameter that needs to be tuned.