Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [bias]

99 questions
32
votes
5 answers

Why underfitting is called high bias and overfitting is called high variance?

I have been using terms like underfitting/overfitting and bias-variance tradeoff for quite some while in data science discussions and I understand that underfitting is associated with high bias and over fitting is associated with high variance. But…
Vaibhav Thakur
  • 2,403
  • 3
  • 13
  • 9
8
votes
3 answers

What are bias and variance in machine learning?

I am studying machine learning, and I have encountered the concept of bias and variance. I am a university student and in the slides of my professor, the bias is defined as: $bias = E[error_s(h)]-error_d(h)$ where $h$ is the hypotesis and…
J.D.
  • 941
  • 6
  • 20
  • 33
8
votes
1 answer

Question on bias-variance tradeoff and means of optimization

So I was wondering how does one, for example, can best optimize the model they are trying to build when confronted with issues presented by high bias or high variance. Now, of course, you can play with the regularization parameter to get to a…
Zer0k
  • 155
  • 6
5
votes
2 answers

Bagging vs pasting in ensemble learning

This is a citation from "Hands-on machine learning with Scikit-Learn, Keras and TensorFlow" by Aurelien Geron: "Bootstrapping introduces a bit more diversity in the subsets that each predictor is trained on, so bagging ends up with a slightly higher…
chekhovana
  • 51
  • 2
5
votes
2 answers

Bagging vs Boosting, Bias vs Variance, Depth of trees

I understand the main principle of bagging and boosting for classification and regression trees. My doubts are about the optimization of the hyperparameters, especially the depth of the trees First question: why we are supposed to use weak learners…
K.Hua
  • 153
  • 1
  • 6
5
votes
1 answer

Whether add bias or not in a perceptron

In some places, perceptron is described as having added bias, while in some places, bias is not added. Which one is right for you?
Developer
  • 1,109
  • 2
  • 9
  • 11
4
votes
2 answers

Trade off between Bias and Variance

What are the best ideas or approaches to trade off between bias and variance in Machine Learning models.
deepguy
  • 1,471
  • 8
  • 21
  • 39
4
votes
3 answers

Can cross validation for tuning and LOO for evaluation on the exact same dataset cause bias?

I read two articles by the same guy where he uses the whole dataset for hyperparameter optimisation using with CV and then evaluates the model with the best hyperparameters using leave one out on the same dataset. This seems fishy, from what I know…
Lisana Daniel
  • 55
  • 4
3
votes
4 answers

Difference between ethics and bias in Machine Learning

I'm confused about the difference between "ethics" and "bias" when those concepts are discussed in the context of Machine Learning (ML). In my understanding, ethical issue in ML is pretty much exactly the same thing as "bias": say, the model…
Qwerty
  • 31
  • 1
3
votes
0 answers

Unbiasedness of random forests

Suppose that I am trying to build a random forest by subsampling the data and choosing a single feature per tree randomly. For example, suppose there is some dataset, $D = \{(x_{1},y_{1}), ......(x_{N},y_{N})\}$ where we have that $x_{i} \in…
user1234
  • 131
  • 1
3
votes
2 answers

Predictive modeling when output affects future input

Assume I have a model which predicts the outcome of the number of icecreams sold in a store. The model is trained on data for the last 5 years while keeping the last year as a validation set and has produced very good results. We now put the model…
CutePoison
  • 520
  • 3
  • 10
3
votes
1 answer

How to measure deviance resulting from different random seeds in machine learning?

I'm running an xgboost model to predict probabilities to a binary classification problem. Then I aggregate the results based on the Age variable (what is the aggregated risk of getting the sickness for Age x). I made a mistake and did not set the…
user100740
  • 91
  • 2
3
votes
1 answer

Learning curve using micro F-score and macro F-score

I plotted the learning curves using micro and macro F-scores for a Multinomial Naive Bayes classifier. The first plot is made using micro F-score, and the second using macro F-score. I find it quite difficult to interpret both of them. The…
Ana
  • 31
  • 1
3
votes
1 answer

Bias-variance tradeoff and the uncertainty principle

Bias variance tradeoff seems to behave like the uncertainty principle, is it just another name for the same principle?
Eugen
  • 457
  • 2
  • 7
3
votes
1 answer

Is normalizing the validation set of time series a kind of look ahead bias?

Here's the data normalization process of a time series in a paper about stock prediction using LSTM: Split train and test set based on time (e.g. training set: 2001-2010, test set:2011-2012). This looks fine to me. Normalize the training set by…
TQA
  • 536
  • 2
  • 14
1
2 3 4 5 6 7