Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [roc]

64 questions
7
votes
1 answer

Micro Average vs Macro Average for Class Imbalance

I have a dataset consisting of around 30'000 data points and 3 classes. The classes are imbalanced (around 5'000 in class 1, 10'000 in class 2 and 15'000 in class 3). I'm building a convolutional neural network model for classification of the data.…
machinery
  • 246
  • 2
  • 9
6
votes
4 answers

Can Micro-Average Roc Auc Score be larger than Class Roc Auc Scores

I'm working with an imbalanced data set. There are 11567 negative and 3737 positive samples in train data. There are 2892 negative and 935 positive samples in validation data. It is a binary classification problem and I used Micro and Macro averaged…
Angerato
  • 63
  • 1
  • 4
5
votes
1 answer

Is meta-analysis necessary?

I have a large dataset that covers 5 countries. I plan to build a prediction model using this dataset. I would like to compute a ROC curve for each country, and then one overall ROC curve for all countries. To go about this, I plan to either: Build…
Laurence
  • 53
  • 3
4
votes
2 answers

Interpreting vertical and horizontal parts of ROC curve

It's not clear to me how I can interpret vertical and horizontal parts of the ROC curve. What important information can I gain from this? This is a text from the book "Human-in-the-Loop Machine Learning" by Robert Monarch: In this example, we can…
Mykola Zotko
  • 77
  • 2
  • 14
4
votes
0 answers

Fast PR / ROC curves and corespondings AUPR / AUROC

I find myself in a position of calculating numerous PR / ROC curves and their associated area under the PR curves (AUPR) / area under the ROC curve (AUROC). Its is quite easy to perform those calculations with standards implementations (I am using…
Lucas Morin
  • 2,775
  • 5
  • 25
  • 47
4
votes
2 answers

What is a discrimination threshold of binary classifier?

With respect to ROC can anyone please tell me what the phrase "discrimination threshold of binary classifier system" means? I know what a binary classifier is.
girl101
  • 1,161
  • 2
  • 11
  • 26
4
votes
1 answer

How should I calculate AUROC if my (TPR,FPR) doesn't go till (1,1)? Should it be area just under the curve or should I include 1 and calculate?

I am running a model where it generates song detections with a confidence value. I then validate it across an annotated dataset. I then plot the values of TPR and FPR at each confidence threshold, starting with 0 till 1 with a stepping of 0.01. This…
Aditya_Panigrahy
  • 41
  • 1
4
votes
1 answer

Does a random classifier have a diagonal ROC (received operator characteristic) curve even when the data is biased toward negatives?

About 9% of the US population have a diabetes diagnosis. So a binary random classifier that just guesses 50% positive and 50% negative would likely be incorrect when it guesses positive (leading to more False Positives (FP) than True Positives…
joseville
  • 143
  • 3
3
votes
1 answer

At what stage are ROC curves used when building machine learning model?

When developing a machine learning model, at what stage are ROC curve with AUC used? Typically I have three data sets train - validation - final test I do K-Fold cross validation using the combined train + validation set During that phase we can…
erotavlas
  • 131
  • 4
3
votes
2 answers

Lower-than-random ROC

If I have an ROC for a single classifier [y(x) in the range 0...1] that is 'worse than random', namely the AUC of the ROC is less than 0.5, would a classifier that reversed the class predictions [y'(x)=1-y(x)] be better than random to the same…
jeremy_rutman
  • 234
  • 1
  • 4
  • 13
3
votes
1 answer

Is it sensible to use the ROC curve with an KNN model? And if so why?

I am a beginner doing my first ML project. I am doing a binary supervised classification on an unbalanced dataset and want to use the ROC curve as a performance metric of my models. I am using Logistic Regression, Support Vector Machine and K…
Ludger
  • 33
  • 1
  • 3
3
votes
1 answer

What does precision-recall curve and ROC curve tell us abouth threshold invariance

Consider a binary classification problem. Intuitively, a value for the area under the curve (for both curves) very close to 1, shows that the curve is almost L-shaped. Thus, this means that the value on y axis stays rather consistent despite changes…
liakoyras
  • 636
  • 4
  • 15
3
votes
1 answer

Interpreting ROC curves across k-fold cross-validation

I have used a MARS model (multivariate adaptive regression splines) and I have used k fold cross validation for the evaluation of the model, obtaining the following graph: How would be the interpretation of this model? I understand that in the 6…
PicaR
  • 334
  • 2
  • 13
3
votes
2 answers

Uncertainty about shape of ROC curve

I am working on a binary classification and the plotted ROC curves that I am using for evaluation together with AUC, have seemed strange to me. Here is an example. I understand that ROC is a visual representation of the true positive rate versus…
lazarea
  • 299
  • 1
  • 15
2
votes
1 answer

Identify optimal thresholds for one-vs-one/one-vs-rest ROC-curve for multiclass classification

Say I have a multiclass classification problem with N classes. I have trained a classifier on a training set, I use a validation set and a One-vs-rest ROC-curve to give me N ROC curves. Since the ROC curve is created based on different thresholds of…
CutePoison
  • 520
  • 3
  • 10
1
2 3 4 5