ROC curves and AUC
Suppose you have a continuous measurement, a biomarker level, an assay readout, and you want to use it to tell two groups apart, diseased from healthy, responder from non-responder. A ROC curve shows how well that single measurement separates the two groups across every possible threshold, and the AUC sums it up in one number. This page covers reading the curve, the AUC, and choosing a cut point.
The problem a ROC curve solves
A continuous marker rarely separates two groups perfectly, the distributions overlap. Wherever you set a threshold to call a sample positive, you make two kinds of mistake, and they trade off. Set the threshold low and you catch almost every true positive but also flag many negatives. Set it high and you avoid false alarms but miss real cases. A ROC curve (receiver operating characteristic) draws that whole tradeoff so you can see it at once.
Sensitivity and specificity
The two axes are the two things you are trading.
- Sensitivity is the fraction of true positives the test catches. High sensitivity means few missed cases. It is on the vertical axis.
- Specificity is the fraction of true negatives the test correctly clears. High specificity means few false alarms. The horizontal axis is one minus specificity, the false-positive rate.
Each point on the ROC curve is one possible threshold, plotting the sensitivity and false-positive rate you would get if you drew the line there. A curve that bows up toward the top-left corner is a good separator, it is achieving high sensitivity without paying much in false positives. A curve along the diagonal is no better than a coin flip.
AUC, one number for separation
The AUC is the area under the ROC curve, a single number from 0.5 to 1. An AUC of 0.5 means the marker has no discriminating power, the diagonal. An AUC of 1.0 is perfect separation. A useful plain reading, the AUC is the probability that a randomly chosen positive scores higher than a randomly chosen negative. So an AUC of 0.85 means that 85% of the time the marker correctly ranks a true case above a true non-case.
The Data Hub reports the AUC with its standard error (Hanley and McNeil 1982 closed form, the same formula GraphPad Prism and pROC use) and its 95% confidence interval. It also reports the number of positives and negatives in the dataset, which tells you whether the confidence interval is likely to be meaningful, a very small positive or negative count gives a wide interval regardless of what the AUC is. An interval whose lower end stays well above 0.5 means the marker genuinely separates the groups, and its width tells you how sure you are, read it as on the effect sizes page.
Choosing a cut point
The ROC curve shows every threshold, but eventually you have to pick one to actually use. A common, balanced choice is the Youden cut point, the threshold that maximizes sensitivity plus specificity minus one, which is the point on the curve sitting farthest above the diagonal. It is the threshold that does best at both jobs at once, and the Data Hub reports it along with the sensitivity and specificity you would get there.
A worked example
A blood marker separating responders from non-responders gives an AUC of 0.82 (95% CI 0.74 to 0.90). The Youden cut point sits at 3.1 ng/mL, where sensitivity is 0.78 and specificity is 0.80. You would write "the marker discriminated responders from non-responders with an AUC of 0.82 (95% CI 0.74 to 0.90); at the Youden-optimal cut of 3.1 ng/mL, sensitivity was 78% and specificity 80%."
ResearchOS validates the ROC curve, the AUC and its interval, and the cut point against scikit-learn and R's pROC package on the transparency page.