Using ROC curves to identify the most accurate test

receiver operating characteristic curve, or ROC curve, is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied.

The relationship between sensitivity and specificity can be represented in a ROC graph. This is created by plotting the sensitivity (true positive rate) on the vertical axis against the false positive rate (1-specifcity) on the horizontal axis, for every observed data value. The area under this curve (AUC) represent the overall accuracy of the test. The AUC value lies between 0 and 1 and the closer the value is to 1 the better the test. An AUC with a value of 0.5 suggests no discrimination.

The relationship between sensitivity and specificity can be represented in a ROC graph. This is created by plotting the sensitivity (true positive rate) on the vertical axis against the false positive rate (1-specifcity) on the horizontal axis, for every observed data value. The area under this curve (AUC) represent the overall accuracy of the test. The AUC value lies between 0 and 1 and the closer the value is to 1 the better the test. An AUC with a value of 0.5 suggests no discrimination.

By way of illustration a hypothetical scenario is presented below.

Measurement of serum levels of Prostate Specific Antigen (PSA) is used as a test for detecting prostate cancer. Improvements in technology have led to two new biomarkers that could potentially detect prostate cancer.

A hypothetical investigation was carried out to discover whether biomarker A (Apsa) and B (Bpsa) can be used to detect prostate cancer and how they compare to the standard test (Spsa). Blood samples from 100,000 men were tested at time 0 with all three tests. These men were then followed up for 15 years and deaths from prostate cancer were recorded.

A ROC curve was used to compare biomarker A and B to the standard test (Spsa). The ROC curve shows the trade-off between sensitivity and specificity. It shows that the standard test is still the most accurate test (AUC=0.9125) in identifying men with prostate cancer, and biomarker B (AUC=0.5123) is the worst and shouldn’t be considered as a test for identifying men with prostate cancer.

ROC analysis of three hypothetical tests to identify prostate cancer