Statistical analysis
We used localization receiver operating characteristic (LROC) to analyze the data for differences in reader performance between reading with and without using interactive CAD, for individual readers, as well as for the average reader. To determine a LROC, the decision threshold is varied and the correct localization fraction is plotted as a function of the false-positive fraction. The false-positive fraction is defined as the fraction of normal cases recalled as a function of the decision threshold.
For every reader, we determined the cutoff point at which the false-positive recall rate was 10%, by thresholding the scores the observer had given to the findings. The primary metric of detection performance was the mean correct localization fraction in the false-positive fraction interval ranging from 0 to 0.1. This interval is chosen because in screening programs radiologists usually have recall rates below 10%.
The location of each finding was indicated in the MLO view and CC view. A finding was considered a true positive (TP), if it had a correct location in at least one of the views. We defined a location to be correct if the distance between the observers’ marked location and the true cancer location was less than 2 cm. The false-positive fraction was estimated from the observers’ marked locations in the normal cases. We computed significance of differences between sessions with and without CAD for the average reader by using the Wilcoxon signed rank test. Differences with a P value of less than 0.05 were considered significant. The statistical analysis was performed by using R data analysis software (version 2.9.0; R Foundation for Statistical Computing, Vienna, Austria). The number of times reported and unreported TP and false-positive (FP) CAD regions were queried was computed for every reader. A CAD region was considered queried if the distance between the observers’ query location and the center point of the CAD region was less than 0.5 cm, or if the query location was within the CAD region.