Id |
Subject |
Object |
Predicate |
Lexical cue |
T468 |
62-289 |
Sentence |
denotes |
Multiple evaluation indicators (PRC, ROC, AUPRC, AUROC, sensitivity, specificity, precision, kappa index, and F1 with a fixed threshold) were computed for a comprehensive and accurate assessment of the classification framework. |
T469 |
290-406 |
Sentence |
denotes |
Multiple threshold values were in the range from 0 to 1 with a step value of 0.005 to obtain the ROC and PRC curves. |
T470 |
407-573 |
Sentence |
denotes |
The PRC showed the relationship between the precision and the sensitivity (or recall), and the ROC indicated the relationship between the sensitivity and specificity. |
T471 |
574-661 |
Sentence |
denotes |
The two curves reflected the comprehensive performance of the classification framework. |
T472 |
662-766 |
Sentence |
denotes |
The kappa index is a statistical method for assessing the degree of agreement between different methods. |
T473 |
767-846 |
Sentence |
denotes |
In our use case, the indicator was used to measure the stability of the method. |
T474 |
847-939 |
Sentence |
denotes |
The F1 score is a harmonic average of precision and sensitivity and considers the FP and FN. |
T475 |
940-1032 |
Sentence |
denotes |
The bootstrapping method was used to calculate the empirical distribution of each indicator. |
T476 |
1033-1226 |
Sentence |
denotes |
The detailed calculation process was as follows: we conducted random sampling with replacement to generate 1000 new test data sets with the same number of samples as the original test data set. |
T477 |
1227-1300 |
Sentence |
denotes |
The evaluation indicators were calculated to determine the distributions. |
T478 |
1301-1374 |
Sentence |
denotes |
The results were displayed in boxplots (Fig. 5 and Supplementary Fig. 2). |