PMC:7782580 / 62358-63732 JSONTXT

{"target":"https://pubannotation.org/docs/sourcedb/PMC/sourceid/7782580","sourcedb":"PMC","sourceid":"7782580","source_url":"https://www.ncbi.nlm.nih.gov/pmc/7782580","text":"Statistical indices to evaluate the classification framework. Multiple evaluation indicators (PRC, ROC, AUPRC, AUROC, sensitivity, specificity, precision, kappa index, and F1 with a fixed threshold) were computed for a comprehensive and accurate assessment of the classification framework. Multiple threshold values were in the range from 0 to 1 with a step value of 0.005 to obtain the ROC and PRC curves. The PRC showed the relationship between the precision and the sensitivity (or recall), and the ROC indicated the relationship between the sensitivity and specificity. The two curves reflected the comprehensive performance of the classification framework. The kappa index is a statistical method for assessing the degree of agreement between different methods. In our use case, the indicator was used to measure the stability of the method. The F1 score is a harmonic average of precision and sensitivity and considers the FP and FN. The bootstrapping method was used to calculate the empirical distribution of each indicator. The detailed calculation process was as follows: we conducted random sampling with replacement to generate 1000 new test data sets with the same number of samples as the original test data set. The evaluation indicators were calculated to determine the distributions. The results were displayed in boxplots (Fig. 5 and Supplementary Fig. 2).","tracks":[{"project":"LitCovid-sentences","denotations":[{"id":"T468","span":{"begin":62,"end":289},"obj":"Sentence"},{"id":"T469","span":{"begin":290,"end":406},"obj":"Sentence"},{"id":"T470","span":{"begin":407,"end":573},"obj":"Sentence"},{"id":"T471","span":{"begin":574,"end":661},"obj":"Sentence"},{"id":"T472","span":{"begin":662,"end":766},"obj":"Sentence"},{"id":"T473","span":{"begin":767,"end":846},"obj":"Sentence"},{"id":"T474","span":{"begin":847,"end":939},"obj":"Sentence"},{"id":"T475","span":{"begin":940,"end":1032},"obj":"Sentence"},{"id":"T476","span":{"begin":1033,"end":1226},"obj":"Sentence"},{"id":"T477","span":{"begin":1227,"end":1300},"obj":"Sentence"},{"id":"T478","span":{"begin":1301,"end":1374},"obj":"Sentence"}],"namespaces":[{"prefix":"_base","uri":"http://pubannotation.org/ontology/tao.owl#"}],"attributes":[{"subj":"T468","pred":"source","obj":"LitCovid-sentences"},{"subj":"T469","pred":"source","obj":"LitCovid-sentences"},{"subj":"T470","pred":"source","obj":"LitCovid-sentences"},{"subj":"T471","pred":"source","obj":"LitCovid-sentences"},{"subj":"T472","pred":"source","obj":"LitCovid-sentences"},{"subj":"T473","pred":"source","obj":"LitCovid-sentences"},{"subj":"T474","pred":"source","obj":"LitCovid-sentences"},{"subj":"T475","pred":"source","obj":"LitCovid-sentences"},{"subj":"T476","pred":"source","obj":"LitCovid-sentences"},{"subj":"T477","pred":"source","obj":"LitCovid-sentences"},{"subj":"T478","pred":"source","obj":"LitCovid-sentences"}]}],"config":{"attribute types":[{"pred":"source","value type":"selection","values":[{"id":"LitCovid-sentences","color":"#ecc293","default":true}]}]}}