| pubmed-sentences-benchmark | | A benchmark data for text segmentation into sentences.
The source of annotation is the GENIA treebank v1.0.
Following is the process taken.
began with the GENIA treebank v1.0.
sentence annotations were extracted and converted to PubAnnotation JSON.
uploaded. 12 abstracts met alignment failure.
among the 12 failure cases, 4 had a dot('.') character where there should be colon (':'). They were manually fixed then successfully uploaded: 7903907, 8053950, 8508358, 9415639.
among the 12 failed abstracts, 8 were "250 word truncation" cases. They were manually fixed and successfully uploaded. During the fixing, manual annotations were added for the missing pieces of text.
30 abstracts had extra text in the end, indicating copyright statement, e.g., "Copyright 1998 Academic Press." They were annotated as a sentence in GTB. However, the text did not exist anymore in PubMed. Therefore, the extra texts were removed, together with the sentence annotation to them.
| 18.4 K | 2023-11-28 | Released | |
