It is the benchmark test data set of the BioNLP-ST 2016 GE task. It includes Genia-style event annotations to 14 full paper articles which are about NFκB proteins. For testing purpose, however, annotations are all blinded, which means users cannot see the annotations in this project. Instead, annotations in any other project can be compared to the hidden annotations in this project, then the annotations in the project will be automatically evaluated based on the comparison.
A participant of GE task can get the evaluation of his/her result of automatic annotation, through following process:
- Create a new project.
- Import documents from the project, bionlp-st-2016-test-proteins to your project.
- Import annotations from the project, bionlp-st-2016-test-proteins to your project.
- At this point, you may want to compare you project to this project, the benchmark data set. It will show that protein annotations in your project is 100% correct, but other annotations, e.g., events, are 0%.
- Produce event annotations, using your system, upon the protein annotations.
- Upload your event annotations to your project.
- Compare your project to this project, to get evaluation.
GE 2016 benchmark data set is provided as multi-layer annotations which include:
Following is supporting resources:
A SPARQL-driven search interface is provided at http://bionlp.dbcls.jp/sparql.