PMC:7799291 / 57937-58707 JSONTXT

{"target":"http://pubannotation.org/docs/sourcedb/PMC/sourceid/7799291","sourcedb":"PMC","sourceid":"7799291","source_url":"https://www.ncbi.nlm.nih.gov/pmc/7799291","text":"It also takes time to identify how best to involve medical experts in assessment. For TREC-COVID, the task of ad hoc retrieval is well defined and has historically been recognized as a useful and important text mining task. The TREC-COVID assessments, though completed in a narrower time window than typical, were still relatively easy for the expert assessors. In the case of Kaggle, however, the first round tasks were very open-ended and submissions were correspondingly diverse and difficult to compare. Medical experts were asked to manually assess more than 500 of these submissions, which was quite time-consuming. As Kaggle converged on a more structured table completion task in Round 2, these assessments became easier and arguably a better use of expert time.","tracks":[]}