| Id |
Subject |
Object |
Predicate |
Lexical cue |
| T332 |
0-28 |
Sentence |
denotes |
Data collection and analysis |
| T333 |
30-50 |
Sentence |
denotes |
Selection of studies |
| T334 |
51-106 |
Sentence |
denotes |
Pairs of review authors independently screened studies. |
| T335 |
107-309 |
Sentence |
denotes |
We resolved disagreements by discussion with a third, experienced review author for initial title and abstract screening, and through discussion between three review authors for eligibility assessments. |
| T336 |
311-341 |
Sentence |
denotes |
Data extraction and management |
| T337 |
342-406 |
Sentence |
denotes |
Pairs of review authors independently performed data extraction. |
| T338 |
407-476 |
Sentence |
denotes |
We resolved disagreements by discussion between three review authors. |
| T339 |
477-579 |
Sentence |
denotes |
We intended to contact study authors where we needed to clarify details or obtain missing information. |
| T340 |
581-617 |
Sentence |
denotes |
Assessment of methodological quality |
| T341 |
618-897 |
Sentence |
denotes |
Pairs of review authors independently assessed risk of bias and applicability concerns using the QUADAS‐2 (Quality Assessment tool for Diagnostic Accuracy Studies) checklist, which was common to the suite of reviews but tailored to each particular review (Whiting 2011; Table 2). |
| T342 |
898-1048 |
Sentence |
denotes |
For this review, we excluded the questions on the nature of the samples as these were not relevant, and we added a question on who assessed the signs. |
| T343 |
1049-1118 |
Sentence |
denotes |
We resolved disagreements by discussion between three review authors. |
| T344 |
1119-1140 |
Sentence |
denotes |
1 QUADAS‐2 checklist |
| T345 |
1141-1175 |
Sentence |
denotes |
Index test(s) Signs and symptoms |
| T346 |
1176-1440 |
Sentence |
denotes |
Patients (setting, intended use of index test, presentation, prior testing) Primary care, hospital outpatient settings including emergency departmentsInpatients presenting with suspected COVID‐19No prior testingSigns and symptoms often used for triage or referral |
| T347 |
1441-1560 |
Sentence |
denotes |
Reference standard and target condition The focus will be on the diagnosis of COVID‐19 disease and COVID‐19 pneumonia. |
| T348 |
1561-1613 |
Sentence |
denotes |
For this review, the focus will not be on prognosis. |
| T349 |
1614-1635 |
Sentence |
denotes |
Participant selection |
| T350 |
1636-1692 |
Sentence |
denotes |
Was a consecutive or random sample of patients enrolled? |
| T351 |
1694-2158 |
Sentence |
denotes |
This will be similar for all index tests, target conditions, and populations.YES: if a study explicitly stated that all participants within a certain time frame were included; that this was done consecutively; or that a random selection was done.NO: if it was clear that a different selection procedure was employed; for example, selection based on clinician's preference, or based on institutions.UNCLEAR: if the selection procedure was not clear or not reported. |
| T352 |
2159-2193 |
Sentence |
denotes |
Was a case‐control design avoided? |
| T353 |
2195-2612 |
Sentence |
denotes |
This will be similar for all index tests, target conditions, and populations.YES: if a study explicitly stated that all participants came from the same group of (suspected) patients.NO: if it was clear that a different selection procedure was employed for the participants depending on their COVID‐19 (pneumonia) status or SARS‐CoV‐2 infection status.UNCLEAR: if the selection procedure was not clear or not reported. |
| T354 |
2613-2658 |
Sentence |
denotes |
Did the study avoid inappropriate exclusions? |
| T355 |
2660-2831 |
Sentence |
denotes |
Studies may have excluded participants, or selected participants in such a way that they avoided including those who were difficult to diagnose or likely to be borderline. |
| T356 |
2832-3097 |
Sentence |
denotes |
Although the inclusion and exclusion criteria will be different for the different index tests, inappropriate exclusions and inclusions will be similar for all index tests: for example, only elderly patients excluded, or children (as sampling may be more difficult). |
| T357 |
3098-3609 |
Sentence |
denotes |
This needs to be addressed on a case‐by‐case basis.YES: if a high proportion of eligible patients was included without clear selection.NO: if a high proportion of eligible patients was excluded without providing a reason; if, in a retrospective study, participants without index test or reference standard results were excluded; if exclusion was based on severity assessment post‐factum or comorbidities (cardiovascular disease, diabetes, immunosuppression).UNCLEAR: if the exclusion criteria were not reported. |
| T358 |
3610-3655 |
Sentence |
denotes |
Did the study avoid inappropriate inclusions? |
| T359 |
3657-3904 |
Sentence |
denotes |
YES: if samples included were likely to be representative of the spectrum of disease.NO: if the study oversampled patients with particular characteristics likely to affect estimates of accuracy.UNCLEAR: if the exclusion criteria were not reported. |
| T360 |
3905-3958 |
Sentence |
denotes |
Could the selection of patients have introduced bias? |
| T361 |
3960-4171 |
Sentence |
denotes |
HIGH: if one or more signalling questions were answered with NO, as any deviation from the selection process may lead to bias.LOW: if all signalling questions were answered with YES.UNCLEAR: all other instances. |
| T362 |
4172-4249 |
Sentence |
denotes |
Is there concern that the included patients do not match the review question? |
| T363 |
4251-4629 |
Sentence |
denotes |
HIGH: if accuracy of signs and symptoms were assessed in a case‐control design, or in an already highly selected group of participants, or the study was able to only estimate sensitivity or specificity.LOW: any situation where signs and symptoms were the first assessment/test to be done on the included participants.UNCLEAR: if a description about the participants was lacking. |
| T364 |
4630-4641 |
Sentence |
denotes |
Index tests |
| T365 |
4642-4741 |
Sentence |
denotes |
Were the index test results interpreted without knowledge of the results of the reference standard? |
| T366 |
4743-5125 |
Sentence |
denotes |
This will be similar for all index tests, target conditions, and populations.YES: if blinding was explicitly stated or index test was recorded before the results from the reference standard were available.NO: if it was explicitly stated that the index test results were interpreted with knowledge of the results of the reference standard.UNCLEAR: if blinding was unclearly reported. |
| T367 |
5126-5171 |
Sentence |
denotes |
If a threshold was used, was it prespecified? |
| T368 |
5173-5699 |
Sentence |
denotes |
This will be similar for all index tests, target conditions, and populations.YES: if the test was dichotomous by nature, or if the threshold was stated in the methods section, or if authors stated that the threshold as recommended by the manufacturer was used.NO: if a receiver operating characteristic curve was drawn or multiple threshold reported in the results section; and the final result was based on one of these thresholds; if fever was not defined beforehand.UNCLEAR: if threshold selection was not clearly reported. |
| T369 |
5700-5775 |
Sentence |
denotes |
Could the conduct or interpretation of the index test have introduced bias? |
| T370 |
5777-6014 |
Sentence |
denotes |
HIGH: if one or more signalling questions were answered with NO, as even in a laboratory situation knowledge of the reference standard may lead to bias.LOW: if all signalling questions were answered with YES.UNCLEAR: all other instances. |
| T371 |
6015-6116 |
Sentence |
denotes |
Is there concern that the index test, its conduct, or interpretation differ from the review question? |
| T372 |
6118-6271 |
Sentence |
denotes |
This will probably be answered 'LOW' in all cases except when assessments were made in a different setting, or using personnel not available in practice. |
| T373 |
6272-6290 |
Sentence |
denotes |
Reference standard |
| T374 |
6291-6367 |
Sentence |
denotes |
Is the reference standard likely to correctly classify the target condition? |
| T375 |
6369-6649 |
Sentence |
denotes |
We will define acceptable reference standards using a consensus process once the list of reference standards that have been used has been obtained from the eligible studies.For severe pneumonia, we will consider how well processes adhered to the WHO case definition in Appendix 1. |
| T376 |
6650-6749 |
Sentence |
denotes |
Were the reference standard results interpreted without knowledge of the results of the index test? |
| T377 |
6751-7201 |
Sentence |
denotes |
YES: if it was explicitly stated that the reference standard results were interpreted without knowledge of the results of the index test, or if the result of the index test was obtained after the reference standard.NO: if it was explicitly stated that the reference standard results were interpreted with knowledge of the results of the index test or if the index test was used to make the final diagnosis.UNCLEAR: if blinding was unclearly reported. |
| T378 |
7202-7290 |
Sentence |
denotes |
Did the definition of the reference standard incorporate results from the index test(s)? |
| T379 |
7292-7559 |
Sentence |
denotes |
YES: if results from the index test were a component of the reference standard definition.NO: if the reference standard did not incorporate the index standard test.UNCLEAR: if it was unclear whether the results of the index test formed part of the reference standard. |
| T380 |
7560-7643 |
Sentence |
denotes |
Could the conduct or interpretation of the reference standard have introduced bias? |
| T381 |
7645-7794 |
Sentence |
denotes |
HIGH: if one or more signalling questions were answered with NO.LOW: if all signalling questions were answered with YES.UNCLEAR: all other instances. |
| T382 |
7795-7910 |
Sentence |
denotes |
Is there concern that the target condition as defined by the reference standard does not match the review question? |
| T383 |
7912-8321 |
Sentence |
denotes |
HIGH: if the target condition was COVID‐19 pneumonia, but only RT‐PCR was used; if alternative diagnosis was highly likely and not excluded (will happen in paediatric cases, where exclusion of other respiratory pathogens is also necessary); if tests used to follow up viral load in known test‐positives.LOW: if above situations were not present.UNCLEAR: if intention for testing was not reported in the study. |
| T384 |
8322-8337 |
Sentence |
denotes |
Flow and timing |
| T385 |
8338-8417 |
Sentence |
denotes |
Was there an appropriate interval between index test(s) and reference standard? |
| T386 |
8419-9006 |
Sentence |
denotes |
YES: this will be similar for all index tests, populations for the current infection target conditions: as the situation of a patient, including clinical presentation and disease progress, evolves rapidly and new/ongoing exposure can result in case status change, an appropriate time interval will be within 24 hours.NO: if there was more than 24 hours between the index test and the reference standard or if participants were otherwise reported to be assessed with the index versus reference standard test at moments of different severity.UNCLEAR: if the time interval was not reported. |
| T387 |
9007-9053 |
Sentence |
denotes |
Did all patients receive a reference standard? |
| T388 |
9055-9288 |
Sentence |
denotes |
YES: if all participants received a reference standard (clearly no partial verification).NO: if only (part of) the index test‐positives or index test‐negatives received the complete reference standard.UNCLEAR: if it was not reported. |
| T389 |
9289-9342 |
Sentence |
denotes |
Did all patients receive the same reference standard? |
| T390 |
9344-9583 |
Sentence |
denotes |
YES: if all participants received the same reference standard (clearly no differential verification).NO: if (part of) the index test‐positives or index test‐negatives received a different reference standard.UNCLEAR: if it was not reported. |
| T391 |
9584-9627 |
Sentence |
denotes |
Were all patients included in the analysis? |
| T392 |
9629-10055 |
Sentence |
denotes |
YES: if all included participants were included in the analyses.NO: if after the inclusion/exclusion process, participants were removed from the analyses for different reasons: no reference standard done, no index test done, intermediate results of both index test or reference standard, indeterminate results of both index test or reference standard, samples unusable.UNCLEAR: if this was not clear from the reported numbers. |
| T393 |
10056-10100 |
Sentence |
denotes |
Could the patient flow have introduced bias? |
| T394 |
10102-10251 |
Sentence |
denotes |
HIGH: if one or more signalling questions were answered with NO.LOW: if all signalling questions were answered with YES.UNCLEAR: all other instances. |
| T395 |
10252-10400 |
Sentence |
denotes |
ICU: intensive care unit; RT‐PCR: reverse transcription polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2; WHO: |
| T396 |
10401-10426 |
Sentence |
denotes |
World Health Organization |
| T397 |
10428-10467 |
Sentence |
denotes |
Statistical analysis and data synthesis |
| T398 |
10468-10594 |
Sentence |
denotes |
We present results of estimated sensitivity and specificity using paired forest plots and summarised in tables as appropriate. |
| T399 |
10595-10936 |
Sentence |
denotes |
We present the results without meta‐analysis, due to the small numbers of studies currently available, considerable heterogeneity across studies and the high risk of bias that we identified, as we felt doing so would otherwise produce a seemingly more accurate estimate than the underlying evidence is able to provide at this moment in time. |
| T400 |
10937-11147 |
Sentence |
denotes |
We present results of estimated sensitivity and specificity using paired forest plots in Review Manager 2014, and dumbbell plots to display the change in disease probability after a positive or negative result. |
| T401 |
11148-11391 |
Sentence |
denotes |
We disaggregated data by study design and organised by target condition, reporting results from cross‐sectional studies separately from studies that used a diagnostic case‐control or other design that we assessed as prone to high risk of bias. |
| T402 |
11392-11613 |
Sentence |
denotes |
When pooling does become possible in a future update of this review, we will estimate mean sensitivity and specificity using hierarchical models where tests either report binary results or at commonly reported thresholds. |
| T403 |
11614-11732 |
Sentence |
denotes |
Where data are sparse, we will use methods described by Takwoingi 2017 for obtaining estimates from simplified models. |
| T404 |
11733-11853 |
Sentence |
denotes |
We anticipate that over time sufficient data will accumulate to provide clear estimates of test accuracy for some tests. |
| T405 |
11854-12053 |
Sentence |
denotes |
We will undertake meta‐analysis in STATA version 16.0 (STATA), or SAS (SAS 2015), as detailed in the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy (Chapter 10; Macaskill 2013). |
| T406 |
12055-12086 |
Sentence |
denotes |
Investigations of heterogeneity |
| T407 |
12087-12208 |
Sentence |
denotes |
We have listed sources of heterogeneity that we investigated if adequate data were available in the Secondary objectives. |
| T408 |
12209-12349 |
Sentence |
denotes |
In this version of the review, we used stratification to investigate heterogeneity as we considered it was inappropriate to combine studies. |
| T409 |
12350-12462 |
Sentence |
denotes |
In future updates, if meta‐analysis becomes possible, we will investigate heterogeneity through meta‐regression. |
| T410 |
12463-12519 |
Sentence |
denotes |
We will stratify by reference standard and study design. |
| T411 |
12520-12652 |
Sentence |
denotes |
In this version of the review we have stratified by study design only, as stratification by reference standard was not yet possible. |
| T412 |
12654-12674 |
Sentence |
denotes |
Sensitivity analyses |
| T413 |
12675-12744 |
Sentence |
denotes |
We aimed to undertake sensitivity analyses considering the impact of: |
| T414 |
12745-12765 |
Sentence |
denotes |
unpublished studies; |
| T415 |
12766-12810 |
Sentence |
denotes |
studies with inadequate reference standards. |
| T416 |
12811-12872 |
Sentence |
denotes |
However, neither were possible in this version of the review. |
| T417 |
12874-12902 |
Sentence |
denotes |
Assessment of reporting bias |
| T418 |
12903-13075 |
Sentence |
denotes |
We aimed to publish lists of studies that we know exist but for which we have not managed to locate reports, and request information to include in updates of these reviews. |
| T419 |
13076-13174 |
Sentence |
denotes |
However, at the time of writing this version of the review, we are unaware of unpublished studies. |
| T420 |
13176-13195 |
Sentence |
denotes |
Summary of findings |
| T421 |
13196-13375 |
Sentence |
denotes |
We have listed our key findings in a 'Summary of findings' table to determine the strength of evidence for each test and findings, and to highlight important gaps in the evidence. |
| T422 |
13377-13385 |
Sentence |
denotes |
Updating |
| T423 |
13386-13605 |
Sentence |
denotes |
We will undertake the searches of published literature and preprints bi‐weekly, and, dependent on the number of new and important studies found, we will consider updating each review with each search if resources allow. |
