Assessment of methodological quality
Pairs of review authors independently assessed risk of bias and applicability concerns using the QUADAS‐2 (Quality Assessment tool for Diagnostic Accuracy Studies) checklist, which was common to the suite of reviews but tailored to each particular review (Whiting 2011; Table 2). For this review, we excluded the questions on the nature of the samples as these were not relevant, and we added a question on who assessed the signs. We resolved disagreements by discussion between three review authors.
1  QUADAS‐2 checklist
Index test(s)   Signs and symptoms
Patients (setting, intended use of index test, presentation, prior testing)  Primary care, hospital outpatient settings including emergency departmentsInpatients presenting with suspected COVID‐19No prior testingSigns and symptoms often used for triage or referral
Reference standard and target condition  The focus will be on the diagnosis of COVID‐19 disease and COVID‐19 pneumonia. For this review, the focus will not be on prognosis.
Participant selection
Was a consecutive or random sample of patients enrolled?  This will be similar for all index tests, target conditions, and populations.YES: if a study explicitly stated that all participants within a certain time frame were included; that this was done consecutively; or that a random selection was done.NO: if it was clear that a different selection procedure was employed; for example, selection based on clinician's preference, or based on institutions.UNCLEAR: if the selection procedure was not clear or not reported.
Was a case‐control design avoided?  This will be similar for all index tests, target conditions, and populations.YES: if a study explicitly stated that all participants came from the same group of (suspected) patients.NO: if it was clear that a different selection procedure was employed for the participants depending on their COVID‐19 (pneumonia) status or SARS‐CoV‐2 infection status.UNCLEAR: if the selection procedure was not clear or not reported.
Did the study avoid inappropriate exclusions?  Studies may have excluded participants, or selected participants in such a way that they avoided including those who were difficult to diagnose or likely to be borderline. Although the inclusion and exclusion criteria will be different for the different index tests, inappropriate exclusions and inclusions will be similar for all index tests: for example, only elderly patients excluded, or children (as sampling may be more difficult). This needs to be addressed on a case‐by‐case basis.YES: if a high proportion of eligible patients was included without clear selection.NO: if a high proportion of eligible patients was excluded without providing a reason; if, in a retrospective study, participants without index test or reference standard results were excluded; if exclusion was based on severity assessment post‐factum or comorbidities (cardiovascular disease, diabetes, immunosuppression).UNCLEAR: if the exclusion criteria were not reported.
Did the study avoid inappropriate inclusions?  YES: if samples included were likely to be representative of the spectrum of disease.NO: if the study oversampled patients with particular characteristics likely to affect estimates of accuracy.UNCLEAR: if the exclusion criteria were not reported.
Could the selection of patients have introduced bias?  HIGH: if one or more signalling questions were answered with NO, as any deviation from the selection process may lead to bias.LOW: if all signalling questions were answered with YES.UNCLEAR: all other instances.
Is there concern that the included patients do not match the review question?  HIGH: if accuracy of signs and symptoms were assessed in a case‐control design, or in an already highly selected group of participants, or the study was able to only estimate sensitivity or specificity.LOW: any situation where signs and symptoms were the first assessment/test to be done on the included participants.UNCLEAR: if a description about the participants was lacking.
Index tests
Were the index test results interpreted without knowledge of the results of the reference standard?  This will be similar for all index tests, target conditions, and populations.YES: if blinding was explicitly stated or index test was recorded before the results from the reference standard were available.NO: if it was explicitly stated that the index test results were interpreted with knowledge of the results of the reference standard.UNCLEAR: if blinding was unclearly reported.
If a threshold was used, was it prespecified?  This will be similar for all index tests, target conditions, and populations.YES: if the test was dichotomous by nature, or if the threshold was stated in the methods section, or if authors stated that the threshold as recommended by the manufacturer was used.NO: if a receiver operating characteristic curve was drawn or multiple threshold reported in the results section; and the final result was based on one of these thresholds; if fever was not defined beforehand.UNCLEAR: if threshold selection was not clearly reported.
Could the conduct or interpretation of the index test have introduced bias?  HIGH: if one or more signalling questions were answered with NO, as even in a laboratory situation knowledge of the reference standard may lead to bias.LOW: if all signalling questions were answered with YES.UNCLEAR: all other instances.
Is there concern that the index test, its conduct, or interpretation differ from the review question?  This will probably be answered 'LOW' in all cases except when assessments were made in a different setting, or using personnel not available in practice.
Reference standard
Is the reference standard likely to correctly classify the target condition?  We will define acceptable reference standards using a consensus process once the list of reference standards that have been used has been obtained from the eligible studies.For severe pneumonia, we will consider how well processes adhered to the WHO case definition in Appendix 1.
Were the reference standard results interpreted without knowledge of the results of the index test?  YES: if it was explicitly stated that the reference standard results were interpreted without knowledge of the results of the index test, or if the result of the index test was obtained after the reference standard.NO: if it was explicitly stated that the reference standard results were interpreted with knowledge of the results of the index test or if the index test was used to make the final diagnosis.UNCLEAR: if blinding was unclearly reported.
Did the definition of the reference standard incorporate results from the index test(s)?  YES: if results from the index test were a component of the reference standard definition.NO: if the reference standard did not incorporate the index standard test.UNCLEAR: if it was unclear whether the results of the index test formed part of the reference standard.
Could the conduct or interpretation of the reference standard have introduced bias?  HIGH: if one or more signalling questions were answered with NO.LOW: if all signalling questions were answered with YES.UNCLEAR: all other instances.
Is there concern that the target condition as defined by the reference standard does not match the review question?  HIGH: if the target condition was COVID‐19 pneumonia, but only RT‐PCR was used; if alternative diagnosis was highly likely and not excluded (will happen in paediatric cases, where exclusion of other respiratory pathogens is also necessary); if tests used to follow up viral load in known test‐positives.LOW: if above situations were not present.UNCLEAR: if intention for testing was not reported in the study.
Flow and timing
Was there an appropriate interval between index test(s) and reference standard?  YES: this will be similar for all index tests, populations for the current infection target conditions: as the situation of a patient, including clinical presentation and disease progress, evolves rapidly and new/ongoing exposure can result in case status change, an appropriate time interval will be within 24 hours.NO: if there was more than 24 hours between the index test and the reference standard or if participants were otherwise reported to be assessed with the index versus reference standard test at moments of different severity.UNCLEAR: if the time interval was not reported.
Did all patients receive a reference standard?  YES: if all participants received a reference standard (clearly no partial verification).NO: if only (part of) the index test‐positives or index test‐negatives received the complete reference standard.UNCLEAR: if it was not reported.
Did all patients receive the same reference standard?  YES: if all participants received the same reference standard (clearly no differential verification).NO: if (part of) the index test‐positives or index test‐negatives received a different reference standard.UNCLEAR: if it was not reported.
Were all patients included in the analysis?  YES: if all included participants were included in the analyses.NO: if after the inclusion/exclusion process, participants were removed from the analyses for different reasons: no reference standard done, no index test done, intermediate results of both index test or reference standard, indeterminate results of both index test or reference standard, samples unusable.UNCLEAR: if this was not clear from the reported numbers.
Could the patient flow have introduced bias?  HIGH: if one or more signalling questions were answered with NO.LOW: if all signalling questions were answered with YES.UNCLEAR: all other instances.
ICU: intensive care unit; RT‐PCR: reverse transcription polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2; WHO: World Health Organization