PMC:7386785 / 43934-59505 JSONTXT

Annnotations TAB JSON ListView MergeView

    LitCovid-PD-FMA-UBERON

    {"project":"LitCovid-PD-FMA-UBERON","denotations":[{"id":"T7762","span":{"begin":2145,"end":2151},"obj":"Body_part"},{"id":"T89828","span":{"begin":2306,"end":2312},"obj":"Body_part"},{"id":"T46","span":{"begin":3192,"end":3198},"obj":"Body_part"},{"id":"T47","span":{"begin":3436,"end":3442},"obj":"Body_part"},{"id":"T48","span":{"begin":3491,"end":3497},"obj":"Body_part"},{"id":"T49","span":{"begin":4425,"end":4431},"obj":"Body_part"},{"id":"T50","span":{"begin":4435,"end":4441},"obj":"Body_part"},{"id":"T51","span":{"begin":10941,"end":10947},"obj":"Body_part"},{"id":"T52","span":{"begin":11683,"end":11688},"obj":"Body_part"},{"id":"T53","span":{"begin":12123,"end":12129},"obj":"Body_part"},{"id":"T54","span":{"begin":13393,"end":13399},"obj":"Body_part"},{"id":"T55","span":{"begin":13404,"end":13410},"obj":"Body_part"}],"attributes":[{"id":"A89297","pred":"fma_id","subj":"T7762","obj":"http://purl.org/sig/ont/fma/fma228738"},{"id":"A24107","pred":"fma_id","subj":"T89828","obj":"http://purl.org/sig/ont/fma/fma228738"},{"id":"A46","pred":"fma_id","subj":"T46","obj":"http://purl.org/sig/ont/fma/fma312401"},{"id":"A47","pred":"fma_id","subj":"T47","obj":"http://purl.org/sig/ont/fma/fma312401"},{"id":"A48","pred":"fma_id","subj":"T48","obj":"http://purl.org/sig/ont/fma/fma228738"},{"id":"A49","pred":"fma_id","subj":"T49","obj":"http://purl.org/sig/ont/fma/fma228738"},{"id":"A50","pred":"fma_id","subj":"T50","obj":"http://purl.org/sig/ont/fma/fma312401"},{"id":"A51","pred":"fma_id","subj":"T51","obj":"http://purl.org/sig/ont/fma/fma228738"},{"id":"A52","pred":"fma_id","subj":"T52","obj":"http://purl.org/sig/ont/fma/fma9576"},{"id":"A53","pred":"fma_id","subj":"T53","obj":"http://purl.org/sig/ont/fma/fma228738"},{"id":"A54","pred":"fma_id","subj":"T54","obj":"http://purl.org/sig/ont/fma/fma228738"},{"id":"A55","pred":"fma_id","subj":"T55","obj":"http://purl.org/sig/ont/fma/fma312401"}],"text":"Results\n\nResults of the search\nThe search yielded 10,965 records after removing duplicates. The first selection resulted in 658 records that were potentially eligible for this review on signs and symptoms. After screening on title and abstract, we excluded 457 records, leaving 201 to be assessed on full text. Of these, we included 16 studies in this review. The reasons for excluding 185 records are listed in the PRISMA flow chart (see Figure 1; Moher 2009).\n1 Flow diagram Two studies reported on the same cases while using a different control group (Chen X 2020; Yang 2020d). Chen X 2020 used a concurrent control group of pneumonia cases negative for SARS‐CoV‐2 on PCR testing but Yang 2020d used a historic control group of influenza pneumonia patients. For this reason we only included the Chen X 2020 results in the analyses.\nOne study reported a study that included a derivation and validation part for the development of a prediction rule (Song 2020b). The two parts are identical in set‐up and only differ in respect to the time of data collection, that is, the derivation part recruited participants up to 5 February 2020 and the validation part recruited participants from 6 February 2020 onwards. As a result, we consider this to be one study and have entered all data on signs and symptoms as such.\nFour studies were conducted in the USA, all other studies were from China. A summary of the main study characteristics can be found in Table 3.\n2 Summary of study characteristics\nStudy ID Target condition Sample size Prevalence Setting Population Design Reference standard\nAi 2020a COVID‐19 pneumonia 53 38% Hospital inpatientsa Patients hospitalised with pneumonia diagnosed by imaging Cross‐sectional PCR on nasopharyngeal swabs\nChen X 2020 COVID‐19 pneumonia 136 Not applicable Hospital inpatientsa Patients admitted with pneumonia Cases selected cross‐sectionally in 5 hospitals, non‐cases from 1 hospital only PCR, samples not specified\nCheng 2020a COVID‐19 pneumonia 33 33% Hospital outpatients Patients presenting to a fever observation department with pneumonia Cross‐sectional PCR on throat swabs\nFeng 2020a COVID‐19 pneumonia 132 5% Emergency department Patients presenting to fever clinic of emergency department Cross‐sectional PCR on throat swabs\nLiang 2020 COVID‐19 pneumonia 88 24% Hospital outpatients Patients with pneumonia and presenting to fever clinic Cross‐sectional PCR, sample not specified; conducted after panel discussion\nNobel 2020 COVID‐19 disease 516 Not applicable Hospital outpatients Patients who underwent SARS‐CoV‐2 testing with intent to hospitalise or in essential personnel Case‐control PCR on nasopharyngeal swabs\nPeng 2020a COVID‐19 disease 86 13% Hospital outpatients Patients clinically suspected and referred for testing Cross‐sectional PCR on nasopharyngeal swabs\nRentsch 2020 COVID‐19 disease 3789 15% Unclear Patients tested for SARS‐CoV‐2 in the Veterans Affairs Cohort born between 1945 and 1965 Cross‐sectional PCR on nasopharyngeal swabs\nSong 2020b COVID‐19 disease 399 7% Hospital outpatients Patients tested for SARS‐CoV‐2 Cross‐sectional PCR on sputum samples\nSun 2020a COVID‐19 disease 788 7% Hospital outpatients Patients presenting to testing centre, either self‐referred, referred from primary care or at‐risk cases identified by national contact tracing Cross‐sectional PCR on sputum, endotracheal aspirate, nasopharyngeal swabs or throat swabs\nTolia 2020 COVID‐19 disease 283 10% Emergency department Patients presenting with symptoms, travel history, risk factors or healthcare workers Cross‐sectional PCR on nasopharyngeal swabs\nWee 2020 COVID‐19 disease 870 18% Emergency department Patients presenting with respiratory symptoms or travel history Cross‐sectional PCR on oropharyngeal swabs\nYan 2020a COVID‐19 disease 262 23% Hospital outpatient Patients presenting hospital for SARS‐CoV‐2 testing, not otherwise specified Internet survey after presentation PCR, samples not specified\nYang 2020d COVID‐19 pneumonia 121 Not applicable Hospital inpatientsa Patient with pneumonia from SARS‐CoV‐2 and patients with pneumonia from influenza in 2015‐2019 Case‐control PCR, samples not specified\nZhao 2020a COVID‐19 pneumonia 34 Not applicable Hospital inpatientsa Patients with pneumonia and admitted to hospital Case‐control PCR on throat or sputum swabs\nZhu 2020b COVID‐19 disease 116 28% Emergency department Patients suspected of SARS‐CoV‐2 and presenting to the emergency department Cross‐sectional PCR, samples not specified\nPCR: polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2\na'Hospital inpatients' refers to studies that recruited patients admitted to hospital with COVID‐19 disease and in whom the signs and symptoms were assessed on admission.\n\nMethodological quality of included studies\nThe results of the quality assessment are summarised in Figure 2 and Figure 3. We rated participant selection as introducing high risk of bias in seven studies. In five studies this was because a CT scan or other imaging was used to diagnose patients with pneumonia prior to inclusion in the study, leading to a highly selected patient population (Ai 2020a; Chen X 2020; Cheng 2020a; Liang 2020; Yang 2020d); RT‐PCR results were subsequently used to distinguish between COVID‐19 pneumonia and pneumonia from other causes. For all studies, testing was highly dependent on the local case definition and testing criteria that were in effect at the time of the study, meaning all patients that were included in studies had already gone through a referral/selection filter, which was not always described. The most extreme example of this is the study by Liang 2020, in which patients with radiological evidence of pneumonia and a clinical presentation compatible with COVID‐19 were only tested for SARS‐CoV‐2 after a panel discussion.\n2 Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies\n3 Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study Of the 16 studies included in this first version of the review, five studies did not use a cross‐sectional design. Three studies were diagnostic case‐control studies (Nobel 2020; Yang 2020d; Zhao 2020a), one study selected cases cross‐sectionally in five hospitals but only selected cases in one hospital (Chen X 2020), and one study emailed patients who had undergone testing for SARS‐CoV‐2 about olfactory symptoms prior to the SARS‐CoV‐2 test, with a response rate of 58% in SARS‐CoV‐2 positive cases and 15% in negative cases (Yan 2020a).\nWe rated all studies except two as carrying a high risk of bias for the index tests because there was little to no detail on how, by whom, and when the signs and symptoms were measured. In addition, there is considerable uncertainty around the reference standard, with some studies providing little detail on the RT‐PCR tests that they used or lack of clarity on blinding.\nParticipant flow was unclear in four studies (Yan 2020a; Yang 2020d; Zhao 2020a; Zhu 2020b), either because the timing of recording signs and symptoms and conduct of the reference standard was unclear, or because some tests received a second or third reference standard at unclear time points during hospital admission.\nWe rated applicability for participant selection as high risk when there was a risk of selection bias or studies did not describe selection. As for the applicability of the index tests and reference standard, we always scored this as low risk except for Chen X 2020, because blinding of the index tests was unclear, and Yang 2020d, because blinding and sample of the reference standard were unclear.\n\nFindings\nThe main characteristics of all included studies are listed in Table 3. There were four studies in hospital inpatients (Ai 2020a; Chen X 2020; Yang 2020d; Zhao 2020a), seven studies in hospital outpatients (Cheng 2020a; Liang 2020; Nobel 2020; Peng 2020a; Song 2020b; Sun 2020a; Yan 2020a), and four studies in emergency departments (Feng 2020a; Tolia 2020; Wee 2020; Zhu 2020b). The setting was not specified in one study (Rentsch 2020); in the 'Summary of findings' table, we classified this study setting as being hospital outpatient under the assumption that at that time in the pandemic (February 2020 to March 2020) tests were not commonly available outside hospital clinics. There were no studies conducted in community primary care services.\nSeven studies assessed the accuracy of signs and symptoms for the diagnosis of COVID‐19 pneumonia (Ai 2020a; Chen X 2020; Cheng 2020a; Feng 2020a; Liang 2020; Yang 2020d; Zhao 2020a); the remaining studies had COVID‐19 disease as the target condition, with no further description of the severity, meaning some patients could have suffered from mild or moderate COVID‐19 disease and others from COVID‐19 pneumonia. The distinction between these two target conditions was not always very clear, and a degree of overlap is to be assumed. All studies used RT‐PCR testing as the reference standard, with some variation in the samples that were used.\nIn all, 7706 patients were included, the median number of participants was 134. Prevalence of infection varied from 5% to 38% with a median of 17%. There were no studies in children or elderly populations, except for Rentsch 2020, which included a cohort of a median age of 65.7 years old from the Veterans Affairs Healthcare System database.\nWe found data on 27 signs and symptoms, which fall into four different categories: systemic, respiratory, gastrointestinal and cardiovascular signs and symptoms. There were no analyses for combinations of tests, only for individual signs and symptoms. The results are summarised in Table 3. Results for the cross‐sectional studies are presented in forest plots (Figure 4; Figure 5; Figure 6; Figure 7), and are plotted in ROC (receiver operating characteristic) space (Figure 8; Figure 9; Figure 10; Figure 11), results for the other studies are only listed in forest plots (Figure 12; Figure 13; Figure 14; Figure 15).\n4 Forest plot of respiratory signs and symptoms (cross‐sectional studies)\n5 Forest plot of systemic signs and symptoms (cross‐sectional studies)\n6 Forest plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n7 Forest plot of cardiovascular signs and symptoms (cross‐sectional studies)\n8 Summary ROC plot of respiratory signs and symptoms (cross‐sectional studies)\n9 Summary ROC plot of systemic signs and symptoms (cross‐sectional studies)\n10 Summary ROC Plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n11 Summary ROC plot of cardiovascular signs and symptoms (cross‐sectional studies)\n12 Forest plot of tests: 27 cough (non‐cross‐sectional study), 28 sore throat (non‐cross‐sectional study), 29 rhinorrhoea (non‐cross‐sectional study), 30 nasal obstruction (non‐cross‐sectional study), 34 dyspnoea (non‐cross‐sectional study), 31 loss of sense of smell (non‐cross‐sectional study), 32 loss of taste (non‐cross‐sectional study), 33 positive auscultation findings (non‐cross‐sectional study)\n13 Forest plot of tests: 37 fatigue (non‐cross‐sectional study), 36 fever (non‐cross‐sectional study), 39 headache (non‐cross‐sectional study), 38 myalgia or arthralgia (non‐cross‐sectional study)\n14 Forest plot of tests: 40 diarrhoea (non‐cross‐sectional study), 41 nausea/vomiting (non‐cross‐sectional study), 42 gastrointestinal symptoms, not specified (non‐cross‐sectional study)\n15 Forest plot of 35 chest tightness (non‐cross‐sectional study) Overall, diagnostic accuracy of individual signs and symptoms is low, especially sensitivity. In addition, results were highly variable across studies, making it difficult to draw firm conclusions.\nSigns and symptoms for which sensitivity was reported above 50% in at least one cross‐sectional study are the following.\nCough: sensitivity between 43% and 71%, specificity between 14% and 54%\nSore throat: sensitivity between 5% and 71%, specificity between 55% and 80%\nFever: sensitivity between 7% and 91%, specificity between 16% and 94%\nMyalgia or arthralgia: sensitivity between 19% and 86%, specificity between 45% and 91%\nFatigue: sensitivity between 10% and 57%, specificity between 60% and 94%\nHeadache: sensitivity between 3% and 71%, specificity between 78% and 98%\nFor fever, six of nine studies report a sensitivity of at least 80%, which is unsurprising considering fever was a key feature of COVID‐19 that was used in selecting patients for further testing. As a result, most participants in these studies would have fever, both cases and non‐cases. The same applies to cough, which was also listed as one of the main criteria for SARS‐CoV‐2 testing and may have contributed to inflated sensitivity estimates.\nSpecificity of at least 90% was achieved for 19 signs and symptoms. In only four signs and symptoms did this go along with sensitivity of at least 50% which would correspond to a positive likelihood ratio of at least 5, a commonly used arbitrary definition of a red flag. Using this definition, fever, myalgia or arthralgia, fatigue, or headache are to be considered red flags.\nStrikingly, most of the respiratory symptoms such as cough, sore throat and sputum production are below the diagonal in ROC space (Figure 8). The diagonal line in ROC space is where sensitivity equals 1‐specificity, meaning a test that is on the diagonal line has a positive likelihood ratio of 1 and is therefore not diagnostic because disease probability is left unchanged after conducting the test. Tests that lie below the diagonal line have a positive likelihood ratio that is smaller than 1, meaning the probability of COVID‐19 disease decreases when this test is positive. For example, in Sun 2020a, pretest probability of COVID‐19 is 6.9%; probability decreases to 6.4% when the patient has a cough and increases to 8.0% when the patient does not have a cough. We hypothesise on the reason for this counterintuitive finding in the discussion section. In contrast to respiratory features, systemic features are mostly above the diagonal line (Figure 9), suggesting that they do increase the probability of COVID‐19 when present. Gastrointestinal symptoms and cardiovascular features are clustered in the bottom left corner or on the diagonal line suggesting that they have very little diagnostic value (Figure 10; Figure 11).\nTo further illustrate the systemic features' ability to either rule in or rule out COVID‐19 disease or COVID‐19 pneumonia, we constructed dumbbell plots showing pre‐ and post‐test probabilities for each feature in each study (Figure 16). For each test, we have plotted the pre‐test probability, which is the prevalence of COVID‐19 disease (blue dot). Probability then changes depending on a positive test result (red dot marked +) or a negative test result (green dot marked ‐). The plot shows that fever, for example, increases the probability of COVID‐19 in two studies (Ai 2020a; Rentsch 2020), makes little to no difference in five studies (Feng 2020; Liang 2020; Peng 2020; Song 2020; Zhu 2020), and decreases the probability of COVID‐19 in two studies (Cheng 2020a; Tolia 2020).\n16 Dumbbell plot: this plot shows how disease probability changes after a positive test result (red dot with plus sign) or after a negative test (green dot with minus sign). Pre‐test probability or prevalence is the blue dot"}

    LitCovid-PD-UBERON

    {"project":"LitCovid-PD-UBERON","denotations":[{"id":"T27","span":{"begin":2145,"end":2151},"obj":"Body_part"},{"id":"T28","span":{"begin":2306,"end":2312},"obj":"Body_part"},{"id":"T29","span":{"begin":3192,"end":3198},"obj":"Body_part"},{"id":"T30","span":{"begin":3436,"end":3442},"obj":"Body_part"},{"id":"T31","span":{"begin":3491,"end":3497},"obj":"Body_part"},{"id":"T32","span":{"begin":4425,"end":4431},"obj":"Body_part"},{"id":"T33","span":{"begin":4435,"end":4441},"obj":"Body_part"},{"id":"T34","span":{"begin":10941,"end":10947},"obj":"Body_part"},{"id":"T35","span":{"begin":11683,"end":11688},"obj":"Body_part"},{"id":"T36","span":{"begin":12123,"end":12129},"obj":"Body_part"},{"id":"T37","span":{"begin":13393,"end":13399},"obj":"Body_part"},{"id":"T38","span":{"begin":13404,"end":13410},"obj":"Body_part"}],"attributes":[{"id":"A27","pred":"uberon_id","subj":"T27","obj":"http://purl.obolibrary.org/obo/UBERON_0000341"},{"id":"A28","pred":"uberon_id","subj":"T28","obj":"http://purl.obolibrary.org/obo/UBERON_0000341"},{"id":"A29","pred":"uberon_id","subj":"T29","obj":"http://purl.obolibrary.org/obo/UBERON_0007311"},{"id":"A30","pred":"uberon_id","subj":"T30","obj":"http://purl.obolibrary.org/obo/UBERON_0007311"},{"id":"A31","pred":"uberon_id","subj":"T31","obj":"http://purl.obolibrary.org/obo/UBERON_0000341"},{"id":"A32","pred":"uberon_id","subj":"T32","obj":"http://purl.obolibrary.org/obo/UBERON_0000341"},{"id":"A33","pred":"uberon_id","subj":"T33","obj":"http://purl.obolibrary.org/obo/UBERON_0007311"},{"id":"A34","pred":"uberon_id","subj":"T34","obj":"http://purl.obolibrary.org/obo/UBERON_0000341"},{"id":"A35","pred":"uberon_id","subj":"T35","obj":"http://purl.obolibrary.org/obo/UBERON_0001443"},{"id":"A36","pred":"uberon_id","subj":"T36","obj":"http://purl.obolibrary.org/obo/UBERON_0000341"},{"id":"A37","pred":"uberon_id","subj":"T37","obj":"http://purl.obolibrary.org/obo/UBERON_0000341"},{"id":"A38","pred":"uberon_id","subj":"T38","obj":"http://purl.obolibrary.org/obo/UBERON_0007311"}],"text":"Results\n\nResults of the search\nThe search yielded 10,965 records after removing duplicates. The first selection resulted in 658 records that were potentially eligible for this review on signs and symptoms. After screening on title and abstract, we excluded 457 records, leaving 201 to be assessed on full text. Of these, we included 16 studies in this review. The reasons for excluding 185 records are listed in the PRISMA flow chart (see Figure 1; Moher 2009).\n1 Flow diagram Two studies reported on the same cases while using a different control group (Chen X 2020; Yang 2020d). Chen X 2020 used a concurrent control group of pneumonia cases negative for SARS‐CoV‐2 on PCR testing but Yang 2020d used a historic control group of influenza pneumonia patients. For this reason we only included the Chen X 2020 results in the analyses.\nOne study reported a study that included a derivation and validation part for the development of a prediction rule (Song 2020b). The two parts are identical in set‐up and only differ in respect to the time of data collection, that is, the derivation part recruited participants up to 5 February 2020 and the validation part recruited participants from 6 February 2020 onwards. As a result, we consider this to be one study and have entered all data on signs and symptoms as such.\nFour studies were conducted in the USA, all other studies were from China. A summary of the main study characteristics can be found in Table 3.\n2 Summary of study characteristics\nStudy ID Target condition Sample size Prevalence Setting Population Design Reference standard\nAi 2020a COVID‐19 pneumonia 53 38% Hospital inpatientsa Patients hospitalised with pneumonia diagnosed by imaging Cross‐sectional PCR on nasopharyngeal swabs\nChen X 2020 COVID‐19 pneumonia 136 Not applicable Hospital inpatientsa Patients admitted with pneumonia Cases selected cross‐sectionally in 5 hospitals, non‐cases from 1 hospital only PCR, samples not specified\nCheng 2020a COVID‐19 pneumonia 33 33% Hospital outpatients Patients presenting to a fever observation department with pneumonia Cross‐sectional PCR on throat swabs\nFeng 2020a COVID‐19 pneumonia 132 5% Emergency department Patients presenting to fever clinic of emergency department Cross‐sectional PCR on throat swabs\nLiang 2020 COVID‐19 pneumonia 88 24% Hospital outpatients Patients with pneumonia and presenting to fever clinic Cross‐sectional PCR, sample not specified; conducted after panel discussion\nNobel 2020 COVID‐19 disease 516 Not applicable Hospital outpatients Patients who underwent SARS‐CoV‐2 testing with intent to hospitalise or in essential personnel Case‐control PCR on nasopharyngeal swabs\nPeng 2020a COVID‐19 disease 86 13% Hospital outpatients Patients clinically suspected and referred for testing Cross‐sectional PCR on nasopharyngeal swabs\nRentsch 2020 COVID‐19 disease 3789 15% Unclear Patients tested for SARS‐CoV‐2 in the Veterans Affairs Cohort born between 1945 and 1965 Cross‐sectional PCR on nasopharyngeal swabs\nSong 2020b COVID‐19 disease 399 7% Hospital outpatients Patients tested for SARS‐CoV‐2 Cross‐sectional PCR on sputum samples\nSun 2020a COVID‐19 disease 788 7% Hospital outpatients Patients presenting to testing centre, either self‐referred, referred from primary care or at‐risk cases identified by national contact tracing Cross‐sectional PCR on sputum, endotracheal aspirate, nasopharyngeal swabs or throat swabs\nTolia 2020 COVID‐19 disease 283 10% Emergency department Patients presenting with symptoms, travel history, risk factors or healthcare workers Cross‐sectional PCR on nasopharyngeal swabs\nWee 2020 COVID‐19 disease 870 18% Emergency department Patients presenting with respiratory symptoms or travel history Cross‐sectional PCR on oropharyngeal swabs\nYan 2020a COVID‐19 disease 262 23% Hospital outpatient Patients presenting hospital for SARS‐CoV‐2 testing, not otherwise specified Internet survey after presentation PCR, samples not specified\nYang 2020d COVID‐19 pneumonia 121 Not applicable Hospital inpatientsa Patient with pneumonia from SARS‐CoV‐2 and patients with pneumonia from influenza in 2015‐2019 Case‐control PCR, samples not specified\nZhao 2020a COVID‐19 pneumonia 34 Not applicable Hospital inpatientsa Patients with pneumonia and admitted to hospital Case‐control PCR on throat or sputum swabs\nZhu 2020b COVID‐19 disease 116 28% Emergency department Patients suspected of SARS‐CoV‐2 and presenting to the emergency department Cross‐sectional PCR, samples not specified\nPCR: polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2\na'Hospital inpatients' refers to studies that recruited patients admitted to hospital with COVID‐19 disease and in whom the signs and symptoms were assessed on admission.\n\nMethodological quality of included studies\nThe results of the quality assessment are summarised in Figure 2 and Figure 3. We rated participant selection as introducing high risk of bias in seven studies. In five studies this was because a CT scan or other imaging was used to diagnose patients with pneumonia prior to inclusion in the study, leading to a highly selected patient population (Ai 2020a; Chen X 2020; Cheng 2020a; Liang 2020; Yang 2020d); RT‐PCR results were subsequently used to distinguish between COVID‐19 pneumonia and pneumonia from other causes. For all studies, testing was highly dependent on the local case definition and testing criteria that were in effect at the time of the study, meaning all patients that were included in studies had already gone through a referral/selection filter, which was not always described. The most extreme example of this is the study by Liang 2020, in which patients with radiological evidence of pneumonia and a clinical presentation compatible with COVID‐19 were only tested for SARS‐CoV‐2 after a panel discussion.\n2 Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies\n3 Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study Of the 16 studies included in this first version of the review, five studies did not use a cross‐sectional design. Three studies were diagnostic case‐control studies (Nobel 2020; Yang 2020d; Zhao 2020a), one study selected cases cross‐sectionally in five hospitals but only selected cases in one hospital (Chen X 2020), and one study emailed patients who had undergone testing for SARS‐CoV‐2 about olfactory symptoms prior to the SARS‐CoV‐2 test, with a response rate of 58% in SARS‐CoV‐2 positive cases and 15% in negative cases (Yan 2020a).\nWe rated all studies except two as carrying a high risk of bias for the index tests because there was little to no detail on how, by whom, and when the signs and symptoms were measured. In addition, there is considerable uncertainty around the reference standard, with some studies providing little detail on the RT‐PCR tests that they used or lack of clarity on blinding.\nParticipant flow was unclear in four studies (Yan 2020a; Yang 2020d; Zhao 2020a; Zhu 2020b), either because the timing of recording signs and symptoms and conduct of the reference standard was unclear, or because some tests received a second or third reference standard at unclear time points during hospital admission.\nWe rated applicability for participant selection as high risk when there was a risk of selection bias or studies did not describe selection. As for the applicability of the index tests and reference standard, we always scored this as low risk except for Chen X 2020, because blinding of the index tests was unclear, and Yang 2020d, because blinding and sample of the reference standard were unclear.\n\nFindings\nThe main characteristics of all included studies are listed in Table 3. There were four studies in hospital inpatients (Ai 2020a; Chen X 2020; Yang 2020d; Zhao 2020a), seven studies in hospital outpatients (Cheng 2020a; Liang 2020; Nobel 2020; Peng 2020a; Song 2020b; Sun 2020a; Yan 2020a), and four studies in emergency departments (Feng 2020a; Tolia 2020; Wee 2020; Zhu 2020b). The setting was not specified in one study (Rentsch 2020); in the 'Summary of findings' table, we classified this study setting as being hospital outpatient under the assumption that at that time in the pandemic (February 2020 to March 2020) tests were not commonly available outside hospital clinics. There were no studies conducted in community primary care services.\nSeven studies assessed the accuracy of signs and symptoms for the diagnosis of COVID‐19 pneumonia (Ai 2020a; Chen X 2020; Cheng 2020a; Feng 2020a; Liang 2020; Yang 2020d; Zhao 2020a); the remaining studies had COVID‐19 disease as the target condition, with no further description of the severity, meaning some patients could have suffered from mild or moderate COVID‐19 disease and others from COVID‐19 pneumonia. The distinction between these two target conditions was not always very clear, and a degree of overlap is to be assumed. All studies used RT‐PCR testing as the reference standard, with some variation in the samples that were used.\nIn all, 7706 patients were included, the median number of participants was 134. Prevalence of infection varied from 5% to 38% with a median of 17%. There were no studies in children or elderly populations, except for Rentsch 2020, which included a cohort of a median age of 65.7 years old from the Veterans Affairs Healthcare System database.\nWe found data on 27 signs and symptoms, which fall into four different categories: systemic, respiratory, gastrointestinal and cardiovascular signs and symptoms. There were no analyses for combinations of tests, only for individual signs and symptoms. The results are summarised in Table 3. Results for the cross‐sectional studies are presented in forest plots (Figure 4; Figure 5; Figure 6; Figure 7), and are plotted in ROC (receiver operating characteristic) space (Figure 8; Figure 9; Figure 10; Figure 11), results for the other studies are only listed in forest plots (Figure 12; Figure 13; Figure 14; Figure 15).\n4 Forest plot of respiratory signs and symptoms (cross‐sectional studies)\n5 Forest plot of systemic signs and symptoms (cross‐sectional studies)\n6 Forest plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n7 Forest plot of cardiovascular signs and symptoms (cross‐sectional studies)\n8 Summary ROC plot of respiratory signs and symptoms (cross‐sectional studies)\n9 Summary ROC plot of systemic signs and symptoms (cross‐sectional studies)\n10 Summary ROC Plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n11 Summary ROC plot of cardiovascular signs and symptoms (cross‐sectional studies)\n12 Forest plot of tests: 27 cough (non‐cross‐sectional study), 28 sore throat (non‐cross‐sectional study), 29 rhinorrhoea (non‐cross‐sectional study), 30 nasal obstruction (non‐cross‐sectional study), 34 dyspnoea (non‐cross‐sectional study), 31 loss of sense of smell (non‐cross‐sectional study), 32 loss of taste (non‐cross‐sectional study), 33 positive auscultation findings (non‐cross‐sectional study)\n13 Forest plot of tests: 37 fatigue (non‐cross‐sectional study), 36 fever (non‐cross‐sectional study), 39 headache (non‐cross‐sectional study), 38 myalgia or arthralgia (non‐cross‐sectional study)\n14 Forest plot of tests: 40 diarrhoea (non‐cross‐sectional study), 41 nausea/vomiting (non‐cross‐sectional study), 42 gastrointestinal symptoms, not specified (non‐cross‐sectional study)\n15 Forest plot of 35 chest tightness (non‐cross‐sectional study) Overall, diagnostic accuracy of individual signs and symptoms is low, especially sensitivity. In addition, results were highly variable across studies, making it difficult to draw firm conclusions.\nSigns and symptoms for which sensitivity was reported above 50% in at least one cross‐sectional study are the following.\nCough: sensitivity between 43% and 71%, specificity between 14% and 54%\nSore throat: sensitivity between 5% and 71%, specificity between 55% and 80%\nFever: sensitivity between 7% and 91%, specificity between 16% and 94%\nMyalgia or arthralgia: sensitivity between 19% and 86%, specificity between 45% and 91%\nFatigue: sensitivity between 10% and 57%, specificity between 60% and 94%\nHeadache: sensitivity between 3% and 71%, specificity between 78% and 98%\nFor fever, six of nine studies report a sensitivity of at least 80%, which is unsurprising considering fever was a key feature of COVID‐19 that was used in selecting patients for further testing. As a result, most participants in these studies would have fever, both cases and non‐cases. The same applies to cough, which was also listed as one of the main criteria for SARS‐CoV‐2 testing and may have contributed to inflated sensitivity estimates.\nSpecificity of at least 90% was achieved for 19 signs and symptoms. In only four signs and symptoms did this go along with sensitivity of at least 50% which would correspond to a positive likelihood ratio of at least 5, a commonly used arbitrary definition of a red flag. Using this definition, fever, myalgia or arthralgia, fatigue, or headache are to be considered red flags.\nStrikingly, most of the respiratory symptoms such as cough, sore throat and sputum production are below the diagonal in ROC space (Figure 8). The diagonal line in ROC space is where sensitivity equals 1‐specificity, meaning a test that is on the diagonal line has a positive likelihood ratio of 1 and is therefore not diagnostic because disease probability is left unchanged after conducting the test. Tests that lie below the diagonal line have a positive likelihood ratio that is smaller than 1, meaning the probability of COVID‐19 disease decreases when this test is positive. For example, in Sun 2020a, pretest probability of COVID‐19 is 6.9%; probability decreases to 6.4% when the patient has a cough and increases to 8.0% when the patient does not have a cough. We hypothesise on the reason for this counterintuitive finding in the discussion section. In contrast to respiratory features, systemic features are mostly above the diagonal line (Figure 9), suggesting that they do increase the probability of COVID‐19 when present. Gastrointestinal symptoms and cardiovascular features are clustered in the bottom left corner or on the diagonal line suggesting that they have very little diagnostic value (Figure 10; Figure 11).\nTo further illustrate the systemic features' ability to either rule in or rule out COVID‐19 disease or COVID‐19 pneumonia, we constructed dumbbell plots showing pre‐ and post‐test probabilities for each feature in each study (Figure 16). For each test, we have plotted the pre‐test probability, which is the prevalence of COVID‐19 disease (blue dot). Probability then changes depending on a positive test result (red dot marked +) or a negative test result (green dot marked ‐). The plot shows that fever, for example, increases the probability of COVID‐19 in two studies (Ai 2020a; Rentsch 2020), makes little to no difference in five studies (Feng 2020; Liang 2020; Peng 2020; Song 2020; Zhu 2020), and decreases the probability of COVID‐19 in two studies (Cheng 2020a; Tolia 2020).\n16 Dumbbell plot: this plot shows how disease probability changes after a positive test result (red dot with plus sign) or after a negative test (green dot with minus sign). Pre‐test probability or prevalence is the blue dot"}

    LitCovid-PD-MONDO

    {"project":"LitCovid-PD-MONDO","denotations":[{"id":"T177","span":{"begin":629,"end":638},"obj":"Disease"},{"id":"T178","span":{"begin":658,"end":662},"obj":"Disease"},{"id":"T179","span":{"begin":732,"end":741},"obj":"Disease"},{"id":"T180","span":{"begin":742,"end":751},"obj":"Disease"},{"id":"T181","span":{"begin":1614,"end":1622},"obj":"Disease"},{"id":"T182","span":{"begin":1623,"end":1632},"obj":"Disease"},{"id":"T183","span":{"begin":1692,"end":1701},"obj":"Disease"},{"id":"T184","span":{"begin":1782,"end":1790},"obj":"Disease"},{"id":"T185","span":{"begin":1791,"end":1800},"obj":"Disease"},{"id":"T186","span":{"begin":1868,"end":1877},"obj":"Disease"},{"id":"T187","span":{"begin":2000,"end":2008},"obj":"Disease"},{"id":"T188","span":{"begin":2009,"end":2018},"obj":"Disease"},{"id":"T189","span":{"begin":2110,"end":2119},"obj":"Disease"},{"id":"T190","span":{"begin":2170,"end":2178},"obj":"Disease"},{"id":"T191","span":{"begin":2179,"end":2188},"obj":"Disease"},{"id":"T192","span":{"begin":2331,"end":2339},"obj":"Disease"},{"id":"T193","span":{"begin":2340,"end":2349},"obj":"Disease"},{"id":"T194","span":{"begin":2396,"end":2405},"obj":"Disease"},{"id":"T195","span":{"begin":2527,"end":2535},"obj":"Disease"},{"id":"T196","span":{"begin":2611,"end":2615},"obj":"Disease"},{"id":"T197","span":{"begin":2738,"end":2746},"obj":"Disease"},{"id":"T198","span":{"begin":2902,"end":2910},"obj":"Disease"},{"id":"T199","span":{"begin":2960,"end":2964},"obj":"Disease"},{"id":"T200","span":{"begin":3087,"end":3095},"obj":"Disease"},{"id":"T201","span":{"begin":3156,"end":3160},"obj":"Disease"},{"id":"T202","span":{"begin":3218,"end":3226},"obj":"Disease"},{"id":"T203","span":{"begin":3516,"end":3524},"obj":"Disease"},{"id":"T204","span":{"begin":3708,"end":3716},"obj":"Disease"},{"id":"T205","span":{"begin":3878,"end":3886},"obj":"Disease"},{"id":"T206","span":{"begin":3960,"end":3964},"obj":"Disease"},{"id":"T207","span":{"begin":4080,"end":4088},"obj":"Disease"},{"id":"T208","span":{"begin":4089,"end":4098},"obj":"Disease"},{"id":"T209","span":{"begin":4156,"end":4165},"obj":"Disease"},{"id":"T210","span":{"begin":4171,"end":4175},"obj":"Disease"},{"id":"T211","span":{"begin":4200,"end":4209},"obj":"Disease"},{"id":"T212","span":{"begin":4215,"end":4224},"obj":"Disease"},{"id":"T213","span":{"begin":4292,"end":4300},"obj":"Disease"},{"id":"T214","span":{"begin":4301,"end":4310},"obj":"Disease"},{"id":"T215","span":{"begin":4368,"end":4377},"obj":"Disease"},{"id":"T216","span":{"begin":4459,"end":4467},"obj":"Disease"},{"id":"T217","span":{"begin":4531,"end":4535},"obj":"Disease"},{"id":"T218","span":{"begin":4662,"end":4666},"obj":"Disease"},{"id":"T219","span":{"begin":4674,"end":4721},"obj":"Disease"},{"id":"T220","span":{"begin":4674,"end":4707},"obj":"Disease"},{"id":"T221","span":{"begin":4813,"end":4821},"obj":"Disease"},{"id":"T222","span":{"begin":5193,"end":5202},"obj":"Disease"},{"id":"T223","span":{"begin":5407,"end":5415},"obj":"Disease"},{"id":"T224","span":{"begin":5416,"end":5425},"obj":"Disease"},{"id":"T225","span":{"begin":5430,"end":5439},"obj":"Disease"},{"id":"T226","span":{"begin":5847,"end":5856},"obj":"Disease"},{"id":"T227","span":{"begin":5901,"end":5909},"obj":"Disease"},{"id":"T228","span":{"begin":5931,"end":5935},"obj":"Disease"},{"id":"T229","span":{"begin":6614,"end":6618},"obj":"Disease"},{"id":"T230","span":{"begin":6663,"end":6667},"obj":"Disease"},{"id":"T231","span":{"begin":6711,"end":6715},"obj":"Disease"},{"id":"T232","span":{"begin":8708,"end":8716},"obj":"Disease"},{"id":"T233","span":{"begin":8717,"end":8726},"obj":"Disease"},{"id":"T234","span":{"begin":8839,"end":8847},"obj":"Disease"},{"id":"T235","span":{"begin":8990,"end":8998},"obj":"Disease"},{"id":"T236","span":{"begin":9023,"end":9031},"obj":"Disease"},{"id":"T237","span":{"begin":9032,"end":9041},"obj":"Disease"},{"id":"T238","span":{"begin":9368,"end":9377},"obj":"Disease"},{"id":"T239","span":{"begin":10936,"end":10947},"obj":"Disease"},{"id":"T240","span":{"begin":11502,"end":11511},"obj":"Disease"},{"id":"T241","span":{"begin":12118,"end":12129},"obj":"Disease"},{"id":"T242","span":{"begin":12632,"end":12640},"obj":"Disease"},{"id":"T243","span":{"begin":12871,"end":12875},"obj":"Disease"},{"id":"T244","span":{"begin":13388,"end":13399},"obj":"Disease"},{"id":"T245","span":{"begin":13853,"end":13861},"obj":"Disease"},{"id":"T246","span":{"begin":13958,"end":13966},"obj":"Disease"},{"id":"T247","span":{"begin":14341,"end":14349},"obj":"Disease"},{"id":"T248","span":{"begin":14644,"end":14652},"obj":"Disease"},{"id":"T249","span":{"begin":14664,"end":14672},"obj":"Disease"},{"id":"T250","span":{"begin":14673,"end":14682},"obj":"Disease"},{"id":"T251","span":{"begin":14883,"end":14891},"obj":"Disease"},{"id":"T252","span":{"begin":15109,"end":15117},"obj":"Disease"},{"id":"T253","span":{"begin":15295,"end":15303},"obj":"Disease"}],"attributes":[{"id":"A177","pred":"mondo_id","subj":"T177","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A178","pred":"mondo_id","subj":"T178","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A179","pred":"mondo_id","subj":"T179","obj":"http://purl.obolibrary.org/obo/MONDO_0005812"},{"id":"A180","pred":"mondo_id","subj":"T180","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A181","pred":"mondo_id","subj":"T181","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A182","pred":"mondo_id","subj":"T182","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A183","pred":"mondo_id","subj":"T183","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A184","pred":"mondo_id","subj":"T184","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A185","pred":"mondo_id","subj":"T185","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A186","pred":"mondo_id","subj":"T186","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A187","pred":"mondo_id","subj":"T187","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A188","pred":"mondo_id","subj":"T188","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A189","pred":"mondo_id","subj":"T189","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A190","pred":"mondo_id","subj":"T190","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A191","pred":"mondo_id","subj":"T191","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A192","pred":"mondo_id","subj":"T192","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A193","pred":"mondo_id","subj":"T193","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A194","pred":"mondo_id","subj":"T194","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A195","pred":"mondo_id","subj":"T195","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A196","pred":"mondo_id","subj":"T196","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A197","pred":"mondo_id","subj":"T197","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A198","pred":"mondo_id","subj":"T198","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A199","pred":"mondo_id","subj":"T199","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A200","pred":"mondo_id","subj":"T200","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A201","pred":"mondo_id","subj":"T201","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A202","pred":"mondo_id","subj":"T202","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A203","pred":"mondo_id","subj":"T203","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A204","pred":"mondo_id","subj":"T204","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A205","pred":"mondo_id","subj":"T205","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A206","pred":"mondo_id","subj":"T206","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A207","pred":"mondo_id","subj":"T207","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A208","pred":"mondo_id","subj":"T208","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A209","pred":"mondo_id","subj":"T209","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A210","pred":"mondo_id","subj":"T210","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A211","pred":"mondo_id","subj":"T211","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A212","pred":"mondo_id","subj":"T212","obj":"http://purl.obolibrary.org/obo/MONDO_0005812"},{"id":"A213","pred":"mondo_id","subj":"T213","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A214","pred":"mondo_id","subj":"T214","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A215","pred":"mondo_id","subj":"T215","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A216","pred":"mondo_id","subj":"T216","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A217","pred":"mondo_id","subj":"T217","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A218","pred":"mondo_id","subj":"T218","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A219","pred":"mondo_id","subj":"T219","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A220","pred":"mondo_id","subj":"T220","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A221","pred":"mondo_id","subj":"T221","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A222","pred":"mondo_id","subj":"T222","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A223","pred":"mondo_id","subj":"T223","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A224","pred":"mondo_id","subj":"T224","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A225","pred":"mondo_id","subj":"T225","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A226","pred":"mondo_id","subj":"T226","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A227","pred":"mondo_id","subj":"T227","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A228","pred":"mondo_id","subj":"T228","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A229","pred":"mondo_id","subj":"T229","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A230","pred":"mondo_id","subj":"T230","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A231","pred":"mondo_id","subj":"T231","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A232","pred":"mondo_id","subj":"T232","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A233","pred":"mondo_id","subj":"T233","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A234","pred":"mondo_id","subj":"T234","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A235","pred":"mondo_id","subj":"T235","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A236","pred":"mondo_id","subj":"T236","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A237","pred":"mondo_id","subj":"T237","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A238","pred":"mondo_id","subj":"T238","obj":"http://purl.obolibrary.org/obo/MONDO_0005550"},{"id":"A239","pred":"mondo_id","subj":"T239","obj":"http://purl.obolibrary.org/obo/MONDO_0002258"},{"id":"A240","pred":"mondo_id","subj":"T240","obj":"http://purl.obolibrary.org/obo/MONDO_0001673"},{"id":"A241","pred":"mondo_id","subj":"T241","obj":"http://purl.obolibrary.org/obo/MONDO_0002258"},{"id":"A242","pred":"mondo_id","subj":"T242","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A243","pred":"mondo_id","subj":"T243","obj":"http://purl.obolibrary.org/obo/MONDO_0005091"},{"id":"A244","pred":"mondo_id","subj":"T244","obj":"http://purl.obolibrary.org/obo/MONDO_0002258"},{"id":"A245","pred":"mondo_id","subj":"T245","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A246","pred":"mondo_id","subj":"T246","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A247","pred":"mondo_id","subj":"T247","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A248","pred":"mondo_id","subj":"T248","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A249","pred":"mondo_id","subj":"T249","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A250","pred":"mondo_id","subj":"T250","obj":"http://purl.obolibrary.org/obo/MONDO_0005249"},{"id":"A251","pred":"mondo_id","subj":"T251","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A252","pred":"mondo_id","subj":"T252","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"},{"id":"A253","pred":"mondo_id","subj":"T253","obj":"http://purl.obolibrary.org/obo/MONDO_0100096"}],"text":"Results\n\nResults of the search\nThe search yielded 10,965 records after removing duplicates. The first selection resulted in 658 records that were potentially eligible for this review on signs and symptoms. After screening on title and abstract, we excluded 457 records, leaving 201 to be assessed on full text. Of these, we included 16 studies in this review. The reasons for excluding 185 records are listed in the PRISMA flow chart (see Figure 1; Moher 2009).\n1 Flow diagram Two studies reported on the same cases while using a different control group (Chen X 2020; Yang 2020d). Chen X 2020 used a concurrent control group of pneumonia cases negative for SARS‐CoV‐2 on PCR testing but Yang 2020d used a historic control group of influenza pneumonia patients. For this reason we only included the Chen X 2020 results in the analyses.\nOne study reported a study that included a derivation and validation part for the development of a prediction rule (Song 2020b). The two parts are identical in set‐up and only differ in respect to the time of data collection, that is, the derivation part recruited participants up to 5 February 2020 and the validation part recruited participants from 6 February 2020 onwards. As a result, we consider this to be one study and have entered all data on signs and symptoms as such.\nFour studies were conducted in the USA, all other studies were from China. A summary of the main study characteristics can be found in Table 3.\n2 Summary of study characteristics\nStudy ID Target condition Sample size Prevalence Setting Population Design Reference standard\nAi 2020a COVID‐19 pneumonia 53 38% Hospital inpatientsa Patients hospitalised with pneumonia diagnosed by imaging Cross‐sectional PCR on nasopharyngeal swabs\nChen X 2020 COVID‐19 pneumonia 136 Not applicable Hospital inpatientsa Patients admitted with pneumonia Cases selected cross‐sectionally in 5 hospitals, non‐cases from 1 hospital only PCR, samples not specified\nCheng 2020a COVID‐19 pneumonia 33 33% Hospital outpatients Patients presenting to a fever observation department with pneumonia Cross‐sectional PCR on throat swabs\nFeng 2020a COVID‐19 pneumonia 132 5% Emergency department Patients presenting to fever clinic of emergency department Cross‐sectional PCR on throat swabs\nLiang 2020 COVID‐19 pneumonia 88 24% Hospital outpatients Patients with pneumonia and presenting to fever clinic Cross‐sectional PCR, sample not specified; conducted after panel discussion\nNobel 2020 COVID‐19 disease 516 Not applicable Hospital outpatients Patients who underwent SARS‐CoV‐2 testing with intent to hospitalise or in essential personnel Case‐control PCR on nasopharyngeal swabs\nPeng 2020a COVID‐19 disease 86 13% Hospital outpatients Patients clinically suspected and referred for testing Cross‐sectional PCR on nasopharyngeal swabs\nRentsch 2020 COVID‐19 disease 3789 15% Unclear Patients tested for SARS‐CoV‐2 in the Veterans Affairs Cohort born between 1945 and 1965 Cross‐sectional PCR on nasopharyngeal swabs\nSong 2020b COVID‐19 disease 399 7% Hospital outpatients Patients tested for SARS‐CoV‐2 Cross‐sectional PCR on sputum samples\nSun 2020a COVID‐19 disease 788 7% Hospital outpatients Patients presenting to testing centre, either self‐referred, referred from primary care or at‐risk cases identified by national contact tracing Cross‐sectional PCR on sputum, endotracheal aspirate, nasopharyngeal swabs or throat swabs\nTolia 2020 COVID‐19 disease 283 10% Emergency department Patients presenting with symptoms, travel history, risk factors or healthcare workers Cross‐sectional PCR on nasopharyngeal swabs\nWee 2020 COVID‐19 disease 870 18% Emergency department Patients presenting with respiratory symptoms or travel history Cross‐sectional PCR on oropharyngeal swabs\nYan 2020a COVID‐19 disease 262 23% Hospital outpatient Patients presenting hospital for SARS‐CoV‐2 testing, not otherwise specified Internet survey after presentation PCR, samples not specified\nYang 2020d COVID‐19 pneumonia 121 Not applicable Hospital inpatientsa Patient with pneumonia from SARS‐CoV‐2 and patients with pneumonia from influenza in 2015‐2019 Case‐control PCR, samples not specified\nZhao 2020a COVID‐19 pneumonia 34 Not applicable Hospital inpatientsa Patients with pneumonia and admitted to hospital Case‐control PCR on throat or sputum swabs\nZhu 2020b COVID‐19 disease 116 28% Emergency department Patients suspected of SARS‐CoV‐2 and presenting to the emergency department Cross‐sectional PCR, samples not specified\nPCR: polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2\na'Hospital inpatients' refers to studies that recruited patients admitted to hospital with COVID‐19 disease and in whom the signs and symptoms were assessed on admission.\n\nMethodological quality of included studies\nThe results of the quality assessment are summarised in Figure 2 and Figure 3. We rated participant selection as introducing high risk of bias in seven studies. In five studies this was because a CT scan or other imaging was used to diagnose patients with pneumonia prior to inclusion in the study, leading to a highly selected patient population (Ai 2020a; Chen X 2020; Cheng 2020a; Liang 2020; Yang 2020d); RT‐PCR results were subsequently used to distinguish between COVID‐19 pneumonia and pneumonia from other causes. For all studies, testing was highly dependent on the local case definition and testing criteria that were in effect at the time of the study, meaning all patients that were included in studies had already gone through a referral/selection filter, which was not always described. The most extreme example of this is the study by Liang 2020, in which patients with radiological evidence of pneumonia and a clinical presentation compatible with COVID‐19 were only tested for SARS‐CoV‐2 after a panel discussion.\n2 Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies\n3 Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study Of the 16 studies included in this first version of the review, five studies did not use a cross‐sectional design. Three studies were diagnostic case‐control studies (Nobel 2020; Yang 2020d; Zhao 2020a), one study selected cases cross‐sectionally in five hospitals but only selected cases in one hospital (Chen X 2020), and one study emailed patients who had undergone testing for SARS‐CoV‐2 about olfactory symptoms prior to the SARS‐CoV‐2 test, with a response rate of 58% in SARS‐CoV‐2 positive cases and 15% in negative cases (Yan 2020a).\nWe rated all studies except two as carrying a high risk of bias for the index tests because there was little to no detail on how, by whom, and when the signs and symptoms were measured. In addition, there is considerable uncertainty around the reference standard, with some studies providing little detail on the RT‐PCR tests that they used or lack of clarity on blinding.\nParticipant flow was unclear in four studies (Yan 2020a; Yang 2020d; Zhao 2020a; Zhu 2020b), either because the timing of recording signs and symptoms and conduct of the reference standard was unclear, or because some tests received a second or third reference standard at unclear time points during hospital admission.\nWe rated applicability for participant selection as high risk when there was a risk of selection bias or studies did not describe selection. As for the applicability of the index tests and reference standard, we always scored this as low risk except for Chen X 2020, because blinding of the index tests was unclear, and Yang 2020d, because blinding and sample of the reference standard were unclear.\n\nFindings\nThe main characteristics of all included studies are listed in Table 3. There were four studies in hospital inpatients (Ai 2020a; Chen X 2020; Yang 2020d; Zhao 2020a), seven studies in hospital outpatients (Cheng 2020a; Liang 2020; Nobel 2020; Peng 2020a; Song 2020b; Sun 2020a; Yan 2020a), and four studies in emergency departments (Feng 2020a; Tolia 2020; Wee 2020; Zhu 2020b). The setting was not specified in one study (Rentsch 2020); in the 'Summary of findings' table, we classified this study setting as being hospital outpatient under the assumption that at that time in the pandemic (February 2020 to March 2020) tests were not commonly available outside hospital clinics. There were no studies conducted in community primary care services.\nSeven studies assessed the accuracy of signs and symptoms for the diagnosis of COVID‐19 pneumonia (Ai 2020a; Chen X 2020; Cheng 2020a; Feng 2020a; Liang 2020; Yang 2020d; Zhao 2020a); the remaining studies had COVID‐19 disease as the target condition, with no further description of the severity, meaning some patients could have suffered from mild or moderate COVID‐19 disease and others from COVID‐19 pneumonia. The distinction between these two target conditions was not always very clear, and a degree of overlap is to be assumed. All studies used RT‐PCR testing as the reference standard, with some variation in the samples that were used.\nIn all, 7706 patients were included, the median number of participants was 134. Prevalence of infection varied from 5% to 38% with a median of 17%. There were no studies in children or elderly populations, except for Rentsch 2020, which included a cohort of a median age of 65.7 years old from the Veterans Affairs Healthcare System database.\nWe found data on 27 signs and symptoms, which fall into four different categories: systemic, respiratory, gastrointestinal and cardiovascular signs and symptoms. There were no analyses for combinations of tests, only for individual signs and symptoms. The results are summarised in Table 3. Results for the cross‐sectional studies are presented in forest plots (Figure 4; Figure 5; Figure 6; Figure 7), and are plotted in ROC (receiver operating characteristic) space (Figure 8; Figure 9; Figure 10; Figure 11), results for the other studies are only listed in forest plots (Figure 12; Figure 13; Figure 14; Figure 15).\n4 Forest plot of respiratory signs and symptoms (cross‐sectional studies)\n5 Forest plot of systemic signs and symptoms (cross‐sectional studies)\n6 Forest plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n7 Forest plot of cardiovascular signs and symptoms (cross‐sectional studies)\n8 Summary ROC plot of respiratory signs and symptoms (cross‐sectional studies)\n9 Summary ROC plot of systemic signs and symptoms (cross‐sectional studies)\n10 Summary ROC Plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n11 Summary ROC plot of cardiovascular signs and symptoms (cross‐sectional studies)\n12 Forest plot of tests: 27 cough (non‐cross‐sectional study), 28 sore throat (non‐cross‐sectional study), 29 rhinorrhoea (non‐cross‐sectional study), 30 nasal obstruction (non‐cross‐sectional study), 34 dyspnoea (non‐cross‐sectional study), 31 loss of sense of smell (non‐cross‐sectional study), 32 loss of taste (non‐cross‐sectional study), 33 positive auscultation findings (non‐cross‐sectional study)\n13 Forest plot of tests: 37 fatigue (non‐cross‐sectional study), 36 fever (non‐cross‐sectional study), 39 headache (non‐cross‐sectional study), 38 myalgia or arthralgia (non‐cross‐sectional study)\n14 Forest plot of tests: 40 diarrhoea (non‐cross‐sectional study), 41 nausea/vomiting (non‐cross‐sectional study), 42 gastrointestinal symptoms, not specified (non‐cross‐sectional study)\n15 Forest plot of 35 chest tightness (non‐cross‐sectional study) Overall, diagnostic accuracy of individual signs and symptoms is low, especially sensitivity. In addition, results were highly variable across studies, making it difficult to draw firm conclusions.\nSigns and symptoms for which sensitivity was reported above 50% in at least one cross‐sectional study are the following.\nCough: sensitivity between 43% and 71%, specificity between 14% and 54%\nSore throat: sensitivity between 5% and 71%, specificity between 55% and 80%\nFever: sensitivity between 7% and 91%, specificity between 16% and 94%\nMyalgia or arthralgia: sensitivity between 19% and 86%, specificity between 45% and 91%\nFatigue: sensitivity between 10% and 57%, specificity between 60% and 94%\nHeadache: sensitivity between 3% and 71%, specificity between 78% and 98%\nFor fever, six of nine studies report a sensitivity of at least 80%, which is unsurprising considering fever was a key feature of COVID‐19 that was used in selecting patients for further testing. As a result, most participants in these studies would have fever, both cases and non‐cases. The same applies to cough, which was also listed as one of the main criteria for SARS‐CoV‐2 testing and may have contributed to inflated sensitivity estimates.\nSpecificity of at least 90% was achieved for 19 signs and symptoms. In only four signs and symptoms did this go along with sensitivity of at least 50% which would correspond to a positive likelihood ratio of at least 5, a commonly used arbitrary definition of a red flag. Using this definition, fever, myalgia or arthralgia, fatigue, or headache are to be considered red flags.\nStrikingly, most of the respiratory symptoms such as cough, sore throat and sputum production are below the diagonal in ROC space (Figure 8). The diagonal line in ROC space is where sensitivity equals 1‐specificity, meaning a test that is on the diagonal line has a positive likelihood ratio of 1 and is therefore not diagnostic because disease probability is left unchanged after conducting the test. Tests that lie below the diagonal line have a positive likelihood ratio that is smaller than 1, meaning the probability of COVID‐19 disease decreases when this test is positive. For example, in Sun 2020a, pretest probability of COVID‐19 is 6.9%; probability decreases to 6.4% when the patient has a cough and increases to 8.0% when the patient does not have a cough. We hypothesise on the reason for this counterintuitive finding in the discussion section. In contrast to respiratory features, systemic features are mostly above the diagonal line (Figure 9), suggesting that they do increase the probability of COVID‐19 when present. Gastrointestinal symptoms and cardiovascular features are clustered in the bottom left corner or on the diagonal line suggesting that they have very little diagnostic value (Figure 10; Figure 11).\nTo further illustrate the systemic features' ability to either rule in or rule out COVID‐19 disease or COVID‐19 pneumonia, we constructed dumbbell plots showing pre‐ and post‐test probabilities for each feature in each study (Figure 16). For each test, we have plotted the pre‐test probability, which is the prevalence of COVID‐19 disease (blue dot). Probability then changes depending on a positive test result (red dot marked +) or a negative test result (green dot marked ‐). The plot shows that fever, for example, increases the probability of COVID‐19 in two studies (Ai 2020a; Rentsch 2020), makes little to no difference in five studies (Feng 2020; Liang 2020; Peng 2020; Song 2020; Zhu 2020), and decreases the probability of COVID‐19 in two studies (Cheng 2020a; Tolia 2020).\n16 Dumbbell plot: this plot shows how disease probability changes after a positive test result (red dot with plus sign) or after a negative test (green dot with minus sign). Pre‐test probability or prevalence is the blue dot"}

    LitCovid-PD-CLO

    {"project":"LitCovid-PD-CLO","denotations":[{"id":"T45385","span":{"begin":529,"end":530},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T88385","span":{"begin":599,"end":600},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T11909","span":{"begin":676,"end":683},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T40143","span":{"begin":704,"end":705},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T69760","span":{"begin":855,"end":856},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T21019","span":{"begin":877,"end":878},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T86778","span":{"begin":933,"end":934},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T34676","span":{"begin":1216,"end":1217},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T34180","span":{"begin":1391,"end":1392},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T303","span":{"begin":2074,"end":2075},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T304","span":{"begin":2190,"end":2193},"obj":"http://purl.obolibrary.org/obo/CLO_0054061"},{"id":"T305","span":{"begin":2622,"end":2629},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T306","span":{"begin":2834,"end":2841},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T307","span":{"begin":2949,"end":2955},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T308","span":{"begin":3145,"end":3151},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T309","span":{"begin":3290,"end":3297},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T310","span":{"begin":3731,"end":3733},"obj":"http://purl.obolibrary.org/obo/CLO_0050510"},{"id":"T311","span":{"begin":3971,"end":3978},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T312","span":{"begin":4100,"end":4103},"obj":"http://purl.obolibrary.org/obo/CLO_0001053"},{"id":"T313","span":{"begin":4312,"end":4314},"obj":"http://purl.obolibrary.org/obo/CLO_0001302"},{"id":"T314","span":{"begin":4477,"end":4480},"obj":"http://purl.obolibrary.org/obo/CLO_0001046"},{"id":"T315","span":{"begin":5131,"end":5132},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T316","span":{"begin":5247,"end":5248},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T317","span":{"begin":5476,"end":5487},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T318","span":{"begin":5538,"end":5545},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T319","span":{"begin":5677,"end":5678},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T320","span":{"begin":5747,"end":5754},"obj":"http://www.ebi.ac.uk/efo/EFO_0000876"},{"id":"T321","span":{"begin":5861,"end":5862},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T322","span":{"begin":5920,"end":5926},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T323","span":{"begin":5948,"end":5949},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T324","span":{"begin":6322,"end":6323},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T325","span":{"begin":6602,"end":6609},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T326","span":{"begin":6674,"end":6678},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T327","span":{"begin":6685,"end":6686},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T328","span":{"begin":6820,"end":6821},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T329","span":{"begin":6854,"end":6859},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T330","span":{"begin":7096,"end":7101},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T331","span":{"begin":7367,"end":7372},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T332","span":{"begin":7382,"end":7383},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T333","span":{"begin":7546,"end":7547},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T334","span":{"begin":7648,"end":7653},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T335","span":{"begin":7766,"end":7771},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T336","span":{"begin":8501,"end":8506},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T337","span":{"begin":9126,"end":9127},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T338","span":{"begin":9188,"end":9198},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T339","span":{"begin":9405,"end":9406},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T340","span":{"begin":9520,"end":9521},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T341","span":{"begin":9532,"end":9533},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T342","span":{"begin":9634,"end":9636},"obj":"http://purl.obolibrary.org/obo/CLO_0050509"},{"id":"T343","span":{"begin":9822,"end":9827},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T344","span":{"begin":10124,"end":10126},"obj":"http://purl.obolibrary.org/obo/CLO_0053733"},{"id":"T345","span":{"begin":10785,"end":10787},"obj":"http://purl.obolibrary.org/obo/CLO_0053733"},{"id":"T346","span":{"begin":10888,"end":10893},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T347","span":{"begin":10895,"end":10897},"obj":"http://purl.obolibrary.org/obo/CLO_0050509"},{"id":"T348","span":{"begin":11071,"end":11073},"obj":"http://purl.obolibrary.org/obo/CLO_0001302"},{"id":"T349","span":{"begin":11294,"end":11299},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T350","span":{"begin":11341,"end":11343},"obj":"http://purl.obolibrary.org/obo/CLO_0001313"},{"id":"T351","span":{"begin":11492,"end":11497},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T352","span":{"begin":11541,"end":11543},"obj":"http://purl.obolibrary.org/obo/CLO_0053794"},{"id":"T353","span":{"begin":11680,"end":11682},"obj":"http://purl.obolibrary.org/obo/CLO_0001000"},{"id":"T354","span":{"begin":11683,"end":11688},"obj":"http://www.ebi.ac.uk/efo/EFO_0000965"},{"id":"T355","span":{"begin":12081,"end":12083},"obj":"http://purl.obolibrary.org/obo/CLO_0054055"},{"id":"T356","span":{"begin":12158,"end":12160},"obj":"http://purl.obolibrary.org/obo/CLO_0054055"},{"id":"T357","span":{"begin":12262,"end":12264},"obj":"http://purl.obolibrary.org/obo/CLO_0001527"},{"id":"T358","span":{"begin":12342,"end":12344},"obj":"http://purl.obolibrary.org/obo/CLO_0053799"},{"id":"T359","span":{"begin":12424,"end":12426},"obj":"http://purl.obolibrary.org/obo/CLO_0001527"},{"id":"T360","span":{"begin":12465,"end":12467},"obj":"http://purl.obolibrary.org/obo/CLO_0054055"},{"id":"T361","span":{"begin":12540,"end":12541},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T362","span":{"begin":12615,"end":12616},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T363","span":{"begin":12689,"end":12696},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T364","span":{"begin":12701,"end":12702},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T365","span":{"begin":12882,"end":12889},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T366","span":{"begin":13127,"end":13128},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T367","span":{"begin":13170,"end":13171},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T368","span":{"begin":13210,"end":13211},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T369","span":{"begin":13552,"end":13553},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T370","span":{"begin":13554,"end":13558},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T371","span":{"begin":13588,"end":13591},"obj":"http://purl.obolibrary.org/obo/CLO_0051582"},{"id":"T73125","span":{"begin":13592,"end":13593},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T373","span":{"begin":13724,"end":13728},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T374","span":{"begin":13730,"end":13735},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T375","span":{"begin":13774,"end":13775},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T376","span":{"begin":13890,"end":13897},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T377","span":{"begin":14023,"end":14026},"obj":"http://purl.obolibrary.org/obo/CLO_0051582"},{"id":"T378","span":{"begin":14027,"end":14028},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T379","span":{"begin":14088,"end":14089},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T380","span":{"begin":14556,"end":14558},"obj":"http://purl.obolibrary.org/obo/CLO_0053733"},{"id":"T381","span":{"begin":14736,"end":14740},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T382","span":{"begin":14808,"end":14812},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T383","span":{"begin":14838,"end":14842},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T384","span":{"begin":14950,"end":14951},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T385","span":{"begin":14961,"end":14965},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T386","span":{"begin":14995,"end":14996},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T387","span":{"begin":15006,"end":15010},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T388","span":{"begin":15419,"end":15420},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T389","span":{"begin":15430,"end":15434},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T390","span":{"begin":15476,"end":15477},"obj":"http://purl.obolibrary.org/obo/CLO_0001020"},{"id":"T391","span":{"begin":15487,"end":15491},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"},{"id":"T392","span":{"begin":15525,"end":15529},"obj":"http://purl.obolibrary.org/obo/UBERON_0000473"}],"text":"Results\n\nResults of the search\nThe search yielded 10,965 records after removing duplicates. The first selection resulted in 658 records that were potentially eligible for this review on signs and symptoms. After screening on title and abstract, we excluded 457 records, leaving 201 to be assessed on full text. Of these, we included 16 studies in this review. The reasons for excluding 185 records are listed in the PRISMA flow chart (see Figure 1; Moher 2009).\n1 Flow diagram Two studies reported on the same cases while using a different control group (Chen X 2020; Yang 2020d). Chen X 2020 used a concurrent control group of pneumonia cases negative for SARS‐CoV‐2 on PCR testing but Yang 2020d used a historic control group of influenza pneumonia patients. For this reason we only included the Chen X 2020 results in the analyses.\nOne study reported a study that included a derivation and validation part for the development of a prediction rule (Song 2020b). The two parts are identical in set‐up and only differ in respect to the time of data collection, that is, the derivation part recruited participants up to 5 February 2020 and the validation part recruited participants from 6 February 2020 onwards. As a result, we consider this to be one study and have entered all data on signs and symptoms as such.\nFour studies were conducted in the USA, all other studies were from China. A summary of the main study characteristics can be found in Table 3.\n2 Summary of study characteristics\nStudy ID Target condition Sample size Prevalence Setting Population Design Reference standard\nAi 2020a COVID‐19 pneumonia 53 38% Hospital inpatientsa Patients hospitalised with pneumonia diagnosed by imaging Cross‐sectional PCR on nasopharyngeal swabs\nChen X 2020 COVID‐19 pneumonia 136 Not applicable Hospital inpatientsa Patients admitted with pneumonia Cases selected cross‐sectionally in 5 hospitals, non‐cases from 1 hospital only PCR, samples not specified\nCheng 2020a COVID‐19 pneumonia 33 33% Hospital outpatients Patients presenting to a fever observation department with pneumonia Cross‐sectional PCR on throat swabs\nFeng 2020a COVID‐19 pneumonia 132 5% Emergency department Patients presenting to fever clinic of emergency department Cross‐sectional PCR on throat swabs\nLiang 2020 COVID‐19 pneumonia 88 24% Hospital outpatients Patients with pneumonia and presenting to fever clinic Cross‐sectional PCR, sample not specified; conducted after panel discussion\nNobel 2020 COVID‐19 disease 516 Not applicable Hospital outpatients Patients who underwent SARS‐CoV‐2 testing with intent to hospitalise or in essential personnel Case‐control PCR on nasopharyngeal swabs\nPeng 2020a COVID‐19 disease 86 13% Hospital outpatients Patients clinically suspected and referred for testing Cross‐sectional PCR on nasopharyngeal swabs\nRentsch 2020 COVID‐19 disease 3789 15% Unclear Patients tested for SARS‐CoV‐2 in the Veterans Affairs Cohort born between 1945 and 1965 Cross‐sectional PCR on nasopharyngeal swabs\nSong 2020b COVID‐19 disease 399 7% Hospital outpatients Patients tested for SARS‐CoV‐2 Cross‐sectional PCR on sputum samples\nSun 2020a COVID‐19 disease 788 7% Hospital outpatients Patients presenting to testing centre, either self‐referred, referred from primary care or at‐risk cases identified by national contact tracing Cross‐sectional PCR on sputum, endotracheal aspirate, nasopharyngeal swabs or throat swabs\nTolia 2020 COVID‐19 disease 283 10% Emergency department Patients presenting with symptoms, travel history, risk factors or healthcare workers Cross‐sectional PCR on nasopharyngeal swabs\nWee 2020 COVID‐19 disease 870 18% Emergency department Patients presenting with respiratory symptoms or travel history Cross‐sectional PCR on oropharyngeal swabs\nYan 2020a COVID‐19 disease 262 23% Hospital outpatient Patients presenting hospital for SARS‐CoV‐2 testing, not otherwise specified Internet survey after presentation PCR, samples not specified\nYang 2020d COVID‐19 pneumonia 121 Not applicable Hospital inpatientsa Patient with pneumonia from SARS‐CoV‐2 and patients with pneumonia from influenza in 2015‐2019 Case‐control PCR, samples not specified\nZhao 2020a COVID‐19 pneumonia 34 Not applicable Hospital inpatientsa Patients with pneumonia and admitted to hospital Case‐control PCR on throat or sputum swabs\nZhu 2020b COVID‐19 disease 116 28% Emergency department Patients suspected of SARS‐CoV‐2 and presenting to the emergency department Cross‐sectional PCR, samples not specified\nPCR: polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2\na'Hospital inpatients' refers to studies that recruited patients admitted to hospital with COVID‐19 disease and in whom the signs and symptoms were assessed on admission.\n\nMethodological quality of included studies\nThe results of the quality assessment are summarised in Figure 2 and Figure 3. We rated participant selection as introducing high risk of bias in seven studies. In five studies this was because a CT scan or other imaging was used to diagnose patients with pneumonia prior to inclusion in the study, leading to a highly selected patient population (Ai 2020a; Chen X 2020; Cheng 2020a; Liang 2020; Yang 2020d); RT‐PCR results were subsequently used to distinguish between COVID‐19 pneumonia and pneumonia from other causes. For all studies, testing was highly dependent on the local case definition and testing criteria that were in effect at the time of the study, meaning all patients that were included in studies had already gone through a referral/selection filter, which was not always described. The most extreme example of this is the study by Liang 2020, in which patients with radiological evidence of pneumonia and a clinical presentation compatible with COVID‐19 were only tested for SARS‐CoV‐2 after a panel discussion.\n2 Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies\n3 Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study Of the 16 studies included in this first version of the review, five studies did not use a cross‐sectional design. Three studies were diagnostic case‐control studies (Nobel 2020; Yang 2020d; Zhao 2020a), one study selected cases cross‐sectionally in five hospitals but only selected cases in one hospital (Chen X 2020), and one study emailed patients who had undergone testing for SARS‐CoV‐2 about olfactory symptoms prior to the SARS‐CoV‐2 test, with a response rate of 58% in SARS‐CoV‐2 positive cases and 15% in negative cases (Yan 2020a).\nWe rated all studies except two as carrying a high risk of bias for the index tests because there was little to no detail on how, by whom, and when the signs and symptoms were measured. In addition, there is considerable uncertainty around the reference standard, with some studies providing little detail on the RT‐PCR tests that they used or lack of clarity on blinding.\nParticipant flow was unclear in four studies (Yan 2020a; Yang 2020d; Zhao 2020a; Zhu 2020b), either because the timing of recording signs and symptoms and conduct of the reference standard was unclear, or because some tests received a second or third reference standard at unclear time points during hospital admission.\nWe rated applicability for participant selection as high risk when there was a risk of selection bias or studies did not describe selection. As for the applicability of the index tests and reference standard, we always scored this as low risk except for Chen X 2020, because blinding of the index tests was unclear, and Yang 2020d, because blinding and sample of the reference standard were unclear.\n\nFindings\nThe main characteristics of all included studies are listed in Table 3. There were four studies in hospital inpatients (Ai 2020a; Chen X 2020; Yang 2020d; Zhao 2020a), seven studies in hospital outpatients (Cheng 2020a; Liang 2020; Nobel 2020; Peng 2020a; Song 2020b; Sun 2020a; Yan 2020a), and four studies in emergency departments (Feng 2020a; Tolia 2020; Wee 2020; Zhu 2020b). The setting was not specified in one study (Rentsch 2020); in the 'Summary of findings' table, we classified this study setting as being hospital outpatient under the assumption that at that time in the pandemic (February 2020 to March 2020) tests were not commonly available outside hospital clinics. There were no studies conducted in community primary care services.\nSeven studies assessed the accuracy of signs and symptoms for the diagnosis of COVID‐19 pneumonia (Ai 2020a; Chen X 2020; Cheng 2020a; Feng 2020a; Liang 2020; Yang 2020d; Zhao 2020a); the remaining studies had COVID‐19 disease as the target condition, with no further description of the severity, meaning some patients could have suffered from mild or moderate COVID‐19 disease and others from COVID‐19 pneumonia. The distinction between these two target conditions was not always very clear, and a degree of overlap is to be assumed. All studies used RT‐PCR testing as the reference standard, with some variation in the samples that were used.\nIn all, 7706 patients were included, the median number of participants was 134. Prevalence of infection varied from 5% to 38% with a median of 17%. There were no studies in children or elderly populations, except for Rentsch 2020, which included a cohort of a median age of 65.7 years old from the Veterans Affairs Healthcare System database.\nWe found data on 27 signs and symptoms, which fall into four different categories: systemic, respiratory, gastrointestinal and cardiovascular signs and symptoms. There were no analyses for combinations of tests, only for individual signs and symptoms. The results are summarised in Table 3. Results for the cross‐sectional studies are presented in forest plots (Figure 4; Figure 5; Figure 6; Figure 7), and are plotted in ROC (receiver operating characteristic) space (Figure 8; Figure 9; Figure 10; Figure 11), results for the other studies are only listed in forest plots (Figure 12; Figure 13; Figure 14; Figure 15).\n4 Forest plot of respiratory signs and symptoms (cross‐sectional studies)\n5 Forest plot of systemic signs and symptoms (cross‐sectional studies)\n6 Forest plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n7 Forest plot of cardiovascular signs and symptoms (cross‐sectional studies)\n8 Summary ROC plot of respiratory signs and symptoms (cross‐sectional studies)\n9 Summary ROC plot of systemic signs and symptoms (cross‐sectional studies)\n10 Summary ROC Plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n11 Summary ROC plot of cardiovascular signs and symptoms (cross‐sectional studies)\n12 Forest plot of tests: 27 cough (non‐cross‐sectional study), 28 sore throat (non‐cross‐sectional study), 29 rhinorrhoea (non‐cross‐sectional study), 30 nasal obstruction (non‐cross‐sectional study), 34 dyspnoea (non‐cross‐sectional study), 31 loss of sense of smell (non‐cross‐sectional study), 32 loss of taste (non‐cross‐sectional study), 33 positive auscultation findings (non‐cross‐sectional study)\n13 Forest plot of tests: 37 fatigue (non‐cross‐sectional study), 36 fever (non‐cross‐sectional study), 39 headache (non‐cross‐sectional study), 38 myalgia or arthralgia (non‐cross‐sectional study)\n14 Forest plot of tests: 40 diarrhoea (non‐cross‐sectional study), 41 nausea/vomiting (non‐cross‐sectional study), 42 gastrointestinal symptoms, not specified (non‐cross‐sectional study)\n15 Forest plot of 35 chest tightness (non‐cross‐sectional study) Overall, diagnostic accuracy of individual signs and symptoms is low, especially sensitivity. In addition, results were highly variable across studies, making it difficult to draw firm conclusions.\nSigns and symptoms for which sensitivity was reported above 50% in at least one cross‐sectional study are the following.\nCough: sensitivity between 43% and 71%, specificity between 14% and 54%\nSore throat: sensitivity between 5% and 71%, specificity between 55% and 80%\nFever: sensitivity between 7% and 91%, specificity between 16% and 94%\nMyalgia or arthralgia: sensitivity between 19% and 86%, specificity between 45% and 91%\nFatigue: sensitivity between 10% and 57%, specificity between 60% and 94%\nHeadache: sensitivity between 3% and 71%, specificity between 78% and 98%\nFor fever, six of nine studies report a sensitivity of at least 80%, which is unsurprising considering fever was a key feature of COVID‐19 that was used in selecting patients for further testing. As a result, most participants in these studies would have fever, both cases and non‐cases. The same applies to cough, which was also listed as one of the main criteria for SARS‐CoV‐2 testing and may have contributed to inflated sensitivity estimates.\nSpecificity of at least 90% was achieved for 19 signs and symptoms. In only four signs and symptoms did this go along with sensitivity of at least 50% which would correspond to a positive likelihood ratio of at least 5, a commonly used arbitrary definition of a red flag. Using this definition, fever, myalgia or arthralgia, fatigue, or headache are to be considered red flags.\nStrikingly, most of the respiratory symptoms such as cough, sore throat and sputum production are below the diagonal in ROC space (Figure 8). The diagonal line in ROC space is where sensitivity equals 1‐specificity, meaning a test that is on the diagonal line has a positive likelihood ratio of 1 and is therefore not diagnostic because disease probability is left unchanged after conducting the test. Tests that lie below the diagonal line have a positive likelihood ratio that is smaller than 1, meaning the probability of COVID‐19 disease decreases when this test is positive. For example, in Sun 2020a, pretest probability of COVID‐19 is 6.9%; probability decreases to 6.4% when the patient has a cough and increases to 8.0% when the patient does not have a cough. We hypothesise on the reason for this counterintuitive finding in the discussion section. In contrast to respiratory features, systemic features are mostly above the diagonal line (Figure 9), suggesting that they do increase the probability of COVID‐19 when present. Gastrointestinal symptoms and cardiovascular features are clustered in the bottom left corner or on the diagonal line suggesting that they have very little diagnostic value (Figure 10; Figure 11).\nTo further illustrate the systemic features' ability to either rule in or rule out COVID‐19 disease or COVID‐19 pneumonia, we constructed dumbbell plots showing pre‐ and post‐test probabilities for each feature in each study (Figure 16). For each test, we have plotted the pre‐test probability, which is the prevalence of COVID‐19 disease (blue dot). Probability then changes depending on a positive test result (red dot marked +) or a negative test result (green dot marked ‐). The plot shows that fever, for example, increases the probability of COVID‐19 in two studies (Ai 2020a; Rentsch 2020), makes little to no difference in five studies (Feng 2020; Liang 2020; Peng 2020; Song 2020; Zhu 2020), and decreases the probability of COVID‐19 in two studies (Cheng 2020a; Tolia 2020).\n16 Dumbbell plot: this plot shows how disease probability changes after a positive test result (red dot with plus sign) or after a negative test (green dot with minus sign). Pre‐test probability or prevalence is the blue dot"}

    LitCovid-PD-CHEBI

    {"project":"LitCovid-PD-CHEBI","denotations":[{"id":"T9","span":{"begin":549,"end":554},"obj":"Chemical"},{"id":"T10","span":{"begin":620,"end":625},"obj":"Chemical"},{"id":"T11","span":{"begin":723,"end":728},"obj":"Chemical"},{"id":"T12","span":{"begin":1502,"end":1504},"obj":"Chemical"},{"id":"T13","span":{"begin":2995,"end":3001},"obj":"Chemical"}],"attributes":[{"id":"A9","pred":"chebi_id","subj":"T9","obj":"http://purl.obolibrary.org/obo/CHEBI_24433"},{"id":"A10","pred":"chebi_id","subj":"T10","obj":"http://purl.obolibrary.org/obo/CHEBI_24433"},{"id":"A11","pred":"chebi_id","subj":"T11","obj":"http://purl.obolibrary.org/obo/CHEBI_24433"},{"id":"A12","pred":"chebi_id","subj":"T12","obj":"http://purl.obolibrary.org/obo/CHEBI_141439"},{"id":"A13","pred":"chebi_id","subj":"T13","obj":"http://purl.obolibrary.org/obo/CHEBI_34935"}],"text":"Results\n\nResults of the search\nThe search yielded 10,965 records after removing duplicates. The first selection resulted in 658 records that were potentially eligible for this review on signs and symptoms. After screening on title and abstract, we excluded 457 records, leaving 201 to be assessed on full text. Of these, we included 16 studies in this review. The reasons for excluding 185 records are listed in the PRISMA flow chart (see Figure 1; Moher 2009).\n1 Flow diagram Two studies reported on the same cases while using a different control group (Chen X 2020; Yang 2020d). Chen X 2020 used a concurrent control group of pneumonia cases negative for SARS‐CoV‐2 on PCR testing but Yang 2020d used a historic control group of influenza pneumonia patients. For this reason we only included the Chen X 2020 results in the analyses.\nOne study reported a study that included a derivation and validation part for the development of a prediction rule (Song 2020b). The two parts are identical in set‐up and only differ in respect to the time of data collection, that is, the derivation part recruited participants up to 5 February 2020 and the validation part recruited participants from 6 February 2020 onwards. As a result, we consider this to be one study and have entered all data on signs and symptoms as such.\nFour studies were conducted in the USA, all other studies were from China. A summary of the main study characteristics can be found in Table 3.\n2 Summary of study characteristics\nStudy ID Target condition Sample size Prevalence Setting Population Design Reference standard\nAi 2020a COVID‐19 pneumonia 53 38% Hospital inpatientsa Patients hospitalised with pneumonia diagnosed by imaging Cross‐sectional PCR on nasopharyngeal swabs\nChen X 2020 COVID‐19 pneumonia 136 Not applicable Hospital inpatientsa Patients admitted with pneumonia Cases selected cross‐sectionally in 5 hospitals, non‐cases from 1 hospital only PCR, samples not specified\nCheng 2020a COVID‐19 pneumonia 33 33% Hospital outpatients Patients presenting to a fever observation department with pneumonia Cross‐sectional PCR on throat swabs\nFeng 2020a COVID‐19 pneumonia 132 5% Emergency department Patients presenting to fever clinic of emergency department Cross‐sectional PCR on throat swabs\nLiang 2020 COVID‐19 pneumonia 88 24% Hospital outpatients Patients with pneumonia and presenting to fever clinic Cross‐sectional PCR, sample not specified; conducted after panel discussion\nNobel 2020 COVID‐19 disease 516 Not applicable Hospital outpatients Patients who underwent SARS‐CoV‐2 testing with intent to hospitalise or in essential personnel Case‐control PCR on nasopharyngeal swabs\nPeng 2020a COVID‐19 disease 86 13% Hospital outpatients Patients clinically suspected and referred for testing Cross‐sectional PCR on nasopharyngeal swabs\nRentsch 2020 COVID‐19 disease 3789 15% Unclear Patients tested for SARS‐CoV‐2 in the Veterans Affairs Cohort born between 1945 and 1965 Cross‐sectional PCR on nasopharyngeal swabs\nSong 2020b COVID‐19 disease 399 7% Hospital outpatients Patients tested for SARS‐CoV‐2 Cross‐sectional PCR on sputum samples\nSun 2020a COVID‐19 disease 788 7% Hospital outpatients Patients presenting to testing centre, either self‐referred, referred from primary care or at‐risk cases identified by national contact tracing Cross‐sectional PCR on sputum, endotracheal aspirate, nasopharyngeal swabs or throat swabs\nTolia 2020 COVID‐19 disease 283 10% Emergency department Patients presenting with symptoms, travel history, risk factors or healthcare workers Cross‐sectional PCR on nasopharyngeal swabs\nWee 2020 COVID‐19 disease 870 18% Emergency department Patients presenting with respiratory symptoms or travel history Cross‐sectional PCR on oropharyngeal swabs\nYan 2020a COVID‐19 disease 262 23% Hospital outpatient Patients presenting hospital for SARS‐CoV‐2 testing, not otherwise specified Internet survey after presentation PCR, samples not specified\nYang 2020d COVID‐19 pneumonia 121 Not applicable Hospital inpatientsa Patient with pneumonia from SARS‐CoV‐2 and patients with pneumonia from influenza in 2015‐2019 Case‐control PCR, samples not specified\nZhao 2020a COVID‐19 pneumonia 34 Not applicable Hospital inpatientsa Patients with pneumonia and admitted to hospital Case‐control PCR on throat or sputum swabs\nZhu 2020b COVID‐19 disease 116 28% Emergency department Patients suspected of SARS‐CoV‐2 and presenting to the emergency department Cross‐sectional PCR, samples not specified\nPCR: polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2\na'Hospital inpatients' refers to studies that recruited patients admitted to hospital with COVID‐19 disease and in whom the signs and symptoms were assessed on admission.\n\nMethodological quality of included studies\nThe results of the quality assessment are summarised in Figure 2 and Figure 3. We rated participant selection as introducing high risk of bias in seven studies. In five studies this was because a CT scan or other imaging was used to diagnose patients with pneumonia prior to inclusion in the study, leading to a highly selected patient population (Ai 2020a; Chen X 2020; Cheng 2020a; Liang 2020; Yang 2020d); RT‐PCR results were subsequently used to distinguish between COVID‐19 pneumonia and pneumonia from other causes. For all studies, testing was highly dependent on the local case definition and testing criteria that were in effect at the time of the study, meaning all patients that were included in studies had already gone through a referral/selection filter, which was not always described. The most extreme example of this is the study by Liang 2020, in which patients with radiological evidence of pneumonia and a clinical presentation compatible with COVID‐19 were only tested for SARS‐CoV‐2 after a panel discussion.\n2 Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies\n3 Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study Of the 16 studies included in this first version of the review, five studies did not use a cross‐sectional design. Three studies were diagnostic case‐control studies (Nobel 2020; Yang 2020d; Zhao 2020a), one study selected cases cross‐sectionally in five hospitals but only selected cases in one hospital (Chen X 2020), and one study emailed patients who had undergone testing for SARS‐CoV‐2 about olfactory symptoms prior to the SARS‐CoV‐2 test, with a response rate of 58% in SARS‐CoV‐2 positive cases and 15% in negative cases (Yan 2020a).\nWe rated all studies except two as carrying a high risk of bias for the index tests because there was little to no detail on how, by whom, and when the signs and symptoms were measured. In addition, there is considerable uncertainty around the reference standard, with some studies providing little detail on the RT‐PCR tests that they used or lack of clarity on blinding.\nParticipant flow was unclear in four studies (Yan 2020a; Yang 2020d; Zhao 2020a; Zhu 2020b), either because the timing of recording signs and symptoms and conduct of the reference standard was unclear, or because some tests received a second or third reference standard at unclear time points during hospital admission.\nWe rated applicability for participant selection as high risk when there was a risk of selection bias or studies did not describe selection. As for the applicability of the index tests and reference standard, we always scored this as low risk except for Chen X 2020, because blinding of the index tests was unclear, and Yang 2020d, because blinding and sample of the reference standard were unclear.\n\nFindings\nThe main characteristics of all included studies are listed in Table 3. There were four studies in hospital inpatients (Ai 2020a; Chen X 2020; Yang 2020d; Zhao 2020a), seven studies in hospital outpatients (Cheng 2020a; Liang 2020; Nobel 2020; Peng 2020a; Song 2020b; Sun 2020a; Yan 2020a), and four studies in emergency departments (Feng 2020a; Tolia 2020; Wee 2020; Zhu 2020b). The setting was not specified in one study (Rentsch 2020); in the 'Summary of findings' table, we classified this study setting as being hospital outpatient under the assumption that at that time in the pandemic (February 2020 to March 2020) tests were not commonly available outside hospital clinics. There were no studies conducted in community primary care services.\nSeven studies assessed the accuracy of signs and symptoms for the diagnosis of COVID‐19 pneumonia (Ai 2020a; Chen X 2020; Cheng 2020a; Feng 2020a; Liang 2020; Yang 2020d; Zhao 2020a); the remaining studies had COVID‐19 disease as the target condition, with no further description of the severity, meaning some patients could have suffered from mild or moderate COVID‐19 disease and others from COVID‐19 pneumonia. The distinction between these two target conditions was not always very clear, and a degree of overlap is to be assumed. All studies used RT‐PCR testing as the reference standard, with some variation in the samples that were used.\nIn all, 7706 patients were included, the median number of participants was 134. Prevalence of infection varied from 5% to 38% with a median of 17%. There were no studies in children or elderly populations, except for Rentsch 2020, which included a cohort of a median age of 65.7 years old from the Veterans Affairs Healthcare System database.\nWe found data on 27 signs and symptoms, which fall into four different categories: systemic, respiratory, gastrointestinal and cardiovascular signs and symptoms. There were no analyses for combinations of tests, only for individual signs and symptoms. The results are summarised in Table 3. Results for the cross‐sectional studies are presented in forest plots (Figure 4; Figure 5; Figure 6; Figure 7), and are plotted in ROC (receiver operating characteristic) space (Figure 8; Figure 9; Figure 10; Figure 11), results for the other studies are only listed in forest plots (Figure 12; Figure 13; Figure 14; Figure 15).\n4 Forest plot of respiratory signs and symptoms (cross‐sectional studies)\n5 Forest plot of systemic signs and symptoms (cross‐sectional studies)\n6 Forest plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n7 Forest plot of cardiovascular signs and symptoms (cross‐sectional studies)\n8 Summary ROC plot of respiratory signs and symptoms (cross‐sectional studies)\n9 Summary ROC plot of systemic signs and symptoms (cross‐sectional studies)\n10 Summary ROC Plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n11 Summary ROC plot of cardiovascular signs and symptoms (cross‐sectional studies)\n12 Forest plot of tests: 27 cough (non‐cross‐sectional study), 28 sore throat (non‐cross‐sectional study), 29 rhinorrhoea (non‐cross‐sectional study), 30 nasal obstruction (non‐cross‐sectional study), 34 dyspnoea (non‐cross‐sectional study), 31 loss of sense of smell (non‐cross‐sectional study), 32 loss of taste (non‐cross‐sectional study), 33 positive auscultation findings (non‐cross‐sectional study)\n13 Forest plot of tests: 37 fatigue (non‐cross‐sectional study), 36 fever (non‐cross‐sectional study), 39 headache (non‐cross‐sectional study), 38 myalgia or arthralgia (non‐cross‐sectional study)\n14 Forest plot of tests: 40 diarrhoea (non‐cross‐sectional study), 41 nausea/vomiting (non‐cross‐sectional study), 42 gastrointestinal symptoms, not specified (non‐cross‐sectional study)\n15 Forest plot of 35 chest tightness (non‐cross‐sectional study) Overall, diagnostic accuracy of individual signs and symptoms is low, especially sensitivity. In addition, results were highly variable across studies, making it difficult to draw firm conclusions.\nSigns and symptoms for which sensitivity was reported above 50% in at least one cross‐sectional study are the following.\nCough: sensitivity between 43% and 71%, specificity between 14% and 54%\nSore throat: sensitivity between 5% and 71%, specificity between 55% and 80%\nFever: sensitivity between 7% and 91%, specificity between 16% and 94%\nMyalgia or arthralgia: sensitivity between 19% and 86%, specificity between 45% and 91%\nFatigue: sensitivity between 10% and 57%, specificity between 60% and 94%\nHeadache: sensitivity between 3% and 71%, specificity between 78% and 98%\nFor fever, six of nine studies report a sensitivity of at least 80%, which is unsurprising considering fever was a key feature of COVID‐19 that was used in selecting patients for further testing. As a result, most participants in these studies would have fever, both cases and non‐cases. The same applies to cough, which was also listed as one of the main criteria for SARS‐CoV‐2 testing and may have contributed to inflated sensitivity estimates.\nSpecificity of at least 90% was achieved for 19 signs and symptoms. In only four signs and symptoms did this go along with sensitivity of at least 50% which would correspond to a positive likelihood ratio of at least 5, a commonly used arbitrary definition of a red flag. Using this definition, fever, myalgia or arthralgia, fatigue, or headache are to be considered red flags.\nStrikingly, most of the respiratory symptoms such as cough, sore throat and sputum production are below the diagonal in ROC space (Figure 8). The diagonal line in ROC space is where sensitivity equals 1‐specificity, meaning a test that is on the diagonal line has a positive likelihood ratio of 1 and is therefore not diagnostic because disease probability is left unchanged after conducting the test. Tests that lie below the diagonal line have a positive likelihood ratio that is smaller than 1, meaning the probability of COVID‐19 disease decreases when this test is positive. For example, in Sun 2020a, pretest probability of COVID‐19 is 6.9%; probability decreases to 6.4% when the patient has a cough and increases to 8.0% when the patient does not have a cough. We hypothesise on the reason for this counterintuitive finding in the discussion section. In contrast to respiratory features, systemic features are mostly above the diagonal line (Figure 9), suggesting that they do increase the probability of COVID‐19 when present. Gastrointestinal symptoms and cardiovascular features are clustered in the bottom left corner or on the diagonal line suggesting that they have very little diagnostic value (Figure 10; Figure 11).\nTo further illustrate the systemic features' ability to either rule in or rule out COVID‐19 disease or COVID‐19 pneumonia, we constructed dumbbell plots showing pre‐ and post‐test probabilities for each feature in each study (Figure 16). For each test, we have plotted the pre‐test probability, which is the prevalence of COVID‐19 disease (blue dot). Probability then changes depending on a positive test result (red dot marked +) or a negative test result (green dot marked ‐). The plot shows that fever, for example, increases the probability of COVID‐19 in two studies (Ai 2020a; Rentsch 2020), makes little to no difference in five studies (Feng 2020; Liang 2020; Peng 2020; Song 2020; Zhu 2020), and decreases the probability of COVID‐19 in two studies (Cheng 2020a; Tolia 2020).\n16 Dumbbell plot: this plot shows how disease probability changes after a positive test result (red dot with plus sign) or after a negative test (green dot with minus sign). Pre‐test probability or prevalence is the blue dot"}

    LitCovid-PD-HP

    {"project":"LitCovid-PD-HP","denotations":[{"id":"T93","span":{"begin":629,"end":638},"obj":"Phenotype"},{"id":"T94","span":{"begin":742,"end":751},"obj":"Phenotype"},{"id":"T95","span":{"begin":1623,"end":1632},"obj":"Phenotype"},{"id":"T96","span":{"begin":1692,"end":1701},"obj":"Phenotype"},{"id":"T97","span":{"begin":1791,"end":1800},"obj":"Phenotype"},{"id":"T98","span":{"begin":1868,"end":1877},"obj":"Phenotype"},{"id":"T99","span":{"begin":2009,"end":2018},"obj":"Phenotype"},{"id":"T100","span":{"begin":2076,"end":2081},"obj":"Phenotype"},{"id":"T101","span":{"begin":2110,"end":2119},"obj":"Phenotype"},{"id":"T102","span":{"begin":2179,"end":2188},"obj":"Phenotype"},{"id":"T103","span":{"begin":2244,"end":2249},"obj":"Phenotype"},{"id":"T104","span":{"begin":2340,"end":2349},"obj":"Phenotype"},{"id":"T105","span":{"begin":2396,"end":2405},"obj":"Phenotype"},{"id":"T106","span":{"begin":2424,"end":2429},"obj":"Phenotype"},{"id":"T107","span":{"begin":4089,"end":4098},"obj":"Phenotype"},{"id":"T108","span":{"begin":4156,"end":4165},"obj":"Phenotype"},{"id":"T109","span":{"begin":4200,"end":4209},"obj":"Phenotype"},{"id":"T110","span":{"begin":4301,"end":4310},"obj":"Phenotype"},{"id":"T111","span":{"begin":4368,"end":4377},"obj":"Phenotype"},{"id":"T112","span":{"begin":5193,"end":5202},"obj":"Phenotype"},{"id":"T113","span":{"begin":5416,"end":5425},"obj":"Phenotype"},{"id":"T114","span":{"begin":5430,"end":5439},"obj":"Phenotype"},{"id":"T115","span":{"begin":5847,"end":5856},"obj":"Phenotype"},{"id":"T116","span":{"begin":8717,"end":8726},"obj":"Phenotype"},{"id":"T117","span":{"begin":9032,"end":9041},"obj":"Phenotype"},{"id":"T118","span":{"begin":10898,"end":10903},"obj":"Phenotype"},{"id":"T119","span":{"begin":10936,"end":10947},"obj":"Phenotype"},{"id":"T120","span":{"begin":11024,"end":11041},"obj":"Phenotype"},{"id":"T121","span":{"begin":11074,"end":11082},"obj":"Phenotype"},{"id":"T122","span":{"begin":11170,"end":11183},"obj":"Phenotype"},{"id":"T123","span":{"begin":11304,"end":11311},"obj":"Phenotype"},{"id":"T124","span":{"begin":11344,"end":11349},"obj":"Phenotype"},{"id":"T125","span":{"begin":11382,"end":11390},"obj":"Phenotype"},{"id":"T126","span":{"begin":11423,"end":11430},"obj":"Phenotype"},{"id":"T127","span":{"begin":11434,"end":11444},"obj":"Phenotype"},{"id":"T128","span":{"begin":11502,"end":11511},"obj":"Phenotype"},{"id":"T129","span":{"begin":11544,"end":11559},"obj":"Phenotype"},{"id":"T130","span":{"begin":11683,"end":11698},"obj":"Phenotype"},{"id":"T131","span":{"begin":12046,"end":12051},"obj":"Phenotype"},{"id":"T132","span":{"begin":12118,"end":12129},"obj":"Phenotype"},{"id":"T133","span":{"begin":12195,"end":12200},"obj":"Phenotype"},{"id":"T134","span":{"begin":12266,"end":12273},"obj":"Phenotype"},{"id":"T135","span":{"begin":12277,"end":12287},"obj":"Phenotype"},{"id":"T136","span":{"begin":12354,"end":12361},"obj":"Phenotype"},{"id":"T137","span":{"begin":12428,"end":12436},"obj":"Phenotype"},{"id":"T138","span":{"begin":12506,"end":12511},"obj":"Phenotype"},{"id":"T139","span":{"begin":12605,"end":12610},"obj":"Phenotype"},{"id":"T140","span":{"begin":12757,"end":12762},"obj":"Phenotype"},{"id":"T141","span":{"begin":12810,"end":12815},"obj":"Phenotype"},{"id":"T142","span":{"begin":13245,"end":13250},"obj":"Phenotype"},{"id":"T143","span":{"begin":13252,"end":13259},"obj":"Phenotype"},{"id":"T144","span":{"begin":13263,"end":13273},"obj":"Phenotype"},{"id":"T145","span":{"begin":13275,"end":13282},"obj":"Phenotype"},{"id":"T146","span":{"begin":13287,"end":13295},"obj":"Phenotype"},{"id":"T147","span":{"begin":13381,"end":13386},"obj":"Phenotype"},{"id":"T148","span":{"begin":13388,"end":13399},"obj":"Phenotype"},{"id":"T149","span":{"begin":14029,"end":14034},"obj":"Phenotype"},{"id":"T150","span":{"begin":14090,"end":14095},"obj":"Phenotype"},{"id":"T151","span":{"begin":14673,"end":14682},"obj":"Phenotype"},{"id":"T152","span":{"begin":15060,"end":15065},"obj":"Phenotype"}],"attributes":[{"id":"A93","pred":"hp_id","subj":"T93","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A94","pred":"hp_id","subj":"T94","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A95","pred":"hp_id","subj":"T95","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A96","pred":"hp_id","subj":"T96","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A97","pred":"hp_id","subj":"T97","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A98","pred":"hp_id","subj":"T98","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A99","pred":"hp_id","subj":"T99","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A100","pred":"hp_id","subj":"T100","obj":"http://purl.obolibrary.org/obo/HP_0001945"},{"id":"A101","pred":"hp_id","subj":"T101","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A102","pred":"hp_id","subj":"T102","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A103","pred":"hp_id","subj":"T103","obj":"http://purl.obolibrary.org/obo/HP_0001945"},{"id":"A104","pred":"hp_id","subj":"T104","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A105","pred":"hp_id","subj":"T105","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A106","pred":"hp_id","subj":"T106","obj":"http://purl.obolibrary.org/obo/HP_0001945"},{"id":"A107","pred":"hp_id","subj":"T107","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A108","pred":"hp_id","subj":"T108","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A109","pred":"hp_id","subj":"T109","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A110","pred":"hp_id","subj":"T110","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A111","pred":"hp_id","subj":"T111","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A112","pred":"hp_id","subj":"T112","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A113","pred":"hp_id","subj":"T113","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A114","pred":"hp_id","subj":"T114","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A115","pred":"hp_id","subj":"T115","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A116","pred":"hp_id","subj":"T116","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A117","pred":"hp_id","subj":"T117","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A118","pred":"hp_id","subj":"T118","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A119","pred":"hp_id","subj":"T119","obj":"http://purl.obolibrary.org/obo/HP_0033050"},{"id":"A120","pred":"hp_id","subj":"T120","obj":"http://purl.obolibrary.org/obo/HP_0001742"},{"id":"A121","pred":"hp_id","subj":"T121","obj":"http://purl.obolibrary.org/obo/HP_0002094"},{"id":"A122","pred":"hp_id","subj":"T122","obj":"http://purl.obolibrary.org/obo/HP_0000223"},{"id":"A123","pred":"hp_id","subj":"T123","obj":"http://purl.obolibrary.org/obo/HP_0012378"},{"id":"A124","pred":"hp_id","subj":"T124","obj":"http://purl.obolibrary.org/obo/HP_0001945"},{"id":"A125","pred":"hp_id","subj":"T125","obj":"http://purl.obolibrary.org/obo/HP_0002315"},{"id":"A126","pred":"hp_id","subj":"T126","obj":"http://purl.obolibrary.org/obo/HP_0003326"},{"id":"A127","pred":"hp_id","subj":"T127","obj":"http://purl.obolibrary.org/obo/HP_0002829"},{"id":"A128","pred":"hp_id","subj":"T128","obj":"http://purl.obolibrary.org/obo/HP_0002014"},{"id":"A129","pred":"hp_id","subj":"T129","obj":"http://purl.obolibrary.org/obo/HP_0002017"},{"id":"A130","pred":"hp_id","subj":"T130","obj":"http://purl.obolibrary.org/obo/HP_0031352"},{"id":"A131","pred":"hp_id","subj":"T131","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A132","pred":"hp_id","subj":"T132","obj":"http://purl.obolibrary.org/obo/HP_0033050"},{"id":"A133","pred":"hp_id","subj":"T133","obj":"http://purl.obolibrary.org/obo/HP_0001945"},{"id":"A134","pred":"hp_id","subj":"T134","obj":"http://purl.obolibrary.org/obo/HP_0003326"},{"id":"A135","pred":"hp_id","subj":"T135","obj":"http://purl.obolibrary.org/obo/HP_0002829"},{"id":"A136","pred":"hp_id","subj":"T136","obj":"http://purl.obolibrary.org/obo/HP_0012378"},{"id":"A137","pred":"hp_id","subj":"T137","obj":"http://purl.obolibrary.org/obo/HP_0002315"},{"id":"A138","pred":"hp_id","subj":"T138","obj":"http://purl.obolibrary.org/obo/HP_0001945"},{"id":"A139","pred":"hp_id","subj":"T139","obj":"http://purl.obolibrary.org/obo/HP_0001945"},{"id":"A140","pred":"hp_id","subj":"T140","obj":"http://purl.obolibrary.org/obo/HP_0001945"},{"id":"A141","pred":"hp_id","subj":"T141","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A142","pred":"hp_id","subj":"T142","obj":"http://purl.obolibrary.org/obo/HP_0001945"},{"id":"A143","pred":"hp_id","subj":"T143","obj":"http://purl.obolibrary.org/obo/HP_0003326"},{"id":"A144","pred":"hp_id","subj":"T144","obj":"http://purl.obolibrary.org/obo/HP_0002829"},{"id":"A145","pred":"hp_id","subj":"T145","obj":"http://purl.obolibrary.org/obo/HP_0012378"},{"id":"A146","pred":"hp_id","subj":"T146","obj":"http://purl.obolibrary.org/obo/HP_0002315"},{"id":"A147","pred":"hp_id","subj":"T147","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A148","pred":"hp_id","subj":"T148","obj":"http://purl.obolibrary.org/obo/HP_0033050"},{"id":"A149","pred":"hp_id","subj":"T149","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A150","pred":"hp_id","subj":"T150","obj":"http://purl.obolibrary.org/obo/HP_0012735"},{"id":"A151","pred":"hp_id","subj":"T151","obj":"http://purl.obolibrary.org/obo/HP_0002090"},{"id":"A152","pred":"hp_id","subj":"T152","obj":"http://purl.obolibrary.org/obo/HP_0001945"}],"text":"Results\n\nResults of the search\nThe search yielded 10,965 records after removing duplicates. The first selection resulted in 658 records that were potentially eligible for this review on signs and symptoms. After screening on title and abstract, we excluded 457 records, leaving 201 to be assessed on full text. Of these, we included 16 studies in this review. The reasons for excluding 185 records are listed in the PRISMA flow chart (see Figure 1; Moher 2009).\n1 Flow diagram Two studies reported on the same cases while using a different control group (Chen X 2020; Yang 2020d). Chen X 2020 used a concurrent control group of pneumonia cases negative for SARS‐CoV‐2 on PCR testing but Yang 2020d used a historic control group of influenza pneumonia patients. For this reason we only included the Chen X 2020 results in the analyses.\nOne study reported a study that included a derivation and validation part for the development of a prediction rule (Song 2020b). The two parts are identical in set‐up and only differ in respect to the time of data collection, that is, the derivation part recruited participants up to 5 February 2020 and the validation part recruited participants from 6 February 2020 onwards. As a result, we consider this to be one study and have entered all data on signs and symptoms as such.\nFour studies were conducted in the USA, all other studies were from China. A summary of the main study characteristics can be found in Table 3.\n2 Summary of study characteristics\nStudy ID Target condition Sample size Prevalence Setting Population Design Reference standard\nAi 2020a COVID‐19 pneumonia 53 38% Hospital inpatientsa Patients hospitalised with pneumonia diagnosed by imaging Cross‐sectional PCR on nasopharyngeal swabs\nChen X 2020 COVID‐19 pneumonia 136 Not applicable Hospital inpatientsa Patients admitted with pneumonia Cases selected cross‐sectionally in 5 hospitals, non‐cases from 1 hospital only PCR, samples not specified\nCheng 2020a COVID‐19 pneumonia 33 33% Hospital outpatients Patients presenting to a fever observation department with pneumonia Cross‐sectional PCR on throat swabs\nFeng 2020a COVID‐19 pneumonia 132 5% Emergency department Patients presenting to fever clinic of emergency department Cross‐sectional PCR on throat swabs\nLiang 2020 COVID‐19 pneumonia 88 24% Hospital outpatients Patients with pneumonia and presenting to fever clinic Cross‐sectional PCR, sample not specified; conducted after panel discussion\nNobel 2020 COVID‐19 disease 516 Not applicable Hospital outpatients Patients who underwent SARS‐CoV‐2 testing with intent to hospitalise or in essential personnel Case‐control PCR on nasopharyngeal swabs\nPeng 2020a COVID‐19 disease 86 13% Hospital outpatients Patients clinically suspected and referred for testing Cross‐sectional PCR on nasopharyngeal swabs\nRentsch 2020 COVID‐19 disease 3789 15% Unclear Patients tested for SARS‐CoV‐2 in the Veterans Affairs Cohort born between 1945 and 1965 Cross‐sectional PCR on nasopharyngeal swabs\nSong 2020b COVID‐19 disease 399 7% Hospital outpatients Patients tested for SARS‐CoV‐2 Cross‐sectional PCR on sputum samples\nSun 2020a COVID‐19 disease 788 7% Hospital outpatients Patients presenting to testing centre, either self‐referred, referred from primary care or at‐risk cases identified by national contact tracing Cross‐sectional PCR on sputum, endotracheal aspirate, nasopharyngeal swabs or throat swabs\nTolia 2020 COVID‐19 disease 283 10% Emergency department Patients presenting with symptoms, travel history, risk factors or healthcare workers Cross‐sectional PCR on nasopharyngeal swabs\nWee 2020 COVID‐19 disease 870 18% Emergency department Patients presenting with respiratory symptoms or travel history Cross‐sectional PCR on oropharyngeal swabs\nYan 2020a COVID‐19 disease 262 23% Hospital outpatient Patients presenting hospital for SARS‐CoV‐2 testing, not otherwise specified Internet survey after presentation PCR, samples not specified\nYang 2020d COVID‐19 pneumonia 121 Not applicable Hospital inpatientsa Patient with pneumonia from SARS‐CoV‐2 and patients with pneumonia from influenza in 2015‐2019 Case‐control PCR, samples not specified\nZhao 2020a COVID‐19 pneumonia 34 Not applicable Hospital inpatientsa Patients with pneumonia and admitted to hospital Case‐control PCR on throat or sputum swabs\nZhu 2020b COVID‐19 disease 116 28% Emergency department Patients suspected of SARS‐CoV‐2 and presenting to the emergency department Cross‐sectional PCR, samples not specified\nPCR: polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2\na'Hospital inpatients' refers to studies that recruited patients admitted to hospital with COVID‐19 disease and in whom the signs and symptoms were assessed on admission.\n\nMethodological quality of included studies\nThe results of the quality assessment are summarised in Figure 2 and Figure 3. We rated participant selection as introducing high risk of bias in seven studies. In five studies this was because a CT scan or other imaging was used to diagnose patients with pneumonia prior to inclusion in the study, leading to a highly selected patient population (Ai 2020a; Chen X 2020; Cheng 2020a; Liang 2020; Yang 2020d); RT‐PCR results were subsequently used to distinguish between COVID‐19 pneumonia and pneumonia from other causes. For all studies, testing was highly dependent on the local case definition and testing criteria that were in effect at the time of the study, meaning all patients that were included in studies had already gone through a referral/selection filter, which was not always described. The most extreme example of this is the study by Liang 2020, in which patients with radiological evidence of pneumonia and a clinical presentation compatible with COVID‐19 were only tested for SARS‐CoV‐2 after a panel discussion.\n2 Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies\n3 Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study Of the 16 studies included in this first version of the review, five studies did not use a cross‐sectional design. Three studies were diagnostic case‐control studies (Nobel 2020; Yang 2020d; Zhao 2020a), one study selected cases cross‐sectionally in five hospitals but only selected cases in one hospital (Chen X 2020), and one study emailed patients who had undergone testing for SARS‐CoV‐2 about olfactory symptoms prior to the SARS‐CoV‐2 test, with a response rate of 58% in SARS‐CoV‐2 positive cases and 15% in negative cases (Yan 2020a).\nWe rated all studies except two as carrying a high risk of bias for the index tests because there was little to no detail on how, by whom, and when the signs and symptoms were measured. In addition, there is considerable uncertainty around the reference standard, with some studies providing little detail on the RT‐PCR tests that they used or lack of clarity on blinding.\nParticipant flow was unclear in four studies (Yan 2020a; Yang 2020d; Zhao 2020a; Zhu 2020b), either because the timing of recording signs and symptoms and conduct of the reference standard was unclear, or because some tests received a second or third reference standard at unclear time points during hospital admission.\nWe rated applicability for participant selection as high risk when there was a risk of selection bias or studies did not describe selection. As for the applicability of the index tests and reference standard, we always scored this as low risk except for Chen X 2020, because blinding of the index tests was unclear, and Yang 2020d, because blinding and sample of the reference standard were unclear.\n\nFindings\nThe main characteristics of all included studies are listed in Table 3. There were four studies in hospital inpatients (Ai 2020a; Chen X 2020; Yang 2020d; Zhao 2020a), seven studies in hospital outpatients (Cheng 2020a; Liang 2020; Nobel 2020; Peng 2020a; Song 2020b; Sun 2020a; Yan 2020a), and four studies in emergency departments (Feng 2020a; Tolia 2020; Wee 2020; Zhu 2020b). The setting was not specified in one study (Rentsch 2020); in the 'Summary of findings' table, we classified this study setting as being hospital outpatient under the assumption that at that time in the pandemic (February 2020 to March 2020) tests were not commonly available outside hospital clinics. There were no studies conducted in community primary care services.\nSeven studies assessed the accuracy of signs and symptoms for the diagnosis of COVID‐19 pneumonia (Ai 2020a; Chen X 2020; Cheng 2020a; Feng 2020a; Liang 2020; Yang 2020d; Zhao 2020a); the remaining studies had COVID‐19 disease as the target condition, with no further description of the severity, meaning some patients could have suffered from mild or moderate COVID‐19 disease and others from COVID‐19 pneumonia. The distinction between these two target conditions was not always very clear, and a degree of overlap is to be assumed. All studies used RT‐PCR testing as the reference standard, with some variation in the samples that were used.\nIn all, 7706 patients were included, the median number of participants was 134. Prevalence of infection varied from 5% to 38% with a median of 17%. There were no studies in children or elderly populations, except for Rentsch 2020, which included a cohort of a median age of 65.7 years old from the Veterans Affairs Healthcare System database.\nWe found data on 27 signs and symptoms, which fall into four different categories: systemic, respiratory, gastrointestinal and cardiovascular signs and symptoms. There were no analyses for combinations of tests, only for individual signs and symptoms. The results are summarised in Table 3. Results for the cross‐sectional studies are presented in forest plots (Figure 4; Figure 5; Figure 6; Figure 7), and are plotted in ROC (receiver operating characteristic) space (Figure 8; Figure 9; Figure 10; Figure 11), results for the other studies are only listed in forest plots (Figure 12; Figure 13; Figure 14; Figure 15).\n4 Forest plot of respiratory signs and symptoms (cross‐sectional studies)\n5 Forest plot of systemic signs and symptoms (cross‐sectional studies)\n6 Forest plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n7 Forest plot of cardiovascular signs and symptoms (cross‐sectional studies)\n8 Summary ROC plot of respiratory signs and symptoms (cross‐sectional studies)\n9 Summary ROC plot of systemic signs and symptoms (cross‐sectional studies)\n10 Summary ROC Plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n11 Summary ROC plot of cardiovascular signs and symptoms (cross‐sectional studies)\n12 Forest plot of tests: 27 cough (non‐cross‐sectional study), 28 sore throat (non‐cross‐sectional study), 29 rhinorrhoea (non‐cross‐sectional study), 30 nasal obstruction (non‐cross‐sectional study), 34 dyspnoea (non‐cross‐sectional study), 31 loss of sense of smell (non‐cross‐sectional study), 32 loss of taste (non‐cross‐sectional study), 33 positive auscultation findings (non‐cross‐sectional study)\n13 Forest plot of tests: 37 fatigue (non‐cross‐sectional study), 36 fever (non‐cross‐sectional study), 39 headache (non‐cross‐sectional study), 38 myalgia or arthralgia (non‐cross‐sectional study)\n14 Forest plot of tests: 40 diarrhoea (non‐cross‐sectional study), 41 nausea/vomiting (non‐cross‐sectional study), 42 gastrointestinal symptoms, not specified (non‐cross‐sectional study)\n15 Forest plot of 35 chest tightness (non‐cross‐sectional study) Overall, diagnostic accuracy of individual signs and symptoms is low, especially sensitivity. In addition, results were highly variable across studies, making it difficult to draw firm conclusions.\nSigns and symptoms for which sensitivity was reported above 50% in at least one cross‐sectional study are the following.\nCough: sensitivity between 43% and 71%, specificity between 14% and 54%\nSore throat: sensitivity between 5% and 71%, specificity between 55% and 80%\nFever: sensitivity between 7% and 91%, specificity between 16% and 94%\nMyalgia or arthralgia: sensitivity between 19% and 86%, specificity between 45% and 91%\nFatigue: sensitivity between 10% and 57%, specificity between 60% and 94%\nHeadache: sensitivity between 3% and 71%, specificity between 78% and 98%\nFor fever, six of nine studies report a sensitivity of at least 80%, which is unsurprising considering fever was a key feature of COVID‐19 that was used in selecting patients for further testing. As a result, most participants in these studies would have fever, both cases and non‐cases. The same applies to cough, which was also listed as one of the main criteria for SARS‐CoV‐2 testing and may have contributed to inflated sensitivity estimates.\nSpecificity of at least 90% was achieved for 19 signs and symptoms. In only four signs and symptoms did this go along with sensitivity of at least 50% which would correspond to a positive likelihood ratio of at least 5, a commonly used arbitrary definition of a red flag. Using this definition, fever, myalgia or arthralgia, fatigue, or headache are to be considered red flags.\nStrikingly, most of the respiratory symptoms such as cough, sore throat and sputum production are below the diagonal in ROC space (Figure 8). The diagonal line in ROC space is where sensitivity equals 1‐specificity, meaning a test that is on the diagonal line has a positive likelihood ratio of 1 and is therefore not diagnostic because disease probability is left unchanged after conducting the test. Tests that lie below the diagonal line have a positive likelihood ratio that is smaller than 1, meaning the probability of COVID‐19 disease decreases when this test is positive. For example, in Sun 2020a, pretest probability of COVID‐19 is 6.9%; probability decreases to 6.4% when the patient has a cough and increases to 8.0% when the patient does not have a cough. We hypothesise on the reason for this counterintuitive finding in the discussion section. In contrast to respiratory features, systemic features are mostly above the diagonal line (Figure 9), suggesting that they do increase the probability of COVID‐19 when present. Gastrointestinal symptoms and cardiovascular features are clustered in the bottom left corner or on the diagonal line suggesting that they have very little diagnostic value (Figure 10; Figure 11).\nTo further illustrate the systemic features' ability to either rule in or rule out COVID‐19 disease or COVID‐19 pneumonia, we constructed dumbbell plots showing pre‐ and post‐test probabilities for each feature in each study (Figure 16). For each test, we have plotted the pre‐test probability, which is the prevalence of COVID‐19 disease (blue dot). Probability then changes depending on a positive test result (red dot marked +) or a negative test result (green dot marked ‐). The plot shows that fever, for example, increases the probability of COVID‐19 in two studies (Ai 2020a; Rentsch 2020), makes little to no difference in five studies (Feng 2020; Liang 2020; Peng 2020; Song 2020; Zhu 2020), and decreases the probability of COVID‐19 in two studies (Cheng 2020a; Tolia 2020).\n16 Dumbbell plot: this plot shows how disease probability changes after a positive test result (red dot with plus sign) or after a negative test (green dot with minus sign). Pre‐test probability or prevalence is the blue dot"}

    LitCovid-PD-GO-BP

    {"project":"LitCovid-PD-GO-BP","denotations":[{"id":"T26","span":{"begin":11123,"end":11137},"obj":"http://purl.obolibrary.org/obo/GO_0007608"},{"id":"T27","span":{"begin":11178,"end":11183},"obj":"http://purl.obolibrary.org/obo/GO_0050909"}],"text":"Results\n\nResults of the search\nThe search yielded 10,965 records after removing duplicates. The first selection resulted in 658 records that were potentially eligible for this review on signs and symptoms. After screening on title and abstract, we excluded 457 records, leaving 201 to be assessed on full text. Of these, we included 16 studies in this review. The reasons for excluding 185 records are listed in the PRISMA flow chart (see Figure 1; Moher 2009).\n1 Flow diagram Two studies reported on the same cases while using a different control group (Chen X 2020; Yang 2020d). Chen X 2020 used a concurrent control group of pneumonia cases negative for SARS‐CoV‐2 on PCR testing but Yang 2020d used a historic control group of influenza pneumonia patients. For this reason we only included the Chen X 2020 results in the analyses.\nOne study reported a study that included a derivation and validation part for the development of a prediction rule (Song 2020b). The two parts are identical in set‐up and only differ in respect to the time of data collection, that is, the derivation part recruited participants up to 5 February 2020 and the validation part recruited participants from 6 February 2020 onwards. As a result, we consider this to be one study and have entered all data on signs and symptoms as such.\nFour studies were conducted in the USA, all other studies were from China. A summary of the main study characteristics can be found in Table 3.\n2 Summary of study characteristics\nStudy ID Target condition Sample size Prevalence Setting Population Design Reference standard\nAi 2020a COVID‐19 pneumonia 53 38% Hospital inpatientsa Patients hospitalised with pneumonia diagnosed by imaging Cross‐sectional PCR on nasopharyngeal swabs\nChen X 2020 COVID‐19 pneumonia 136 Not applicable Hospital inpatientsa Patients admitted with pneumonia Cases selected cross‐sectionally in 5 hospitals, non‐cases from 1 hospital only PCR, samples not specified\nCheng 2020a COVID‐19 pneumonia 33 33% Hospital outpatients Patients presenting to a fever observation department with pneumonia Cross‐sectional PCR on throat swabs\nFeng 2020a COVID‐19 pneumonia 132 5% Emergency department Patients presenting to fever clinic of emergency department Cross‐sectional PCR on throat swabs\nLiang 2020 COVID‐19 pneumonia 88 24% Hospital outpatients Patients with pneumonia and presenting to fever clinic Cross‐sectional PCR, sample not specified; conducted after panel discussion\nNobel 2020 COVID‐19 disease 516 Not applicable Hospital outpatients Patients who underwent SARS‐CoV‐2 testing with intent to hospitalise or in essential personnel Case‐control PCR on nasopharyngeal swabs\nPeng 2020a COVID‐19 disease 86 13% Hospital outpatients Patients clinically suspected and referred for testing Cross‐sectional PCR on nasopharyngeal swabs\nRentsch 2020 COVID‐19 disease 3789 15% Unclear Patients tested for SARS‐CoV‐2 in the Veterans Affairs Cohort born between 1945 and 1965 Cross‐sectional PCR on nasopharyngeal swabs\nSong 2020b COVID‐19 disease 399 7% Hospital outpatients Patients tested for SARS‐CoV‐2 Cross‐sectional PCR on sputum samples\nSun 2020a COVID‐19 disease 788 7% Hospital outpatients Patients presenting to testing centre, either self‐referred, referred from primary care or at‐risk cases identified by national contact tracing Cross‐sectional PCR on sputum, endotracheal aspirate, nasopharyngeal swabs or throat swabs\nTolia 2020 COVID‐19 disease 283 10% Emergency department Patients presenting with symptoms, travel history, risk factors or healthcare workers Cross‐sectional PCR on nasopharyngeal swabs\nWee 2020 COVID‐19 disease 870 18% Emergency department Patients presenting with respiratory symptoms or travel history Cross‐sectional PCR on oropharyngeal swabs\nYan 2020a COVID‐19 disease 262 23% Hospital outpatient Patients presenting hospital for SARS‐CoV‐2 testing, not otherwise specified Internet survey after presentation PCR, samples not specified\nYang 2020d COVID‐19 pneumonia 121 Not applicable Hospital inpatientsa Patient with pneumonia from SARS‐CoV‐2 and patients with pneumonia from influenza in 2015‐2019 Case‐control PCR, samples not specified\nZhao 2020a COVID‐19 pneumonia 34 Not applicable Hospital inpatientsa Patients with pneumonia and admitted to hospital Case‐control PCR on throat or sputum swabs\nZhu 2020b COVID‐19 disease 116 28% Emergency department Patients suspected of SARS‐CoV‐2 and presenting to the emergency department Cross‐sectional PCR, samples not specified\nPCR: polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2\na'Hospital inpatients' refers to studies that recruited patients admitted to hospital with COVID‐19 disease and in whom the signs and symptoms were assessed on admission.\n\nMethodological quality of included studies\nThe results of the quality assessment are summarised in Figure 2 and Figure 3. We rated participant selection as introducing high risk of bias in seven studies. In five studies this was because a CT scan or other imaging was used to diagnose patients with pneumonia prior to inclusion in the study, leading to a highly selected patient population (Ai 2020a; Chen X 2020; Cheng 2020a; Liang 2020; Yang 2020d); RT‐PCR results were subsequently used to distinguish between COVID‐19 pneumonia and pneumonia from other causes. For all studies, testing was highly dependent on the local case definition and testing criteria that were in effect at the time of the study, meaning all patients that were included in studies had already gone through a referral/selection filter, which was not always described. The most extreme example of this is the study by Liang 2020, in which patients with radiological evidence of pneumonia and a clinical presentation compatible with COVID‐19 were only tested for SARS‐CoV‐2 after a panel discussion.\n2 Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies\n3 Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study Of the 16 studies included in this first version of the review, five studies did not use a cross‐sectional design. Three studies were diagnostic case‐control studies (Nobel 2020; Yang 2020d; Zhao 2020a), one study selected cases cross‐sectionally in five hospitals but only selected cases in one hospital (Chen X 2020), and one study emailed patients who had undergone testing for SARS‐CoV‐2 about olfactory symptoms prior to the SARS‐CoV‐2 test, with a response rate of 58% in SARS‐CoV‐2 positive cases and 15% in negative cases (Yan 2020a).\nWe rated all studies except two as carrying a high risk of bias for the index tests because there was little to no detail on how, by whom, and when the signs and symptoms were measured. In addition, there is considerable uncertainty around the reference standard, with some studies providing little detail on the RT‐PCR tests that they used or lack of clarity on blinding.\nParticipant flow was unclear in four studies (Yan 2020a; Yang 2020d; Zhao 2020a; Zhu 2020b), either because the timing of recording signs and symptoms and conduct of the reference standard was unclear, or because some tests received a second or third reference standard at unclear time points during hospital admission.\nWe rated applicability for participant selection as high risk when there was a risk of selection bias or studies did not describe selection. As for the applicability of the index tests and reference standard, we always scored this as low risk except for Chen X 2020, because blinding of the index tests was unclear, and Yang 2020d, because blinding and sample of the reference standard were unclear.\n\nFindings\nThe main characteristics of all included studies are listed in Table 3. There were four studies in hospital inpatients (Ai 2020a; Chen X 2020; Yang 2020d; Zhao 2020a), seven studies in hospital outpatients (Cheng 2020a; Liang 2020; Nobel 2020; Peng 2020a; Song 2020b; Sun 2020a; Yan 2020a), and four studies in emergency departments (Feng 2020a; Tolia 2020; Wee 2020; Zhu 2020b). The setting was not specified in one study (Rentsch 2020); in the 'Summary of findings' table, we classified this study setting as being hospital outpatient under the assumption that at that time in the pandemic (February 2020 to March 2020) tests were not commonly available outside hospital clinics. There were no studies conducted in community primary care services.\nSeven studies assessed the accuracy of signs and symptoms for the diagnosis of COVID‐19 pneumonia (Ai 2020a; Chen X 2020; Cheng 2020a; Feng 2020a; Liang 2020; Yang 2020d; Zhao 2020a); the remaining studies had COVID‐19 disease as the target condition, with no further description of the severity, meaning some patients could have suffered from mild or moderate COVID‐19 disease and others from COVID‐19 pneumonia. The distinction between these two target conditions was not always very clear, and a degree of overlap is to be assumed. All studies used RT‐PCR testing as the reference standard, with some variation in the samples that were used.\nIn all, 7706 patients were included, the median number of participants was 134. Prevalence of infection varied from 5% to 38% with a median of 17%. There were no studies in children or elderly populations, except for Rentsch 2020, which included a cohort of a median age of 65.7 years old from the Veterans Affairs Healthcare System database.\nWe found data on 27 signs and symptoms, which fall into four different categories: systemic, respiratory, gastrointestinal and cardiovascular signs and symptoms. There were no analyses for combinations of tests, only for individual signs and symptoms. The results are summarised in Table 3. Results for the cross‐sectional studies are presented in forest plots (Figure 4; Figure 5; Figure 6; Figure 7), and are plotted in ROC (receiver operating characteristic) space (Figure 8; Figure 9; Figure 10; Figure 11), results for the other studies are only listed in forest plots (Figure 12; Figure 13; Figure 14; Figure 15).\n4 Forest plot of respiratory signs and symptoms (cross‐sectional studies)\n5 Forest plot of systemic signs and symptoms (cross‐sectional studies)\n6 Forest plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n7 Forest plot of cardiovascular signs and symptoms (cross‐sectional studies)\n8 Summary ROC plot of respiratory signs and symptoms (cross‐sectional studies)\n9 Summary ROC plot of systemic signs and symptoms (cross‐sectional studies)\n10 Summary ROC Plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n11 Summary ROC plot of cardiovascular signs and symptoms (cross‐sectional studies)\n12 Forest plot of tests: 27 cough (non‐cross‐sectional study), 28 sore throat (non‐cross‐sectional study), 29 rhinorrhoea (non‐cross‐sectional study), 30 nasal obstruction (non‐cross‐sectional study), 34 dyspnoea (non‐cross‐sectional study), 31 loss of sense of smell (non‐cross‐sectional study), 32 loss of taste (non‐cross‐sectional study), 33 positive auscultation findings (non‐cross‐sectional study)\n13 Forest plot of tests: 37 fatigue (non‐cross‐sectional study), 36 fever (non‐cross‐sectional study), 39 headache (non‐cross‐sectional study), 38 myalgia or arthralgia (non‐cross‐sectional study)\n14 Forest plot of tests: 40 diarrhoea (non‐cross‐sectional study), 41 nausea/vomiting (non‐cross‐sectional study), 42 gastrointestinal symptoms, not specified (non‐cross‐sectional study)\n15 Forest plot of 35 chest tightness (non‐cross‐sectional study) Overall, diagnostic accuracy of individual signs and symptoms is low, especially sensitivity. In addition, results were highly variable across studies, making it difficult to draw firm conclusions.\nSigns and symptoms for which sensitivity was reported above 50% in at least one cross‐sectional study are the following.\nCough: sensitivity between 43% and 71%, specificity between 14% and 54%\nSore throat: sensitivity between 5% and 71%, specificity between 55% and 80%\nFever: sensitivity between 7% and 91%, specificity between 16% and 94%\nMyalgia or arthralgia: sensitivity between 19% and 86%, specificity between 45% and 91%\nFatigue: sensitivity between 10% and 57%, specificity between 60% and 94%\nHeadache: sensitivity between 3% and 71%, specificity between 78% and 98%\nFor fever, six of nine studies report a sensitivity of at least 80%, which is unsurprising considering fever was a key feature of COVID‐19 that was used in selecting patients for further testing. As a result, most participants in these studies would have fever, both cases and non‐cases. The same applies to cough, which was also listed as one of the main criteria for SARS‐CoV‐2 testing and may have contributed to inflated sensitivity estimates.\nSpecificity of at least 90% was achieved for 19 signs and symptoms. In only four signs and symptoms did this go along with sensitivity of at least 50% which would correspond to a positive likelihood ratio of at least 5, a commonly used arbitrary definition of a red flag. Using this definition, fever, myalgia or arthralgia, fatigue, or headache are to be considered red flags.\nStrikingly, most of the respiratory symptoms such as cough, sore throat and sputum production are below the diagonal in ROC space (Figure 8). The diagonal line in ROC space is where sensitivity equals 1‐specificity, meaning a test that is on the diagonal line has a positive likelihood ratio of 1 and is therefore not diagnostic because disease probability is left unchanged after conducting the test. Tests that lie below the diagonal line have a positive likelihood ratio that is smaller than 1, meaning the probability of COVID‐19 disease decreases when this test is positive. For example, in Sun 2020a, pretest probability of COVID‐19 is 6.9%; probability decreases to 6.4% when the patient has a cough and increases to 8.0% when the patient does not have a cough. We hypothesise on the reason for this counterintuitive finding in the discussion section. In contrast to respiratory features, systemic features are mostly above the diagonal line (Figure 9), suggesting that they do increase the probability of COVID‐19 when present. Gastrointestinal symptoms and cardiovascular features are clustered in the bottom left corner or on the diagonal line suggesting that they have very little diagnostic value (Figure 10; Figure 11).\nTo further illustrate the systemic features' ability to either rule in or rule out COVID‐19 disease or COVID‐19 pneumonia, we constructed dumbbell plots showing pre‐ and post‐test probabilities for each feature in each study (Figure 16). For each test, we have plotted the pre‐test probability, which is the prevalence of COVID‐19 disease (blue dot). Probability then changes depending on a positive test result (red dot marked +) or a negative test result (green dot marked ‐). The plot shows that fever, for example, increases the probability of COVID‐19 in two studies (Ai 2020a; Rentsch 2020), makes little to no difference in five studies (Feng 2020; Liang 2020; Peng 2020; Song 2020; Zhu 2020), and decreases the probability of COVID‐19 in two studies (Cheng 2020a; Tolia 2020).\n16 Dumbbell plot: this plot shows how disease probability changes after a positive test result (red dot with plus sign) or after a negative test (green dot with minus sign). Pre‐test probability or prevalence is the blue dot"}

    LitCovid-sentences

    {"project":"LitCovid-sentences","denotations":[{"id":"T424","span":{"begin":0,"end":7},"obj":"Sentence"},{"id":"T425","span":{"begin":9,"end":30},"obj":"Sentence"},{"id":"T426","span":{"begin":31,"end":91},"obj":"Sentence"},{"id":"T427","span":{"begin":92,"end":205},"obj":"Sentence"},{"id":"T428","span":{"begin":206,"end":310},"obj":"Sentence"},{"id":"T429","span":{"begin":311,"end":359},"obj":"Sentence"},{"id":"T430","span":{"begin":360,"end":461},"obj":"Sentence"},{"id":"T431","span":{"begin":462,"end":581},"obj":"Sentence"},{"id":"T432","span":{"begin":582,"end":761},"obj":"Sentence"},{"id":"T433","span":{"begin":762,"end":835},"obj":"Sentence"},{"id":"T434","span":{"begin":836,"end":964},"obj":"Sentence"},{"id":"T435","span":{"begin":965,"end":1212},"obj":"Sentence"},{"id":"T436","span":{"begin":1213,"end":1315},"obj":"Sentence"},{"id":"T437","span":{"begin":1316,"end":1390},"obj":"Sentence"},{"id":"T438","span":{"begin":1391,"end":1459},"obj":"Sentence"},{"id":"T439","span":{"begin":1460,"end":1495},"obj":"Sentence"},{"id":"T440","span":{"begin":1496,"end":1603},"obj":"Sentence"},{"id":"T441","span":{"begin":1604,"end":1768},"obj":"Sentence"},{"id":"T442","span":{"begin":1769,"end":1986},"obj":"Sentence"},{"id":"T443","span":{"begin":1987,"end":2157},"obj":"Sentence"},{"id":"T444","span":{"begin":2158,"end":2318},"obj":"Sentence"},{"id":"T445","span":{"begin":2319,"end":2514},"obj":"Sentence"},{"id":"T446","span":{"begin":2515,"end":2725},"obj":"Sentence"},{"id":"T447","span":{"begin":2726,"end":2887},"obj":"Sentence"},{"id":"T448","span":{"begin":2888,"end":3074},"obj":"Sentence"},{"id":"T449","span":{"begin":3075,"end":3206},"obj":"Sentence"},{"id":"T450","span":{"begin":3207,"end":3503},"obj":"Sentence"},{"id":"T451","span":{"begin":3504,"end":3697},"obj":"Sentence"},{"id":"T452","span":{"begin":3698,"end":3866},"obj":"Sentence"},{"id":"T453","span":{"begin":3867,"end":4067},"obj":"Sentence"},{"id":"T454","span":{"begin":4068,"end":4279},"obj":"Sentence"},{"id":"T455","span":{"begin":4280,"end":4447},"obj":"Sentence"},{"id":"T456","span":{"begin":4448,"end":4629},"obj":"Sentence"},{"id":"T457","span":{"begin":4630,"end":4721},"obj":"Sentence"},{"id":"T458","span":{"begin":4722,"end":4892},"obj":"Sentence"},{"id":"T459","span":{"begin":4894,"end":4936},"obj":"Sentence"},{"id":"T460","span":{"begin":4937,"end":5015},"obj":"Sentence"},{"id":"T461","span":{"begin":5016,"end":5097},"obj":"Sentence"},{"id":"T462","span":{"begin":5098,"end":5458},"obj":"Sentence"},{"id":"T463","span":{"begin":5459,"end":5737},"obj":"Sentence"},{"id":"T464","span":{"begin":5738,"end":5967},"obj":"Sentence"},{"id":"T465","span":{"begin":5968,"end":6111},"obj":"Sentence"},{"id":"T466","span":{"begin":6112,"end":6347},"obj":"Sentence"},{"id":"T467","span":{"begin":6348,"end":6775},"obj":"Sentence"},{"id":"T468","span":{"begin":6776,"end":6961},"obj":"Sentence"},{"id":"T469","span":{"begin":6962,"end":7148},"obj":"Sentence"},{"id":"T470","span":{"begin":7149,"end":7468},"obj":"Sentence"},{"id":"T471","span":{"begin":7469,"end":7609},"obj":"Sentence"},{"id":"T472","span":{"begin":7610,"end":7868},"obj":"Sentence"},{"id":"T473","span":{"begin":7870,"end":7878},"obj":"Sentence"},{"id":"T474","span":{"begin":7879,"end":7950},"obj":"Sentence"},{"id":"T475","span":{"begin":7951,"end":8258},"obj":"Sentence"},{"id":"T476","span":{"begin":8259,"end":8560},"obj":"Sentence"},{"id":"T477","span":{"begin":8561,"end":8628},"obj":"Sentence"},{"id":"T478","span":{"begin":8629,"end":9042},"obj":"Sentence"},{"id":"T479","span":{"begin":9043,"end":9163},"obj":"Sentence"},{"id":"T480","span":{"begin":9164,"end":9273},"obj":"Sentence"},{"id":"T481","span":{"begin":9274,"end":9353},"obj":"Sentence"},{"id":"T482","span":{"begin":9354,"end":9421},"obj":"Sentence"},{"id":"T483","span":{"begin":9422,"end":9616},"obj":"Sentence"},{"id":"T484","span":{"begin":9617,"end":9778},"obj":"Sentence"},{"id":"T485","span":{"begin":9779,"end":9868},"obj":"Sentence"},{"id":"T486","span":{"begin":9869,"end":9907},"obj":"Sentence"},{"id":"T487","span":{"begin":9908,"end":10236},"obj":"Sentence"},{"id":"T488","span":{"begin":10237,"end":10311},"obj":"Sentence"},{"id":"T489","span":{"begin":10312,"end":10383},"obj":"Sentence"},{"id":"T490","span":{"begin":10384,"end":10463},"obj":"Sentence"},{"id":"T491","span":{"begin":10464,"end":10541},"obj":"Sentence"},{"id":"T492","span":{"begin":10542,"end":10621},"obj":"Sentence"},{"id":"T493","span":{"begin":10622,"end":10698},"obj":"Sentence"},{"id":"T494","span":{"begin":10699,"end":10784},"obj":"Sentence"},{"id":"T495","span":{"begin":10785,"end":10868},"obj":"Sentence"},{"id":"T496","span":{"begin":10869,"end":10894},"obj":"Sentence"},{"id":"T497","span":{"begin":10895,"end":11274},"obj":"Sentence"},{"id":"T498","span":{"begin":11275,"end":11300},"obj":"Sentence"},{"id":"T499","span":{"begin":11301,"end":11472},"obj":"Sentence"},{"id":"T500","span":{"begin":11473,"end":11498},"obj":"Sentence"},{"id":"T501","span":{"begin":11499,"end":11660},"obj":"Sentence"},{"id":"T502","span":{"begin":11661,"end":11820},"obj":"Sentence"},{"id":"T503","span":{"begin":11821,"end":11924},"obj":"Sentence"},{"id":"T504","span":{"begin":11925,"end":12045},"obj":"Sentence"},{"id":"T505","span":{"begin":12046,"end":12117},"obj":"Sentence"},{"id":"T506","span":{"begin":12118,"end":12194},"obj":"Sentence"},{"id":"T507","span":{"begin":12195,"end":12265},"obj":"Sentence"},{"id":"T508","span":{"begin":12266,"end":12353},"obj":"Sentence"},{"id":"T509","span":{"begin":12354,"end":12427},"obj":"Sentence"},{"id":"T510","span":{"begin":12428,"end":12501},"obj":"Sentence"},{"id":"T511","span":{"begin":12502,"end":12697},"obj":"Sentence"},{"id":"T512","span":{"begin":12698,"end":12789},"obj":"Sentence"},{"id":"T513","span":{"begin":12790,"end":12949},"obj":"Sentence"},{"id":"T514","span":{"begin":12950,"end":13017},"obj":"Sentence"},{"id":"T515","span":{"begin":13018,"end":13221},"obj":"Sentence"},{"id":"T516","span":{"begin":13222,"end":13327},"obj":"Sentence"},{"id":"T517","span":{"begin":13328,"end":13469},"obj":"Sentence"},{"id":"T518","span":{"begin":13470,"end":13729},"obj":"Sentence"},{"id":"T519","span":{"begin":13730,"end":13907},"obj":"Sentence"},{"id":"T520","span":{"begin":13908,"end":14096},"obj":"Sentence"},{"id":"T521","span":{"begin":14097,"end":14186},"obj":"Sentence"},{"id":"T522","span":{"begin":14187,"end":14363},"obj":"Sentence"},{"id":"T523","span":{"begin":14364,"end":14560},"obj":"Sentence"},{"id":"T524","span":{"begin":14561,"end":14798},"obj":"Sentence"},{"id":"T525","span":{"begin":14799,"end":14911},"obj":"Sentence"},{"id":"T526","span":{"begin":14912,"end":15039},"obj":"Sentence"},{"id":"T527","span":{"begin":15040,"end":15345},"obj":"Sentence"},{"id":"T528","span":{"begin":15346,"end":15520},"obj":"Sentence"},{"id":"T529","span":{"begin":15521,"end":15571},"obj":"Sentence"}],"namespaces":[{"prefix":"_base","uri":"http://pubannotation.org/ontology/tao.owl#"}],"text":"Results\n\nResults of the search\nThe search yielded 10,965 records after removing duplicates. The first selection resulted in 658 records that were potentially eligible for this review on signs and symptoms. After screening on title and abstract, we excluded 457 records, leaving 201 to be assessed on full text. Of these, we included 16 studies in this review. The reasons for excluding 185 records are listed in the PRISMA flow chart (see Figure 1; Moher 2009).\n1 Flow diagram Two studies reported on the same cases while using a different control group (Chen X 2020; Yang 2020d). Chen X 2020 used a concurrent control group of pneumonia cases negative for SARS‐CoV‐2 on PCR testing but Yang 2020d used a historic control group of influenza pneumonia patients. For this reason we only included the Chen X 2020 results in the analyses.\nOne study reported a study that included a derivation and validation part for the development of a prediction rule (Song 2020b). The two parts are identical in set‐up and only differ in respect to the time of data collection, that is, the derivation part recruited participants up to 5 February 2020 and the validation part recruited participants from 6 February 2020 onwards. As a result, we consider this to be one study and have entered all data on signs and symptoms as such.\nFour studies were conducted in the USA, all other studies were from China. A summary of the main study characteristics can be found in Table 3.\n2 Summary of study characteristics\nStudy ID Target condition Sample size Prevalence Setting Population Design Reference standard\nAi 2020a COVID‐19 pneumonia 53 38% Hospital inpatientsa Patients hospitalised with pneumonia diagnosed by imaging Cross‐sectional PCR on nasopharyngeal swabs\nChen X 2020 COVID‐19 pneumonia 136 Not applicable Hospital inpatientsa Patients admitted with pneumonia Cases selected cross‐sectionally in 5 hospitals, non‐cases from 1 hospital only PCR, samples not specified\nCheng 2020a COVID‐19 pneumonia 33 33% Hospital outpatients Patients presenting to a fever observation department with pneumonia Cross‐sectional PCR on throat swabs\nFeng 2020a COVID‐19 pneumonia 132 5% Emergency department Patients presenting to fever clinic of emergency department Cross‐sectional PCR on throat swabs\nLiang 2020 COVID‐19 pneumonia 88 24% Hospital outpatients Patients with pneumonia and presenting to fever clinic Cross‐sectional PCR, sample not specified; conducted after panel discussion\nNobel 2020 COVID‐19 disease 516 Not applicable Hospital outpatients Patients who underwent SARS‐CoV‐2 testing with intent to hospitalise or in essential personnel Case‐control PCR on nasopharyngeal swabs\nPeng 2020a COVID‐19 disease 86 13% Hospital outpatients Patients clinically suspected and referred for testing Cross‐sectional PCR on nasopharyngeal swabs\nRentsch 2020 COVID‐19 disease 3789 15% Unclear Patients tested for SARS‐CoV‐2 in the Veterans Affairs Cohort born between 1945 and 1965 Cross‐sectional PCR on nasopharyngeal swabs\nSong 2020b COVID‐19 disease 399 7% Hospital outpatients Patients tested for SARS‐CoV‐2 Cross‐sectional PCR on sputum samples\nSun 2020a COVID‐19 disease 788 7% Hospital outpatients Patients presenting to testing centre, either self‐referred, referred from primary care or at‐risk cases identified by national contact tracing Cross‐sectional PCR on sputum, endotracheal aspirate, nasopharyngeal swabs or throat swabs\nTolia 2020 COVID‐19 disease 283 10% Emergency department Patients presenting with symptoms, travel history, risk factors or healthcare workers Cross‐sectional PCR on nasopharyngeal swabs\nWee 2020 COVID‐19 disease 870 18% Emergency department Patients presenting with respiratory symptoms or travel history Cross‐sectional PCR on oropharyngeal swabs\nYan 2020a COVID‐19 disease 262 23% Hospital outpatient Patients presenting hospital for SARS‐CoV‐2 testing, not otherwise specified Internet survey after presentation PCR, samples not specified\nYang 2020d COVID‐19 pneumonia 121 Not applicable Hospital inpatientsa Patient with pneumonia from SARS‐CoV‐2 and patients with pneumonia from influenza in 2015‐2019 Case‐control PCR, samples not specified\nZhao 2020a COVID‐19 pneumonia 34 Not applicable Hospital inpatientsa Patients with pneumonia and admitted to hospital Case‐control PCR on throat or sputum swabs\nZhu 2020b COVID‐19 disease 116 28% Emergency department Patients suspected of SARS‐CoV‐2 and presenting to the emergency department Cross‐sectional PCR, samples not specified\nPCR: polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2\na'Hospital inpatients' refers to studies that recruited patients admitted to hospital with COVID‐19 disease and in whom the signs and symptoms were assessed on admission.\n\nMethodological quality of included studies\nThe results of the quality assessment are summarised in Figure 2 and Figure 3. We rated participant selection as introducing high risk of bias in seven studies. In five studies this was because a CT scan or other imaging was used to diagnose patients with pneumonia prior to inclusion in the study, leading to a highly selected patient population (Ai 2020a; Chen X 2020; Cheng 2020a; Liang 2020; Yang 2020d); RT‐PCR results were subsequently used to distinguish between COVID‐19 pneumonia and pneumonia from other causes. For all studies, testing was highly dependent on the local case definition and testing criteria that were in effect at the time of the study, meaning all patients that were included in studies had already gone through a referral/selection filter, which was not always described. The most extreme example of this is the study by Liang 2020, in which patients with radiological evidence of pneumonia and a clinical presentation compatible with COVID‐19 were only tested for SARS‐CoV‐2 after a panel discussion.\n2 Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies\n3 Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study Of the 16 studies included in this first version of the review, five studies did not use a cross‐sectional design. Three studies were diagnostic case‐control studies (Nobel 2020; Yang 2020d; Zhao 2020a), one study selected cases cross‐sectionally in five hospitals but only selected cases in one hospital (Chen X 2020), and one study emailed patients who had undergone testing for SARS‐CoV‐2 about olfactory symptoms prior to the SARS‐CoV‐2 test, with a response rate of 58% in SARS‐CoV‐2 positive cases and 15% in negative cases (Yan 2020a).\nWe rated all studies except two as carrying a high risk of bias for the index tests because there was little to no detail on how, by whom, and when the signs and symptoms were measured. In addition, there is considerable uncertainty around the reference standard, with some studies providing little detail on the RT‐PCR tests that they used or lack of clarity on blinding.\nParticipant flow was unclear in four studies (Yan 2020a; Yang 2020d; Zhao 2020a; Zhu 2020b), either because the timing of recording signs and symptoms and conduct of the reference standard was unclear, or because some tests received a second or third reference standard at unclear time points during hospital admission.\nWe rated applicability for participant selection as high risk when there was a risk of selection bias or studies did not describe selection. As for the applicability of the index tests and reference standard, we always scored this as low risk except for Chen X 2020, because blinding of the index tests was unclear, and Yang 2020d, because blinding and sample of the reference standard were unclear.\n\nFindings\nThe main characteristics of all included studies are listed in Table 3. There were four studies in hospital inpatients (Ai 2020a; Chen X 2020; Yang 2020d; Zhao 2020a), seven studies in hospital outpatients (Cheng 2020a; Liang 2020; Nobel 2020; Peng 2020a; Song 2020b; Sun 2020a; Yan 2020a), and four studies in emergency departments (Feng 2020a; Tolia 2020; Wee 2020; Zhu 2020b). The setting was not specified in one study (Rentsch 2020); in the 'Summary of findings' table, we classified this study setting as being hospital outpatient under the assumption that at that time in the pandemic (February 2020 to March 2020) tests were not commonly available outside hospital clinics. There were no studies conducted in community primary care services.\nSeven studies assessed the accuracy of signs and symptoms for the diagnosis of COVID‐19 pneumonia (Ai 2020a; Chen X 2020; Cheng 2020a; Feng 2020a; Liang 2020; Yang 2020d; Zhao 2020a); the remaining studies had COVID‐19 disease as the target condition, with no further description of the severity, meaning some patients could have suffered from mild or moderate COVID‐19 disease and others from COVID‐19 pneumonia. The distinction between these two target conditions was not always very clear, and a degree of overlap is to be assumed. All studies used RT‐PCR testing as the reference standard, with some variation in the samples that were used.\nIn all, 7706 patients were included, the median number of participants was 134. Prevalence of infection varied from 5% to 38% with a median of 17%. There were no studies in children or elderly populations, except for Rentsch 2020, which included a cohort of a median age of 65.7 years old from the Veterans Affairs Healthcare System database.\nWe found data on 27 signs and symptoms, which fall into four different categories: systemic, respiratory, gastrointestinal and cardiovascular signs and symptoms. There were no analyses for combinations of tests, only for individual signs and symptoms. The results are summarised in Table 3. Results for the cross‐sectional studies are presented in forest plots (Figure 4; Figure 5; Figure 6; Figure 7), and are plotted in ROC (receiver operating characteristic) space (Figure 8; Figure 9; Figure 10; Figure 11), results for the other studies are only listed in forest plots (Figure 12; Figure 13; Figure 14; Figure 15).\n4 Forest plot of respiratory signs and symptoms (cross‐sectional studies)\n5 Forest plot of systemic signs and symptoms (cross‐sectional studies)\n6 Forest plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n7 Forest plot of cardiovascular signs and symptoms (cross‐sectional studies)\n8 Summary ROC plot of respiratory signs and symptoms (cross‐sectional studies)\n9 Summary ROC plot of systemic signs and symptoms (cross‐sectional studies)\n10 Summary ROC Plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n11 Summary ROC plot of cardiovascular signs and symptoms (cross‐sectional studies)\n12 Forest plot of tests: 27 cough (non‐cross‐sectional study), 28 sore throat (non‐cross‐sectional study), 29 rhinorrhoea (non‐cross‐sectional study), 30 nasal obstruction (non‐cross‐sectional study), 34 dyspnoea (non‐cross‐sectional study), 31 loss of sense of smell (non‐cross‐sectional study), 32 loss of taste (non‐cross‐sectional study), 33 positive auscultation findings (non‐cross‐sectional study)\n13 Forest plot of tests: 37 fatigue (non‐cross‐sectional study), 36 fever (non‐cross‐sectional study), 39 headache (non‐cross‐sectional study), 38 myalgia or arthralgia (non‐cross‐sectional study)\n14 Forest plot of tests: 40 diarrhoea (non‐cross‐sectional study), 41 nausea/vomiting (non‐cross‐sectional study), 42 gastrointestinal symptoms, not specified (non‐cross‐sectional study)\n15 Forest plot of 35 chest tightness (non‐cross‐sectional study) Overall, diagnostic accuracy of individual signs and symptoms is low, especially sensitivity. In addition, results were highly variable across studies, making it difficult to draw firm conclusions.\nSigns and symptoms for which sensitivity was reported above 50% in at least one cross‐sectional study are the following.\nCough: sensitivity between 43% and 71%, specificity between 14% and 54%\nSore throat: sensitivity between 5% and 71%, specificity between 55% and 80%\nFever: sensitivity between 7% and 91%, specificity between 16% and 94%\nMyalgia or arthralgia: sensitivity between 19% and 86%, specificity between 45% and 91%\nFatigue: sensitivity between 10% and 57%, specificity between 60% and 94%\nHeadache: sensitivity between 3% and 71%, specificity between 78% and 98%\nFor fever, six of nine studies report a sensitivity of at least 80%, which is unsurprising considering fever was a key feature of COVID‐19 that was used in selecting patients for further testing. As a result, most participants in these studies would have fever, both cases and non‐cases. The same applies to cough, which was also listed as one of the main criteria for SARS‐CoV‐2 testing and may have contributed to inflated sensitivity estimates.\nSpecificity of at least 90% was achieved for 19 signs and symptoms. In only four signs and symptoms did this go along with sensitivity of at least 50% which would correspond to a positive likelihood ratio of at least 5, a commonly used arbitrary definition of a red flag. Using this definition, fever, myalgia or arthralgia, fatigue, or headache are to be considered red flags.\nStrikingly, most of the respiratory symptoms such as cough, sore throat and sputum production are below the diagonal in ROC space (Figure 8). The diagonal line in ROC space is where sensitivity equals 1‐specificity, meaning a test that is on the diagonal line has a positive likelihood ratio of 1 and is therefore not diagnostic because disease probability is left unchanged after conducting the test. Tests that lie below the diagonal line have a positive likelihood ratio that is smaller than 1, meaning the probability of COVID‐19 disease decreases when this test is positive. For example, in Sun 2020a, pretest probability of COVID‐19 is 6.9%; probability decreases to 6.4% when the patient has a cough and increases to 8.0% when the patient does not have a cough. We hypothesise on the reason for this counterintuitive finding in the discussion section. In contrast to respiratory features, systemic features are mostly above the diagonal line (Figure 9), suggesting that they do increase the probability of COVID‐19 when present. Gastrointestinal symptoms and cardiovascular features are clustered in the bottom left corner or on the diagonal line suggesting that they have very little diagnostic value (Figure 10; Figure 11).\nTo further illustrate the systemic features' ability to either rule in or rule out COVID‐19 disease or COVID‐19 pneumonia, we constructed dumbbell plots showing pre‐ and post‐test probabilities for each feature in each study (Figure 16). For each test, we have plotted the pre‐test probability, which is the prevalence of COVID‐19 disease (blue dot). Probability then changes depending on a positive test result (red dot marked +) or a negative test result (green dot marked ‐). The plot shows that fever, for example, increases the probability of COVID‐19 in two studies (Ai 2020a; Rentsch 2020), makes little to no difference in five studies (Feng 2020; Liang 2020; Peng 2020; Song 2020; Zhu 2020), and decreases the probability of COVID‐19 in two studies (Cheng 2020a; Tolia 2020).\n16 Dumbbell plot: this plot shows how disease probability changes after a positive test result (red dot with plus sign) or after a negative test (green dot with minus sign). Pre‐test probability or prevalence is the blue dot"}

    LitCovid-PubTator

    {"project":"LitCovid-PubTator","denotations":[{"id":"391","span":{"begin":4778,"end":4786},"obj":"Species"},{"id":"393","span":{"begin":4813,"end":4821},"obj":"Disease"},{"id":"724","span":{"begin":658,"end":668},"obj":"Species"},{"id":"725","span":{"begin":752,"end":760},"obj":"Species"},{"id":"726","span":{"begin":629,"end":638},"obj":"Disease"},{"id":"727","span":{"begin":742,"end":751},"obj":"Disease"},{"id":"730","span":{"begin":1101,"end":1113},"obj":"Species"},{"id":"731","span":{"begin":1170,"end":1182},"obj":"Species"},{"id":"790","span":{"begin":1665,"end":1673},"obj":"Species"},{"id":"791","span":{"begin":1845,"end":1853},"obj":"Species"},{"id":"792","span":{"begin":1973,"end":1986},"obj":"Species"},{"id":"793","span":{"begin":2051,"end":2059},"obj":"Species"},{"id":"794","span":{"begin":2221,"end":2229},"obj":"Species"},{"id":"795","span":{"begin":2382,"end":2390},"obj":"Species"},{"id":"796","span":{"begin":2467,"end":2480},"obj":"Species"},{"id":"797","span":{"begin":2588,"end":2596},"obj":"Species"},{"id":"798","span":{"begin":2611,"end":2621},"obj":"Species"},{"id":"799","span":{"begin":2787,"end":2795},"obj":"Species"},{"id":"800","span":{"begin":2940,"end":2948},"obj":"Species"},{"id":"801","span":{"begin":2960,"end":2970},"obj":"Species"},{"id":"802","span":{"begin":3136,"end":3144},"obj":"Species"},{"id":"803","span":{"begin":3156,"end":3166},"obj":"Species"},{"id":"804","span":{"begin":3267,"end":3275},"obj":"Species"},{"id":"805","span":{"begin":3566,"end":3574},"obj":"Species"},{"id":"806","span":{"begin":3758,"end":3766},"obj":"Species"},{"id":"807","span":{"begin":3915,"end":3925},"obj":"Species"},{"id":"808","span":{"begin":3927,"end":3935},"obj":"Species"},{"id":"809","span":{"begin":3960,"end":3970},"obj":"Species"},{"id":"810","span":{"begin":4054,"end":4067},"obj":"Species"},{"id":"811","span":{"begin":4143,"end":4150},"obj":"Species"},{"id":"812","span":{"begin":4171,"end":4181},"obj":"Species"},{"id":"813","span":{"begin":4186,"end":4194},"obj":"Species"},{"id":"814","span":{"begin":4266,"end":4279},"obj":"Species"},{"id":"815","span":{"begin":4354,"end":4362},"obj":"Species"},{"id":"816","span":{"begin":4509,"end":4517},"obj":"Species"},{"id":"817","span":{"begin":4531,"end":4541},"obj":"Species"},{"id":"818","span":{"begin":4616,"end":4629},"obj":"Species"},{"id":"819","span":{"begin":4662,"end":4672},"obj":"Species"},{"id":"820","span":{"begin":4674,"end":4721},"obj":"Species"},{"id":"821","span":{"begin":1614,"end":1632},"obj":"Disease"},{"id":"822","span":{"begin":1692,"end":1701},"obj":"Disease"},{"id":"823","span":{"begin":1782,"end":1800},"obj":"Disease"},{"id":"824","span":{"begin":1868,"end":1877},"obj":"Disease"},{"id":"825","span":{"begin":2000,"end":2018},"obj":"Disease"},{"id":"826","span":{"begin":2076,"end":2081},"obj":"Disease"},{"id":"827","span":{"begin":2110,"end":2119},"obj":"Disease"},{"id":"828","span":{"begin":2170,"end":2188},"obj":"Disease"},{"id":"829","span":{"begin":2244,"end":2249},"obj":"Disease"},{"id":"830","span":{"begin":2331,"end":2349},"obj":"Disease"},{"id":"831","span":{"begin":2396,"end":2405},"obj":"Disease"},{"id":"832","span":{"begin":2424,"end":2429},"obj":"Disease"},{"id":"833","span":{"begin":2527,"end":2535},"obj":"Disease"},{"id":"834","span":{"begin":2738,"end":2746},"obj":"Disease"},{"id":"835","span":{"begin":2902,"end":2910},"obj":"Disease"},{"id":"836","span":{"begin":3087,"end":3095},"obj":"Disease"},{"id":"837","span":{"begin":3218,"end":3226},"obj":"Disease"},{"id":"838","span":{"begin":3516,"end":3524},"obj":"Disease"},{"id":"839","span":{"begin":3708,"end":3716},"obj":"Disease"},{"id":"840","span":{"begin":3783,"end":3803},"obj":"Disease"},{"id":"841","span":{"begin":3878,"end":3886},"obj":"Disease"},{"id":"842","span":{"begin":4080,"end":4098},"obj":"Disease"},{"id":"843","span":{"begin":4156,"end":4165},"obj":"Disease"},{"id":"844","span":{"begin":4200,"end":4209},"obj":"Disease"},{"id":"845","span":{"begin":4292,"end":4310},"obj":"Disease"},{"id":"846","span":{"begin":4368,"end":4377},"obj":"Disease"},{"id":"847","span":{"begin":4459,"end":4467},"obj":"Disease"},{"id":"863","span":{"begin":5025,"end":5036},"obj":"Species"},{"id":"864","span":{"begin":5179,"end":5187},"obj":"Species"},{"id":"865","span":{"begin":5265,"end":5272},"obj":"Species"},{"id":"866","span":{"begin":5613,"end":5621},"obj":"Species"},{"id":"867","span":{"begin":5808,"end":5816},"obj":"Species"},{"id":"868","span":{"begin":5931,"end":5941},"obj":"Species"},{"id":"869","span":{"begin":5193,"end":5202},"obj":"Disease"},{"id":"870","span":{"begin":5407,"end":5425},"obj":"Disease"},{"id":"871","span":{"begin":5430,"end":5439},"obj":"Disease"},{"id":"872","span":{"begin":5847,"end":5856},"obj":"Disease"},{"id":"873","span":{"begin":5901,"end":5909},"obj":"Disease"},{"id":"878","span":{"begin":6575,"end":6583},"obj":"Species"},{"id":"879","span":{"begin":6614,"end":6624},"obj":"Species"},{"id":"880","span":{"begin":6663,"end":6673},"obj":"Species"},{"id":"881","span":{"begin":6711,"end":6721},"obj":"Species"},{"id":"883","span":{"begin":7149,"end":7160},"obj":"Species"},{"id":"885","span":{"begin":7496,"end":7507},"obj":"Species"},{"id":"888","span":{"begin":8275,"end":8288},"obj":"Species"},{"id":"889","span":{"begin":8405,"end":8415},"obj":"Species"},{"id":"895","span":{"begin":8939,"end":8947},"obj":"Species"},{"id":"896","span":{"begin":8708,"end":8726},"obj":"Disease"},{"id":"897","span":{"begin":8839,"end":8847},"obj":"Disease"},{"id":"898","span":{"begin":8990,"end":8998},"obj":"Disease"},{"id":"899","span":{"begin":9023,"end":9041},"obj":"Disease"},{"id":"904","span":{"begin":9287,"end":9295},"obj":"Species"},{"id":"905","span":{"begin":9332,"end":9344},"obj":"Species"},{"id":"906","span":{"begin":9447,"end":9455},"obj":"Species"},{"id":"907","span":{"begin":9368,"end":9377},"obj":"Disease"},{"id":"913","span":{"begin":10898,"end":10903},"obj":"Disease"},{"id":"914","span":{"begin":10936,"end":10947},"obj":"Disease"},{"id":"915","span":{"begin":10980,"end":10991},"obj":"Disease"},{"id":"916","span":{"begin":11030,"end":11041},"obj":"Disease"},{"id":"917","span":{"begin":11074,"end":11082},"obj":"Disease"},{"id":"923","span":{"begin":11304,"end":11311},"obj":"Disease"},{"id":"924","span":{"begin":11344,"end":11349},"obj":"Disease"},{"id":"925","span":{"begin":11382,"end":11390},"obj":"Disease"},{"id":"926","span":{"begin":11423,"end":11430},"obj":"Disease"},{"id":"927","span":{"begin":11434,"end":11444},"obj":"Disease"},{"id":"933","span":{"begin":11619,"end":11632},"obj":"Species"},{"id":"934","span":{"begin":11502,"end":11511},"obj":"Disease"},{"id":"935","span":{"begin":11544,"end":11550},"obj":"Disease"},{"id":"936","span":{"begin":11551,"end":11559},"obj":"Disease"},{"id":"937","span":{"begin":11592,"end":11617},"obj":"Disease"},{"id":"939","span":{"begin":11683,"end":11698},"obj":"Disease"},{"id":"941","span":{"begin":12046,"end":12051},"obj":"Disease"},{"id":"943","span":{"begin":12195,"end":12200},"obj":"Disease"},{"id":"946","span":{"begin":12266,"end":12273},"obj":"Disease"},{"id":"947","span":{"begin":12277,"end":12287},"obj":"Disease"},{"id":"949","span":{"begin":12354,"end":12361},"obj":"Disease"},{"id":"951","span":{"begin":12428,"end":12436},"obj":"Disease"},{"id":"960","span":{"begin":12668,"end":12676},"obj":"Species"},{"id":"961","span":{"begin":12716,"end":12728},"obj":"Species"},{"id":"962","span":{"begin":12871,"end":12881},"obj":"Species"},{"id":"963","span":{"begin":12506,"end":12511},"obj":"Disease"},{"id":"964","span":{"begin":12605,"end":12610},"obj":"Disease"},{"id":"965","span":{"begin":12632,"end":12640},"obj":"Disease"},{"id":"966","span":{"begin":12757,"end":12762},"obj":"Disease"},{"id":"967","span":{"begin":12810,"end":12815},"obj":"Disease"},{"id":"974","span":{"begin":13321,"end":13326},"obj":"Species"},{"id":"975","span":{"begin":13245,"end":13250},"obj":"Disease"},{"id":"976","span":{"begin":13252,"end":13259},"obj":"Disease"},{"id":"977","span":{"begin":13263,"end":13273},"obj":"Disease"},{"id":"978","span":{"begin":13275,"end":13282},"obj":"Disease"},{"id":"979","span":{"begin":13287,"end":13295},"obj":"Disease"},{"id":"991","span":{"begin":14015,"end":14022},"obj":"Species"},{"id":"992","span":{"begin":14066,"end":14073},"obj":"Species"},{"id":"993","span":{"begin":13352,"end":13372},"obj":"Disease"},{"id":"994","span":{"begin":13381,"end":13386},"obj":"Disease"},{"id":"995","span":{"begin":13388,"end":13399},"obj":"Disease"},{"id":"996","span":{"begin":13853,"end":13861},"obj":"Disease"},{"id":"997","span":{"begin":13958,"end":13966},"obj":"Disease"},{"id":"998","span":{"begin":14029,"end":14034},"obj":"Disease"},{"id":"999","span":{"begin":14090,"end":14095},"obj":"Disease"},{"id":"1000","span":{"begin":14341,"end":14349},"obj":"Disease"},{"id":"1001","span":{"begin":14364,"end":14389},"obj":"Disease"},{"id":"1008","span":{"begin":14644,"end":14652},"obj":"Disease"},{"id":"1009","span":{"begin":14664,"end":14682},"obj":"Disease"},{"id":"1010","span":{"begin":14883,"end":14891},"obj":"Disease"},{"id":"1011","span":{"begin":15060,"end":15065},"obj":"Disease"},{"id":"1012","span":{"begin":15109,"end":15117},"obj":"Disease"},{"id":"1013","span":{"begin":15295,"end":15303},"obj":"Disease"}],"attributes":[{"id":"A391","pred":"tao:has_database_id","subj":"391","obj":"Tax:9606"},{"id":"A393","pred":"tao:has_database_id","subj":"393","obj":"MESH:C000657245"},{"id":"A724","pred":"tao:has_database_id","subj":"724","obj":"Tax:2697049"},{"id":"A725","pred":"tao:has_database_id","subj":"725","obj":"Tax:9606"},{"id":"A726","pred":"tao:has_database_id","subj":"726","obj":"MESH:D011014"},{"id":"A727","pred":"tao:has_database_id","subj":"727","obj":"MESH:D011014"},{"id":"A730","pred":"tao:has_database_id","subj":"730","obj":"Tax:9606"},{"id":"A731","pred":"tao:has_database_id","subj":"731","obj":"Tax:9606"},{"id":"A790","pred":"tao:has_database_id","subj":"790","obj":"Tax:9606"},{"id":"A791","pred":"tao:has_database_id","subj":"791","obj":"Tax:9606"},{"id":"A792","pred":"tao:has_database_id","subj":"792","obj":"Tax:32644"},{"id":"A793","pred":"tao:has_database_id","subj":"793","obj":"Tax:9606"},{"id":"A794","pred":"tao:has_database_id","subj":"794","obj":"Tax:9606"},{"id":"A795","pred":"tao:has_database_id","subj":"795","obj":"Tax:9606"},{"id":"A796","pred":"tao:has_database_id","subj":"796","obj":"Tax:32644"},{"id":"A797","pred":"tao:has_database_id","subj":"797","obj":"Tax:9606"},{"id":"A798","pred":"tao:has_database_id","subj":"798","obj":"Tax:2697049"},{"id":"A799","pred":"tao:has_database_id","subj":"799","obj":"Tax:9606"},{"id":"A800","pred":"tao:has_database_id","subj":"800","obj":"Tax:9606"},{"id":"A801","pred":"tao:has_database_id","subj":"801","obj":"Tax:2697049"},{"id":"A802","pred":"tao:has_database_id","subj":"802","obj":"Tax:9606"},{"id":"A803","pred":"tao:has_database_id","subj":"803","obj":"Tax:2697049"},{"id":"A804","pred":"tao:has_database_id","subj":"804","obj":"Tax:9606"},{"id":"A805","pred":"tao:has_database_id","subj":"805","obj":"Tax:9606"},{"id":"A806","pred":"tao:has_database_id","subj":"806","obj":"Tax:9606"},{"id":"A807","pred":"tao:has_database_id","subj":"807","obj":"Tax:9606"},{"id":"A808","pred":"tao:has_database_id","subj":"808","obj":"Tax:9606"},{"id":"A809","pred":"tao:has_database_id","subj":"809","obj":"Tax:2697049"},{"id":"A810","pred":"tao:has_database_id","subj":"810","obj":"Tax:32644"},{"id":"A811","pred":"tao:has_database_id","subj":"811","obj":"Tax:9606"},{"id":"A812","pred":"tao:has_database_id","subj":"812","obj":"Tax:2697049"},{"id":"A813","pred":"tao:has_database_id","subj":"813","obj":"Tax:9606"},{"id":"A814","pred":"tao:has_database_id","subj":"814","obj":"Tax:32644"},{"id":"A815","pred":"tao:has_database_id","subj":"815","obj":"Tax:9606"},{"id":"A816","pred":"tao:has_database_id","subj":"816","obj":"Tax:9606"},{"id":"A817","pred":"tao:has_database_id","subj":"817","obj":"Tax:2697049"},{"id":"A818","pred":"tao:has_database_id","subj":"818","obj":"Tax:32644"},{"id":"A819","pred":"tao:has_database_id","subj":"819","obj":"Tax:2697049"},{"id":"A820","pred":"tao:has_database_id","subj":"820","obj":"Tax:2697049"},{"id":"A821","pred":"tao:has_database_id","subj":"821","obj":"MESH:C000657245"},{"id":"A822","pred":"tao:has_database_id","subj":"822","obj":"MESH:D011014"},{"id":"A823","pred":"tao:has_database_id","subj":"823","obj":"MESH:C000657245"},{"id":"A824","pred":"tao:has_database_id","subj":"824","obj":"MESH:D011014"},{"id":"A825","pred":"tao:has_database_id","subj":"825","obj":"MESH:C000657245"},{"id":"A826","pred":"tao:has_database_id","subj":"826","obj":"MESH:D005334"},{"id":"A827","pred":"tao:has_database_id","subj":"827","obj":"MESH:D011014"},{"id":"A828","pred":"tao:has_database_id","subj":"828","obj":"MESH:C000657245"},{"id":"A829","pred":"tao:has_database_id","subj":"829","obj":"MESH:D005334"},{"id":"A830","pred":"tao:has_database_id","subj":"830","obj":"MESH:C000657245"},{"id":"A831","pred":"tao:has_database_id","subj":"831","obj":"MESH:D011014"},{"id":"A832","pred":"tao:has_database_id","subj":"832","obj":"MESH:D005334"},{"id":"A833","pred":"tao:has_database_id","subj":"833","obj":"MESH:C000657245"},{"id":"A834","pred":"tao:has_database_id","subj":"834","obj":"MESH:C000657245"},{"id":"A835","pred":"tao:has_database_id","subj":"835","obj":"MESH:C000657245"},{"id":"A836","pred":"tao:has_database_id","subj":"836","obj":"MESH:C000657245"},{"id":"A837","pred":"tao:has_database_id","subj":"837","obj":"MESH:C000657245"},{"id":"A838","pred":"tao:has_database_id","subj":"838","obj":"MESH:C000657245"},{"id":"A839","pred":"tao:has_database_id","subj":"839","obj":"MESH:C000657245"},{"id":"A840","pred":"tao:has_database_id","subj":"840","obj":"MESH:D012818"},{"id":"A841","pred":"tao:has_database_id","subj":"841","obj":"MESH:C000657245"},{"id":"A842","pred":"tao:has_database_id","subj":"842","obj":"MESH:C000657245"},{"id":"A843","pred":"tao:has_database_id","subj":"843","obj":"MESH:D011014"},{"id":"A844","pred":"tao:has_database_id","subj":"844","obj":"MESH:D011014"},{"id":"A845","pred":"tao:has_database_id","subj":"845","obj":"MESH:C000657245"},{"id":"A846","pred":"tao:has_database_id","subj":"846","obj":"MESH:D011014"},{"id":"A847","pred":"tao:has_database_id","subj":"847","obj":"MESH:C000657245"},{"id":"A863","pred":"tao:has_database_id","subj":"863","obj":"Tax:9606"},{"id":"A864","pred":"tao:has_database_id","subj":"864","obj":"Tax:9606"},{"id":"A865","pred":"tao:has_database_id","subj":"865","obj":"Tax:9606"},{"id":"A866","pred":"tao:has_database_id","subj":"866","obj":"Tax:9606"},{"id":"A867","pred":"tao:has_database_id","subj":"867","obj":"Tax:9606"},{"id":"A868","pred":"tao:has_database_id","subj":"868","obj":"Tax:2697049"},{"id":"A869","pred":"tao:has_database_id","subj":"869","obj":"MESH:D011014"},{"id":"A870","pred":"tao:has_database_id","subj":"870","obj":"MESH:C000657245"},{"id":"A871","pred":"tao:has_database_id","subj":"871","obj":"MESH:D011014"},{"id":"A872","pred":"tao:has_database_id","subj":"872","obj":"MESH:D011014"},{"id":"A873","pred":"tao:has_database_id","subj":"873","obj":"MESH:C000657245"},{"id":"A878","pred":"tao:has_database_id","subj":"878","obj":"Tax:9606"},{"id":"A879","pred":"tao:has_database_id","subj":"879","obj":"Tax:2697049"},{"id":"A880","pred":"tao:has_database_id","subj":"880","obj":"Tax:2697049"},{"id":"A881","pred":"tao:has_database_id","subj":"881","obj":"Tax:2697049"},{"id":"A883","pred":"tao:has_database_id","subj":"883","obj":"Tax:9606"},{"id":"A885","pred":"tao:has_database_id","subj":"885","obj":"Tax:9606"},{"id":"A888","pred":"tao:has_database_id","subj":"888","obj":"Tax:32644"},{"id":"A889","pred":"tao:has_database_id","subj":"889","obj":"Tax:9606"},{"id":"A895","pred":"tao:has_database_id","subj":"895","obj":"Tax:9606"},{"id":"A896","pred":"tao:has_database_id","subj":"896","obj":"MESH:C000657245"},{"id":"A897","pred":"tao:has_database_id","subj":"897","obj":"MESH:C000657245"},{"id":"A898","pred":"tao:has_database_id","subj":"898","obj":"MESH:C000657245"},{"id":"A899","pred":"tao:has_database_id","subj":"899","obj":"MESH:C000657245"},{"id":"A904","pred":"tao:has_database_id","subj":"904","obj":"Tax:9606"},{"id":"A905","pred":"tao:has_database_id","subj":"905","obj":"Tax:9606"},{"id":"A906","pred":"tao:has_database_id","subj":"906","obj":"Tax:9606"},{"id":"A907","pred":"tao:has_database_id","subj":"907","obj":"MESH:D007239"},{"id":"A913","pred":"tao:has_database_id","subj":"913","obj":"MESH:D003371"},{"id":"A914","pred":"tao:has_database_id","subj":"914","obj":"MESH:D010608"},{"id":"A916","pred":"tao:has_database_id","subj":"916","obj":"MESH:D000402"},{"id":"A917","pred":"tao:has_database_id","subj":"917","obj":"MESH:D004417"},{"id":"A923","pred":"tao:has_database_id","subj":"923","obj":"MESH:D005221"},{"id":"A924","pred":"tao:has_database_id","subj":"924","obj":"MESH:D005334"},{"id":"A925","pred":"tao:has_database_id","subj":"925","obj":"MESH:D006261"},{"id":"A926","pred":"tao:has_database_id","subj":"926","obj":"MESH:D063806"},{"id":"A927","pred":"tao:has_database_id","subj":"927","obj":"MESH:D018771"},{"id":"A933","pred":"tao:has_database_id","subj":"933","obj":"Tax:32644"},{"id":"A934","pred":"tao:has_database_id","subj":"934","obj":"MESH:D003967"},{"id":"A935","pred":"tao:has_database_id","subj":"935","obj":"MESH:D009325"},{"id":"A936","pred":"tao:has_database_id","subj":"936","obj":"MESH:D014839"},{"id":"A937","pred":"tao:has_database_id","subj":"937","obj":"MESH:D012817"},{"id":"A939","pred":"tao:has_database_id","subj":"939","obj":"MESH:D002637"},{"id":"A941","pred":"tao:has_database_id","subj":"941","obj":"MESH:D003371"},{"id":"A943","pred":"tao:has_database_id","subj":"943","obj":"MESH:D005334"},{"id":"A946","pred":"tao:has_database_id","subj":"946","obj":"MESH:D063806"},{"id":"A947","pred":"tao:has_database_id","subj":"947","obj":"MESH:D018771"},{"id":"A949","pred":"tao:has_database_id","subj":"949","obj":"MESH:D005221"},{"id":"A951","pred":"tao:has_database_id","subj":"951","obj":"MESH:D006261"},{"id":"A960","pred":"tao:has_database_id","subj":"960","obj":"Tax:9606"},{"id":"A961","pred":"tao:has_database_id","subj":"961","obj":"Tax:9606"},{"id":"A962","pred":"tao:has_database_id","subj":"962","obj":"Tax:2697049"},{"id":"A963","pred":"tao:has_database_id","subj":"963","obj":"MESH:D005334"},{"id":"A964","pred":"tao:has_database_id","subj":"964","obj":"MESH:D005334"},{"id":"A965","pred":"tao:has_database_id","subj":"965","obj":"MESH:C000657245"},{"id":"A966","pred":"tao:has_database_id","subj":"966","obj":"MESH:D005334"},{"id":"A967","pred":"tao:has_database_id","subj":"967","obj":"MESH:D003371"},{"id":"A974","pred":"tao:has_database_id","subj":"974","obj":"Tax:34205"},{"id":"A975","pred":"tao:has_database_id","subj":"975","obj":"MESH:D005334"},{"id":"A976","pred":"tao:has_database_id","subj":"976","obj":"MESH:D063806"},{"id":"A977","pred":"tao:has_database_id","subj":"977","obj":"MESH:D018771"},{"id":"A978","pred":"tao:has_database_id","subj":"978","obj":"MESH:D005221"},{"id":"A979","pred":"tao:has_database_id","subj":"979","obj":"MESH:D006261"},{"id":"A991","pred":"tao:has_database_id","subj":"991","obj":"Tax:9606"},{"id":"A992","pred":"tao:has_database_id","subj":"992","obj":"Tax:9606"},{"id":"A993","pred":"tao:has_database_id","subj":"993","obj":"MESH:D012818"},{"id":"A994","pred":"tao:has_database_id","subj":"994","obj":"MESH:D003371"},{"id":"A995","pred":"tao:has_database_id","subj":"995","obj":"MESH:D010608"},{"id":"A996","pred":"tao:has_database_id","subj":"996","obj":"MESH:C000657245"},{"id":"A997","pred":"tao:has_database_id","subj":"997","obj":"MESH:C000657245"},{"id":"A998","pred":"tao:has_database_id","subj":"998","obj":"MESH:D003371"},{"id":"A999","pred":"tao:has_database_id","subj":"999","obj":"MESH:D003371"},{"id":"A1000","pred":"tao:has_database_id","subj":"1000","obj":"MESH:C000657245"},{"id":"A1001","pred":"tao:has_database_id","subj":"1001","obj":"MESH:D012817"},{"id":"A1008","pred":"tao:has_database_id","subj":"1008","obj":"MESH:C000657245"},{"id":"A1009","pred":"tao:has_database_id","subj":"1009","obj":"MESH:C000657245"},{"id":"A1010","pred":"tao:has_database_id","subj":"1010","obj":"MESH:C000657245"},{"id":"A1011","pred":"tao:has_database_id","subj":"1011","obj":"MESH:D005334"},{"id":"A1012","pred":"tao:has_database_id","subj":"1012","obj":"MESH:C000657245"},{"id":"A1013","pred":"tao:has_database_id","subj":"1013","obj":"MESH:C000657245"}],"namespaces":[{"prefix":"Tax","uri":"https://www.ncbi.nlm.nih.gov/taxonomy/"},{"prefix":"MESH","uri":"https://id.nlm.nih.gov/mesh/"},{"prefix":"Gene","uri":"https://www.ncbi.nlm.nih.gov/gene/"},{"prefix":"CVCL","uri":"https://web.expasy.org/cellosaurus/CVCL_"}],"text":"Results\n\nResults of the search\nThe search yielded 10,965 records after removing duplicates. The first selection resulted in 658 records that were potentially eligible for this review on signs and symptoms. After screening on title and abstract, we excluded 457 records, leaving 201 to be assessed on full text. Of these, we included 16 studies in this review. The reasons for excluding 185 records are listed in the PRISMA flow chart (see Figure 1; Moher 2009).\n1 Flow diagram Two studies reported on the same cases while using a different control group (Chen X 2020; Yang 2020d). Chen X 2020 used a concurrent control group of pneumonia cases negative for SARS‐CoV‐2 on PCR testing but Yang 2020d used a historic control group of influenza pneumonia patients. For this reason we only included the Chen X 2020 results in the analyses.\nOne study reported a study that included a derivation and validation part for the development of a prediction rule (Song 2020b). The two parts are identical in set‐up and only differ in respect to the time of data collection, that is, the derivation part recruited participants up to 5 February 2020 and the validation part recruited participants from 6 February 2020 onwards. As a result, we consider this to be one study and have entered all data on signs and symptoms as such.\nFour studies were conducted in the USA, all other studies were from China. A summary of the main study characteristics can be found in Table 3.\n2 Summary of study characteristics\nStudy ID Target condition Sample size Prevalence Setting Population Design Reference standard\nAi 2020a COVID‐19 pneumonia 53 38% Hospital inpatientsa Patients hospitalised with pneumonia diagnosed by imaging Cross‐sectional PCR on nasopharyngeal swabs\nChen X 2020 COVID‐19 pneumonia 136 Not applicable Hospital inpatientsa Patients admitted with pneumonia Cases selected cross‐sectionally in 5 hospitals, non‐cases from 1 hospital only PCR, samples not specified\nCheng 2020a COVID‐19 pneumonia 33 33% Hospital outpatients Patients presenting to a fever observation department with pneumonia Cross‐sectional PCR on throat swabs\nFeng 2020a COVID‐19 pneumonia 132 5% Emergency department Patients presenting to fever clinic of emergency department Cross‐sectional PCR on throat swabs\nLiang 2020 COVID‐19 pneumonia 88 24% Hospital outpatients Patients with pneumonia and presenting to fever clinic Cross‐sectional PCR, sample not specified; conducted after panel discussion\nNobel 2020 COVID‐19 disease 516 Not applicable Hospital outpatients Patients who underwent SARS‐CoV‐2 testing with intent to hospitalise or in essential personnel Case‐control PCR on nasopharyngeal swabs\nPeng 2020a COVID‐19 disease 86 13% Hospital outpatients Patients clinically suspected and referred for testing Cross‐sectional PCR on nasopharyngeal swabs\nRentsch 2020 COVID‐19 disease 3789 15% Unclear Patients tested for SARS‐CoV‐2 in the Veterans Affairs Cohort born between 1945 and 1965 Cross‐sectional PCR on nasopharyngeal swabs\nSong 2020b COVID‐19 disease 399 7% Hospital outpatients Patients tested for SARS‐CoV‐2 Cross‐sectional PCR on sputum samples\nSun 2020a COVID‐19 disease 788 7% Hospital outpatients Patients presenting to testing centre, either self‐referred, referred from primary care or at‐risk cases identified by national contact tracing Cross‐sectional PCR on sputum, endotracheal aspirate, nasopharyngeal swabs or throat swabs\nTolia 2020 COVID‐19 disease 283 10% Emergency department Patients presenting with symptoms, travel history, risk factors or healthcare workers Cross‐sectional PCR on nasopharyngeal swabs\nWee 2020 COVID‐19 disease 870 18% Emergency department Patients presenting with respiratory symptoms or travel history Cross‐sectional PCR on oropharyngeal swabs\nYan 2020a COVID‐19 disease 262 23% Hospital outpatient Patients presenting hospital for SARS‐CoV‐2 testing, not otherwise specified Internet survey after presentation PCR, samples not specified\nYang 2020d COVID‐19 pneumonia 121 Not applicable Hospital inpatientsa Patient with pneumonia from SARS‐CoV‐2 and patients with pneumonia from influenza in 2015‐2019 Case‐control PCR, samples not specified\nZhao 2020a COVID‐19 pneumonia 34 Not applicable Hospital inpatientsa Patients with pneumonia and admitted to hospital Case‐control PCR on throat or sputum swabs\nZhu 2020b COVID‐19 disease 116 28% Emergency department Patients suspected of SARS‐CoV‐2 and presenting to the emergency department Cross‐sectional PCR, samples not specified\nPCR: polymerase chain reaction; SARS‐CoV‐2: severe acute respiratory syndrome coronavirus 2\na'Hospital inpatients' refers to studies that recruited patients admitted to hospital with COVID‐19 disease and in whom the signs and symptoms were assessed on admission.\n\nMethodological quality of included studies\nThe results of the quality assessment are summarised in Figure 2 and Figure 3. We rated participant selection as introducing high risk of bias in seven studies. In five studies this was because a CT scan or other imaging was used to diagnose patients with pneumonia prior to inclusion in the study, leading to a highly selected patient population (Ai 2020a; Chen X 2020; Cheng 2020a; Liang 2020; Yang 2020d); RT‐PCR results were subsequently used to distinguish between COVID‐19 pneumonia and pneumonia from other causes. For all studies, testing was highly dependent on the local case definition and testing criteria that were in effect at the time of the study, meaning all patients that were included in studies had already gone through a referral/selection filter, which was not always described. The most extreme example of this is the study by Liang 2020, in which patients with radiological evidence of pneumonia and a clinical presentation compatible with COVID‐19 were only tested for SARS‐CoV‐2 after a panel discussion.\n2 Risk of bias and applicability concerns graph: review authors' judgements about each domain presented as percentages across included studies\n3 Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study Of the 16 studies included in this first version of the review, five studies did not use a cross‐sectional design. Three studies were diagnostic case‐control studies (Nobel 2020; Yang 2020d; Zhao 2020a), one study selected cases cross‐sectionally in five hospitals but only selected cases in one hospital (Chen X 2020), and one study emailed patients who had undergone testing for SARS‐CoV‐2 about olfactory symptoms prior to the SARS‐CoV‐2 test, with a response rate of 58% in SARS‐CoV‐2 positive cases and 15% in negative cases (Yan 2020a).\nWe rated all studies except two as carrying a high risk of bias for the index tests because there was little to no detail on how, by whom, and when the signs and symptoms were measured. In addition, there is considerable uncertainty around the reference standard, with some studies providing little detail on the RT‐PCR tests that they used or lack of clarity on blinding.\nParticipant flow was unclear in four studies (Yan 2020a; Yang 2020d; Zhao 2020a; Zhu 2020b), either because the timing of recording signs and symptoms and conduct of the reference standard was unclear, or because some tests received a second or third reference standard at unclear time points during hospital admission.\nWe rated applicability for participant selection as high risk when there was a risk of selection bias or studies did not describe selection. As for the applicability of the index tests and reference standard, we always scored this as low risk except for Chen X 2020, because blinding of the index tests was unclear, and Yang 2020d, because blinding and sample of the reference standard were unclear.\n\nFindings\nThe main characteristics of all included studies are listed in Table 3. There were four studies in hospital inpatients (Ai 2020a; Chen X 2020; Yang 2020d; Zhao 2020a), seven studies in hospital outpatients (Cheng 2020a; Liang 2020; Nobel 2020; Peng 2020a; Song 2020b; Sun 2020a; Yan 2020a), and four studies in emergency departments (Feng 2020a; Tolia 2020; Wee 2020; Zhu 2020b). The setting was not specified in one study (Rentsch 2020); in the 'Summary of findings' table, we classified this study setting as being hospital outpatient under the assumption that at that time in the pandemic (February 2020 to March 2020) tests were not commonly available outside hospital clinics. There were no studies conducted in community primary care services.\nSeven studies assessed the accuracy of signs and symptoms for the diagnosis of COVID‐19 pneumonia (Ai 2020a; Chen X 2020; Cheng 2020a; Feng 2020a; Liang 2020; Yang 2020d; Zhao 2020a); the remaining studies had COVID‐19 disease as the target condition, with no further description of the severity, meaning some patients could have suffered from mild or moderate COVID‐19 disease and others from COVID‐19 pneumonia. The distinction between these two target conditions was not always very clear, and a degree of overlap is to be assumed. All studies used RT‐PCR testing as the reference standard, with some variation in the samples that were used.\nIn all, 7706 patients were included, the median number of participants was 134. Prevalence of infection varied from 5% to 38% with a median of 17%. There were no studies in children or elderly populations, except for Rentsch 2020, which included a cohort of a median age of 65.7 years old from the Veterans Affairs Healthcare System database.\nWe found data on 27 signs and symptoms, which fall into four different categories: systemic, respiratory, gastrointestinal and cardiovascular signs and symptoms. There were no analyses for combinations of tests, only for individual signs and symptoms. The results are summarised in Table 3. Results for the cross‐sectional studies are presented in forest plots (Figure 4; Figure 5; Figure 6; Figure 7), and are plotted in ROC (receiver operating characteristic) space (Figure 8; Figure 9; Figure 10; Figure 11), results for the other studies are only listed in forest plots (Figure 12; Figure 13; Figure 14; Figure 15).\n4 Forest plot of respiratory signs and symptoms (cross‐sectional studies)\n5 Forest plot of systemic signs and symptoms (cross‐sectional studies)\n6 Forest plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n7 Forest plot of cardiovascular signs and symptoms (cross‐sectional studies)\n8 Summary ROC plot of respiratory signs and symptoms (cross‐sectional studies)\n9 Summary ROC plot of systemic signs and symptoms (cross‐sectional studies)\n10 Summary ROC Plot of gastrointestinal signs and symptoms (cross‐sectional studies)\n11 Summary ROC plot of cardiovascular signs and symptoms (cross‐sectional studies)\n12 Forest plot of tests: 27 cough (non‐cross‐sectional study), 28 sore throat (non‐cross‐sectional study), 29 rhinorrhoea (non‐cross‐sectional study), 30 nasal obstruction (non‐cross‐sectional study), 34 dyspnoea (non‐cross‐sectional study), 31 loss of sense of smell (non‐cross‐sectional study), 32 loss of taste (non‐cross‐sectional study), 33 positive auscultation findings (non‐cross‐sectional study)\n13 Forest plot of tests: 37 fatigue (non‐cross‐sectional study), 36 fever (non‐cross‐sectional study), 39 headache (non‐cross‐sectional study), 38 myalgia or arthralgia (non‐cross‐sectional study)\n14 Forest plot of tests: 40 diarrhoea (non‐cross‐sectional study), 41 nausea/vomiting (non‐cross‐sectional study), 42 gastrointestinal symptoms, not specified (non‐cross‐sectional study)\n15 Forest plot of 35 chest tightness (non‐cross‐sectional study) Overall, diagnostic accuracy of individual signs and symptoms is low, especially sensitivity. In addition, results were highly variable across studies, making it difficult to draw firm conclusions.\nSigns and symptoms for which sensitivity was reported above 50% in at least one cross‐sectional study are the following.\nCough: sensitivity between 43% and 71%, specificity between 14% and 54%\nSore throat: sensitivity between 5% and 71%, specificity between 55% and 80%\nFever: sensitivity between 7% and 91%, specificity between 16% and 94%\nMyalgia or arthralgia: sensitivity between 19% and 86%, specificity between 45% and 91%\nFatigue: sensitivity between 10% and 57%, specificity between 60% and 94%\nHeadache: sensitivity between 3% and 71%, specificity between 78% and 98%\nFor fever, six of nine studies report a sensitivity of at least 80%, which is unsurprising considering fever was a key feature of COVID‐19 that was used in selecting patients for further testing. As a result, most participants in these studies would have fever, both cases and non‐cases. The same applies to cough, which was also listed as one of the main criteria for SARS‐CoV‐2 testing and may have contributed to inflated sensitivity estimates.\nSpecificity of at least 90% was achieved for 19 signs and symptoms. In only four signs and symptoms did this go along with sensitivity of at least 50% which would correspond to a positive likelihood ratio of at least 5, a commonly used arbitrary definition of a red flag. Using this definition, fever, myalgia or arthralgia, fatigue, or headache are to be considered red flags.\nStrikingly, most of the respiratory symptoms such as cough, sore throat and sputum production are below the diagonal in ROC space (Figure 8). The diagonal line in ROC space is where sensitivity equals 1‐specificity, meaning a test that is on the diagonal line has a positive likelihood ratio of 1 and is therefore not diagnostic because disease probability is left unchanged after conducting the test. Tests that lie below the diagonal line have a positive likelihood ratio that is smaller than 1, meaning the probability of COVID‐19 disease decreases when this test is positive. For example, in Sun 2020a, pretest probability of COVID‐19 is 6.9%; probability decreases to 6.4% when the patient has a cough and increases to 8.0% when the patient does not have a cough. We hypothesise on the reason for this counterintuitive finding in the discussion section. In contrast to respiratory features, systemic features are mostly above the diagonal line (Figure 9), suggesting that they do increase the probability of COVID‐19 when present. Gastrointestinal symptoms and cardiovascular features are clustered in the bottom left corner or on the diagonal line suggesting that they have very little diagnostic value (Figure 10; Figure 11).\nTo further illustrate the systemic features' ability to either rule in or rule out COVID‐19 disease or COVID‐19 pneumonia, we constructed dumbbell plots showing pre‐ and post‐test probabilities for each feature in each study (Figure 16). For each test, we have plotted the pre‐test probability, which is the prevalence of COVID‐19 disease (blue dot). Probability then changes depending on a positive test result (red dot marked +) or a negative test result (green dot marked ‐). The plot shows that fever, for example, increases the probability of COVID‐19 in two studies (Ai 2020a; Rentsch 2020), makes little to no difference in five studies (Feng 2020; Liang 2020; Peng 2020; Song 2020; Zhu 2020), and decreases the probability of COVID‐19 in two studies (Cheng 2020a; Tolia 2020).\n16 Dumbbell plot: this plot shows how disease probability changes after a positive test result (red dot with plus sign) or after a negative test (green dot with minus sign). Pre‐test probability or prevalence is the blue dot"}