| Id |
Subject |
Object |
Predicate |
Lexical cue |
| T118 |
0-7 |
Sentence |
denotes |
Results |
| T119 |
8-184 |
Sentence |
denotes |
We collected data from 35 463 patients between 6 February 2020 and 20 May 2020 in the derivation cohort; 1275 (3.6%) patients had no outcome recorded and were considered alive. |
| T120 |
185-240 |
Sentence |
denotes |
The overall mortality rate was 32.2% (11 426 patients). |
| T121 |
241-399 |
Sentence |
denotes |
The median age of patients in the cohort was 73 years (interquartile range 59-83); 41.7% (14 741) were female and 76.0% (26 966) had at least one comorbidity. |
| T122 |
400-498 |
Sentence |
denotes |
Table 1 shows demographic and clinical characteristics for the derivation and validation datasets. |
| T123 |
500-517 |
Sentence |
denotes |
Model development |
| T124 |
518-635 |
Sentence |
denotes |
We identified 41 candidate predictor variables measured at hospital admission for model creation (fig 1, appendix 2). |
| T125 |
636-821 |
Sentence |
denotes |
After the creation of a composite variable containing all seven individual comorbidities and the exclusion of 13 variables owing to high levels of missing values, 21 variables remained. |
| T126 |
822-1135 |
Sentence |
denotes |
We identified eight important predictors of mortality by using generalised additive modelling with multiply imputed datasets: age, sex, number of comorbidities, respiratory rate, peripheral oxygen saturation, Glasgow coma scale, urea level, and C reactive protein (for variable selection process, see appendix 3). |
| T127 |
1136-1386 |
Sentence |
denotes |
Given the need for a pragmatic score for use at the bedside, continuous variables were converted to factors with cut-off values chosen by using component smoothed functions (on linear predictor scale) from generalised additive modelling (appendix 4). |
| T128 |
1387-1567 |
Sentence |
denotes |
On entering variables into a penalised logistic regression model (least absolute shrinkage and selection operator), all variables were retained within the final model (appendix 5). |
| T129 |
1568-1716 |
Sentence |
denotes |
We converted penalised regression coefficients into a prognostic index by using appropriate scaling (4C Mortality Score range 0-21 points; table 2). |
| T130 |
1717-1884 |
Sentence |
denotes |
The 4C Mortality Score showed good discrimination for death in hospital within the derivation cohort (table 3), with performance approaching that of the XGBoost model. |
| T131 |
1885-2069 |
Sentence |
denotes |
The 4C Mortality Score showed good calibration (calibration intercept=0, slope=1, Brier score 0.170) across the range of risk and no adjustment to the model was required (appendix 11). |
| T132 |
2071-2087 |
Sentence |
denotes |
Model validation |
| T133 |
2088-2304 |
Sentence |
denotes |
The validation cohort included data from 22 361 patients collected between 21 May 2020 and 29 June 2020 who had at least four weeks of follow-up; 743 (3.3%) patients had no outcome recorded and were considered alive. |
| T134 |
2305-2358 |
Sentence |
denotes |
The overall mortality rate was 30.1% (6729 patients). |
| T135 |
2359-2525 |
Sentence |
denotes |
The median age of patients in the cohort was 76 (interquartile range 60-85) years; 10 178 (45.6%) were female and 17 263 (77%) had at least one comorbidity (table 1). |
| T136 |
2526-2643 |
Sentence |
denotes |
Discrimination of the 4C Mortality Score in the validation cohort was similar to that of the XGBoost model (table 3). |
| T137 |
2644-2901 |
Sentence |
denotes |
Calibration was also found to be excellent in the validation cohort: overall observed (30.1%) versus predicted (30.1%) mortality was equal (calibration-in-the-large=0) and calibration was excellent over the range of risk (slope=1, Brier score 0.171; fig 2). |
| T138 |
2902-3023 |
Sentence |
denotes |
The 4C Mortality Score showed good performance in clinically relevant metrics across a range of cut-off values (table 4). |
| T139 |
3024-3260 |
Sentence |
denotes |
Four risk groups were defined with corresponding mortality rates determined (table 5): low risk (0-3 score, mortality rate 1.2%), intermediate risk (4-8 score, 9.9%), high risk (9-14 score, 31.4%), and very high risk (≥15 score, 61.5%). |
| T140 |
3261-3448 |
Sentence |
denotes |
Performance metrics showed a high sensitivity (99.7%) and negative predictive value (98.8%) for the low risk group, covering 7.4% of the cohort and a corresponding mortality rate of 1.2%. |
| T141 |
3449-3579 |
Sentence |
denotes |
Patients in the intermediate risk group (score 4-8, n=4889, 21.9%) had a mortality rate of 9.9% (negative predictive value 90.1%). |
| T142 |
3580-3823 |
Sentence |
denotes |
Patients in the high risk group (score 9-14, n=11 664, 52.2%) had a mortality rate of 31.4% (negative predictive value 68.6%), while patients scoring 15 or higher (n=4158, 18.6%) had a mortality rate of 61.5% (positive predictive value 61.5%). |
| T143 |
3824-3892 |
Sentence |
denotes |
An interactive infographic is available at https://isaric4c.net/risk |
| T144 |
3894-3924 |
Sentence |
denotes |
Comparison with existing tools |
| T145 |
3925-4211 |
Sentence |
denotes |
We performed a systematic literature search and identified 15 risk stratification scores that could beapplied to these data.62228-40 The 4C Mortality Score compared well against these existing risk stratification scores in predicting in-hospital mortality (table 6, fig 3, upper panel). |
| T146 |
4212-4595 |
Sentence |
denotes |
Risk stratification scores originally validated in patients with community acquired pneumonia (n=9) generally had higher discrimination for inhospital mortality in the validation cohort (eg, A-DROP (area under the receiver operating characteristic curve 0.74, 95% confidence interval 0.73 to 0.74) and E-CURB65 (0.76, 0.74 to 0.79)) than those developed within covid-19 cohorts (n=4: |
| T147 |
4596-4730 |
Sentence |
denotes |
Surgisphere (0.63, 0.62 to 0.64), DL score (0.67, 0.66 to 0.68), COVID-GRAM (0.71, 0.68 to 0.74), and Xie score (0.73, 0.70 to 0.75)). |
| T148 |
4731-4878 |
Sentence |
denotes |
Performance metrics for the 4C Mortality Score compared well against existing risk stratification scores at specified cut-off values (appendix 13). |
| T149 |
4879-5110 |
Sentence |
denotes |
The number of patients in whom risk stratification scores could be applied differed owing to certain variables not being available, either because of missingness or because they were not tested for or recorded in clinical practice. |
| T150 |
5111-5330 |
Sentence |
denotes |
Seven scores could be applied to fewer than 2000 patients (<10%) in the validation cohort owing to the requirement for biomarkers or physiological parameters that were not routinely captured (eg, lactate dehydrogenase). |
| T151 |
5331-5600 |
Sentence |
denotes |
Decision curve analysis showed that the 4C Mortality Score had better clinical utility across a wide range of threshold risks compared with the best performing existing scores applicable to more than 50% of the validation cohort (A-DROP and CURB65; fig 3, lower panel). |
| T152 |
5602-5622 |
Sentence |
denotes |
Sensitivity analysis |
| T153 |
5623-5803 |
Sentence |
denotes |
Sensitivity analyses that used complete case data showed similar discrimination (appendix 7) and performance metrics (appendices 8 and 9) to analyses that used the imputed dataset. |
| T154 |
5804-6152 |
Sentence |
denotes |
After stratification of the validation cohort into two geographical cohorts (validation north and south; appendix 14), discrimination remained similar for the 4C Mortality Score in the north subset (area under the receiver operating characteristic curve 0.77, 95% confidence interval 0.76 to 0.78) and south subset (0.76, 0.75 to 0.77; appendix 6). |
| T155 |
6153-6252 |
Sentence |
denotes |
Finally, we checked discrimination of the 4C Mortality Score by sex and ethnic group (appendix 10). |
| T156 |
6253-6419 |
Sentence |
denotes |
Discrimination was the same in men (area under the receiver operating characteristic curve 0.77, 95% confidence interval 0.76 to 0.78) and women (0.76, 0.75 to 0.77). |
| T157 |
6420-6506 |
Sentence |
denotes |
Discrimination was better in all nonwhite ethnic groups compared with the white group: |
| T158 |
6507-6649 |
Sentence |
denotes |
South Asian (0.82, 0.80 to 0.85), East Asian (0.85, 0.79 to 0.91), Black (0.83, 0.80 to 0.86), and other ethnic minority (0.81, 0.79 to 0.84). |
