0_colil

PMC:2940023 / 13246-13248 JSON TXT

Incremental value of the CT coronary calcium score for the prediction of coronary artery disease Abstract Objectives: To validate published prediction models for the presence of obstructive coronary artery disease (CAD) in patients with new onset stable typical or atypical angina pectoris and to assess the incremental value of the CT coronary calcium score (CTCS). Methods: We searched the literature for clinical prediction rules for the diagnosis of obstructive CAD, defined as ≥50% stenosis in at least one vessel on conventional coronary angiography. Significant variables were re-analysed in our dataset of 254 patients with logistic regression. CTCS was subsequently included in the models. The area under the receiver operating characteristic curve (AUC) was calculated to assess diagnostic performance. Results: Re-analysing the variables used by Diamond & Forrester yielded an AUC of 0.798, which increased to 0.890 by adding CTCS. For Pryor, Morise 1994, Morise 1997 and Shaw the AUC increased from 0.838 to 0.901, 0.831 to 0.899, 0.840 to 0.898 and 0.833 to 0.899. CTCS significantly improved model performance in each model. Conclusions: Validation demonstrated good diagnostic performance across all models. CTCS improves the prediction of the presence of obstructive CAD, independent of clinical predictors, and should be considered in its diagnostic work-up. Introduction The CT coronary calcium score (CTCS) is used in both the diagnosis of coronary artery disease (CAD) [1–6] and the prediction of cardiovascular events [7–13]. Although substantial evidence is available on the incremental value of CTCS in predicting future cardiovascular events and mortality in asymptomatic individuals [10–13], the diagnostic value in symptomatic patients is less clear. A meta-analysis revealed that the presence of coronary calcium had a sensitivity and specificity of 98% and 40%, respectively, in detecting significant stenoses [8]. Furthermore, numerous studies have reported on the value of CTCS in the prediction of the probability of obstructive CAD [1–4]. However, the incremental value (i.e. in addition to all known clinical predictors of CAD) of the CTCS as a continuous predictor of prevalent obstructive CAD is less well studied. The purpose of this study was to validate previously published clinical prediction models and to determine the incremental value of CTCS for the prediction of prevalent obstructive CAD in patients with new onset stable typical or atypical angina pectoris. Materials and methods Study population The study population was derived from an existing database, which consisted of 402 patients with chest pain suggestive of stable angina pectoris and suspected of having CAD. All patients were prospectively included in a large study evaluating 64-slice CT coronary angiography (CTCA) at our institution. All patients were referred for conventional coronary angiography (CCA) based on their presentation or functional testing that suggested the presence of ischaemia and all patients underwent multidetector CT angiography within a week before CCA. Inclusion criteria for this study were: informed consent, sinus heart rhythm and the ability to hold their breath for 15 s. Patients with a history of percutaneous coronary intervention or coronary artery bypass surgery, impaired renal function (serum creatinine >120 μmol/L) or a known intolerance to iodinated contrast medium were excluded. The Institutional Review Board approved the study and all patients signed informed consent. As this paper focuses on patients with new onset stable chest pain, we also excluded patients with acute coronary syndromes and patients with a previous myocardial infarction (Fig. 1). Fig. 1 Flow chart of patients in the study. CTCA computed tomography coronary angiography. *Data from an existing database were used. All patients were referred for conventional coronary angiography based on their presentation or functional testing that suggested the presence of cardiac ischaemia. See Materials and methods CT coronary calcium images Metoprolol (100 mg, Selokeen, AstraZeneca, London, UK) was administered orally 1 h before CT in patients with heart rates >65 beats per minute. A 64-slice single source CT system (Sensation 64; Siemens, Forchheim, Germany) with a gantry rotation time of 330 ms, acquisition time of 165 ms and voxel size of 0.4 mm3 was used to acquire standard spiral low-dose and ECG-gated coronary calcium CT images. CT parameters were 32 × 2 slices per rotation, individual detector width of 0.6 mm, 3.8-mm/rotation table feed, 120-kV tube voltage, 150-mAs tube current, with activated prospective x-ray tube modulation. Overlapping slices were reconstructed at 65% of the R–R interval using the B35f convolution kernel. Reconstructed slice thickness was 3.0 mm with an increment of 1.5 mm. The radiation exposure, estimated using dedicated software (ImPACT, version 0.99x, St. George’s Hospital, Tooting, London, UK), was 1.4 mSv in men and 1.8 mSv in women. One observer (with more than 3 years’ experience), who was blinded to the CCA and clinical data, measured the coronary calcium. Conventional coronary angiography The CTCS and CCA were carried out within 1 week. Coronary segments were assessed on CCA following a 17-segment modified American Heart Association (AHA) classification model [14] by a single observer (with more than 10 years’ experience), who was blinded to the CT and clinical data. A mean luminal narrowing of ≥50% was considered to be a significant stenosis. Validated quantitative coronary angiography software (CAAS II®, Pie Medical, Maastricht, the Netherlands) was used. Clinical variables and outcome All patients were interviewed at enrollment in the prospective cohort study. Clinical parameters recorded were: age (years), sex (male/female), type of chest pain (atypical vs. typical), body mass index (BMI) (defined as weight/height2 in kg/m2), smoking status (past or current smoker, yes/no), hypertension (present/absent), dyslipidaemia (serum cholesterol >200 mg/dL or 5.18 mmol/L, present/absent), diabetes (plasma glucose ≥126 mg/dL or 7.0 mmol, present/absent) and family history of CAD (present/absent). The CTCS was measured by the Agatston method [15] using dedicated software (syngo Calcium Scoring VE31H, Siemens, Germany). The outcome of interest was the presence of obstructive CAD defined as ≥50% stenosis in at least one vessel (present/absent) on CCA. Sample size As a general rule, 10 patients with the condition of interest per analysed variable are required for regression analysis. In our dataset (n = 254), 123 patients were identified as having obstructive CAD on CCA. This allowed for the analysis of 12 variables. Our sample meets the required number of cases and non-cases that has been suggested for external validation of prediction models [16]. Systematic literature search We searched the English-language medical literature in PubMed up to October 14, 2009 for diagnostic prediction models. See the Appendix for a detailed description of the search strategy. From the included articles, clinical variables that were identified as significant predictors of CAD were extracted. Data analysis Age was analysed as a continuous variable. To account for the skewed distribution of the coronary calcium scoring, CTCS was transformed by taking the natural logarithm of CTCS + 1. All other variables were dichotomous. Oestrogen status was not available in our dataset. Therefore, we assumed women below the age of 50 to be oestrogen positive, women of 50 years and above to be oestrogen negative and all men to be oestrogen neutral. Obesity was considered in the model by Morise (1997) only. We defined obesity as a BMI >27 kg/m2, corresponding to their definition [17]. The extracted sets of clinical variables were analysed with multivariate logistic regression analysis, fitting new regression coefficients. No attempt was made to validate original regression coefficients, as such coefficients were often not reported. CTCS was subsequently included in each of the models. Models without CTCS were compared with corresponding models including CTCS using the likelihood ratio test. The level of significance was defined at a p value less than 0.05. Diagnostic performance was assessed by calculating the area under the receiver operating characteristic (ROC) curve, the c-index. The c-index is a measure of discrimination and is interpreted as being the probability that a randomly chosen patient with CAD will have a higher predicted probability of disease than a randomly chosen patient without CAD [18]. An area under the ROC curve (AUC) of 0.5 corresponds to a model that provides no diagnostic information, whereas an AUC of 1.0 corresponds to a perfect diagnostic model. STATA statistical analysis software v10.0 (StataCorp, Texas, USA) was used for logistic regression analysis. Next, we quantified the effect of adding CTCS to the model on the classification of patients into probability categories of CAD. Four probability categories were defined: <30%, ≥30–50%, ≥50–70% and ≥70%. Reclassification tables were constructed for the Diamond & Forrester model and the Pryor model (see Tables 4 and 5) [19]. We computed the reclassification calibration statistic (RCS) [20] which is equivalent to the Hosmer–Lemeshow statistic, applied to the cross-classified cells of the reclassification table with at least 20 observations. A significant result indicates a lack of fit. Furthermore, the following reclassification measures were calculated for each model: the overall (correct) percentage of reclassification, the net reclassification improvement (NRI) [21] and the integrated discrimination improvement (IDI) [20]. The NRI is the difference in proportions reclassifying to higher and lower probability categories among cases and non-cases. It is interpreted as the percentage reclassified, adjusted for the reclassification direction. A significant NRI indicates that classification improves when CTCS is included. The IDI compares the difference in the average regression slope of cases and non-cases among the models with and without CTCS. A significant IDI indicates that the new model performs better in discriminating cases and non-cases. Reclassification computations were executed by using syntax made available by Cook and Ridker [20] in SAS Enterprise Guide v3 (SAS Inc, North Carolina, USA). Results Study population During a 24-month period, 402 patients were enrolled. Patients with acute coronary syndrome or a history of myocardial infarction were excluded and 254 patients were left for data analysis (Table 1). Of these, 123 (48%) patients had obstructive CAD on CCA. CTCS ranged from 0 to 3,839 with a median of 4 for patients without obstructive CAD and a median of 337 for patients with obstructive CAD. Of 131 patients without obstructive CAD, 44 (33.6%) patients had no coronary calcification; whereas of 123 patients diagnosed with obstructive CAD, 3 (2.4%) patients did not have any coronary calcification. Table 1 Patient characteristics Total (n = 254) no. Patients without CAD (n = 131) no. Patients with CAD (n = 123) no. p value Mean age (SD) 59 (11) 56 (12) 62 (10) <0.001 Male sex (%) 171 (67%) 78 (60%) 93 (76%) 0.01 Typical chest pain (%) 118 (46%) 33 (25%) 85 (69%) < 0.001 Mean BMIa (SD) 27 (4) 27 (4) 28 (5) 0.11 Smokingb (%) 63 (25%) 30 (23%) 33 (27%) 0.47 Hypertension (%) 140 (55%) 59 (45%) 81 (66%) <0.001 Dyslipidaemiac (%) 136 (54%) 47 (36%) 89 (72%) <0.001 Diabetesd (%) 32 (13%) 10 (8%) 22 (18%) 0.01 Family history (%) 126 (50%) 57 (44%) 69 (56%) 0.05 Mean calcium scoree (SD) 346 (572) 132 (320) 574 (685) <0.001 Median calcium score 138 4 337 – CADf on CCA (%) 123 (48%) 0 (0%) 123 (100%) CAD obstructive coronary artery disease, CCA conventional coronary angiography, SD standard deviation aBody Mass Index, defined as weight/height2 (in kg/m2) bPast or current cSerum cholesterol >200 mg/dL or 5.18 mmol/L dPlasma glucose ≥126 mg/dL or 7.0 mmol eMeasured according to Agatston [15] fDefined as ≥50% stenosis in at least one vessel Systematic literature search We obtained 649 articles in our literature search of which 632 articles were excluded based on title or abstract. After excluding 11 articles based on the full text, 6 studies were left for analysis. Multivariate logistic regression analysis Five published prediction rules were validated (Tables 2 and 3): Diamond & Forrester [22], Pryor et al. [23], Morise et al. 1994 [24], Morise et al. 1997 [17] and Shaw et al. [25]. The Diamond & Forrester prediction rule includes age, sex and type of chest pain, all of which were significant predictors of obstructive CAD in our dataset and with an area under the ROC curve (AUC) of 0.798. Including CTCS increased the AUC to 0.890, which was a statistically significant improvement (p < 0.001). In the expanded model age and sex were no longer significant predictors (Table 2). Table 2 Comparison of multivariate logistic regression models Variables Model 1: Diamond & Forrester 1979 Model 2: Pryor et al. 1993 Model 3: Morise et al. 1994 No calcium score Calcium score No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.97 1.60, 2.41 1.93 1.55, 2.41 1.87 1.52, 2.29 Age 1.05 1.02, 1.08 0.98 0.95, 1.02 1.02 0.95, 1.09 0.93 0.86, 1.01 1.05 1.01, 1.08 0.98 0.95, 1.02 Male sex 2.97 1.57, 5.61 1.64 0.79, 3.42 0.19 0.00, 12.20 0.02 0.00, 1.91 3.48 1.77, 6.83 2.05 0.94, 4.45 Typical chest pain 6.61 3.67, 11.89 5.34 2.70, 10.56 5.44 2.90, 10.18 5.22 2.52, 10.81 5.44 2.94, 10.05 4.91 2.43, 9.91 Smoking 0.08 0.00, 9.40 0.06 0.00, 11.40 Dyslipidaemia 10.90 0.19, 610.99 2.43 0.03, 195.86 3.03 1.63, 5.61 1.94 0.96, 3.94 Diabetes 2.81 1.07, 7.38 2.14 0.71, 6.42 2.61 1.02, 6.73 2.22 0.74, 6.65 Age–smoking 1.04 0.97, 1.13 1.03 0.95, 1.12 Age–dyslipidaemia 0.98 0.92, 1.05 1.00 0.93, 1.07 Sex–smoking 1.77 0.36, 8.84 3.82 0.66, 22.09 Age–sex 1.05 0.98, 1.12 1.07 1.00, 1.15 Oestrogen Hypertension Family history Dyslipidaemia–family history Obesity BMI AUCb 0.798 0.742, 0.854 0.890 0.851, 0.930 0.838 0.789, 0.887 0.901 0.863, 0.938 0.831 0.780, 0.881 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS Table 3 Comparison of multivariate logistic regression models Variables Model 4: Morise et al. 1997 Model 5: Shaw et al. 1998 No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.87 1.50, 2.31 1.86 1.51, 2.30 Intercept Age 1.05 1.01, 1.09 0.99 0.95, 1.03 1.05 1.02, 1.08 0.98 0.95, 1.02 Male sex 3.15 1.29, 7.70 1.37 0.48, 3.95 3.42 1.74, 6.74 2.05 0.94, 4.45 Typical pain 5.56 2.94, 10.51 4.82 2.36, 9.86 5.50 2.96, 10.21 4.91 2.44, 9.90 Smoking 1.53 0.72, 3.24 1.03 0.44, 2.39 1.63 0.80, 3.30 1.04 0.47, 2.27 Dyslipidaemia 3.20 1.29, 7.95 1.80 0.62, 5.22 3.04 1.63, 5.66 1.95 0.96, 3.94 Diabetes 2.66 0.98, 7.26 2.01 0.61, 6.61 2.85 1.10, 7.39 2.24 0.74, 6.75 Age–smoking Age–dyslipidaemia Sex–smoking Age–sex Oestrogen 0.78 0.33, 1.86 0.53 0.19, 1.48 Hypertension 1.83 0.93, 3.60 1.31 0.62, 2.79 Family History 2.02 0.77, 5.29 1.14 0.38, 3.39 Dyslipidaemia–family history 0.74 0.21, 2.60 1.15 0.28, 4.82 Obesity 0.88 0.31, 2.48 0.65 0.20, 2.12 BMI 0.99 0.88, 1.13 1.05 0.91, 1.22 AUCb 0.840 0.792, 0.889 0.898 0.859, 0.936 0.833 0.783, 0.883 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS Pryor et al. analysed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes and the interaction between age and smoking, age and dyslipidaemia, sex and smoking, and age and sex, of which type of chest pain and the presence of diabetes were significant predictors. This model resulted in an AUC of 0.838. After including CTCS, the AUC increased to 0.901 which was a statistically significant improvement (p < 0.001). In the expanded model diabetes was no longer a significant predictor (Table 2). Morise et al. (1994) included diabetes and dyslipidaemia in addition to the variables used by Diamond & Forrester and resulted in an AUC of 0.831. All variables were significant predictors of the presence of obstructive CAD. After including CTCS, type of chest pain was the only variable that remained significant. After inclusion of CTCS the AUC increased to 0.899, which was a statistically significant improvement (p < 0.001) (Table 2). Morise et al. (1997) assessed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes, oestrogen status, hypertension, family history, obesity, BMI and the interaction between dyslipidaemia and family history. This model resulted in an AUC of 0.840. Age, sex, type of chest pain and dyslipidaemia were significant predictors. After including CTCS, the AUC increased to 0.898, which was a significant model improvement (p < 0.001). Age, sex and dyslipidaemia were no longer significant predictors after the addition of CTCS (Table 3). Shaw et al. considered age, sex, typical chest pain, smoking, dyslipidaemia and diabetes and resulted in an AUC of 0.833. After including CTCS, only type of chest pain remained a significant predictor and the AUC increased to 0.899 which was a statistically significant improvement (p < 0.001) (Table 3). Reclassification tables for the Diamond & Forrester model and the Pryor model are presented in Tables 4 and 5. The addition of CTCS to Diamond & Forrester resulted in reclassification of 47.2% of patients of whom 73.3% were correctly reclassified. The reclassification calibration statistic (RCS) indicated a strong lack of fit for the Diamond & Forrester model (p < 0.00001) which decreased substantially when CTCS was added to the model (p < 0.01). The NRI (net reclassification improvement) was 33.6% (p < 0.0001) and the IDI (integrated discrimination improvement) was also statistically significant (18.8%, p < 0.001) indicating improvement in the classification of cases and non-cases in probability categories and improvement in discrimination between cases and non-cases. Table 4 Probability of coronary artery disease: reclassification table after addition of CTCS to the Diamond and Forrester (model 1) Probability category based on model 1 Probability category based on model 1 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 51 (67.1) 19 (25.0%) 4 (5.2) 2 (2.6) 76 (29.9) Observed probability, % 11.8 36.8 50.0 100.0 22.4 ≥30–50% N (%) 25 (39.7) 13 (20.6) 19 (30.2) 6 (9.5) 63 (28.4) Observed probability, % 0.0 23.1 68.4 50.0 30.2 ≥50–70% N (%) 4 (11.1) 5 (13.9) 6 (16.7) 21 (58.3) 36 (14.2) Observed probability, % 25 80.0 50.0 66.7 61.1 ≥70% N (%) 5 (6.3) 2 (2.5) 8 (10.1) 64 (81.0) 79 (31.1) Observed probability, % 0.0 50.0 37.5 95.3 82.3 Total N (%) 85 (33.5) 39 (15.4) 37 (14.6) 93 (36.6) 254 (100.0) Observed probability, % 8.2 38.5 56.8 86.0 48.4 CTCS computed tomography coronary calcium score Table 5 Probability of coronary artery disease: reclassification table after addition of CTCS to the model published by Pryor et al. (model 2) Probability category based on model 2 Probability category based on model 2 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 76 (85.4) 10 (11.2) 2 (2.3) 1 (1.1) 89 (35.0) Observed probability, % 9.2 40.0 100.0 100.0 15.7 ≥30–50% N (%) 10 (27.0) 10 (27.0) 12 (32.4) 5 (13.5) 37 (14.6) Observed probability, % 10.0 20.0 50.0 100.0 37.8 ≥50–70% N (%) 7 (13.5) 6 (11.5) 15 (28.9) 24 (46.2) 52 (20.5) Observed probability, % 0.0 50.0 73.3 66.7 57.7 ≥70% N (%) 3 (4.0) 3 (4.0) 9 (11.8) 61 (80.3) 76 (29.9) Observed probability, % 33.3 0.0 66.7 95.0 85.5 Total N (%) 96 (37.8) 29 (11.4) 38 (15.0) 91 (35.8) 254 (100.0) Observed probability, % 9.4 31.0 65.8 87.9 48.4 CTCS computed tomography coronary calcium score For the model by Pryor et al., 36.2% of the patients were reclassified, of whom 54.3% were correctly classified. The RCS indicated a lack of fit (p = 0.01), which decreased when CTCS was added to the model (p = 0.03). The NRI was 24.0% (p < 0.0001) and the IDI was 14.8% (p < 0.001). The reclassification measures for all models are presented in Table 6. Table 6 Reclassification measures obtained by adding CTCS to the existing prediction models Model Reclassification percentages % Correcta Chi-squared excluding CTCSb p value Chi-squared including CTCSc p value NRI, % p value for NRI Reclassification improvement (cases)d Reclassification improvement (non-cases)d IDI, % p value for IDI Overall Reclassified from ≥30–50% Reclassified from ≥50–70% 1 47.2 79.4 83.3 73.3 25.01 <0.00001 10.91 <0.01 33.6 <0.0001 <0.0001 0.23 18.8 <0.001 2 36.2 73.0 71.2 54.3 6.57 0.01 4.70 0.03 24.0 0.001 <0.001 0.31 14.8 <0.001 3 38.2 77.5 75.5 63.9 4.14 0.04 3.31 0.07 21.8 0.005 <0.001 0.58 14.9 <0.001 4 34.3 74.4 74.0 83.9 3.82 0.05 1.58 0.21 24.9 <0.001 <0.001 0.29 13.3 <0.001 5 37.8 77.5 75.9 69.8 3.65 0.06 5.14 0.02 22.6 0.003 <0.001 0.58 14.4 <0.001 CTCS computed tomography coronary calcium score, NRI net reclassification improvement, IDI integrated discrimination improvement aIf the predicted probability of obstructive CAD of the model including CTCS was closer to the observed probability of CAD compared with the prediction of the model without CTCS, the reclassification was considered to be correct bReclassification calibration (Hosmer–Lemeshow) statistic for model without CTCS, using cells from the reclassification table with at least 20 observations cReclassification calibration statistic for models including CTCS, using cells from the reclassification table with at least 20 observations dThe reclassification improvement is defined as the difference in proportions of patients moving up and down for cases and non-cases separately Discussion We analysed the incremental value of CTCS in the prediction of prevalent obstructive CAD. We showed CTCS to be a significant predictor, independent of other variables included in the model. Furthermore, we confirm that the prediction of prevalent obstructive CAD is mainly determined by age, sex and type of chest pain. In four of the five models age and sex were significant predictors before the addition of CTCS. However, after including CTCS, both age and sex were no longer significant predictors in each of those models. Type of chest pain was a significant predictor of obstructive CAD, independent of other variables included in the model and independent of whether CTCS was included or not. CTCS proved to be an excellent and significant predictor of CAD with adjusted odds ratios close to 2. Apart from age and sex, most risk factors did not result in significant odds ratios which suggests that these risk factors are of minor importance in the prediction of prevalent obstructive CAD. Furthermore, the risk factors that were significant in the models without CTCS lost their significance after including CTCS. Analysis of reclassification showed that adding CTCS yields reclassification of 34–47% of patients, most of which was correct. The reclassification calibration statistic demonstrated a lack of fit for all models, which decreased when CTCS was added in all models except for Shaw’s. Numerous studies have previously reported on the incremental value of CTCS in the prediction of cardiovascular events and mortality in asymptomatic individuals [10–13]. Some studies have reported on the incremental value of CTCS in the prediction of prevalent obstructive CAD. Both Guerci et al. [1] and Kennedy et al. [2] studied the relationship between obstructive CAD and CTCS, adjusting for various risk factors. Both studies found highly significant odds ratios for the CTCS in predicting the presence of obstructive CAD, but the odds ratios were lower compared to what we found. Budoff et al. [4] evaluated the value of CTCS in diagnosing CAD and showed that the addition of CTCS increased the AUC from 0.672 to 0.842. However, the models included only age and sex as clinical variables. Considering the fact that the type of chest pain was not taken into account, an increase in AUC with CTCS can be expected. However, the resulting area under the ROC curve for the model including CTCS was similar to what we found. Schmermund et al. [3] previously studied the value of CTCS in predicting the extent of CAD. They showed an independent and incremental value for CTCS in multiple linear regression in predicting the total number of segments per patient with ≥50% stenosis. Study limitations Our study assessed the prediction of ≥50% stenosis in at least one vessel. One could argue that physicians are primarily interested in diagnosing severe CAD, as these patients would be eligible for revascularisation whereas others can be adequately treated medically. Likewise, physicians might be primarily interested in predicting future cardiovascular adverse events. However, we did not consider prognosis in this analysis. All patients in our study were referred for CCA based on their presentation or functional testing that suggested the presence of cardiac ischaemia. In this way a high-risk population was selected, which could have biased our results. Unfortunately this limitation is inherent to the study design. Further research is necessary to determine the value of CTCS in other (e.g. lower risk) populations. Also, risk factors such as type of chest pain, smoking status and family history of CAD were obtained by interviewing the patient. Potentially, this method underestimates their predictive effects as compared to the predictive effect of the CTCS, which was directly measured. Tables 4 and 5 illustrate how CTCS influences the classification of patients in probability categories. However, the limitations of reclassification measures in the context of this research should be taken into account. Our sample size was too small to reliably assess reclassification. For example, the RCS only uses the cross-classified cells containing at least 20 observations. In Table 4 only four cells contain 20 or more observations, implying that a substantial amount of (correctly) reclassified patients are ignored. Thus, in our study the reclassification percentages, NRI and IDI indices are more reliable than the RCS. Ideally, the probability categories should be based on clinically relevant cutoffs. However, no well-established clinically relevant probability threshold exists. The probability of CAD is commonly defined as low (<30%), intermediate (≥30–70%) and high (≥70%) [26]. In our view the intermediate category is rather wide, which is why we divided this category into low–intermediate (≥30–50%) and intermediate–high (≥50–70%). It should be noted that the overall percentage of reclassification is highly dependent on the choice and number of probability categories. Clinical implications Our results demonstrate that the estimation of the probability of obstructive CAD can be improved by including CTCS. This implies that clinicians can make better decisions as to whether a particular patient would benefit from further testing, for example CTCA or CCA. In low-risk patients, a CTCS of 0 could exclude CAD and avoid further testing using CTCA. Hereby, one also avoids the intravenous administration of contrast agent, the extra radiation exposure, and extra scan time and costs associated with CTCA. In patients with a low CTCS, CCA can be avoided and further non-invasive testing would be preferred. In patients with an intermediate CTCS, a CTCA might be the optimal next step. In patients with a high CTCS, direct CCA might be justified because of the high probability of CAD. All in all, CTCS could be useful as a triage test for patients who are suspected of having CAD. We confirmed that the prediction of significant CAD is primarily driven by the patient’s symptoms. A detailed history of the patient’s symptoms remains most important in the diagnostic work-up of patients with suspected CAD. However, history taking is difficult and subjective, therefore limiting our ability to accurately predict the presence of CAD. Hence, further diagnostic testing will be important, even in patients with a low to intermediate probability of CAD. On the other hand, the harms and costs of obtaining CTCS should be considered. Kim [27] studied the radiation dose and cancer risk of CTCS screening (every 5 years) in asymptomatic individuals. They concluded that the excess lifetime cancer risk was 42 (62) per 100,000 men (women). It is important to note that our study assessed the value of a single CTCS in symptomatic patients, for whom the excess lifetime cancer risk will be lower and small compared with the risk of missing a CAD diagnosis. Moreover, CTCS could reduce the use of additional testing in patients with a low CTCS and a low probability of CAD, thereby reducing the total radiation exposure. Although performing a CTCS measurement is a fast, low-dose and relatively inexpensive procedure, the harms and benefits should be considered in a cost-effectiveness analysis. Conclusion Our results suggest that CTCS significantly improves the prediction of prevalent obstructive CAD, independent of other clinical variables. Therefore, CTCS should be considered in the diagnostic work-up of CAD. Appendix Studies were included if they met all of the following criteria: (1) Study population were patients with chest pain suggestive of stable angina pectoris, (2) study reported a new multivariate prediction rule that included clinical variables, (3) outcome was the presence of significant CAD defined as ≥50% stenosis in at least one vessel. Studies were excluded if they met one of the following criteria: (1) Article was a review, guideline, or cost-effectiveness analysis, or (2) study did not report a prediction model, or (3) the outcome of interest was prognostic (e.g. event rate after treatment) or the authors did not use ≥50% stenosis in at least one vessel as their outcome, or (4) study population consisted of unstable patients, asymptomatic patients, patients with known CAD or patients with a specific co-morbidity, or (5) main topic was a diagnostic (imaging) test, or (6) a reference standard other than conventional coronary angiography was used, or (7) authors did not use the traditional classification of typical and atypical chest pain, or (8) the study focused on the association between one particular risk factor and CAD. Articles not accessible were excluded from the analysis. Search strategy Coronary artery disease [MeSH] Coronary heart disease [MeSH] Coronary stenosis [MeSH] Coronary disease #1 OR #2 OR 3# OR 4 Chest pain [MeSH] Angina pectoris [MeSH] Suspected [title/abstract] #6 OR #7 OR #8 Logistic models [MeSH] Probability [MeSH] Risk [MeSH] Models, statistical [MeSH] ROC curve [MeSH] #10 OR #11 OR #12 OR #13 OR #14 Medical history taking [MeSH] Physical examination [MeSH] Clinical [title/abstract] #16 OR #17 OR #18 #5 AND #9 AND #15 AND #19 Animals [MeSH] NOT humans [MeSH] Editorial [publication type] Comment [publication type] Letter [publication type] Meta-analysis [publication type] Case reports [publication type]) #22 OR #23 OR #24 OR #25 OR #26 Acute [title/abstract] #20 NOT #21 NOT #27 NOT #28 Acknowledgements This study was funded by the Health Care Efficiency Research grant (number 945–04–263) from the Netherlands Organisation for Health Research and Development, a Dutch governmental organisation, and by internal funding through a Health Care Efficiency grant from the Erasmus University Medical Center, Rotterdam. The authors’ work was independent of the funding organisations. The funding organisations had no involvement in the design or conduct of this study; data management and analysis; or manuscript preparation and review or authorisation for submission. Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Document structure show

article-title	Incremental value of the CT coronary calcium score for the prediction of coronary artery disease
abstract	Objectives: To validate published prediction models for the presence of obstructive coronary artery disease (CAD) in patients with new onset stable typical or atypical angina pectoris and to assess the incremental value of the CT coronary calcium score (CTCS). Methods: We searched the literature for clinical prediction rules for the diagnosis of obstructive CAD, defined as ≥50% stenosis in at least one vessel on conventional coronary angiography. Significant variables were re-analysed in our dataset of 254 patients with logistic regression. CTCS was subsequently included in the models. The area under the receiver operating characteristic curve (AUC) was calculated to assess diagnostic performance. Results: Re-analysing the variables used by Diamond & Forrester yielded an AUC of 0.798, which increased to 0.890 by adding CTCS. For Pryor, Morise 1994, Morise 1997 and Shaw the AUC increased from 0.838 to 0.901, 0.831 to 0.899, 0.840 to 0.898 and 0.833 to 0.899. CTCS significantly improved model performance in each model. Conclusions: Validation demonstrated good diagnostic performance across all models. CTCS improves the prediction of the presence of obstructive CAD, independent of clinical predictors, and should be considered in its diagnostic work-up.
sec	Objectives: To validate published prediction models for the presence of obstructive coronary artery disease (CAD) in patients with new onset stable typical or atypical angina pectoris and to assess the incremental value of the CT coronary calcium score (CTCS).
title	Objectives:
p	To validate published prediction models for the presence of obstructive coronary artery disease (CAD) in patients with new onset stable typical or atypical angina pectoris and to assess the incremental value of the CT coronary calcium score (CTCS).
sec	Methods: We searched the literature for clinical prediction rules for the diagnosis of obstructive CAD, defined as ≥50% stenosis in at least one vessel on conventional coronary angiography. Significant variables were re-analysed in our dataset of 254 patients with logistic regression. CTCS was subsequently included in the models. The area under the receiver operating characteristic curve (AUC) was calculated to assess diagnostic performance.
title	Methods:
p	We searched the literature for clinical prediction rules for the diagnosis of obstructive CAD, defined as ≥50% stenosis in at least one vessel on conventional coronary angiography. Significant variables were re-analysed in our dataset of 254 patients with logistic regression. CTCS was subsequently included in the models. The area under the receiver operating characteristic curve (AUC) was calculated to assess diagnostic performance.
sec	Results: Re-analysing the variables used by Diamond & Forrester yielded an AUC of 0.798, which increased to 0.890 by adding CTCS. For Pryor, Morise 1994, Morise 1997 and Shaw the AUC increased from 0.838 to 0.901, 0.831 to 0.899, 0.840 to 0.898 and 0.833 to 0.899. CTCS significantly improved model performance in each model.
title	Results:
p	Re-analysing the variables used by Diamond & Forrester yielded an AUC of 0.798, which increased to 0.890 by adding CTCS. For Pryor, Morise 1994, Morise 1997 and Shaw the AUC increased from 0.838 to 0.901, 0.831 to 0.899, 0.840 to 0.898 and 0.833 to 0.899. CTCS significantly improved model performance in each model.
sec	Conclusions: Validation demonstrated good diagnostic performance across all models. CTCS improves the prediction of the presence of obstructive CAD, independent of clinical predictors, and should be considered in its diagnostic work-up.
title	Conclusions:
p	Validation demonstrated good diagnostic performance across all models. CTCS improves the prediction of the presence of obstructive CAD, independent of clinical predictors, and should be considered in its diagnostic work-up.
body	Introduction The CT coronary calcium score (CTCS) is used in both the diagnosis of coronary artery disease (CAD) [1–6] and the prediction of cardiovascular events [7–13]. Although substantial evidence is available on the incremental value of CTCS in predicting future cardiovascular events and mortality in asymptomatic individuals [10–13], the diagnostic value in symptomatic patients is less clear. A meta-analysis revealed that the presence of coronary calcium had a sensitivity and specificity of 98% and 40%, respectively, in detecting significant stenoses [8]. Furthermore, numerous studies have reported on the value of CTCS in the prediction of the probability of obstructive CAD [1–4]. However, the incremental value (i.e. in addition to all known clinical predictors of CAD) of the CTCS as a continuous predictor of prevalent obstructive CAD is less well studied. The purpose of this study was to validate previously published clinical prediction models and to determine the incremental value of CTCS for the prediction of prevalent obstructive CAD in patients with new onset stable typical or atypical angina pectoris. Materials and methods Study population The study population was derived from an existing database, which consisted of 402 patients with chest pain suggestive of stable angina pectoris and suspected of having CAD. All patients were prospectively included in a large study evaluating 64-slice CT coronary angiography (CTCA) at our institution. All patients were referred for conventional coronary angiography (CCA) based on their presentation or functional testing that suggested the presence of ischaemia and all patients underwent multidetector CT angiography within a week before CCA. Inclusion criteria for this study were: informed consent, sinus heart rhythm and the ability to hold their breath for 15 s. Patients with a history of percutaneous coronary intervention or coronary artery bypass surgery, impaired renal function (serum creatinine >120 μmol/L) or a known intolerance to iodinated contrast medium were excluded. The Institutional Review Board approved the study and all patients signed informed consent. As this paper focuses on patients with new onset stable chest pain, we also excluded patients with acute coronary syndromes and patients with a previous myocardial infarction (Fig. 1). Fig. 1 Flow chart of patients in the study. CTCA computed tomography coronary angiography. *Data from an existing database were used. All patients were referred for conventional coronary angiography based on their presentation or functional testing that suggested the presence of cardiac ischaemia. See Materials and methods CT coronary calcium images Metoprolol (100 mg, Selokeen, AstraZeneca, London, UK) was administered orally 1 h before CT in patients with heart rates >65 beats per minute. A 64-slice single source CT system (Sensation 64; Siemens, Forchheim, Germany) with a gantry rotation time of 330 ms, acquisition time of 165 ms and voxel size of 0.4 mm3 was used to acquire standard spiral low-dose and ECG-gated coronary calcium CT images. CT parameters were 32 × 2 slices per rotation, individual detector width of 0.6 mm, 3.8-mm/rotation table feed, 120-kV tube voltage, 150-mAs tube current, with activated prospective x-ray tube modulation. Overlapping slices were reconstructed at 65% of the R–R interval using the B35f convolution kernel. Reconstructed slice thickness was 3.0 mm with an increment of 1.5 mm. The radiation exposure, estimated using dedicated software (ImPACT, version 0.99x, St. George’s Hospital, Tooting, London, UK), was 1.4 mSv in men and 1.8 mSv in women. One observer (with more than 3 years’ experience), who was blinded to the CCA and clinical data, measured the coronary calcium. Conventional coronary angiography The CTCS and CCA were carried out within 1 week. Coronary segments were assessed on CCA following a 17-segment modified American Heart Association (AHA) classification model [14] by a single observer (with more than 10 years’ experience), who was blinded to the CT and clinical data. A mean luminal narrowing of ≥50% was considered to be a significant stenosis. Validated quantitative coronary angiography software (CAAS II®, Pie Medical, Maastricht, the Netherlands) was used. Clinical variables and outcome All patients were interviewed at enrollment in the prospective cohort study. Clinical parameters recorded were: age (years), sex (male/female), type of chest pain (atypical vs. typical), body mass index (BMI) (defined as weight/height2 in kg/m2), smoking status (past or current smoker, yes/no), hypertension (present/absent), dyslipidaemia (serum cholesterol >200 mg/dL or 5.18 mmol/L, present/absent), diabetes (plasma glucose ≥126 mg/dL or 7.0 mmol, present/absent) and family history of CAD (present/absent). The CTCS was measured by the Agatston method [15] using dedicated software (syngo Calcium Scoring VE31H, Siemens, Germany). The outcome of interest was the presence of obstructive CAD defined as ≥50% stenosis in at least one vessel (present/absent) on CCA. Sample size As a general rule, 10 patients with the condition of interest per analysed variable are required for regression analysis. In our dataset (n = 254), 123 patients were identified as having obstructive CAD on CCA. This allowed for the analysis of 12 variables. Our sample meets the required number of cases and non-cases that has been suggested for external validation of prediction models [16]. Systematic literature search We searched the English-language medical literature in PubMed up to October 14, 2009 for diagnostic prediction models. See the Appendix for a detailed description of the search strategy. From the included articles, clinical variables that were identified as significant predictors of CAD were extracted. Data analysis Age was analysed as a continuous variable. To account for the skewed distribution of the coronary calcium scoring, CTCS was transformed by taking the natural logarithm of CTCS + 1. All other variables were dichotomous. Oestrogen status was not available in our dataset. Therefore, we assumed women below the age of 50 to be oestrogen positive, women of 50 years and above to be oestrogen negative and all men to be oestrogen neutral. Obesity was considered in the model by Morise (1997) only. We defined obesity as a BMI >27 kg/m2, corresponding to their definition [17]. The extracted sets of clinical variables were analysed with multivariate logistic regression analysis, fitting new regression coefficients. No attempt was made to validate original regression coefficients, as such coefficients were often not reported. CTCS was subsequently included in each of the models. Models without CTCS were compared with corresponding models including CTCS using the likelihood ratio test. The level of significance was defined at a p value less than 0.05. Diagnostic performance was assessed by calculating the area under the receiver operating characteristic (ROC) curve, the c-index. The c-index is a measure of discrimination and is interpreted as being the probability that a randomly chosen patient with CAD will have a higher predicted probability of disease than a randomly chosen patient without CAD [18]. An area under the ROC curve (AUC) of 0.5 corresponds to a model that provides no diagnostic information, whereas an AUC of 1.0 corresponds to a perfect diagnostic model. STATA statistical analysis software v10.0 (StataCorp, Texas, USA) was used for logistic regression analysis. Next, we quantified the effect of adding CTCS to the model on the classification of patients into probability categories of CAD. Four probability categories were defined: <30%, ≥30–50%, ≥50–70% and ≥70%. Reclassification tables were constructed for the Diamond & Forrester model and the Pryor model (see Tables 4 and 5) [19]. We computed the reclassification calibration statistic (RCS) [20] which is equivalent to the Hosmer–Lemeshow statistic, applied to the cross-classified cells of the reclassification table with at least 20 observations. A significant result indicates a lack of fit. Furthermore, the following reclassification measures were calculated for each model: the overall (correct) percentage of reclassification, the net reclassification improvement (NRI) [21] and the integrated discrimination improvement (IDI) [20]. The NRI is the difference in proportions reclassifying to higher and lower probability categories among cases and non-cases. It is interpreted as the percentage reclassified, adjusted for the reclassification direction. A significant NRI indicates that classification improves when CTCS is included. The IDI compares the difference in the average regression slope of cases and non-cases among the models with and without CTCS. A significant IDI indicates that the new model performs better in discriminating cases and non-cases. Reclassification computations were executed by using syntax made available by Cook and Ridker [20] in SAS Enterprise Guide v3 (SAS Inc, North Carolina, USA). Results Study population During a 24-month period, 402 patients were enrolled. Patients with acute coronary syndrome or a history of myocardial infarction were excluded and 254 patients were left for data analysis (Table 1). Of these, 123 (48%) patients had obstructive CAD on CCA. CTCS ranged from 0 to 3,839 with a median of 4 for patients without obstructive CAD and a median of 337 for patients with obstructive CAD. Of 131 patients without obstructive CAD, 44 (33.6%) patients had no coronary calcification; whereas of 123 patients diagnosed with obstructive CAD, 3 (2.4%) patients did not have any coronary calcification. Table 1 Patient characteristics Total (n = 254) no. Patients without CAD (n = 131) no. Patients with CAD (n = 123) no. p value Mean age (SD) 59 (11) 56 (12) 62 (10) <0.001 Male sex (%) 171 (67%) 78 (60%) 93 (76%) 0.01 Typical chest pain (%) 118 (46%) 33 (25%) 85 (69%) < 0.001 Mean BMIa (SD) 27 (4) 27 (4) 28 (5) 0.11 Smokingb (%) 63 (25%) 30 (23%) 33 (27%) 0.47 Hypertension (%) 140 (55%) 59 (45%) 81 (66%) <0.001 Dyslipidaemiac (%) 136 (54%) 47 (36%) 89 (72%) <0.001 Diabetesd (%) 32 (13%) 10 (8%) 22 (18%) 0.01 Family history (%) 126 (50%) 57 (44%) 69 (56%) 0.05 Mean calcium scoree (SD) 346 (572) 132 (320) 574 (685) <0.001 Median calcium score 138 4 337 – CADf on CCA (%) 123 (48%) 0 (0%) 123 (100%) CAD obstructive coronary artery disease, CCA conventional coronary angiography, SD standard deviation aBody Mass Index, defined as weight/height2 (in kg/m2) bPast or current cSerum cholesterol >200 mg/dL or 5.18 mmol/L dPlasma glucose ≥126 mg/dL or 7.0 mmol eMeasured according to Agatston [15] fDefined as ≥50% stenosis in at least one vessel Systematic literature search We obtained 649 articles in our literature search of which 632 articles were excluded based on title or abstract. After excluding 11 articles based on the full text, 6 studies were left for analysis. Multivariate logistic regression analysis Five published prediction rules were validated (Tables 2 and 3): Diamond & Forrester [22], Pryor et al. [23], Morise et al. 1994 [24], Morise et al. 1997 [17] and Shaw et al. [25]. The Diamond & Forrester prediction rule includes age, sex and type of chest pain, all of which were significant predictors of obstructive CAD in our dataset and with an area under the ROC curve (AUC) of 0.798. Including CTCS increased the AUC to 0.890, which was a statistically significant improvement (p < 0.001). In the expanded model age and sex were no longer significant predictors (Table 2). Table 2 Comparison of multivariate logistic regression models Variables Model 1: Diamond & Forrester 1979 Model 2: Pryor et al. 1993 Model 3: Morise et al. 1994 No calcium score Calcium score No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.97 1.60, 2.41 1.93 1.55, 2.41 1.87 1.52, 2.29 Age 1.05 1.02, 1.08 0.98 0.95, 1.02 1.02 0.95, 1.09 0.93 0.86, 1.01 1.05 1.01, 1.08 0.98 0.95, 1.02 Male sex 2.97 1.57, 5.61 1.64 0.79, 3.42 0.19 0.00, 12.20 0.02 0.00, 1.91 3.48 1.77, 6.83 2.05 0.94, 4.45 Typical chest pain 6.61 3.67, 11.89 5.34 2.70, 10.56 5.44 2.90, 10.18 5.22 2.52, 10.81 5.44 2.94, 10.05 4.91 2.43, 9.91 Smoking 0.08 0.00, 9.40 0.06 0.00, 11.40 Dyslipidaemia 10.90 0.19, 610.99 2.43 0.03, 195.86 3.03 1.63, 5.61 1.94 0.96, 3.94 Diabetes 2.81 1.07, 7.38 2.14 0.71, 6.42 2.61 1.02, 6.73 2.22 0.74, 6.65 Age–smoking 1.04 0.97, 1.13 1.03 0.95, 1.12 Age–dyslipidaemia 0.98 0.92, 1.05 1.00 0.93, 1.07 Sex–smoking 1.77 0.36, 8.84 3.82 0.66, 22.09 Age–sex 1.05 0.98, 1.12 1.07 1.00, 1.15 Oestrogen Hypertension Family history Dyslipidaemia–family history Obesity BMI AUCb 0.798 0.742, 0.854 0.890 0.851, 0.930 0.838 0.789, 0.887 0.901 0.863, 0.938 0.831 0.780, 0.881 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS Table 3 Comparison of multivariate logistic regression models Variables Model 4: Morise et al. 1997 Model 5: Shaw et al. 1998 No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.87 1.50, 2.31 1.86 1.51, 2.30 Intercept Age 1.05 1.01, 1.09 0.99 0.95, 1.03 1.05 1.02, 1.08 0.98 0.95, 1.02 Male sex 3.15 1.29, 7.70 1.37 0.48, 3.95 3.42 1.74, 6.74 2.05 0.94, 4.45 Typical pain 5.56 2.94, 10.51 4.82 2.36, 9.86 5.50 2.96, 10.21 4.91 2.44, 9.90 Smoking 1.53 0.72, 3.24 1.03 0.44, 2.39 1.63 0.80, 3.30 1.04 0.47, 2.27 Dyslipidaemia 3.20 1.29, 7.95 1.80 0.62, 5.22 3.04 1.63, 5.66 1.95 0.96, 3.94 Diabetes 2.66 0.98, 7.26 2.01 0.61, 6.61 2.85 1.10, 7.39 2.24 0.74, 6.75 Age–smoking Age–dyslipidaemia Sex–smoking Age–sex Oestrogen 0.78 0.33, 1.86 0.53 0.19, 1.48 Hypertension 1.83 0.93, 3.60 1.31 0.62, 2.79 Family History 2.02 0.77, 5.29 1.14 0.38, 3.39 Dyslipidaemia–family history 0.74 0.21, 2.60 1.15 0.28, 4.82 Obesity 0.88 0.31, 2.48 0.65 0.20, 2.12 BMI 0.99 0.88, 1.13 1.05 0.91, 1.22 AUCb 0.840 0.792, 0.889 0.898 0.859, 0.936 0.833 0.783, 0.883 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS Pryor et al. analysed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes and the interaction between age and smoking, age and dyslipidaemia, sex and smoking, and age and sex, of which type of chest pain and the presence of diabetes were significant predictors. This model resulted in an AUC of 0.838. After including CTCS, the AUC increased to 0.901 which was a statistically significant improvement (p < 0.001). In the expanded model diabetes was no longer a significant predictor (Table 2). Morise et al. (1994) included diabetes and dyslipidaemia in addition to the variables used by Diamond & Forrester and resulted in an AUC of 0.831. All variables were significant predictors of the presence of obstructive CAD. After including CTCS, type of chest pain was the only variable that remained significant. After inclusion of CTCS the AUC increased to 0.899, which was a statistically significant improvement (p < 0.001) (Table 2). Morise et al. (1997) assessed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes, oestrogen status, hypertension, family history, obesity, BMI and the interaction between dyslipidaemia and family history. This model resulted in an AUC of 0.840. Age, sex, type of chest pain and dyslipidaemia were significant predictors. After including CTCS, the AUC increased to 0.898, which was a significant model improvement (p < 0.001). Age, sex and dyslipidaemia were no longer significant predictors after the addition of CTCS (Table 3). Shaw et al. considered age, sex, typical chest pain, smoking, dyslipidaemia and diabetes and resulted in an AUC of 0.833. After including CTCS, only type of chest pain remained a significant predictor and the AUC increased to 0.899 which was a statistically significant improvement (p < 0.001) (Table 3). Reclassification tables for the Diamond & Forrester model and the Pryor model are presented in Tables 4 and 5. The addition of CTCS to Diamond & Forrester resulted in reclassification of 47.2% of patients of whom 73.3% were correctly reclassified. The reclassification calibration statistic (RCS) indicated a strong lack of fit for the Diamond & Forrester model (p < 0.00001) which decreased substantially when CTCS was added to the model (p < 0.01). The NRI (net reclassification improvement) was 33.6% (p < 0.0001) and the IDI (integrated discrimination improvement) was also statistically significant (18.8%, p < 0.001) indicating improvement in the classification of cases and non-cases in probability categories and improvement in discrimination between cases and non-cases. Table 4 Probability of coronary artery disease: reclassification table after addition of CTCS to the Diamond and Forrester (model 1) Probability category based on model 1 Probability category based on model 1 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 51 (67.1) 19 (25.0%) 4 (5.2) 2 (2.6) 76 (29.9) Observed probability, % 11.8 36.8 50.0 100.0 22.4 ≥30–50% N (%) 25 (39.7) 13 (20.6) 19 (30.2) 6 (9.5) 63 (28.4) Observed probability, % 0.0 23.1 68.4 50.0 30.2 ≥50–70% N (%) 4 (11.1) 5 (13.9) 6 (16.7) 21 (58.3) 36 (14.2) Observed probability, % 25 80.0 50.0 66.7 61.1 ≥70% N (%) 5 (6.3) 2 (2.5) 8 (10.1) 64 (81.0) 79 (31.1) Observed probability, % 0.0 50.0 37.5 95.3 82.3 Total N (%) 85 (33.5) 39 (15.4) 37 (14.6) 93 (36.6) 254 (100.0) Observed probability, % 8.2 38.5 56.8 86.0 48.4 CTCS computed tomography coronary calcium score Table 5 Probability of coronary artery disease: reclassification table after addition of CTCS to the model published by Pryor et al. (model 2) Probability category based on model 2 Probability category based on model 2 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 76 (85.4) 10 (11.2) 2 (2.3) 1 (1.1) 89 (35.0) Observed probability, % 9.2 40.0 100.0 100.0 15.7 ≥30–50% N (%) 10 (27.0) 10 (27.0) 12 (32.4) 5 (13.5) 37 (14.6) Observed probability, % 10.0 20.0 50.0 100.0 37.8 ≥50–70% N (%) 7 (13.5) 6 (11.5) 15 (28.9) 24 (46.2) 52 (20.5) Observed probability, % 0.0 50.0 73.3 66.7 57.7 ≥70% N (%) 3 (4.0) 3 (4.0) 9 (11.8) 61 (80.3) 76 (29.9) Observed probability, % 33.3 0.0 66.7 95.0 85.5 Total N (%) 96 (37.8) 29 (11.4) 38 (15.0) 91 (35.8) 254 (100.0) Observed probability, % 9.4 31.0 65.8 87.9 48.4 CTCS computed tomography coronary calcium score For the model by Pryor et al., 36.2% of the patients were reclassified, of whom 54.3% were correctly classified. The RCS indicated a lack of fit (p = 0.01), which decreased when CTCS was added to the model (p = 0.03). The NRI was 24.0% (p < 0.0001) and the IDI was 14.8% (p < 0.001). The reclassification measures for all models are presented in Table 6. Table 6 Reclassification measures obtained by adding CTCS to the existing prediction models Model Reclassification percentages % Correcta Chi-squared excluding CTCSb p value Chi-squared including CTCSc p value NRI, % p value for NRI Reclassification improvement (cases)d Reclassification improvement (non-cases)d IDI, % p value for IDI Overall Reclassified from ≥30–50% Reclassified from ≥50–70% 1 47.2 79.4 83.3 73.3 25.01 <0.00001 10.91 <0.01 33.6 <0.0001 <0.0001 0.23 18.8 <0.001 2 36.2 73.0 71.2 54.3 6.57 0.01 4.70 0.03 24.0 0.001 <0.001 0.31 14.8 <0.001 3 38.2 77.5 75.5 63.9 4.14 0.04 3.31 0.07 21.8 0.005 <0.001 0.58 14.9 <0.001 4 34.3 74.4 74.0 83.9 3.82 0.05 1.58 0.21 24.9 <0.001 <0.001 0.29 13.3 <0.001 5 37.8 77.5 75.9 69.8 3.65 0.06 5.14 0.02 22.6 0.003 <0.001 0.58 14.4 <0.001 CTCS computed tomography coronary calcium score, NRI net reclassification improvement, IDI integrated discrimination improvement aIf the predicted probability of obstructive CAD of the model including CTCS was closer to the observed probability of CAD compared with the prediction of the model without CTCS, the reclassification was considered to be correct bReclassification calibration (Hosmer–Lemeshow) statistic for model without CTCS, using cells from the reclassification table with at least 20 observations cReclassification calibration statistic for models including CTCS, using cells from the reclassification table with at least 20 observations dThe reclassification improvement is defined as the difference in proportions of patients moving up and down for cases and non-cases separately Discussion We analysed the incremental value of CTCS in the prediction of prevalent obstructive CAD. We showed CTCS to be a significant predictor, independent of other variables included in the model. Furthermore, we confirm that the prediction of prevalent obstructive CAD is mainly determined by age, sex and type of chest pain. In four of the five models age and sex were significant predictors before the addition of CTCS. However, after including CTCS, both age and sex were no longer significant predictors in each of those models. Type of chest pain was a significant predictor of obstructive CAD, independent of other variables included in the model and independent of whether CTCS was included or not. CTCS proved to be an excellent and significant predictor of CAD with adjusted odds ratios close to 2. Apart from age and sex, most risk factors did not result in significant odds ratios which suggests that these risk factors are of minor importance in the prediction of prevalent obstructive CAD. Furthermore, the risk factors that were significant in the models without CTCS lost their significance after including CTCS. Analysis of reclassification showed that adding CTCS yields reclassification of 34–47% of patients, most of which was correct. The reclassification calibration statistic demonstrated a lack of fit for all models, which decreased when CTCS was added in all models except for Shaw’s. Numerous studies have previously reported on the incremental value of CTCS in the prediction of cardiovascular events and mortality in asymptomatic individuals [10–13]. Some studies have reported on the incremental value of CTCS in the prediction of prevalent obstructive CAD. Both Guerci et al. [1] and Kennedy et al. [2] studied the relationship between obstructive CAD and CTCS, adjusting for various risk factors. Both studies found highly significant odds ratios for the CTCS in predicting the presence of obstructive CAD, but the odds ratios were lower compared to what we found. Budoff et al. [4] evaluated the value of CTCS in diagnosing CAD and showed that the addition of CTCS increased the AUC from 0.672 to 0.842. However, the models included only age and sex as clinical variables. Considering the fact that the type of chest pain was not taken into account, an increase in AUC with CTCS can be expected. However, the resulting area under the ROC curve for the model including CTCS was similar to what we found. Schmermund et al. [3] previously studied the value of CTCS in predicting the extent of CAD. They showed an independent and incremental value for CTCS in multiple linear regression in predicting the total number of segments per patient with ≥50% stenosis. Study limitations Our study assessed the prediction of ≥50% stenosis in at least one vessel. One could argue that physicians are primarily interested in diagnosing severe CAD, as these patients would be eligible for revascularisation whereas others can be adequately treated medically. Likewise, physicians might be primarily interested in predicting future cardiovascular adverse events. However, we did not consider prognosis in this analysis. All patients in our study were referred for CCA based on their presentation or functional testing that suggested the presence of cardiac ischaemia. In this way a high-risk population was selected, which could have biased our results. Unfortunately this limitation is inherent to the study design. Further research is necessary to determine the value of CTCS in other (e.g. lower risk) populations. Also, risk factors such as type of chest pain, smoking status and family history of CAD were obtained by interviewing the patient. Potentially, this method underestimates their predictive effects as compared to the predictive effect of the CTCS, which was directly measured. Tables 4 and 5 illustrate how CTCS influences the classification of patients in probability categories. However, the limitations of reclassification measures in the context of this research should be taken into account. Our sample size was too small to reliably assess reclassification. For example, the RCS only uses the cross-classified cells containing at least 20 observations. In Table 4 only four cells contain 20 or more observations, implying that a substantial amount of (correctly) reclassified patients are ignored. Thus, in our study the reclassification percentages, NRI and IDI indices are more reliable than the RCS. Ideally, the probability categories should be based on clinically relevant cutoffs. However, no well-established clinically relevant probability threshold exists. The probability of CAD is commonly defined as low (<30%), intermediate (≥30–70%) and high (≥70%) [26]. In our view the intermediate category is rather wide, which is why we divided this category into low–intermediate (≥30–50%) and intermediate–high (≥50–70%). It should be noted that the overall percentage of reclassification is highly dependent on the choice and number of probability categories. Clinical implications Our results demonstrate that the estimation of the probability of obstructive CAD can be improved by including CTCS. This implies that clinicians can make better decisions as to whether a particular patient would benefit from further testing, for example CTCA or CCA. In low-risk patients, a CTCS of 0 could exclude CAD and avoid further testing using CTCA. Hereby, one also avoids the intravenous administration of contrast agent, the extra radiation exposure, and extra scan time and costs associated with CTCA. In patients with a low CTCS, CCA can be avoided and further non-invasive testing would be preferred. In patients with an intermediate CTCS, a CTCA might be the optimal next step. In patients with a high CTCS, direct CCA might be justified because of the high probability of CAD. All in all, CTCS could be useful as a triage test for patients who are suspected of having CAD. We confirmed that the prediction of significant CAD is primarily driven by the patient’s symptoms. A detailed history of the patient’s symptoms remains most important in the diagnostic work-up of patients with suspected CAD. However, history taking is difficult and subjective, therefore limiting our ability to accurately predict the presence of CAD. Hence, further diagnostic testing will be important, even in patients with a low to intermediate probability of CAD. On the other hand, the harms and costs of obtaining CTCS should be considered. Kim [27] studied the radiation dose and cancer risk of CTCS screening (every 5 years) in asymptomatic individuals. They concluded that the excess lifetime cancer risk was 42 (62) per 100,000 men (women). It is important to note that our study assessed the value of a single CTCS in symptomatic patients, for whom the excess lifetime cancer risk will be lower and small compared with the risk of missing a CAD diagnosis. Moreover, CTCS could reduce the use of additional testing in patients with a low CTCS and a low probability of CAD, thereby reducing the total radiation exposure. Although performing a CTCS measurement is a fast, low-dose and relatively inexpensive procedure, the harms and benefits should be considered in a cost-effectiveness analysis. Conclusion Our results suggest that CTCS significantly improves the prediction of prevalent obstructive CAD, independent of other clinical variables. Therefore, CTCS should be considered in the diagnostic work-up of CAD.
sec	Introduction The CT coronary calcium score (CTCS) is used in both the diagnosis of coronary artery disease (CAD) [1–6] and the prediction of cardiovascular events [7–13]. Although substantial evidence is available on the incremental value of CTCS in predicting future cardiovascular events and mortality in asymptomatic individuals [10–13], the diagnostic value in symptomatic patients is less clear. A meta-analysis revealed that the presence of coronary calcium had a sensitivity and specificity of 98% and 40%, respectively, in detecting significant stenoses [8]. Furthermore, numerous studies have reported on the value of CTCS in the prediction of the probability of obstructive CAD [1–4]. However, the incremental value (i.e. in addition to all known clinical predictors of CAD) of the CTCS as a continuous predictor of prevalent obstructive CAD is less well studied. The purpose of this study was to validate previously published clinical prediction models and to determine the incremental value of CTCS for the prediction of prevalent obstructive CAD in patients with new onset stable typical or atypical angina pectoris.
title	Introduction
p	The CT coronary calcium score (CTCS) is used in both the diagnosis of coronary artery disease (CAD) [1–6] and the prediction of cardiovascular events [7–13]. Although substantial evidence is available on the incremental value of CTCS in predicting future cardiovascular events and mortality in asymptomatic individuals [10–13], the diagnostic value in symptomatic patients is less clear.
p	A meta-analysis revealed that the presence of coronary calcium had a sensitivity and specificity of 98% and 40%, respectively, in detecting significant stenoses [8]. Furthermore, numerous studies have reported on the value of CTCS in the prediction of the probability of obstructive CAD [1–4]. However, the incremental value (i.e. in addition to all known clinical predictors of CAD) of the CTCS as a continuous predictor of prevalent obstructive CAD is less well studied.
p	The purpose of this study was to validate previously published clinical prediction models and to determine the incremental value of CTCS for the prediction of prevalent obstructive CAD in patients with new onset stable typical or atypical angina pectoris.
sec	Materials and methods Study population The study population was derived from an existing database, which consisted of 402 patients with chest pain suggestive of stable angina pectoris and suspected of having CAD. All patients were prospectively included in a large study evaluating 64-slice CT coronary angiography (CTCA) at our institution. All patients were referred for conventional coronary angiography (CCA) based on their presentation or functional testing that suggested the presence of ischaemia and all patients underwent multidetector CT angiography within a week before CCA. Inclusion criteria for this study were: informed consent, sinus heart rhythm and the ability to hold their breath for 15 s. Patients with a history of percutaneous coronary intervention or coronary artery bypass surgery, impaired renal function (serum creatinine >120 μmol/L) or a known intolerance to iodinated contrast medium were excluded. The Institutional Review Board approved the study and all patients signed informed consent. As this paper focuses on patients with new onset stable chest pain, we also excluded patients with acute coronary syndromes and patients with a previous myocardial infarction (Fig. 1). Fig. 1 Flow chart of patients in the study. CTCA computed tomography coronary angiography. *Data from an existing database were used. All patients were referred for conventional coronary angiography based on their presentation or functional testing that suggested the presence of cardiac ischaemia. See Materials and methods CT coronary calcium images Metoprolol (100 mg, Selokeen, AstraZeneca, London, UK) was administered orally 1 h before CT in patients with heart rates >65 beats per minute. A 64-slice single source CT system (Sensation 64; Siemens, Forchheim, Germany) with a gantry rotation time of 330 ms, acquisition time of 165 ms and voxel size of 0.4 mm3 was used to acquire standard spiral low-dose and ECG-gated coronary calcium CT images. CT parameters were 32 × 2 slices per rotation, individual detector width of 0.6 mm, 3.8-mm/rotation table feed, 120-kV tube voltage, 150-mAs tube current, with activated prospective x-ray tube modulation. Overlapping slices were reconstructed at 65% of the R–R interval using the B35f convolution kernel. Reconstructed slice thickness was 3.0 mm with an increment of 1.5 mm. The radiation exposure, estimated using dedicated software (ImPACT, version 0.99x, St. George’s Hospital, Tooting, London, UK), was 1.4 mSv in men and 1.8 mSv in women. One observer (with more than 3 years’ experience), who was blinded to the CCA and clinical data, measured the coronary calcium. Conventional coronary angiography The CTCS and CCA were carried out within 1 week. Coronary segments were assessed on CCA following a 17-segment modified American Heart Association (AHA) classification model [14] by a single observer (with more than 10 years’ experience), who was blinded to the CT and clinical data. A mean luminal narrowing of ≥50% was considered to be a significant stenosis. Validated quantitative coronary angiography software (CAAS II®, Pie Medical, Maastricht, the Netherlands) was used. Clinical variables and outcome All patients were interviewed at enrollment in the prospective cohort study. Clinical parameters recorded were: age (years), sex (male/female), type of chest pain (atypical vs. typical), body mass index (BMI) (defined as weight/height2 in kg/m2), smoking status (past or current smoker, yes/no), hypertension (present/absent), dyslipidaemia (serum cholesterol >200 mg/dL or 5.18 mmol/L, present/absent), diabetes (plasma glucose ≥126 mg/dL or 7.0 mmol, present/absent) and family history of CAD (present/absent). The CTCS was measured by the Agatston method [15] using dedicated software (syngo Calcium Scoring VE31H, Siemens, Germany). The outcome of interest was the presence of obstructive CAD defined as ≥50% stenosis in at least one vessel (present/absent) on CCA. Sample size As a general rule, 10 patients with the condition of interest per analysed variable are required for regression analysis. In our dataset (n = 254), 123 patients were identified as having obstructive CAD on CCA. This allowed for the analysis of 12 variables. Our sample meets the required number of cases and non-cases that has been suggested for external validation of prediction models [16]. Systematic literature search We searched the English-language medical literature in PubMed up to October 14, 2009 for diagnostic prediction models. See the Appendix for a detailed description of the search strategy. From the included articles, clinical variables that were identified as significant predictors of CAD were extracted. Data analysis Age was analysed as a continuous variable. To account for the skewed distribution of the coronary calcium scoring, CTCS was transformed by taking the natural logarithm of CTCS + 1. All other variables were dichotomous. Oestrogen status was not available in our dataset. Therefore, we assumed women below the age of 50 to be oestrogen positive, women of 50 years and above to be oestrogen negative and all men to be oestrogen neutral. Obesity was considered in the model by Morise (1997) only. We defined obesity as a BMI >27 kg/m2, corresponding to their definition [17]. The extracted sets of clinical variables were analysed with multivariate logistic regression analysis, fitting new regression coefficients. No attempt was made to validate original regression coefficients, as such coefficients were often not reported. CTCS was subsequently included in each of the models. Models without CTCS were compared with corresponding models including CTCS using the likelihood ratio test. The level of significance was defined at a p value less than 0.05. Diagnostic performance was assessed by calculating the area under the receiver operating characteristic (ROC) curve, the c-index. The c-index is a measure of discrimination and is interpreted as being the probability that a randomly chosen patient with CAD will have a higher predicted probability of disease than a randomly chosen patient without CAD [18]. An area under the ROC curve (AUC) of 0.5 corresponds to a model that provides no diagnostic information, whereas an AUC of 1.0 corresponds to a perfect diagnostic model. STATA statistical analysis software v10.0 (StataCorp, Texas, USA) was used for logistic regression analysis. Next, we quantified the effect of adding CTCS to the model on the classification of patients into probability categories of CAD. Four probability categories were defined: <30%, ≥30–50%, ≥50–70% and ≥70%. Reclassification tables were constructed for the Diamond & Forrester model and the Pryor model (see Tables 4 and 5) [19]. We computed the reclassification calibration statistic (RCS) [20] which is equivalent to the Hosmer–Lemeshow statistic, applied to the cross-classified cells of the reclassification table with at least 20 observations. A significant result indicates a lack of fit. Furthermore, the following reclassification measures were calculated for each model: the overall (correct) percentage of reclassification, the net reclassification improvement (NRI) [21] and the integrated discrimination improvement (IDI) [20]. The NRI is the difference in proportions reclassifying to higher and lower probability categories among cases and non-cases. It is interpreted as the percentage reclassified, adjusted for the reclassification direction. A significant NRI indicates that classification improves when CTCS is included. The IDI compares the difference in the average regression slope of cases and non-cases among the models with and without CTCS. A significant IDI indicates that the new model performs better in discriminating cases and non-cases. Reclassification computations were executed by using syntax made available by Cook and Ridker [20] in SAS Enterprise Guide v3 (SAS Inc, North Carolina, USA).
title	Materials and methods
sec	Study population The study population was derived from an existing database, which consisted of 402 patients with chest pain suggestive of stable angina pectoris and suspected of having CAD. All patients were prospectively included in a large study evaluating 64-slice CT coronary angiography (CTCA) at our institution. All patients were referred for conventional coronary angiography (CCA) based on their presentation or functional testing that suggested the presence of ischaemia and all patients underwent multidetector CT angiography within a week before CCA. Inclusion criteria for this study were: informed consent, sinus heart rhythm and the ability to hold their breath for 15 s. Patients with a history of percutaneous coronary intervention or coronary artery bypass surgery, impaired renal function (serum creatinine >120 μmol/L) or a known intolerance to iodinated contrast medium were excluded. The Institutional Review Board approved the study and all patients signed informed consent. As this paper focuses on patients with new onset stable chest pain, we also excluded patients with acute coronary syndromes and patients with a previous myocardial infarction (Fig. 1). Fig. 1 Flow chart of patients in the study. CTCA computed tomography coronary angiography. *Data from an existing database were used. All patients were referred for conventional coronary angiography based on their presentation or functional testing that suggested the presence of cardiac ischaemia. See Materials and methods
title	Study population
p	The study population was derived from an existing database, which consisted of 402 patients with chest pain suggestive of stable angina pectoris and suspected of having CAD. All patients were prospectively included in a large study evaluating 64-slice CT coronary angiography (CTCA) at our institution. All patients were referred for conventional coronary angiography (CCA) based on their presentation or functional testing that suggested the presence of ischaemia and all patients underwent multidetector CT angiography within a week before CCA. Inclusion criteria for this study were: informed consent, sinus heart rhythm and the ability to hold their breath for 15 s. Patients with a history of percutaneous coronary intervention or coronary artery bypass surgery, impaired renal function (serum creatinine >120 μmol/L) or a known intolerance to iodinated contrast medium were excluded. The Institutional Review Board approved the study and all patients signed informed consent. As this paper focuses on patients with new onset stable chest pain, we also excluded patients with acute coronary syndromes and patients with a previous myocardial infarction (Fig. 1). Fig. 1 Flow chart of patients in the study. CTCA computed tomography coronary angiography. *Data from an existing database were used. All patients were referred for conventional coronary angiography based on their presentation or functional testing that suggested the presence of cardiac ischaemia. See Materials and methods
figure	Fig. 1 Flow chart of patients in the study. CTCA computed tomography coronary angiography. *Data from an existing database were used. All patients were referred for conventional coronary angiography based on their presentation or functional testing that suggested the presence of cardiac ischaemia. See Materials and methods
label	Fig. 1
caption	Flow chart of patients in the study. CTCA computed tomography coronary angiography. *Data from an existing database were used. All patients were referred for conventional coronary angiography based on their presentation or functional testing that suggested the presence of cardiac ischaemia. See Materials and methods
p	Flow chart of patients in the study. CTCA computed tomography coronary angiography. *Data from an existing database were used. All patients were referred for conventional coronary angiography based on their presentation or functional testing that suggested the presence of cardiac ischaemia. See Materials and methods
sec	CT coronary calcium images Metoprolol (100 mg, Selokeen, AstraZeneca, London, UK) was administered orally 1 h before CT in patients with heart rates >65 beats per minute. A 64-slice single source CT system (Sensation 64; Siemens, Forchheim, Germany) with a gantry rotation time of 330 ms, acquisition time of 165 ms and voxel size of 0.4 mm3 was used to acquire standard spiral low-dose and ECG-gated coronary calcium CT images. CT parameters were 32 × 2 slices per rotation, individual detector width of 0.6 mm, 3.8-mm/rotation table feed, 120-kV tube voltage, 150-mAs tube current, with activated prospective x-ray tube modulation. Overlapping slices were reconstructed at 65% of the R–R interval using the B35f convolution kernel. Reconstructed slice thickness was 3.0 mm with an increment of 1.5 mm. The radiation exposure, estimated using dedicated software (ImPACT, version 0.99x, St. George’s Hospital, Tooting, London, UK), was 1.4 mSv in men and 1.8 mSv in women. One observer (with more than 3 years’ experience), who was blinded to the CCA and clinical data, measured the coronary calcium.
title	CT coronary calcium images
p	Metoprolol (100 mg, Selokeen, AstraZeneca, London, UK) was administered orally 1 h before CT in patients with heart rates >65 beats per minute. A 64-slice single source CT system (Sensation 64; Siemens, Forchheim, Germany) with a gantry rotation time of 330 ms, acquisition time of 165 ms and voxel size of 0.4 mm3 was used to acquire standard spiral low-dose and ECG-gated coronary calcium CT images. CT parameters were 32 × 2 slices per rotation, individual detector width of 0.6 mm, 3.8-mm/rotation table feed, 120-kV tube voltage, 150-mAs tube current, with activated prospective x-ray tube modulation. Overlapping slices were reconstructed at 65% of the R–R interval using the B35f convolution kernel. Reconstructed slice thickness was 3.0 mm with an increment of 1.5 mm. The radiation exposure, estimated using dedicated software (ImPACT, version 0.99x, St. George’s Hospital, Tooting, London, UK), was 1.4 mSv in men and 1.8 mSv in women. One observer (with more than 3 years’ experience), who was blinded to the CCA and clinical data, measured the coronary calcium.
sec	Conventional coronary angiography The CTCS and CCA were carried out within 1 week. Coronary segments were assessed on CCA following a 17-segment modified American Heart Association (AHA) classification model [14] by a single observer (with more than 10 years’ experience), who was blinded to the CT and clinical data. A mean luminal narrowing of ≥50% was considered to be a significant stenosis. Validated quantitative coronary angiography software (CAAS II®, Pie Medical, Maastricht, the Netherlands) was used.
title	Conventional coronary angiography
p	The CTCS and CCA were carried out within 1 week. Coronary segments were assessed on CCA following a 17-segment modified American Heart Association (AHA) classification model [14] by a single observer (with more than 10 years’ experience), who was blinded to the CT and clinical data. A mean luminal narrowing of ≥50% was considered to be a significant stenosis. Validated quantitative coronary angiography software (CAAS II®, Pie Medical, Maastricht, the Netherlands) was used.
sec	Clinical variables and outcome All patients were interviewed at enrollment in the prospective cohort study. Clinical parameters recorded were: age (years), sex (male/female), type of chest pain (atypical vs. typical), body mass index (BMI) (defined as weight/height2 in kg/m2), smoking status (past or current smoker, yes/no), hypertension (present/absent), dyslipidaemia (serum cholesterol >200 mg/dL or 5.18 mmol/L, present/absent), diabetes (plasma glucose ≥126 mg/dL or 7.0 mmol, present/absent) and family history of CAD (present/absent). The CTCS was measured by the Agatston method [15] using dedicated software (syngo Calcium Scoring VE31H, Siemens, Germany). The outcome of interest was the presence of obstructive CAD defined as ≥50% stenosis in at least one vessel (present/absent) on CCA.
title	Clinical variables and outcome
p	All patients were interviewed at enrollment in the prospective cohort study. Clinical parameters recorded were: age (years), sex (male/female), type of chest pain (atypical vs. typical), body mass index (BMI) (defined as weight/height2 in kg/m2), smoking status (past or current smoker, yes/no), hypertension (present/absent), dyslipidaemia (serum cholesterol >200 mg/dL or 5.18 mmol/L, present/absent), diabetes (plasma glucose ≥126 mg/dL or 7.0 mmol, present/absent) and family history of CAD (present/absent).
p	The CTCS was measured by the Agatston method [15] using dedicated software (syngo Calcium Scoring VE31H, Siemens, Germany). The outcome of interest was the presence of obstructive CAD defined as ≥50% stenosis in at least one vessel (present/absent) on CCA.
sec	Sample size As a general rule, 10 patients with the condition of interest per analysed variable are required for regression analysis. In our dataset (n = 254), 123 patients were identified as having obstructive CAD on CCA. This allowed for the analysis of 12 variables. Our sample meets the required number of cases and non-cases that has been suggested for external validation of prediction models [16].
title	Sample size
p	As a general rule, 10 patients with the condition of interest per analysed variable are required for regression analysis. In our dataset (n = 254), 123 patients were identified as having obstructive CAD on CCA. This allowed for the analysis of 12 variables. Our sample meets the required number of cases and non-cases that has been suggested for external validation of prediction models [16].
sec	Systematic literature search We searched the English-language medical literature in PubMed up to October 14, 2009 for diagnostic prediction models. See the Appendix for a detailed description of the search strategy. From the included articles, clinical variables that were identified as significant predictors of CAD were extracted.
title	Systematic literature search
p	We searched the English-language medical literature in PubMed up to October 14, 2009 for diagnostic prediction models. See the Appendix for a detailed description of the search strategy. From the included articles, clinical variables that were identified as significant predictors of CAD were extracted.
sec	Data analysis Age was analysed as a continuous variable. To account for the skewed distribution of the coronary calcium scoring, CTCS was transformed by taking the natural logarithm of CTCS + 1. All other variables were dichotomous. Oestrogen status was not available in our dataset. Therefore, we assumed women below the age of 50 to be oestrogen positive, women of 50 years and above to be oestrogen negative and all men to be oestrogen neutral. Obesity was considered in the model by Morise (1997) only. We defined obesity as a BMI >27 kg/m2, corresponding to their definition [17]. The extracted sets of clinical variables were analysed with multivariate logistic regression analysis, fitting new regression coefficients. No attempt was made to validate original regression coefficients, as such coefficients were often not reported. CTCS was subsequently included in each of the models. Models without CTCS were compared with corresponding models including CTCS using the likelihood ratio test. The level of significance was defined at a p value less than 0.05. Diagnostic performance was assessed by calculating the area under the receiver operating characteristic (ROC) curve, the c-index. The c-index is a measure of discrimination and is interpreted as being the probability that a randomly chosen patient with CAD will have a higher predicted probability of disease than a randomly chosen patient without CAD [18]. An area under the ROC curve (AUC) of 0.5 corresponds to a model that provides no diagnostic information, whereas an AUC of 1.0 corresponds to a perfect diagnostic model. STATA statistical analysis software v10.0 (StataCorp, Texas, USA) was used for logistic regression analysis. Next, we quantified the effect of adding CTCS to the model on the classification of patients into probability categories of CAD. Four probability categories were defined: <30%, ≥30–50%, ≥50–70% and ≥70%. Reclassification tables were constructed for the Diamond & Forrester model and the Pryor model (see Tables 4 and 5) [19]. We computed the reclassification calibration statistic (RCS) [20] which is equivalent to the Hosmer–Lemeshow statistic, applied to the cross-classified cells of the reclassification table with at least 20 observations. A significant result indicates a lack of fit. Furthermore, the following reclassification measures were calculated for each model: the overall (correct) percentage of reclassification, the net reclassification improvement (NRI) [21] and the integrated discrimination improvement (IDI) [20]. The NRI is the difference in proportions reclassifying to higher and lower probability categories among cases and non-cases. It is interpreted as the percentage reclassified, adjusted for the reclassification direction. A significant NRI indicates that classification improves when CTCS is included. The IDI compares the difference in the average regression slope of cases and non-cases among the models with and without CTCS. A significant IDI indicates that the new model performs better in discriminating cases and non-cases. Reclassification computations were executed by using syntax made available by Cook and Ridker [20] in SAS Enterprise Guide v3 (SAS Inc, North Carolina, USA).
title	Data analysis
p	Age was analysed as a continuous variable. To account for the skewed distribution of the coronary calcium scoring, CTCS was transformed by taking the natural logarithm of CTCS + 1. All other variables were dichotomous. Oestrogen status was not available in our dataset. Therefore, we assumed women below the age of 50 to be oestrogen positive, women of 50 years and above to be oestrogen negative and all men to be oestrogen neutral. Obesity was considered in the model by Morise (1997) only. We defined obesity as a BMI >27 kg/m2, corresponding to their definition [17].
p	The extracted sets of clinical variables were analysed with multivariate logistic regression analysis, fitting new regression coefficients. No attempt was made to validate original regression coefficients, as such coefficients were often not reported. CTCS was subsequently included in each of the models. Models without CTCS were compared with corresponding models including CTCS using the likelihood ratio test. The level of significance was defined at a p value less than 0.05.
p	Diagnostic performance was assessed by calculating the area under the receiver operating characteristic (ROC) curve, the c-index. The c-index is a measure of discrimination and is interpreted as being the probability that a randomly chosen patient with CAD will have a higher predicted probability of disease than a randomly chosen patient without CAD [18]. An area under the ROC curve (AUC) of 0.5 corresponds to a model that provides no diagnostic information, whereas an AUC of 1.0 corresponds to a perfect diagnostic model.
p	STATA statistical analysis software v10.0 (StataCorp, Texas, USA) was used for logistic regression analysis.
p	Next, we quantified the effect of adding CTCS to the model on the classification of patients into probability categories of CAD. Four probability categories were defined: <30%, ≥30–50%, ≥50–70% and ≥70%. Reclassification tables were constructed for the Diamond & Forrester model and the Pryor model (see Tables 4 and 5) [19]. We computed the reclassification calibration statistic (RCS) [20] which is equivalent to the Hosmer–Lemeshow statistic, applied to the cross-classified cells of the reclassification table with at least 20 observations. A significant result indicates a lack of fit.
p	Furthermore, the following reclassification measures were calculated for each model: the overall (correct) percentage of reclassification, the net reclassification improvement (NRI) [21] and the integrated discrimination improvement (IDI) [20]. The NRI is the difference in proportions reclassifying to higher and lower probability categories among cases and non-cases. It is interpreted as the percentage reclassified, adjusted for the reclassification direction. A significant NRI indicates that classification improves when CTCS is included. The IDI compares the difference in the average regression slope of cases and non-cases among the models with and without CTCS. A significant IDI indicates that the new model performs better in discriminating cases and non-cases.
p	Reclassification computations were executed by using syntax made available by Cook and Ridker [20] in SAS Enterprise Guide v3 (SAS Inc, North Carolina, USA).
sec	Results Study population During a 24-month period, 402 patients were enrolled. Patients with acute coronary syndrome or a history of myocardial infarction were excluded and 254 patients were left for data analysis (Table 1). Of these, 123 (48%) patients had obstructive CAD on CCA. CTCS ranged from 0 to 3,839 with a median of 4 for patients without obstructive CAD and a median of 337 for patients with obstructive CAD. Of 131 patients without obstructive CAD, 44 (33.6%) patients had no coronary calcification; whereas of 123 patients diagnosed with obstructive CAD, 3 (2.4%) patients did not have any coronary calcification. Table 1 Patient characteristics Total (n = 254) no. Patients without CAD (n = 131) no. Patients with CAD (n = 123) no. p value Mean age (SD) 59 (11) 56 (12) 62 (10) <0.001 Male sex (%) 171 (67%) 78 (60%) 93 (76%) 0.01 Typical chest pain (%) 118 (46%) 33 (25%) 85 (69%) < 0.001 Mean BMIa (SD) 27 (4) 27 (4) 28 (5) 0.11 Smokingb (%) 63 (25%) 30 (23%) 33 (27%) 0.47 Hypertension (%) 140 (55%) 59 (45%) 81 (66%) <0.001 Dyslipidaemiac (%) 136 (54%) 47 (36%) 89 (72%) <0.001 Diabetesd (%) 32 (13%) 10 (8%) 22 (18%) 0.01 Family history (%) 126 (50%) 57 (44%) 69 (56%) 0.05 Mean calcium scoree (SD) 346 (572) 132 (320) 574 (685) <0.001 Median calcium score 138 4 337 – CADf on CCA (%) 123 (48%) 0 (0%) 123 (100%) CAD obstructive coronary artery disease, CCA conventional coronary angiography, SD standard deviation aBody Mass Index, defined as weight/height2 (in kg/m2) bPast or current cSerum cholesterol >200 mg/dL or 5.18 mmol/L dPlasma glucose ≥126 mg/dL or 7.0 mmol eMeasured according to Agatston [15] fDefined as ≥50% stenosis in at least one vessel Systematic literature search We obtained 649 articles in our literature search of which 632 articles were excluded based on title or abstract. After excluding 11 articles based on the full text, 6 studies were left for analysis. Multivariate logistic regression analysis Five published prediction rules were validated (Tables 2 and 3): Diamond & Forrester [22], Pryor et al. [23], Morise et al. 1994 [24], Morise et al. 1997 [17] and Shaw et al. [25]. The Diamond & Forrester prediction rule includes age, sex and type of chest pain, all of which were significant predictors of obstructive CAD in our dataset and with an area under the ROC curve (AUC) of 0.798. Including CTCS increased the AUC to 0.890, which was a statistically significant improvement (p < 0.001). In the expanded model age and sex were no longer significant predictors (Table 2). Table 2 Comparison of multivariate logistic regression models Variables Model 1: Diamond & Forrester 1979 Model 2: Pryor et al. 1993 Model 3: Morise et al. 1994 No calcium score Calcium score No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.97 1.60, 2.41 1.93 1.55, 2.41 1.87 1.52, 2.29 Age 1.05 1.02, 1.08 0.98 0.95, 1.02 1.02 0.95, 1.09 0.93 0.86, 1.01 1.05 1.01, 1.08 0.98 0.95, 1.02 Male sex 2.97 1.57, 5.61 1.64 0.79, 3.42 0.19 0.00, 12.20 0.02 0.00, 1.91 3.48 1.77, 6.83 2.05 0.94, 4.45 Typical chest pain 6.61 3.67, 11.89 5.34 2.70, 10.56 5.44 2.90, 10.18 5.22 2.52, 10.81 5.44 2.94, 10.05 4.91 2.43, 9.91 Smoking 0.08 0.00, 9.40 0.06 0.00, 11.40 Dyslipidaemia 10.90 0.19, 610.99 2.43 0.03, 195.86 3.03 1.63, 5.61 1.94 0.96, 3.94 Diabetes 2.81 1.07, 7.38 2.14 0.71, 6.42 2.61 1.02, 6.73 2.22 0.74, 6.65 Age–smoking 1.04 0.97, 1.13 1.03 0.95, 1.12 Age–dyslipidaemia 0.98 0.92, 1.05 1.00 0.93, 1.07 Sex–smoking 1.77 0.36, 8.84 3.82 0.66, 22.09 Age–sex 1.05 0.98, 1.12 1.07 1.00, 1.15 Oestrogen Hypertension Family history Dyslipidaemia–family history Obesity BMI AUCb 0.798 0.742, 0.854 0.890 0.851, 0.930 0.838 0.789, 0.887 0.901 0.863, 0.938 0.831 0.780, 0.881 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS Table 3 Comparison of multivariate logistic regression models Variables Model 4: Morise et al. 1997 Model 5: Shaw et al. 1998 No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.87 1.50, 2.31 1.86 1.51, 2.30 Intercept Age 1.05 1.01, 1.09 0.99 0.95, 1.03 1.05 1.02, 1.08 0.98 0.95, 1.02 Male sex 3.15 1.29, 7.70 1.37 0.48, 3.95 3.42 1.74, 6.74 2.05 0.94, 4.45 Typical pain 5.56 2.94, 10.51 4.82 2.36, 9.86 5.50 2.96, 10.21 4.91 2.44, 9.90 Smoking 1.53 0.72, 3.24 1.03 0.44, 2.39 1.63 0.80, 3.30 1.04 0.47, 2.27 Dyslipidaemia 3.20 1.29, 7.95 1.80 0.62, 5.22 3.04 1.63, 5.66 1.95 0.96, 3.94 Diabetes 2.66 0.98, 7.26 2.01 0.61, 6.61 2.85 1.10, 7.39 2.24 0.74, 6.75 Age–smoking Age–dyslipidaemia Sex–smoking Age–sex Oestrogen 0.78 0.33, 1.86 0.53 0.19, 1.48 Hypertension 1.83 0.93, 3.60 1.31 0.62, 2.79 Family History 2.02 0.77, 5.29 1.14 0.38, 3.39 Dyslipidaemia–family history 0.74 0.21, 2.60 1.15 0.28, 4.82 Obesity 0.88 0.31, 2.48 0.65 0.20, 2.12 BMI 0.99 0.88, 1.13 1.05 0.91, 1.22 AUCb 0.840 0.792, 0.889 0.898 0.859, 0.936 0.833 0.783, 0.883 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS Pryor et al. analysed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes and the interaction between age and smoking, age and dyslipidaemia, sex and smoking, and age and sex, of which type of chest pain and the presence of diabetes were significant predictors. This model resulted in an AUC of 0.838. After including CTCS, the AUC increased to 0.901 which was a statistically significant improvement (p < 0.001). In the expanded model diabetes was no longer a significant predictor (Table 2). Morise et al. (1994) included diabetes and dyslipidaemia in addition to the variables used by Diamond & Forrester and resulted in an AUC of 0.831. All variables were significant predictors of the presence of obstructive CAD. After including CTCS, type of chest pain was the only variable that remained significant. After inclusion of CTCS the AUC increased to 0.899, which was a statistically significant improvement (p < 0.001) (Table 2). Morise et al. (1997) assessed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes, oestrogen status, hypertension, family history, obesity, BMI and the interaction between dyslipidaemia and family history. This model resulted in an AUC of 0.840. Age, sex, type of chest pain and dyslipidaemia were significant predictors. After including CTCS, the AUC increased to 0.898, which was a significant model improvement (p < 0.001). Age, sex and dyslipidaemia were no longer significant predictors after the addition of CTCS (Table 3). Shaw et al. considered age, sex, typical chest pain, smoking, dyslipidaemia and diabetes and resulted in an AUC of 0.833. After including CTCS, only type of chest pain remained a significant predictor and the AUC increased to 0.899 which was a statistically significant improvement (p < 0.001) (Table 3). Reclassification tables for the Diamond & Forrester model and the Pryor model are presented in Tables 4 and 5. The addition of CTCS to Diamond & Forrester resulted in reclassification of 47.2% of patients of whom 73.3% were correctly reclassified. The reclassification calibration statistic (RCS) indicated a strong lack of fit for the Diamond & Forrester model (p < 0.00001) which decreased substantially when CTCS was added to the model (p < 0.01). The NRI (net reclassification improvement) was 33.6% (p < 0.0001) and the IDI (integrated discrimination improvement) was also statistically significant (18.8%, p < 0.001) indicating improvement in the classification of cases and non-cases in probability categories and improvement in discrimination between cases and non-cases. Table 4 Probability of coronary artery disease: reclassification table after addition of CTCS to the Diamond and Forrester (model 1) Probability category based on model 1 Probability category based on model 1 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 51 (67.1) 19 (25.0%) 4 (5.2) 2 (2.6) 76 (29.9) Observed probability, % 11.8 36.8 50.0 100.0 22.4 ≥30–50% N (%) 25 (39.7) 13 (20.6) 19 (30.2) 6 (9.5) 63 (28.4) Observed probability, % 0.0 23.1 68.4 50.0 30.2 ≥50–70% N (%) 4 (11.1) 5 (13.9) 6 (16.7) 21 (58.3) 36 (14.2) Observed probability, % 25 80.0 50.0 66.7 61.1 ≥70% N (%) 5 (6.3) 2 (2.5) 8 (10.1) 64 (81.0) 79 (31.1) Observed probability, % 0.0 50.0 37.5 95.3 82.3 Total N (%) 85 (33.5) 39 (15.4) 37 (14.6) 93 (36.6) 254 (100.0) Observed probability, % 8.2 38.5 56.8 86.0 48.4 CTCS computed tomography coronary calcium score Table 5 Probability of coronary artery disease: reclassification table after addition of CTCS to the model published by Pryor et al. (model 2) Probability category based on model 2 Probability category based on model 2 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 76 (85.4) 10 (11.2) 2 (2.3) 1 (1.1) 89 (35.0) Observed probability, % 9.2 40.0 100.0 100.0 15.7 ≥30–50% N (%) 10 (27.0) 10 (27.0) 12 (32.4) 5 (13.5) 37 (14.6) Observed probability, % 10.0 20.0 50.0 100.0 37.8 ≥50–70% N (%) 7 (13.5) 6 (11.5) 15 (28.9) 24 (46.2) 52 (20.5) Observed probability, % 0.0 50.0 73.3 66.7 57.7 ≥70% N (%) 3 (4.0) 3 (4.0) 9 (11.8) 61 (80.3) 76 (29.9) Observed probability, % 33.3 0.0 66.7 95.0 85.5 Total N (%) 96 (37.8) 29 (11.4) 38 (15.0) 91 (35.8) 254 (100.0) Observed probability, % 9.4 31.0 65.8 87.9 48.4 CTCS computed tomography coronary calcium score For the model by Pryor et al., 36.2% of the patients were reclassified, of whom 54.3% were correctly classified. The RCS indicated a lack of fit (p = 0.01), which decreased when CTCS was added to the model (p = 0.03). The NRI was 24.0% (p < 0.0001) and the IDI was 14.8% (p < 0.001). The reclassification measures for all models are presented in Table 6. Table 6 Reclassification measures obtained by adding CTCS to the existing prediction models Model Reclassification percentages % Correcta Chi-squared excluding CTCSb p value Chi-squared including CTCSc p value NRI, % p value for NRI Reclassification improvement (cases)d Reclassification improvement (non-cases)d IDI, % p value for IDI Overall Reclassified from ≥30–50% Reclassified from ≥50–70% 1 47.2 79.4 83.3 73.3 25.01 <0.00001 10.91 <0.01 33.6 <0.0001 <0.0001 0.23 18.8 <0.001 2 36.2 73.0 71.2 54.3 6.57 0.01 4.70 0.03 24.0 0.001 <0.001 0.31 14.8 <0.001 3 38.2 77.5 75.5 63.9 4.14 0.04 3.31 0.07 21.8 0.005 <0.001 0.58 14.9 <0.001 4 34.3 74.4 74.0 83.9 3.82 0.05 1.58 0.21 24.9 <0.001 <0.001 0.29 13.3 <0.001 5 37.8 77.5 75.9 69.8 3.65 0.06 5.14 0.02 22.6 0.003 <0.001 0.58 14.4 <0.001 CTCS computed tomography coronary calcium score, NRI net reclassification improvement, IDI integrated discrimination improvement aIf the predicted probability of obstructive CAD of the model including CTCS was closer to the observed probability of CAD compared with the prediction of the model without CTCS, the reclassification was considered to be correct bReclassification calibration (Hosmer–Lemeshow) statistic for model without CTCS, using cells from the reclassification table with at least 20 observations cReclassification calibration statistic for models including CTCS, using cells from the reclassification table with at least 20 observations dThe reclassification improvement is defined as the difference in proportions of patients moving up and down for cases and non-cases separately
title	Results
sec	Study population During a 24-month period, 402 patients were enrolled. Patients with acute coronary syndrome or a history of myocardial infarction were excluded and 254 patients were left for data analysis (Table 1). Of these, 123 (48%) patients had obstructive CAD on CCA. CTCS ranged from 0 to 3,839 with a median of 4 for patients without obstructive CAD and a median of 337 for patients with obstructive CAD. Of 131 patients without obstructive CAD, 44 (33.6%) patients had no coronary calcification; whereas of 123 patients diagnosed with obstructive CAD, 3 (2.4%) patients did not have any coronary calcification. Table 1 Patient characteristics Total (n = 254) no. Patients without CAD (n = 131) no. Patients with CAD (n = 123) no. p value Mean age (SD) 59 (11) 56 (12) 62 (10) <0.001 Male sex (%) 171 (67%) 78 (60%) 93 (76%) 0.01 Typical chest pain (%) 118 (46%) 33 (25%) 85 (69%) < 0.001 Mean BMIa (SD) 27 (4) 27 (4) 28 (5) 0.11 Smokingb (%) 63 (25%) 30 (23%) 33 (27%) 0.47 Hypertension (%) 140 (55%) 59 (45%) 81 (66%) <0.001 Dyslipidaemiac (%) 136 (54%) 47 (36%) 89 (72%) <0.001 Diabetesd (%) 32 (13%) 10 (8%) 22 (18%) 0.01 Family history (%) 126 (50%) 57 (44%) 69 (56%) 0.05 Mean calcium scoree (SD) 346 (572) 132 (320) 574 (685) <0.001 Median calcium score 138 4 337 – CADf on CCA (%) 123 (48%) 0 (0%) 123 (100%) CAD obstructive coronary artery disease, CCA conventional coronary angiography, SD standard deviation aBody Mass Index, defined as weight/height2 (in kg/m2) bPast or current cSerum cholesterol >200 mg/dL or 5.18 mmol/L dPlasma glucose ≥126 mg/dL or 7.0 mmol eMeasured according to Agatston [15] fDefined as ≥50% stenosis in at least one vessel
title	Study population
p	During a 24-month period, 402 patients were enrolled. Patients with acute coronary syndrome or a history of myocardial infarction were excluded and 254 patients were left for data analysis (Table 1). Of these, 123 (48%) patients had obstructive CAD on CCA. CTCS ranged from 0 to 3,839 with a median of 4 for patients without obstructive CAD and a median of 337 for patients with obstructive CAD. Of 131 patients without obstructive CAD, 44 (33.6%) patients had no coronary calcification; whereas of 123 patients diagnosed with obstructive CAD, 3 (2.4%) patients did not have any coronary calcification. Table 1 Patient characteristics Total (n = 254) no. Patients without CAD (n = 131) no. Patients with CAD (n = 123) no. p value Mean age (SD) 59 (11) 56 (12) 62 (10) <0.001 Male sex (%) 171 (67%) 78 (60%) 93 (76%) 0.01 Typical chest pain (%) 118 (46%) 33 (25%) 85 (69%) < 0.001 Mean BMIa (SD) 27 (4) 27 (4) 28 (5) 0.11 Smokingb (%) 63 (25%) 30 (23%) 33 (27%) 0.47 Hypertension (%) 140 (55%) 59 (45%) 81 (66%) <0.001 Dyslipidaemiac (%) 136 (54%) 47 (36%) 89 (72%) <0.001 Diabetesd (%) 32 (13%) 10 (8%) 22 (18%) 0.01 Family history (%) 126 (50%) 57 (44%) 69 (56%) 0.05 Mean calcium scoree (SD) 346 (572) 132 (320) 574 (685) <0.001 Median calcium score 138 4 337 – CADf on CCA (%) 123 (48%) 0 (0%) 123 (100%) CAD obstructive coronary artery disease, CCA conventional coronary angiography, SD standard deviation aBody Mass Index, defined as weight/height2 (in kg/m2) bPast or current cSerum cholesterol >200 mg/dL or 5.18 mmol/L dPlasma glucose ≥126 mg/dL or 7.0 mmol eMeasured according to Agatston [15] fDefined as ≥50% stenosis in at least one vessel
table-wrap	Table 1 Patient characteristics Total (n = 254) no. Patients without CAD (n = 131) no. Patients with CAD (n = 123) no. p value Mean age (SD) 59 (11) 56 (12) 62 (10) <0.001 Male sex (%) 171 (67%) 78 (60%) 93 (76%) 0.01 Typical chest pain (%) 118 (46%) 33 (25%) 85 (69%) < 0.001 Mean BMIa (SD) 27 (4) 27 (4) 28 (5) 0.11 Smokingb (%) 63 (25%) 30 (23%) 33 (27%) 0.47 Hypertension (%) 140 (55%) 59 (45%) 81 (66%) <0.001 Dyslipidaemiac (%) 136 (54%) 47 (36%) 89 (72%) <0.001 Diabetesd (%) 32 (13%) 10 (8%) 22 (18%) 0.01 Family history (%) 126 (50%) 57 (44%) 69 (56%) 0.05 Mean calcium scoree (SD) 346 (572) 132 (320) 574 (685) <0.001 Median calcium score 138 4 337 – CADf on CCA (%) 123 (48%) 0 (0%) 123 (100%) CAD obstructive coronary artery disease, CCA conventional coronary angiography, SD standard deviation aBody Mass Index, defined as weight/height2 (in kg/m2) bPast or current cSerum cholesterol >200 mg/dL or 5.18 mmol/L dPlasma glucose ≥126 mg/dL or 7.0 mmol eMeasured according to Agatston [15] fDefined as ≥50% stenosis in at least one vessel
label	Table 1
caption	Patient characteristics
p	Patient characteristics
table	Total (n = 254) no. Patients without CAD (n = 131) no. Patients with CAD (n = 123) no. p value Mean age (SD) 59 (11) 56 (12) 62 (10) <0.001 Male sex (%) 171 (67%) 78 (60%) 93 (76%) 0.01 Typical chest pain (%) 118 (46%) 33 (25%) 85 (69%) < 0.001 Mean BMIa (SD) 27 (4) 27 (4) 28 (5) 0.11 Smokingb (%) 63 (25%) 30 (23%) 33 (27%) 0.47 Hypertension (%) 140 (55%) 59 (45%) 81 (66%) <0.001 Dyslipidaemiac (%) 136 (54%) 47 (36%) 89 (72%) <0.001 Diabetesd (%) 32 (13%) 10 (8%) 22 (18%) 0.01 Family history (%) 126 (50%) 57 (44%) 69 (56%) 0.05 Mean calcium scoree (SD) 346 (572) 132 (320) 574 (685) <0.001 Median calcium score 138 4 337 – CADf on CCA (%) 123 (48%) 0 (0%) 123 (100%)
tr	Total (n = 254) no. Patients without CAD (n = 131) no. Patients with CAD (n = 123) no. p value
th	Total (n = 254) no.
th	Patients without CAD (n = 131) no.
th	Patients with CAD (n = 123) no.
th	p value
tr	Mean age (SD) 59 (11) 56 (12) 62 (10) <0.001
td	Mean age (SD)
td	59 (11)
td	56 (12)
td	62 (10)
td	<0.001
tr	Male sex (%) 171 (67%) 78 (60%) 93 (76%) 0.01
td	Male sex (%)
td	171 (67%)
td	78 (60%)
td	93 (76%)
td	0.01
tr	Typical chest pain (%) 118 (46%) 33 (25%) 85 (69%) < 0.001
td	Typical chest pain (%)
td	118 (46%)
td	33 (25%)
td	85 (69%)
td	< 0.001
tr	Mean BMIa (SD) 27 (4) 27 (4) 28 (5) 0.11
td	Mean BMIa (SD)
td	27 (4)
td	27 (4)
td	28 (5)
td	0.11
tr	Smokingb (%) 63 (25%) 30 (23%) 33 (27%) 0.47
td	Smokingb (%)
td	63 (25%)
td	30 (23%)
td	33 (27%)
td	0.47
tr	Hypertension (%) 140 (55%) 59 (45%) 81 (66%) <0.001
td	Hypertension (%)
td	140 (55%)
td	59 (45%)
td	81 (66%)
td	<0.001
tr	Dyslipidaemiac (%) 136 (54%) 47 (36%) 89 (72%) <0.001
td	Dyslipidaemiac (%)
td	136 (54%)
td	47 (36%)
td	89 (72%)
td	<0.001
tr	Diabetesd (%) 32 (13%) 10 (8%) 22 (18%) 0.01
td	Diabetesd (%)
td	32 (13%)
td	10 (8%)
td	22 (18%)
td	0.01
tr	Family history (%) 126 (50%) 57 (44%) 69 (56%) 0.05
td	Family history (%)
td	126 (50%)
td	57 (44%)
td	69 (56%)
td	0.05
tr	Mean calcium scoree (SD) 346 (572) 132 (320) 574 (685) <0.001
td	Mean calcium scoree (SD)
td	346 (572)
td	132 (320)
td	574 (685)
td	<0.001
tr	Median calcium score 138 4 337 –
td	Median calcium score
td	138
td	4
td	337
td	–
tr	CADf on CCA (%) 123 (48%) 0 (0%) 123 (100%)
td	CADf on CCA (%)
td	123 (48%)
td	0 (0%)
td	123 (100%)
table-wrap-foot	CAD obstructive coronary artery disease, CCA conventional coronary angiography, SD standard deviation aBody Mass Index, defined as weight/height2 (in kg/m2) bPast or current cSerum cholesterol >200 mg/dL or 5.18 mmol/L dPlasma glucose ≥126 mg/dL or 7.0 mmol eMeasured according to Agatston [15] fDefined as ≥50% stenosis in at least one vessel
p	CAD obstructive coronary artery disease, CCA conventional coronary angiography, SD standard deviation
p	aBody Mass Index, defined as weight/height2 (in kg/m2)
p	bPast or current
p	cSerum cholesterol >200 mg/dL or 5.18 mmol/L
p	dPlasma glucose ≥126 mg/dL or 7.0 mmol
p	eMeasured according to Agatston [15]
p	fDefined as ≥50% stenosis in at least one vessel
sec	Systematic literature search We obtained 649 articles in our literature search of which 632 articles were excluded based on title or abstract. After excluding 11 articles based on the full text, 6 studies were left for analysis.
title	Systematic literature search
p	We obtained 649 articles in our literature search of which 632 articles were excluded based on title or abstract. After excluding 11 articles based on the full text, 6 studies were left for analysis.
sec	Multivariate logistic regression analysis Five published prediction rules were validated (Tables 2 and 3): Diamond & Forrester [22], Pryor et al. [23], Morise et al. 1994 [24], Morise et al. 1997 [17] and Shaw et al. [25]. The Diamond & Forrester prediction rule includes age, sex and type of chest pain, all of which were significant predictors of obstructive CAD in our dataset and with an area under the ROC curve (AUC) of 0.798. Including CTCS increased the AUC to 0.890, which was a statistically significant improvement (p < 0.001). In the expanded model age and sex were no longer significant predictors (Table 2). Table 2 Comparison of multivariate logistic regression models Variables Model 1: Diamond & Forrester 1979 Model 2: Pryor et al. 1993 Model 3: Morise et al. 1994 No calcium score Calcium score No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.97 1.60, 2.41 1.93 1.55, 2.41 1.87 1.52, 2.29 Age 1.05 1.02, 1.08 0.98 0.95, 1.02 1.02 0.95, 1.09 0.93 0.86, 1.01 1.05 1.01, 1.08 0.98 0.95, 1.02 Male sex 2.97 1.57, 5.61 1.64 0.79, 3.42 0.19 0.00, 12.20 0.02 0.00, 1.91 3.48 1.77, 6.83 2.05 0.94, 4.45 Typical chest pain 6.61 3.67, 11.89 5.34 2.70, 10.56 5.44 2.90, 10.18 5.22 2.52, 10.81 5.44 2.94, 10.05 4.91 2.43, 9.91 Smoking 0.08 0.00, 9.40 0.06 0.00, 11.40 Dyslipidaemia 10.90 0.19, 610.99 2.43 0.03, 195.86 3.03 1.63, 5.61 1.94 0.96, 3.94 Diabetes 2.81 1.07, 7.38 2.14 0.71, 6.42 2.61 1.02, 6.73 2.22 0.74, 6.65 Age–smoking 1.04 0.97, 1.13 1.03 0.95, 1.12 Age–dyslipidaemia 0.98 0.92, 1.05 1.00 0.93, 1.07 Sex–smoking 1.77 0.36, 8.84 3.82 0.66, 22.09 Age–sex 1.05 0.98, 1.12 1.07 1.00, 1.15 Oestrogen Hypertension Family history Dyslipidaemia–family history Obesity BMI AUCb 0.798 0.742, 0.854 0.890 0.851, 0.930 0.838 0.789, 0.887 0.901 0.863, 0.938 0.831 0.780, 0.881 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS Table 3 Comparison of multivariate logistic regression models Variables Model 4: Morise et al. 1997 Model 5: Shaw et al. 1998 No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.87 1.50, 2.31 1.86 1.51, 2.30 Intercept Age 1.05 1.01, 1.09 0.99 0.95, 1.03 1.05 1.02, 1.08 0.98 0.95, 1.02 Male sex 3.15 1.29, 7.70 1.37 0.48, 3.95 3.42 1.74, 6.74 2.05 0.94, 4.45 Typical pain 5.56 2.94, 10.51 4.82 2.36, 9.86 5.50 2.96, 10.21 4.91 2.44, 9.90 Smoking 1.53 0.72, 3.24 1.03 0.44, 2.39 1.63 0.80, 3.30 1.04 0.47, 2.27 Dyslipidaemia 3.20 1.29, 7.95 1.80 0.62, 5.22 3.04 1.63, 5.66 1.95 0.96, 3.94 Diabetes 2.66 0.98, 7.26 2.01 0.61, 6.61 2.85 1.10, 7.39 2.24 0.74, 6.75 Age–smoking Age–dyslipidaemia Sex–smoking Age–sex Oestrogen 0.78 0.33, 1.86 0.53 0.19, 1.48 Hypertension 1.83 0.93, 3.60 1.31 0.62, 2.79 Family History 2.02 0.77, 5.29 1.14 0.38, 3.39 Dyslipidaemia–family history 0.74 0.21, 2.60 1.15 0.28, 4.82 Obesity 0.88 0.31, 2.48 0.65 0.20, 2.12 BMI 0.99 0.88, 1.13 1.05 0.91, 1.22 AUCb 0.840 0.792, 0.889 0.898 0.859, 0.936 0.833 0.783, 0.883 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS Pryor et al. analysed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes and the interaction between age and smoking, age and dyslipidaemia, sex and smoking, and age and sex, of which type of chest pain and the presence of diabetes were significant predictors. This model resulted in an AUC of 0.838. After including CTCS, the AUC increased to 0.901 which was a statistically significant improvement (p < 0.001). In the expanded model diabetes was no longer a significant predictor (Table 2). Morise et al. (1994) included diabetes and dyslipidaemia in addition to the variables used by Diamond & Forrester and resulted in an AUC of 0.831. All variables were significant predictors of the presence of obstructive CAD. After including CTCS, type of chest pain was the only variable that remained significant. After inclusion of CTCS the AUC increased to 0.899, which was a statistically significant improvement (p < 0.001) (Table 2). Morise et al. (1997) assessed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes, oestrogen status, hypertension, family history, obesity, BMI and the interaction between dyslipidaemia and family history. This model resulted in an AUC of 0.840. Age, sex, type of chest pain and dyslipidaemia were significant predictors. After including CTCS, the AUC increased to 0.898, which was a significant model improvement (p < 0.001). Age, sex and dyslipidaemia were no longer significant predictors after the addition of CTCS (Table 3). Shaw et al. considered age, sex, typical chest pain, smoking, dyslipidaemia and diabetes and resulted in an AUC of 0.833. After including CTCS, only type of chest pain remained a significant predictor and the AUC increased to 0.899 which was a statistically significant improvement (p < 0.001) (Table 3). Reclassification tables for the Diamond & Forrester model and the Pryor model are presented in Tables 4 and 5. The addition of CTCS to Diamond & Forrester resulted in reclassification of 47.2% of patients of whom 73.3% were correctly reclassified. The reclassification calibration statistic (RCS) indicated a strong lack of fit for the Diamond & Forrester model (p < 0.00001) which decreased substantially when CTCS was added to the model (p < 0.01). The NRI (net reclassification improvement) was 33.6% (p < 0.0001) and the IDI (integrated discrimination improvement) was also statistically significant (18.8%, p < 0.001) indicating improvement in the classification of cases and non-cases in probability categories and improvement in discrimination between cases and non-cases. Table 4 Probability of coronary artery disease: reclassification table after addition of CTCS to the Diamond and Forrester (model 1) Probability category based on model 1 Probability category based on model 1 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 51 (67.1) 19 (25.0%) 4 (5.2) 2 (2.6) 76 (29.9) Observed probability, % 11.8 36.8 50.0 100.0 22.4 ≥30–50% N (%) 25 (39.7) 13 (20.6) 19 (30.2) 6 (9.5) 63 (28.4) Observed probability, % 0.0 23.1 68.4 50.0 30.2 ≥50–70% N (%) 4 (11.1) 5 (13.9) 6 (16.7) 21 (58.3) 36 (14.2) Observed probability, % 25 80.0 50.0 66.7 61.1 ≥70% N (%) 5 (6.3) 2 (2.5) 8 (10.1) 64 (81.0) 79 (31.1) Observed probability, % 0.0 50.0 37.5 95.3 82.3 Total N (%) 85 (33.5) 39 (15.4) 37 (14.6) 93 (36.6) 254 (100.0) Observed probability, % 8.2 38.5 56.8 86.0 48.4 CTCS computed tomography coronary calcium score Table 5 Probability of coronary artery disease: reclassification table after addition of CTCS to the model published by Pryor et al. (model 2) Probability category based on model 2 Probability category based on model 2 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 76 (85.4) 10 (11.2) 2 (2.3) 1 (1.1) 89 (35.0) Observed probability, % 9.2 40.0 100.0 100.0 15.7 ≥30–50% N (%) 10 (27.0) 10 (27.0) 12 (32.4) 5 (13.5) 37 (14.6) Observed probability, % 10.0 20.0 50.0 100.0 37.8 ≥50–70% N (%) 7 (13.5) 6 (11.5) 15 (28.9) 24 (46.2) 52 (20.5) Observed probability, % 0.0 50.0 73.3 66.7 57.7 ≥70% N (%) 3 (4.0) 3 (4.0) 9 (11.8) 61 (80.3) 76 (29.9) Observed probability, % 33.3 0.0 66.7 95.0 85.5 Total N (%) 96 (37.8) 29 (11.4) 38 (15.0) 91 (35.8) 254 (100.0) Observed probability, % 9.4 31.0 65.8 87.9 48.4 CTCS computed tomography coronary calcium score For the model by Pryor et al., 36.2% of the patients were reclassified, of whom 54.3% were correctly classified. The RCS indicated a lack of fit (p = 0.01), which decreased when CTCS was added to the model (p = 0.03). The NRI was 24.0% (p < 0.0001) and the IDI was 14.8% (p < 0.001). The reclassification measures for all models are presented in Table 6. Table 6 Reclassification measures obtained by adding CTCS to the existing prediction models Model Reclassification percentages % Correcta Chi-squared excluding CTCSb p value Chi-squared including CTCSc p value NRI, % p value for NRI Reclassification improvement (cases)d Reclassification improvement (non-cases)d IDI, % p value for IDI Overall Reclassified from ≥30–50% Reclassified from ≥50–70% 1 47.2 79.4 83.3 73.3 25.01 <0.00001 10.91 <0.01 33.6 <0.0001 <0.0001 0.23 18.8 <0.001 2 36.2 73.0 71.2 54.3 6.57 0.01 4.70 0.03 24.0 0.001 <0.001 0.31 14.8 <0.001 3 38.2 77.5 75.5 63.9 4.14 0.04 3.31 0.07 21.8 0.005 <0.001 0.58 14.9 <0.001 4 34.3 74.4 74.0 83.9 3.82 0.05 1.58 0.21 24.9 <0.001 <0.001 0.29 13.3 <0.001 5 37.8 77.5 75.9 69.8 3.65 0.06 5.14 0.02 22.6 0.003 <0.001 0.58 14.4 <0.001 CTCS computed tomography coronary calcium score, NRI net reclassification improvement, IDI integrated discrimination improvement aIf the predicted probability of obstructive CAD of the model including CTCS was closer to the observed probability of CAD compared with the prediction of the model without CTCS, the reclassification was considered to be correct bReclassification calibration (Hosmer–Lemeshow) statistic for model without CTCS, using cells from the reclassification table with at least 20 observations cReclassification calibration statistic for models including CTCS, using cells from the reclassification table with at least 20 observations dThe reclassification improvement is defined as the difference in proportions of patients moving up and down for cases and non-cases separately
title	Multivariate logistic regression analysis
p	Five published prediction rules were validated (Tables 2 and 3): Diamond & Forrester [22], Pryor et al. [23], Morise et al. 1994 [24], Morise et al. 1997 [17] and Shaw et al. [25]. The Diamond & Forrester prediction rule includes age, sex and type of chest pain, all of which were significant predictors of obstructive CAD in our dataset and with an area under the ROC curve (AUC) of 0.798. Including CTCS increased the AUC to 0.890, which was a statistically significant improvement (p < 0.001). In the expanded model age and sex were no longer significant predictors (Table 2). Table 2 Comparison of multivariate logistic regression models Variables Model 1: Diamond & Forrester 1979 Model 2: Pryor et al. 1993 Model 3: Morise et al. 1994 No calcium score Calcium score No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.97 1.60, 2.41 1.93 1.55, 2.41 1.87 1.52, 2.29 Age 1.05 1.02, 1.08 0.98 0.95, 1.02 1.02 0.95, 1.09 0.93 0.86, 1.01 1.05 1.01, 1.08 0.98 0.95, 1.02 Male sex 2.97 1.57, 5.61 1.64 0.79, 3.42 0.19 0.00, 12.20 0.02 0.00, 1.91 3.48 1.77, 6.83 2.05 0.94, 4.45 Typical chest pain 6.61 3.67, 11.89 5.34 2.70, 10.56 5.44 2.90, 10.18 5.22 2.52, 10.81 5.44 2.94, 10.05 4.91 2.43, 9.91 Smoking 0.08 0.00, 9.40 0.06 0.00, 11.40 Dyslipidaemia 10.90 0.19, 610.99 2.43 0.03, 195.86 3.03 1.63, 5.61 1.94 0.96, 3.94 Diabetes 2.81 1.07, 7.38 2.14 0.71, 6.42 2.61 1.02, 6.73 2.22 0.74, 6.65 Age–smoking 1.04 0.97, 1.13 1.03 0.95, 1.12 Age–dyslipidaemia 0.98 0.92, 1.05 1.00 0.93, 1.07 Sex–smoking 1.77 0.36, 8.84 3.82 0.66, 22.09 Age–sex 1.05 0.98, 1.12 1.07 1.00, 1.15 Oestrogen Hypertension Family history Dyslipidaemia–family history Obesity BMI AUCb 0.798 0.742, 0.854 0.890 0.851, 0.930 0.838 0.789, 0.887 0.901 0.863, 0.938 0.831 0.780, 0.881 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS Table 3 Comparison of multivariate logistic regression models Variables Model 4: Morise et al. 1997 Model 5: Shaw et al. 1998 No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.87 1.50, 2.31 1.86 1.51, 2.30 Intercept Age 1.05 1.01, 1.09 0.99 0.95, 1.03 1.05 1.02, 1.08 0.98 0.95, 1.02 Male sex 3.15 1.29, 7.70 1.37 0.48, 3.95 3.42 1.74, 6.74 2.05 0.94, 4.45 Typical pain 5.56 2.94, 10.51 4.82 2.36, 9.86 5.50 2.96, 10.21 4.91 2.44, 9.90 Smoking 1.53 0.72, 3.24 1.03 0.44, 2.39 1.63 0.80, 3.30 1.04 0.47, 2.27 Dyslipidaemia 3.20 1.29, 7.95 1.80 0.62, 5.22 3.04 1.63, 5.66 1.95 0.96, 3.94 Diabetes 2.66 0.98, 7.26 2.01 0.61, 6.61 2.85 1.10, 7.39 2.24 0.74, 6.75 Age–smoking Age–dyslipidaemia Sex–smoking Age–sex Oestrogen 0.78 0.33, 1.86 0.53 0.19, 1.48 Hypertension 1.83 0.93, 3.60 1.31 0.62, 2.79 Family History 2.02 0.77, 5.29 1.14 0.38, 3.39 Dyslipidaemia–family history 0.74 0.21, 2.60 1.15 0.28, 4.82 Obesity 0.88 0.31, 2.48 0.65 0.20, 2.12 BMI 0.99 0.88, 1.13 1.05 0.91, 1.22 AUCb 0.840 0.792, 0.889 0.898 0.859, 0.936 0.833 0.783, 0.883 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS
table-wrap	Table 2 Comparison of multivariate logistic regression models Variables Model 1: Diamond & Forrester 1979 Model 2: Pryor et al. 1993 Model 3: Morise et al. 1994 No calcium score Calcium score No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.97 1.60, 2.41 1.93 1.55, 2.41 1.87 1.52, 2.29 Age 1.05 1.02, 1.08 0.98 0.95, 1.02 1.02 0.95, 1.09 0.93 0.86, 1.01 1.05 1.01, 1.08 0.98 0.95, 1.02 Male sex 2.97 1.57, 5.61 1.64 0.79, 3.42 0.19 0.00, 12.20 0.02 0.00, 1.91 3.48 1.77, 6.83 2.05 0.94, 4.45 Typical chest pain 6.61 3.67, 11.89 5.34 2.70, 10.56 5.44 2.90, 10.18 5.22 2.52, 10.81 5.44 2.94, 10.05 4.91 2.43, 9.91 Smoking 0.08 0.00, 9.40 0.06 0.00, 11.40 Dyslipidaemia 10.90 0.19, 610.99 2.43 0.03, 195.86 3.03 1.63, 5.61 1.94 0.96, 3.94 Diabetes 2.81 1.07, 7.38 2.14 0.71, 6.42 2.61 1.02, 6.73 2.22 0.74, 6.65 Age–smoking 1.04 0.97, 1.13 1.03 0.95, 1.12 Age–dyslipidaemia 0.98 0.92, 1.05 1.00 0.93, 1.07 Sex–smoking 1.77 0.36, 8.84 3.82 0.66, 22.09 Age–sex 1.05 0.98, 1.12 1.07 1.00, 1.15 Oestrogen Hypertension Family history Dyslipidaemia–family history Obesity BMI AUCb 0.798 0.742, 0.854 0.890 0.851, 0.930 0.838 0.789, 0.887 0.901 0.863, 0.938 0.831 0.780, 0.881 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS
label	Table 2
caption	Comparison of multivariate logistic regression models
p	Comparison of multivariate logistic regression models
table	Variables Model 1: Diamond & Forrester 1979 Model 2: Pryor et al. 1993 Model 3: Morise et al. 1994 No calcium score Calcium score No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.97 1.60, 2.41 1.93 1.55, 2.41 1.87 1.52, 2.29 Age 1.05 1.02, 1.08 0.98 0.95, 1.02 1.02 0.95, 1.09 0.93 0.86, 1.01 1.05 1.01, 1.08 0.98 0.95, 1.02 Male sex 2.97 1.57, 5.61 1.64 0.79, 3.42 0.19 0.00, 12.20 0.02 0.00, 1.91 3.48 1.77, 6.83 2.05 0.94, 4.45 Typical chest pain 6.61 3.67, 11.89 5.34 2.70, 10.56 5.44 2.90, 10.18 5.22 2.52, 10.81 5.44 2.94, 10.05 4.91 2.43, 9.91 Smoking 0.08 0.00, 9.40 0.06 0.00, 11.40 Dyslipidaemia 10.90 0.19, 610.99 2.43 0.03, 195.86 3.03 1.63, 5.61 1.94 0.96, 3.94 Diabetes 2.81 1.07, 7.38 2.14 0.71, 6.42 2.61 1.02, 6.73 2.22 0.74, 6.65 Age–smoking 1.04 0.97, 1.13 1.03 0.95, 1.12 Age–dyslipidaemia 0.98 0.92, 1.05 1.00 0.93, 1.07 Sex–smoking 1.77 0.36, 8.84 3.82 0.66, 22.09 Age–sex 1.05 0.98, 1.12 1.07 1.00, 1.15 Oestrogen Hypertension Family history Dyslipidaemia–family history Obesity BMI AUCb 0.798 0.742, 0.854 0.890 0.851, 0.930 0.838 0.789, 0.887 0.901 0.863, 0.938 0.831 0.780, 0.881 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 p < 0.001
tr	Variables Model 1: Diamond & Forrester 1979 Model 2: Pryor et al. 1993 Model 3: Morise et al. 1994
th	Variables
th	Model 1: Diamond & Forrester 1979
th	Model 2: Pryor et al. 1993
th	Model 3: Morise et al. 1994
tr	No calcium score Calcium score No calcium score Calcium score No calcium score Calcium score
th	No calcium score
th	Calcium score
th	No calcium score
th	Calcium score
th	No calcium score
th	Calcium score
tr	OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI OR 95% CI
th	OR
th	95% CI
th	OR
th	95% CI
th	OR
th	95% CI
th	OR
th	95% CI
th	OR
th	95% CI
th	OR
th	95% CI
tr	ln CTCSa (CT calcium score) 1.97 1.60, 2.41 1.93 1.55, 2.41 1.87 1.52, 2.29
td	ln CTCSa (CT calcium score)
td	1.97
td	1.60, 2.41
td	1.93
td	1.55, 2.41
td	1.87
td	1.52, 2.29
tr	Age 1.05 1.02, 1.08 0.98 0.95, 1.02 1.02 0.95, 1.09 0.93 0.86, 1.01 1.05 1.01, 1.08 0.98 0.95, 1.02
td	Age
td	1.05
td	1.02, 1.08
td	0.98
td	0.95, 1.02
td	1.02
td	0.95, 1.09
td	0.93
td	0.86, 1.01
td	1.05
td	1.01, 1.08
td	0.98
td	0.95, 1.02
tr	Male sex 2.97 1.57, 5.61 1.64 0.79, 3.42 0.19 0.00, 12.20 0.02 0.00, 1.91 3.48 1.77, 6.83 2.05 0.94, 4.45
td	Male sex
td	2.97
td	1.57, 5.61
td	1.64
td	0.79, 3.42
td	0.19
td	0.00, 12.20
td	0.02
td	0.00, 1.91
td	3.48
td	1.77, 6.83
td	2.05
td	0.94, 4.45
tr	Typical chest pain 6.61 3.67, 11.89 5.34 2.70, 10.56 5.44 2.90, 10.18 5.22 2.52, 10.81 5.44 2.94, 10.05 4.91 2.43, 9.91
td	Typical chest pain
td	6.61
td	3.67, 11.89
td	5.34
td	2.70, 10.56
td	5.44
td	2.90, 10.18
td	5.22
td	2.52, 10.81
td	5.44
td	2.94, 10.05
td	4.91
td	2.43, 9.91
tr	Smoking 0.08 0.00, 9.40 0.06 0.00, 11.40
td	Smoking
td	0.08
td	0.00, 9.40
td	0.06
td	0.00, 11.40
tr	Dyslipidaemia 10.90 0.19, 610.99 2.43 0.03, 195.86 3.03 1.63, 5.61 1.94 0.96, 3.94
td	Dyslipidaemia
td	10.90
td	0.19, 610.99
td	2.43
td	0.03, 195.86
td	3.03
td	1.63, 5.61
td	1.94
td	0.96, 3.94
tr	Diabetes 2.81 1.07, 7.38 2.14 0.71, 6.42 2.61 1.02, 6.73 2.22 0.74, 6.65
td	Diabetes
td	2.81
td	1.07, 7.38
td	2.14
td	0.71, 6.42
td	2.61
td	1.02, 6.73
td	2.22
td	0.74, 6.65
tr	Age–smoking 1.04 0.97, 1.13 1.03 0.95, 1.12
td	Age–smoking
td	1.04
td	0.97, 1.13
td	1.03
td	0.95, 1.12
tr	Age–dyslipidaemia 0.98 0.92, 1.05 1.00 0.93, 1.07
td	Age–dyslipidaemia
td	0.98
td	0.92, 1.05
td	1.00
td	0.93, 1.07
tr	Sex–smoking 1.77 0.36, 8.84 3.82 0.66, 22.09
td	Sex–smoking
td	1.77
td	0.36, 8.84
td	3.82
td	0.66, 22.09
tr	Age–sex 1.05 0.98, 1.12 1.07 1.00, 1.15
td	Age–sex
td	1.05
td	0.98, 1.12
td	1.07
td	1.00, 1.15
tr	Oestrogen
td	Oestrogen
tr	Hypertension
td	Hypertension
tr	Family history
td	Family history
tr	Dyslipidaemia–family history
td	Dyslipidaemia–family history
tr	Obesity
td	Obesity
tr	BMI
td	BMI
tr
td
tr	AUCb 0.798 0.742, 0.854 0.890 0.851, 0.930 0.838 0.789, 0.887 0.901 0.863, 0.938 0.831 0.780, 0.881 0.899 0.861, 0.937
td	AUCb
td	0.798
td	0.742, 0.854
td	0.890
td	0.851, 0.930
td	0.838
td	0.789, 0.887
td	0.901
td	0.863, 0.938
td	0.831
td	0.780, 0.881
td	0.899
td	0.861, 0.937
tr	LR testc p < 0.001 p < 0.001 p < 0.001
td	LR testc
td	p < 0.001
td	p < 0.001
td	p < 0.001
table-wrap-foot	Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS
p	Odds ratios (ORs) in bold typeface are statistically significant
p	aNatural logarithm of CTCS + 1
p	bArea under the receiver operating characteristic curve
p	cLikelihood ratio test comparing model without CTCS and model including CTCS
table-wrap	Table 3 Comparison of multivariate logistic regression models Variables Model 4: Morise et al. 1997 Model 5: Shaw et al. 1998 No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.87 1.50, 2.31 1.86 1.51, 2.30 Intercept Age 1.05 1.01, 1.09 0.99 0.95, 1.03 1.05 1.02, 1.08 0.98 0.95, 1.02 Male sex 3.15 1.29, 7.70 1.37 0.48, 3.95 3.42 1.74, 6.74 2.05 0.94, 4.45 Typical pain 5.56 2.94, 10.51 4.82 2.36, 9.86 5.50 2.96, 10.21 4.91 2.44, 9.90 Smoking 1.53 0.72, 3.24 1.03 0.44, 2.39 1.63 0.80, 3.30 1.04 0.47, 2.27 Dyslipidaemia 3.20 1.29, 7.95 1.80 0.62, 5.22 3.04 1.63, 5.66 1.95 0.96, 3.94 Diabetes 2.66 0.98, 7.26 2.01 0.61, 6.61 2.85 1.10, 7.39 2.24 0.74, 6.75 Age–smoking Age–dyslipidaemia Sex–smoking Age–sex Oestrogen 0.78 0.33, 1.86 0.53 0.19, 1.48 Hypertension 1.83 0.93, 3.60 1.31 0.62, 2.79 Family History 2.02 0.77, 5.29 1.14 0.38, 3.39 Dyslipidaemia–family history 0.74 0.21, 2.60 1.15 0.28, 4.82 Obesity 0.88 0.31, 2.48 0.65 0.20, 2.12 BMI 0.99 0.88, 1.13 1.05 0.91, 1.22 AUCb 0.840 0.792, 0.889 0.898 0.859, 0.936 0.833 0.783, 0.883 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001 Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS
label	Table 3
caption	Comparison of multivariate logistic regression models
p	Comparison of multivariate logistic regression models
table	Variables Model 4: Morise et al. 1997 Model 5: Shaw et al. 1998 No calcium score Calcium score No calcium score Calcium score OR 95% CI OR 95% CI OR 95% CI OR 95% CI ln CTCSa (CT calcium score) 1.87 1.50, 2.31 1.86 1.51, 2.30 Intercept Age 1.05 1.01, 1.09 0.99 0.95, 1.03 1.05 1.02, 1.08 0.98 0.95, 1.02 Male sex 3.15 1.29, 7.70 1.37 0.48, 3.95 3.42 1.74, 6.74 2.05 0.94, 4.45 Typical pain 5.56 2.94, 10.51 4.82 2.36, 9.86 5.50 2.96, 10.21 4.91 2.44, 9.90 Smoking 1.53 0.72, 3.24 1.03 0.44, 2.39 1.63 0.80, 3.30 1.04 0.47, 2.27 Dyslipidaemia 3.20 1.29, 7.95 1.80 0.62, 5.22 3.04 1.63, 5.66 1.95 0.96, 3.94 Diabetes 2.66 0.98, 7.26 2.01 0.61, 6.61 2.85 1.10, 7.39 2.24 0.74, 6.75 Age–smoking Age–dyslipidaemia Sex–smoking Age–sex Oestrogen 0.78 0.33, 1.86 0.53 0.19, 1.48 Hypertension 1.83 0.93, 3.60 1.31 0.62, 2.79 Family History 2.02 0.77, 5.29 1.14 0.38, 3.39 Dyslipidaemia–family history 0.74 0.21, 2.60 1.15 0.28, 4.82 Obesity 0.88 0.31, 2.48 0.65 0.20, 2.12 BMI 0.99 0.88, 1.13 1.05 0.91, 1.22 AUCb 0.840 0.792, 0.889 0.898 0.859, 0.936 0.833 0.783, 0.883 0.899 0.861, 0.937 LR testc p < 0.001 p < 0.001
tr	Variables Model 4: Morise et al. 1997 Model 5: Shaw et al. 1998
th	Variables
th	Model 4: Morise et al. 1997
th	Model 5: Shaw et al. 1998
tr	No calcium score Calcium score No calcium score Calcium score
th	No calcium score
th	Calcium score
th	No calcium score
th	Calcium score
tr	OR 95% CI OR 95% CI OR 95% CI OR 95% CI
th	OR
th	95% CI
th	OR
th	95% CI
th	OR
th	95% CI
th	OR
th	95% CI
tr	ln CTCSa (CT calcium score) 1.87 1.50, 2.31 1.86 1.51, 2.30
td	ln CTCSa (CT calcium score)
td	1.87
td	1.50, 2.31
td	1.86
td	1.51, 2.30
tr	Intercept
td	Intercept
tr	Age 1.05 1.01, 1.09 0.99 0.95, 1.03 1.05 1.02, 1.08 0.98 0.95, 1.02
td	Age
td	1.05
td	1.01, 1.09
td	0.99
td	0.95, 1.03
td	1.05
td	1.02, 1.08
td	0.98
td	0.95, 1.02
tr	Male sex 3.15 1.29, 7.70 1.37 0.48, 3.95 3.42 1.74, 6.74 2.05 0.94, 4.45
td	Male sex
td	3.15
td	1.29, 7.70
td	1.37
td	0.48, 3.95
td	3.42
td	1.74, 6.74
td	2.05
td	0.94, 4.45
tr	Typical pain 5.56 2.94, 10.51 4.82 2.36, 9.86 5.50 2.96, 10.21 4.91 2.44, 9.90
td	Typical pain
td	5.56
td	2.94, 10.51
td	4.82
td	2.36, 9.86
td	5.50
td	2.96, 10.21
td	4.91
td	2.44, 9.90
tr	Smoking 1.53 0.72, 3.24 1.03 0.44, 2.39 1.63 0.80, 3.30 1.04 0.47, 2.27
td	Smoking
td	1.53
td	0.72, 3.24
td	1.03
td	0.44, 2.39
td	1.63
td	0.80, 3.30
td	1.04
td	0.47, 2.27
tr	Dyslipidaemia 3.20 1.29, 7.95 1.80 0.62, 5.22 3.04 1.63, 5.66 1.95 0.96, 3.94
td	Dyslipidaemia
td	3.20
td	1.29, 7.95
td	1.80
td	0.62, 5.22
td	3.04
td	1.63, 5.66
td	1.95
td	0.96, 3.94
tr	Diabetes 2.66 0.98, 7.26 2.01 0.61, 6.61 2.85 1.10, 7.39 2.24 0.74, 6.75
td	Diabetes
td	2.66
td	0.98, 7.26
td	2.01
td	0.61, 6.61
td	2.85
td	1.10, 7.39
td	2.24
td	0.74, 6.75
tr	Age–smoking
td	Age–smoking
tr	Age–dyslipidaemia
td	Age–dyslipidaemia
tr	Sex–smoking
td	Sex–smoking
tr	Age–sex
td	Age–sex
tr	Oestrogen 0.78 0.33, 1.86 0.53 0.19, 1.48
td	Oestrogen
td	0.78
td	0.33, 1.86
td	0.53
td	0.19, 1.48
tr	Hypertension 1.83 0.93, 3.60 1.31 0.62, 2.79
td	Hypertension
td	1.83
td	0.93, 3.60
td	1.31
td	0.62, 2.79
tr	Family History 2.02 0.77, 5.29 1.14 0.38, 3.39
td	Family History
td	2.02
td	0.77, 5.29
td	1.14
td	0.38, 3.39
tr	Dyslipidaemia–family history 0.74 0.21, 2.60 1.15 0.28, 4.82
td	Dyslipidaemia–family history
td	0.74
td	0.21, 2.60
td	1.15
td	0.28, 4.82
tr	Obesity 0.88 0.31, 2.48 0.65 0.20, 2.12
td	Obesity
td	0.88
td	0.31, 2.48
td	0.65
td	0.20, 2.12
tr	BMI 0.99 0.88, 1.13 1.05 0.91, 1.22
td	BMI
td	0.99
td	0.88, 1.13
td	1.05
td	0.91, 1.22
tr
td
tr	AUCb 0.840 0.792, 0.889 0.898 0.859, 0.936 0.833 0.783, 0.883 0.899 0.861, 0.937
td	AUCb
td	0.840
td	0.792, 0.889
td	0.898
td	0.859, 0.936
td	0.833
td	0.783, 0.883
td	0.899
td	0.861, 0.937
tr	LR testc p < 0.001 p < 0.001
td	LR testc
td	p < 0.001
td	p < 0.001
table-wrap-foot	Odds ratios (ORs) in bold typeface are statistically significant aNatural logarithm of CTCS + 1 bArea under the receiver operating characteristic curve cLikelihood ratio test comparing model without CTCS and model including CTCS
p	Odds ratios (ORs) in bold typeface are statistically significant
p	aNatural logarithm of CTCS + 1
p	bArea under the receiver operating characteristic curve
p	cLikelihood ratio test comparing model without CTCS and model including CTCS
p	Pryor et al. analysed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes and the interaction between age and smoking, age and dyslipidaemia, sex and smoking, and age and sex, of which type of chest pain and the presence of diabetes were significant predictors. This model resulted in an AUC of 0.838. After including CTCS, the AUC increased to 0.901 which was a statistically significant improvement (p < 0.001). In the expanded model diabetes was no longer a significant predictor (Table 2).
p	Morise et al. (1994) included diabetes and dyslipidaemia in addition to the variables used by Diamond & Forrester and resulted in an AUC of 0.831. All variables were significant predictors of the presence of obstructive CAD. After including CTCS, type of chest pain was the only variable that remained significant. After inclusion of CTCS the AUC increased to 0.899, which was a statistically significant improvement (p < 0.001) (Table 2).
p	Morise et al. (1997) assessed age, sex, type of chest pain, smoking, dyslipidaemia, diabetes, oestrogen status, hypertension, family history, obesity, BMI and the interaction between dyslipidaemia and family history. This model resulted in an AUC of 0.840. Age, sex, type of chest pain and dyslipidaemia were significant predictors. After including CTCS, the AUC increased to 0.898, which was a significant model improvement (p < 0.001). Age, sex and dyslipidaemia were no longer significant predictors after the addition of CTCS (Table 3).
p	Shaw et al. considered age, sex, typical chest pain, smoking, dyslipidaemia and diabetes and resulted in an AUC of 0.833. After including CTCS, only type of chest pain remained a significant predictor and the AUC increased to 0.899 which was a statistically significant improvement (p < 0.001) (Table 3).
p	Reclassification tables for the Diamond & Forrester model and the Pryor model are presented in Tables 4 and 5. The addition of CTCS to Diamond & Forrester resulted in reclassification of 47.2% of patients of whom 73.3% were correctly reclassified. The reclassification calibration statistic (RCS) indicated a strong lack of fit for the Diamond & Forrester model (p < 0.00001) which decreased substantially when CTCS was added to the model (p < 0.01). The NRI (net reclassification improvement) was 33.6% (p < 0.0001) and the IDI (integrated discrimination improvement) was also statistically significant (18.8%, p < 0.001) indicating improvement in the classification of cases and non-cases in probability categories and improvement in discrimination between cases and non-cases. Table 4 Probability of coronary artery disease: reclassification table after addition of CTCS to the Diamond and Forrester (model 1) Probability category based on model 1 Probability category based on model 1 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 51 (67.1) 19 (25.0%) 4 (5.2) 2 (2.6) 76 (29.9) Observed probability, % 11.8 36.8 50.0 100.0 22.4 ≥30–50% N (%) 25 (39.7) 13 (20.6) 19 (30.2) 6 (9.5) 63 (28.4) Observed probability, % 0.0 23.1 68.4 50.0 30.2 ≥50–70% N (%) 4 (11.1) 5 (13.9) 6 (16.7) 21 (58.3) 36 (14.2) Observed probability, % 25 80.0 50.0 66.7 61.1 ≥70% N (%) 5 (6.3) 2 (2.5) 8 (10.1) 64 (81.0) 79 (31.1) Observed probability, % 0.0 50.0 37.5 95.3 82.3 Total N (%) 85 (33.5) 39 (15.4) 37 (14.6) 93 (36.6) 254 (100.0) Observed probability, % 8.2 38.5 56.8 86.0 48.4 CTCS computed tomography coronary calcium score Table 5 Probability of coronary artery disease: reclassification table after addition of CTCS to the model published by Pryor et al. (model 2) Probability category based on model 2 Probability category based on model 2 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 76 (85.4) 10 (11.2) 2 (2.3) 1 (1.1) 89 (35.0) Observed probability, % 9.2 40.0 100.0 100.0 15.7 ≥30–50% N (%) 10 (27.0) 10 (27.0) 12 (32.4) 5 (13.5) 37 (14.6) Observed probability, % 10.0 20.0 50.0 100.0 37.8 ≥50–70% N (%) 7 (13.5) 6 (11.5) 15 (28.9) 24 (46.2) 52 (20.5) Observed probability, % 0.0 50.0 73.3 66.7 57.7 ≥70% N (%) 3 (4.0) 3 (4.0) 9 (11.8) 61 (80.3) 76 (29.9) Observed probability, % 33.3 0.0 66.7 95.0 85.5 Total N (%) 96 (37.8) 29 (11.4) 38 (15.0) 91 (35.8) 254 (100.0) Observed probability, % 9.4 31.0 65.8 87.9 48.4 CTCS computed tomography coronary calcium score
table-wrap	Table 4 Probability of coronary artery disease: reclassification table after addition of CTCS to the Diamond and Forrester (model 1) Probability category based on model 1 Probability category based on model 1 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 51 (67.1) 19 (25.0%) 4 (5.2) 2 (2.6) 76 (29.9) Observed probability, % 11.8 36.8 50.0 100.0 22.4 ≥30–50% N (%) 25 (39.7) 13 (20.6) 19 (30.2) 6 (9.5) 63 (28.4) Observed probability, % 0.0 23.1 68.4 50.0 30.2 ≥50–70% N (%) 4 (11.1) 5 (13.9) 6 (16.7) 21 (58.3) 36 (14.2) Observed probability, % 25 80.0 50.0 66.7 61.1 ≥70% N (%) 5 (6.3) 2 (2.5) 8 (10.1) 64 (81.0) 79 (31.1) Observed probability, % 0.0 50.0 37.5 95.3 82.3 Total N (%) 85 (33.5) 39 (15.4) 37 (14.6) 93 (36.6) 254 (100.0) Observed probability, % 8.2 38.5 56.8 86.0 48.4 CTCS computed tomography coronary calcium score
label	Table 4
caption	Probability of coronary artery disease: reclassification table after addition of CTCS to the Diamond and Forrester (model 1)
p	Probability of coronary artery disease: reclassification table after addition of CTCS to the Diamond and Forrester (model 1)
table	Probability category based on model 1 Probability category based on model 1 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 51 (67.1) 19 (25.0%) 4 (5.2) 2 (2.6) 76 (29.9) Observed probability, % 11.8 36.8 50.0 100.0 22.4 ≥30–50% N (%) 25 (39.7) 13 (20.6) 19 (30.2) 6 (9.5) 63 (28.4) Observed probability, % 0.0 23.1 68.4 50.0 30.2 ≥50–70% N (%) 4 (11.1) 5 (13.9) 6 (16.7) 21 (58.3) 36 (14.2) Observed probability, % 25 80.0 50.0 66.7 61.1 ≥70% N (%) 5 (6.3) 2 (2.5) 8 (10.1) 64 (81.0) 79 (31.1) Observed probability, % 0.0 50.0 37.5 95.3 82.3 Total N (%) 85 (33.5) 39 (15.4) 37 (14.6) 93 (36.6) 254 (100.0) Observed probability, % 8.2 38.5 56.8 86.0 48.4
tr	Probability category based on model 1 Probability category based on model 1 + CTCS Total, n (%)
th	Probability category based on model 1
th	Probability category based on model 1 + CTCS
th	Total, n (%)
tr	<30% ≥30–50% ≥50–70% ≥70%
th	<30%
th	≥30–50%
th	≥50–70%
th	≥70%
tr	<30%
td	<30%
tr	N (%) 51 (67.1) 19 (25.0%) 4 (5.2) 2 (2.6) 76 (29.9)
td	N (%)
td	51 (67.1)
td	19 (25.0%)
td	4 (5.2)
td	2 (2.6)
td	76 (29.9)
tr	Observed probability, % 11.8 36.8 50.0 100.0 22.4
td	Observed probability, %
td	11.8
td	36.8
td	50.0
td	100.0
td	22.4
tr	≥30–50%
td	≥30–50%
tr	N (%) 25 (39.7) 13 (20.6) 19 (30.2) 6 (9.5) 63 (28.4)
td	N (%)
td	25 (39.7)
td	13 (20.6)
td	19 (30.2)
td	6 (9.5)
td	63 (28.4)
tr	Observed probability, % 0.0 23.1 68.4 50.0 30.2
td	Observed probability, %
td	0.0
td	23.1
td	68.4
td	50.0
td	30.2
tr	≥50–70%
td	≥50–70%
tr	N (%) 4 (11.1) 5 (13.9) 6 (16.7) 21 (58.3) 36 (14.2)
td	N (%)
td	4 (11.1)
td	5 (13.9)
td	6 (16.7)
td	21 (58.3)
td	36 (14.2)
tr	Observed probability, % 25 80.0 50.0 66.7 61.1
td	Observed probability, %
td	25
td	80.0
td	50.0
td	66.7
td	61.1
tr	≥70%
td	≥70%
tr	N (%) 5 (6.3) 2 (2.5) 8 (10.1) 64 (81.0) 79 (31.1)
td	N (%)
td	5 (6.3)
td	2 (2.5)
td	8 (10.1)
td	64 (81.0)
td	79 (31.1)
tr	Observed probability, % 0.0 50.0 37.5 95.3 82.3
td	Observed probability, %
td	0.0
td	50.0
td	37.5
td	95.3
td	82.3
tr
td
tr	Total
td	Total
tr	N (%) 85 (33.5) 39 (15.4) 37 (14.6) 93 (36.6) 254 (100.0)
td	N (%)
td	85 (33.5)
td	39 (15.4)
td	37 (14.6)
td	93 (36.6)
td	254 (100.0)
tr	Observed probability, % 8.2 38.5 56.8 86.0 48.4
td	Observed probability, %
td	8.2
td	38.5
td	56.8
td	86.0
td	48.4
table-wrap-foot	CTCS computed tomography coronary calcium score
p	CTCS computed tomography coronary calcium score
table-wrap	Table 5 Probability of coronary artery disease: reclassification table after addition of CTCS to the model published by Pryor et al. (model 2) Probability category based on model 2 Probability category based on model 2 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 76 (85.4) 10 (11.2) 2 (2.3) 1 (1.1) 89 (35.0) Observed probability, % 9.2 40.0 100.0 100.0 15.7 ≥30–50% N (%) 10 (27.0) 10 (27.0) 12 (32.4) 5 (13.5) 37 (14.6) Observed probability, % 10.0 20.0 50.0 100.0 37.8 ≥50–70% N (%) 7 (13.5) 6 (11.5) 15 (28.9) 24 (46.2) 52 (20.5) Observed probability, % 0.0 50.0 73.3 66.7 57.7 ≥70% N (%) 3 (4.0) 3 (4.0) 9 (11.8) 61 (80.3) 76 (29.9) Observed probability, % 33.3 0.0 66.7 95.0 85.5 Total N (%) 96 (37.8) 29 (11.4) 38 (15.0) 91 (35.8) 254 (100.0) Observed probability, % 9.4 31.0 65.8 87.9 48.4 CTCS computed tomography coronary calcium score
label	Table 5
caption	Probability of coronary artery disease: reclassification table after addition of CTCS to the model published by Pryor et al. (model 2)
p	Probability of coronary artery disease: reclassification table after addition of CTCS to the model published by Pryor et al. (model 2)
table	Probability category based on model 2 Probability category based on model 2 + CTCS Total, n (%) <30% ≥30–50% ≥50–70% ≥70% <30% N (%) 76 (85.4) 10 (11.2) 2 (2.3) 1 (1.1) 89 (35.0) Observed probability, % 9.2 40.0 100.0 100.0 15.7 ≥30–50% N (%) 10 (27.0) 10 (27.0) 12 (32.4) 5 (13.5) 37 (14.6) Observed probability, % 10.0 20.0 50.0 100.0 37.8 ≥50–70% N (%) 7 (13.5) 6 (11.5) 15 (28.9) 24 (46.2) 52 (20.5) Observed probability, % 0.0 50.0 73.3 66.7 57.7 ≥70% N (%) 3 (4.0) 3 (4.0) 9 (11.8) 61 (80.3) 76 (29.9) Observed probability, % 33.3 0.0 66.7 95.0 85.5 Total N (%) 96 (37.8) 29 (11.4) 38 (15.0) 91 (35.8) 254 (100.0) Observed probability, % 9.4 31.0 65.8 87.9 48.4
tr	Probability category based on model 2 Probability category based on model 2 + CTCS Total, n (%)
th	Probability category based on model 2
th	Probability category based on model 2 + CTCS
th	Total, n (%)
tr	<30% ≥30–50% ≥50–70% ≥70%
th	<30%
th	≥30–50%
th	≥50–70%
th	≥70%
tr	<30%
td	<30%
tr	N (%) 76 (85.4) 10 (11.2) 2 (2.3) 1 (1.1) 89 (35.0)
td	N (%)
td	76 (85.4)
td	10 (11.2)
td	2 (2.3)
td	1 (1.1)
td	89 (35.0)
tr	Observed probability, % 9.2 40.0 100.0 100.0 15.7
td	Observed probability, %
td	9.2
td	40.0
td	100.0
td	100.0
td	15.7
tr	≥30–50%
td	≥30–50%
tr	N (%) 10 (27.0) 10 (27.0) 12 (32.4) 5 (13.5) 37 (14.6)
td	N (%)
td	10 (27.0)
td	10 (27.0)
td	12 (32.4)
td	5 (13.5)
td	37 (14.6)
tr	Observed probability, % 10.0 20.0 50.0 100.0 37.8
td	Observed probability, %
td	10.0
td	20.0
td	50.0
td	100.0
td	37.8
tr	≥50–70%
td	≥50–70%
tr	N (%) 7 (13.5) 6 (11.5) 15 (28.9) 24 (46.2) 52 (20.5)
td	N (%)
td	7 (13.5)
td	6 (11.5)
td	15 (28.9)
td	24 (46.2)
td	52 (20.5)
tr	Observed probability, % 0.0 50.0 73.3 66.7 57.7
td	Observed probability, %
td	0.0
td	50.0
td	73.3
td	66.7
td	57.7
tr	≥70%
td	≥70%
tr	N (%) 3 (4.0) 3 (4.0) 9 (11.8) 61 (80.3) 76 (29.9)
td	N (%)
td	3 (4.0)
td	3 (4.0)
td	9 (11.8)
td	61 (80.3)
td	76 (29.9)
tr	Observed probability, % 33.3 0.0 66.7 95.0 85.5
td	Observed probability, %
td	33.3
td	0.0
td	66.7
td	95.0
td	85.5
tr
td
tr	Total
td	Total
tr	N (%) 96 (37.8) 29 (11.4) 38 (15.0) 91 (35.8) 254 (100.0)
td	N (%)
td	96 (37.8)
td	29 (11.4)
td	38 (15.0)
td	91 (35.8)
td	254 (100.0)
tr	Observed probability, % 9.4 31.0 65.8 87.9 48.4
td	Observed probability, %
td	9.4
td	31.0
td	65.8
td	87.9
td	48.4
table-wrap-foot	CTCS computed tomography coronary calcium score
p	CTCS computed tomography coronary calcium score
p	For the model by Pryor et al., 36.2% of the patients were reclassified, of whom 54.3% were correctly classified. The RCS indicated a lack of fit (p = 0.01), which decreased when CTCS was added to the model (p = 0.03). The NRI was 24.0% (p < 0.0001) and the IDI was 14.8% (p < 0.001).
p	The reclassification measures for all models are presented in Table 6. Table 6 Reclassification measures obtained by adding CTCS to the existing prediction models Model Reclassification percentages % Correcta Chi-squared excluding CTCSb p value Chi-squared including CTCSc p value NRI, % p value for NRI Reclassification improvement (cases)d Reclassification improvement (non-cases)d IDI, % p value for IDI Overall Reclassified from ≥30–50% Reclassified from ≥50–70% 1 47.2 79.4 83.3 73.3 25.01 <0.00001 10.91 <0.01 33.6 <0.0001 <0.0001 0.23 18.8 <0.001 2 36.2 73.0 71.2 54.3 6.57 0.01 4.70 0.03 24.0 0.001 <0.001 0.31 14.8 <0.001 3 38.2 77.5 75.5 63.9 4.14 0.04 3.31 0.07 21.8 0.005 <0.001 0.58 14.9 <0.001 4 34.3 74.4 74.0 83.9 3.82 0.05 1.58 0.21 24.9 <0.001 <0.001 0.29 13.3 <0.001 5 37.8 77.5 75.9 69.8 3.65 0.06 5.14 0.02 22.6 0.003 <0.001 0.58 14.4 <0.001 CTCS computed tomography coronary calcium score, NRI net reclassification improvement, IDI integrated discrimination improvement aIf the predicted probability of obstructive CAD of the model including CTCS was closer to the observed probability of CAD compared with the prediction of the model without CTCS, the reclassification was considered to be correct bReclassification calibration (Hosmer–Lemeshow) statistic for model without CTCS, using cells from the reclassification table with at least 20 observations cReclassification calibration statistic for models including CTCS, using cells from the reclassification table with at least 20 observations dThe reclassification improvement is defined as the difference in proportions of patients moving up and down for cases and non-cases separately
table-wrap	Table 6 Reclassification measures obtained by adding CTCS to the existing prediction models Model Reclassification percentages % Correcta Chi-squared excluding CTCSb p value Chi-squared including CTCSc p value NRI, % p value for NRI Reclassification improvement (cases)d Reclassification improvement (non-cases)d IDI, % p value for IDI Overall Reclassified from ≥30–50% Reclassified from ≥50–70% 1 47.2 79.4 83.3 73.3 25.01 <0.00001 10.91 <0.01 33.6 <0.0001 <0.0001 0.23 18.8 <0.001 2 36.2 73.0 71.2 54.3 6.57 0.01 4.70 0.03 24.0 0.001 <0.001 0.31 14.8 <0.001 3 38.2 77.5 75.5 63.9 4.14 0.04 3.31 0.07 21.8 0.005 <0.001 0.58 14.9 <0.001 4 34.3 74.4 74.0 83.9 3.82 0.05 1.58 0.21 24.9 <0.001 <0.001 0.29 13.3 <0.001 5 37.8 77.5 75.9 69.8 3.65 0.06 5.14 0.02 22.6 0.003 <0.001 0.58 14.4 <0.001 CTCS computed tomography coronary calcium score, NRI net reclassification improvement, IDI integrated discrimination improvement aIf the predicted probability of obstructive CAD of the model including CTCS was closer to the observed probability of CAD compared with the prediction of the model without CTCS, the reclassification was considered to be correct bReclassification calibration (Hosmer–Lemeshow) statistic for model without CTCS, using cells from the reclassification table with at least 20 observations cReclassification calibration statistic for models including CTCS, using cells from the reclassification table with at least 20 observations dThe reclassification improvement is defined as the difference in proportions of patients moving up and down for cases and non-cases separately
label	Table 6
caption	Reclassification measures obtained by adding CTCS to the existing prediction models
p	Reclassification measures obtained by adding CTCS to the existing prediction models
table	Model Reclassification percentages % Correcta Chi-squared excluding CTCSb p value Chi-squared including CTCSc p value NRI, % p value for NRI Reclassification improvement (cases)d Reclassification improvement (non-cases)d IDI, % p value for IDI Overall Reclassified from ≥30–50% Reclassified from ≥50–70% 1 47.2 79.4 83.3 73.3 25.01 <0.00001 10.91 <0.01 33.6 <0.0001 <0.0001 0.23 18.8 <0.001 2 36.2 73.0 71.2 54.3 6.57 0.01 4.70 0.03 24.0 0.001 <0.001 0.31 14.8 <0.001 3 38.2 77.5 75.5 63.9 4.14 0.04 3.31 0.07 21.8 0.005 <0.001 0.58 14.9 <0.001 4 34.3 74.4 74.0 83.9 3.82 0.05 1.58 0.21 24.9 <0.001 <0.001 0.29 13.3 <0.001 5 37.8 77.5 75.9 69.8 3.65 0.06 5.14 0.02 22.6 0.003 <0.001 0.58 14.4 <0.001
tr	Model Reclassification percentages % Correcta Chi-squared excluding CTCSb p value Chi-squared including CTCSc p value NRI, % p value for NRI Reclassification improvement (cases)d Reclassification improvement (non-cases)d IDI, % p value for IDI
th	Model
th	Reclassification percentages
th	% Correcta
th	Chi-squared excluding CTCSb
th	p value
th	Chi-squared including CTCSc
th	p value
th	NRI, %
th	p value for NRI
th	Reclassification improvement (cases)d
th	Reclassification improvement (non-cases)d
th	IDI, %
th	p value for IDI
tr	Overall Reclassified from ≥30–50% Reclassified from ≥50–70%
th	Overall
th	Reclassified from ≥30–50%
th	Reclassified from ≥50–70%
tr	1 47.2 79.4 83.3 73.3 25.01 <0.00001 10.91 <0.01 33.6 <0.0001 <0.0001 0.23 18.8 <0.001
td	1
td	47.2
td	79.4
td	83.3
td	73.3
td	25.01
td	<0.00001
td	10.91
td	<0.01
td	33.6
td	<0.0001
td	<0.0001
td	0.23
td	18.8
td	<0.001
tr	2 36.2 73.0 71.2 54.3 6.57 0.01 4.70 0.03 24.0 0.001 <0.001 0.31 14.8 <0.001
td	2
td	36.2
td	73.0
td	71.2
td	54.3
td	6.57
td	0.01
td	4.70
td	0.03
td	24.0
td	0.001
td	<0.001
td	0.31
td	14.8
td	<0.001
tr	3 38.2 77.5 75.5 63.9 4.14 0.04 3.31 0.07 21.8 0.005 <0.001 0.58 14.9 <0.001
td	3
td	38.2
td	77.5
td	75.5
td	63.9
td	4.14
td	0.04
td	3.31
td	0.07
td	21.8
td	0.005
td	<0.001
td	0.58
td	14.9
td	<0.001
tr	4 34.3 74.4 74.0 83.9 3.82 0.05 1.58 0.21 24.9 <0.001 <0.001 0.29 13.3 <0.001
td	4
td	34.3
td	74.4
td	74.0
td	83.9
td	3.82
td	0.05
td	1.58
td	0.21
td	24.9
td	<0.001
td	<0.001
td	0.29
td	13.3
td	<0.001
tr	5 37.8 77.5 75.9 69.8 3.65 0.06 5.14 0.02 22.6 0.003 <0.001 0.58 14.4 <0.001
td	5
td	37.8
td	77.5
td	75.9
td	69.8
td	3.65
td	0.06
td	5.14
td	0.02
td	22.6
td	0.003
td	<0.001
td	0.58
td	14.4
td	<0.001
table-wrap-foot	CTCS computed tomography coronary calcium score, NRI net reclassification improvement, IDI integrated discrimination improvement aIf the predicted probability of obstructive CAD of the model including CTCS was closer to the observed probability of CAD compared with the prediction of the model without CTCS, the reclassification was considered to be correct bReclassification calibration (Hosmer–Lemeshow) statistic for model without CTCS, using cells from the reclassification table with at least 20 observations cReclassification calibration statistic for models including CTCS, using cells from the reclassification table with at least 20 observations dThe reclassification improvement is defined as the difference in proportions of patients moving up and down for cases and non-cases separately
p	CTCS computed tomography coronary calcium score, NRI net reclassification improvement, IDI integrated discrimination improvement
p	aIf the predicted probability of obstructive CAD of the model including CTCS was closer to the observed probability of CAD compared with the prediction of the model without CTCS, the reclassification was considered to be correct
p	bReclassification calibration (Hosmer–Lemeshow) statistic for model without CTCS, using cells from the reclassification table with at least 20 observations
p	cReclassification calibration statistic for models including CTCS, using cells from the reclassification table with at least 20 observations
p	dThe reclassification improvement is defined as the difference in proportions of patients moving up and down for cases and non-cases separately
sec	Discussion We analysed the incremental value of CTCS in the prediction of prevalent obstructive CAD. We showed CTCS to be a significant predictor, independent of other variables included in the model. Furthermore, we confirm that the prediction of prevalent obstructive CAD is mainly determined by age, sex and type of chest pain. In four of the five models age and sex were significant predictors before the addition of CTCS. However, after including CTCS, both age and sex were no longer significant predictors in each of those models. Type of chest pain was a significant predictor of obstructive CAD, independent of other variables included in the model and independent of whether CTCS was included or not. CTCS proved to be an excellent and significant predictor of CAD with adjusted odds ratios close to 2. Apart from age and sex, most risk factors did not result in significant odds ratios which suggests that these risk factors are of minor importance in the prediction of prevalent obstructive CAD. Furthermore, the risk factors that were significant in the models without CTCS lost their significance after including CTCS. Analysis of reclassification showed that adding CTCS yields reclassification of 34–47% of patients, most of which was correct. The reclassification calibration statistic demonstrated a lack of fit for all models, which decreased when CTCS was added in all models except for Shaw’s. Numerous studies have previously reported on the incremental value of CTCS in the prediction of cardiovascular events and mortality in asymptomatic individuals [10–13]. Some studies have reported on the incremental value of CTCS in the prediction of prevalent obstructive CAD. Both Guerci et al. [1] and Kennedy et al. [2] studied the relationship between obstructive CAD and CTCS, adjusting for various risk factors. Both studies found highly significant odds ratios for the CTCS in predicting the presence of obstructive CAD, but the odds ratios were lower compared to what we found. Budoff et al. [4] evaluated the value of CTCS in diagnosing CAD and showed that the addition of CTCS increased the AUC from 0.672 to 0.842. However, the models included only age and sex as clinical variables. Considering the fact that the type of chest pain was not taken into account, an increase in AUC with CTCS can be expected. However, the resulting area under the ROC curve for the model including CTCS was similar to what we found. Schmermund et al. [3] previously studied the value of CTCS in predicting the extent of CAD. They showed an independent and incremental value for CTCS in multiple linear regression in predicting the total number of segments per patient with ≥50% stenosis. Study limitations Our study assessed the prediction of ≥50% stenosis in at least one vessel. One could argue that physicians are primarily interested in diagnosing severe CAD, as these patients would be eligible for revascularisation whereas others can be adequately treated medically. Likewise, physicians might be primarily interested in predicting future cardiovascular adverse events. However, we did not consider prognosis in this analysis. All patients in our study were referred for CCA based on their presentation or functional testing that suggested the presence of cardiac ischaemia. In this way a high-risk population was selected, which could have biased our results. Unfortunately this limitation is inherent to the study design. Further research is necessary to determine the value of CTCS in other (e.g. lower risk) populations. Also, risk factors such as type of chest pain, smoking status and family history of CAD were obtained by interviewing the patient. Potentially, this method underestimates their predictive effects as compared to the predictive effect of the CTCS, which was directly measured. Tables 4 and 5 illustrate how CTCS influences the classification of patients in probability categories. However, the limitations of reclassification measures in the context of this research should be taken into account. Our sample size was too small to reliably assess reclassification. For example, the RCS only uses the cross-classified cells containing at least 20 observations. In Table 4 only four cells contain 20 or more observations, implying that a substantial amount of (correctly) reclassified patients are ignored. Thus, in our study the reclassification percentages, NRI and IDI indices are more reliable than the RCS. Ideally, the probability categories should be based on clinically relevant cutoffs. However, no well-established clinically relevant probability threshold exists. The probability of CAD is commonly defined as low (<30%), intermediate (≥30–70%) and high (≥70%) [26]. In our view the intermediate category is rather wide, which is why we divided this category into low–intermediate (≥30–50%) and intermediate–high (≥50–70%). It should be noted that the overall percentage of reclassification is highly dependent on the choice and number of probability categories. Clinical implications Our results demonstrate that the estimation of the probability of obstructive CAD can be improved by including CTCS. This implies that clinicians can make better decisions as to whether a particular patient would benefit from further testing, for example CTCA or CCA. In low-risk patients, a CTCS of 0 could exclude CAD and avoid further testing using CTCA. Hereby, one also avoids the intravenous administration of contrast agent, the extra radiation exposure, and extra scan time and costs associated with CTCA. In patients with a low CTCS, CCA can be avoided and further non-invasive testing would be preferred. In patients with an intermediate CTCS, a CTCA might be the optimal next step. In patients with a high CTCS, direct CCA might be justified because of the high probability of CAD. All in all, CTCS could be useful as a triage test for patients who are suspected of having CAD. We confirmed that the prediction of significant CAD is primarily driven by the patient’s symptoms. A detailed history of the patient’s symptoms remains most important in the diagnostic work-up of patients with suspected CAD. However, history taking is difficult and subjective, therefore limiting our ability to accurately predict the presence of CAD. Hence, further diagnostic testing will be important, even in patients with a low to intermediate probability of CAD. On the other hand, the harms and costs of obtaining CTCS should be considered. Kim [27] studied the radiation dose and cancer risk of CTCS screening (every 5 years) in asymptomatic individuals. They concluded that the excess lifetime cancer risk was 42 (62) per 100,000 men (women). It is important to note that our study assessed the value of a single CTCS in symptomatic patients, for whom the excess lifetime cancer risk will be lower and small compared with the risk of missing a CAD diagnosis. Moreover, CTCS could reduce the use of additional testing in patients with a low CTCS and a low probability of CAD, thereby reducing the total radiation exposure. Although performing a CTCS measurement is a fast, low-dose and relatively inexpensive procedure, the harms and benefits should be considered in a cost-effectiveness analysis.
title	Discussion
p	We analysed the incremental value of CTCS in the prediction of prevalent obstructive CAD. We showed CTCS to be a significant predictor, independent of other variables included in the model. Furthermore, we confirm that the prediction of prevalent obstructive CAD is mainly determined by age, sex and type of chest pain.
p	In four of the five models age and sex were significant predictors before the addition of CTCS. However, after including CTCS, both age and sex were no longer significant predictors in each of those models. Type of chest pain was a significant predictor of obstructive CAD, independent of other variables included in the model and independent of whether CTCS was included or not. CTCS proved to be an excellent and significant predictor of CAD with adjusted odds ratios close to 2. Apart from age and sex, most risk factors did not result in significant odds ratios which suggests that these risk factors are of minor importance in the prediction of prevalent obstructive CAD. Furthermore, the risk factors that were significant in the models without CTCS lost their significance after including CTCS.
p	Analysis of reclassification showed that adding CTCS yields reclassification of 34–47% of patients, most of which was correct. The reclassification calibration statistic demonstrated a lack of fit for all models, which decreased when CTCS was added in all models except for Shaw’s.
p	Numerous studies have previously reported on the incremental value of CTCS in the prediction of cardiovascular events and mortality in asymptomatic individuals [10–13]. Some studies have reported on the incremental value of CTCS in the prediction of prevalent obstructive CAD.
p	Both Guerci et al. [1] and Kennedy et al. [2] studied the relationship between obstructive CAD and CTCS, adjusting for various risk factors. Both studies found highly significant odds ratios for the CTCS in predicting the presence of obstructive CAD, but the odds ratios were lower compared to what we found. Budoff et al. [4] evaluated the value of CTCS in diagnosing CAD and showed that the addition of CTCS increased the AUC from 0.672 to 0.842. However, the models included only age and sex as clinical variables. Considering the fact that the type of chest pain was not taken into account, an increase in AUC with CTCS can be expected. However, the resulting area under the ROC curve for the model including CTCS was similar to what we found. Schmermund et al. [3] previously studied the value of CTCS in predicting the extent of CAD. They showed an independent and incremental value for CTCS in multiple linear regression in predicting the total number of segments per patient with ≥50% stenosis.
sec	Study limitations Our study assessed the prediction of ≥50% stenosis in at least one vessel. One could argue that physicians are primarily interested in diagnosing severe CAD, as these patients would be eligible for revascularisation whereas others can be adequately treated medically. Likewise, physicians might be primarily interested in predicting future cardiovascular adverse events. However, we did not consider prognosis in this analysis. All patients in our study were referred for CCA based on their presentation or functional testing that suggested the presence of cardiac ischaemia. In this way a high-risk population was selected, which could have biased our results. Unfortunately this limitation is inherent to the study design. Further research is necessary to determine the value of CTCS in other (e.g. lower risk) populations. Also, risk factors such as type of chest pain, smoking status and family history of CAD were obtained by interviewing the patient. Potentially, this method underestimates their predictive effects as compared to the predictive effect of the CTCS, which was directly measured. Tables 4 and 5 illustrate how CTCS influences the classification of patients in probability categories. However, the limitations of reclassification measures in the context of this research should be taken into account. Our sample size was too small to reliably assess reclassification. For example, the RCS only uses the cross-classified cells containing at least 20 observations. In Table 4 only four cells contain 20 or more observations, implying that a substantial amount of (correctly) reclassified patients are ignored. Thus, in our study the reclassification percentages, NRI and IDI indices are more reliable than the RCS. Ideally, the probability categories should be based on clinically relevant cutoffs. However, no well-established clinically relevant probability threshold exists. The probability of CAD is commonly defined as low (<30%), intermediate (≥30–70%) and high (≥70%) [26]. In our view the intermediate category is rather wide, which is why we divided this category into low–intermediate (≥30–50%) and intermediate–high (≥50–70%). It should be noted that the overall percentage of reclassification is highly dependent on the choice and number of probability categories.
title	Study limitations
p	Our study assessed the prediction of ≥50% stenosis in at least one vessel. One could argue that physicians are primarily interested in diagnosing severe CAD, as these patients would be eligible for revascularisation whereas others can be adequately treated medically. Likewise, physicians might be primarily interested in predicting future cardiovascular adverse events. However, we did not consider prognosis in this analysis.
p	All patients in our study were referred for CCA based on their presentation or functional testing that suggested the presence of cardiac ischaemia. In this way a high-risk population was selected, which could have biased our results. Unfortunately this limitation is inherent to the study design. Further research is necessary to determine the value of CTCS in other (e.g. lower risk) populations.
p	Also, risk factors such as type of chest pain, smoking status and family history of CAD were obtained by interviewing the patient. Potentially, this method underestimates their predictive effects as compared to the predictive effect of the CTCS, which was directly measured.
p	Tables 4 and 5 illustrate how CTCS influences the classification of patients in probability categories. However, the limitations of reclassification measures in the context of this research should be taken into account. Our sample size was too small to reliably assess reclassification. For example, the RCS only uses the cross-classified cells containing at least 20 observations. In Table 4 only four cells contain 20 or more observations, implying that a substantial amount of (correctly) reclassified patients are ignored. Thus, in our study the reclassification percentages, NRI and IDI indices are more reliable than the RCS.
p	Ideally, the probability categories should be based on clinically relevant cutoffs. However, no well-established clinically relevant probability threshold exists. The probability of CAD is commonly defined as low (<30%), intermediate (≥30–70%) and high (≥70%) [26]. In our view the intermediate category is rather wide, which is why we divided this category into low–intermediate (≥30–50%) and intermediate–high (≥50–70%). It should be noted that the overall percentage of reclassification is highly dependent on the choice and number of probability categories.
sec	Clinical implications Our results demonstrate that the estimation of the probability of obstructive CAD can be improved by including CTCS. This implies that clinicians can make better decisions as to whether a particular patient would benefit from further testing, for example CTCA or CCA. In low-risk patients, a CTCS of 0 could exclude CAD and avoid further testing using CTCA. Hereby, one also avoids the intravenous administration of contrast agent, the extra radiation exposure, and extra scan time and costs associated with CTCA. In patients with a low CTCS, CCA can be avoided and further non-invasive testing would be preferred. In patients with an intermediate CTCS, a CTCA might be the optimal next step. In patients with a high CTCS, direct CCA might be justified because of the high probability of CAD. All in all, CTCS could be useful as a triage test for patients who are suspected of having CAD. We confirmed that the prediction of significant CAD is primarily driven by the patient’s symptoms. A detailed history of the patient’s symptoms remains most important in the diagnostic work-up of patients with suspected CAD. However, history taking is difficult and subjective, therefore limiting our ability to accurately predict the presence of CAD. Hence, further diagnostic testing will be important, even in patients with a low to intermediate probability of CAD. On the other hand, the harms and costs of obtaining CTCS should be considered. Kim [27] studied the radiation dose and cancer risk of CTCS screening (every 5 years) in asymptomatic individuals. They concluded that the excess lifetime cancer risk was 42 (62) per 100,000 men (women). It is important to note that our study assessed the value of a single CTCS in symptomatic patients, for whom the excess lifetime cancer risk will be lower and small compared with the risk of missing a CAD diagnosis. Moreover, CTCS could reduce the use of additional testing in patients with a low CTCS and a low probability of CAD, thereby reducing the total radiation exposure. Although performing a CTCS measurement is a fast, low-dose and relatively inexpensive procedure, the harms and benefits should be considered in a cost-effectiveness analysis.
title	Clinical implications
p	Our results demonstrate that the estimation of the probability of obstructive CAD can be improved by including CTCS. This implies that clinicians can make better decisions as to whether a particular patient would benefit from further testing, for example CTCA or CCA. In low-risk patients, a CTCS of 0 could exclude CAD and avoid further testing using CTCA. Hereby, one also avoids the intravenous administration of contrast agent, the extra radiation exposure, and extra scan time and costs associated with CTCA. In patients with a low CTCS, CCA can be avoided and further non-invasive testing would be preferred. In patients with an intermediate CTCS, a CTCA might be the optimal next step. In patients with a high CTCS, direct CCA might be justified because of the high probability of CAD. All in all, CTCS could be useful as a triage test for patients who are suspected of having CAD.
p	We confirmed that the prediction of significant CAD is primarily driven by the patient’s symptoms. A detailed history of the patient’s symptoms remains most important in the diagnostic work-up of patients with suspected CAD. However, history taking is difficult and subjective, therefore limiting our ability to accurately predict the presence of CAD. Hence, further diagnostic testing will be important, even in patients with a low to intermediate probability of CAD.
p	On the other hand, the harms and costs of obtaining CTCS should be considered. Kim [27] studied the radiation dose and cancer risk of CTCS screening (every 5 years) in asymptomatic individuals. They concluded that the excess lifetime cancer risk was 42 (62) per 100,000 men (women). It is important to note that our study assessed the value of a single CTCS in symptomatic patients, for whom the excess lifetime cancer risk will be lower and small compared with the risk of missing a CAD diagnosis. Moreover, CTCS could reduce the use of additional testing in patients with a low CTCS and a low probability of CAD, thereby reducing the total radiation exposure.
p	Although performing a CTCS measurement is a fast, low-dose and relatively inexpensive procedure, the harms and benefits should be considered in a cost-effectiveness analysis.
sec	Conclusion Our results suggest that CTCS significantly improves the prediction of prevalent obstructive CAD, independent of other clinical variables. Therefore, CTCS should be considered in the diagnostic work-up of CAD.
title	Conclusion
p	Our results suggest that CTCS significantly improves the prediction of prevalent obstructive CAD, independent of other clinical variables. Therefore, CTCS should be considered in the diagnostic work-up of CAD.
back	Appendix Studies were included if they met all of the following criteria: (1) Study population were patients with chest pain suggestive of stable angina pectoris, (2) study reported a new multivariate prediction rule that included clinical variables, (3) outcome was the presence of significant CAD defined as ≥50% stenosis in at least one vessel. Studies were excluded if they met one of the following criteria: (1) Article was a review, guideline, or cost-effectiveness analysis, or (2) study did not report a prediction model, or (3) the outcome of interest was prognostic (e.g. event rate after treatment) or the authors did not use ≥50% stenosis in at least one vessel as their outcome, or (4) study population consisted of unstable patients, asymptomatic patients, patients with known CAD or patients with a specific co-morbidity, or (5) main topic was a diagnostic (imaging) test, or (6) a reference standard other than conventional coronary angiography was used, or (7) authors did not use the traditional classification of typical and atypical chest pain, or (8) the study focused on the association between one particular risk factor and CAD. Articles not accessible were excluded from the analysis. Search strategy Coronary artery disease [MeSH] Coronary heart disease [MeSH] Coronary stenosis [MeSH] Coronary disease #1 OR #2 OR 3# OR 4 Chest pain [MeSH] Angina pectoris [MeSH] Suspected [title/abstract] #6 OR #7 OR #8 Logistic models [MeSH] Probability [MeSH] Risk [MeSH] Models, statistical [MeSH] ROC curve [MeSH] #10 OR #11 OR #12 OR #13 OR #14 Medical history taking [MeSH] Physical examination [MeSH] Clinical [title/abstract] #16 OR #17 OR #18 #5 AND #9 AND #15 AND #19 Animals [MeSH] NOT humans [MeSH] Editorial [publication type] Comment [publication type] Letter [publication type] Meta-analysis [publication type] Case reports [publication type]) #22 OR #23 OR #24 OR #25 OR #26 Acute [title/abstract] #20 NOT #21 NOT #27 NOT #28 Acknowledgements This study was funded by the Health Care Efficiency Research grant (number 945–04–263) from the Netherlands Organisation for Health Research and Development, a Dutch governmental organisation, and by internal funding through a Health Care Efficiency grant from the Erasmus University Medical Center, Rotterdam. The authors’ work was independent of the funding organisations. The funding organisations had no involvement in the design or conduct of this study; data management and analysis; or manuscript preparation and review or authorisation for submission. Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
appendix	Appendix Studies were included if they met all of the following criteria: (1) Study population were patients with chest pain suggestive of stable angina pectoris, (2) study reported a new multivariate prediction rule that included clinical variables, (3) outcome was the presence of significant CAD defined as ≥50% stenosis in at least one vessel. Studies were excluded if they met one of the following criteria: (1) Article was a review, guideline, or cost-effectiveness analysis, or (2) study did not report a prediction model, or (3) the outcome of interest was prognostic (e.g. event rate after treatment) or the authors did not use ≥50% stenosis in at least one vessel as their outcome, or (4) study population consisted of unstable patients, asymptomatic patients, patients with known CAD or patients with a specific co-morbidity, or (5) main topic was a diagnostic (imaging) test, or (6) a reference standard other than conventional coronary angiography was used, or (7) authors did not use the traditional classification of typical and atypical chest pain, or (8) the study focused on the association between one particular risk factor and CAD. Articles not accessible were excluded from the analysis. Search strategy Coronary artery disease [MeSH] Coronary heart disease [MeSH] Coronary stenosis [MeSH] Coronary disease #1 OR #2 OR 3# OR 4 Chest pain [MeSH] Angina pectoris [MeSH] Suspected [title/abstract] #6 OR #7 OR #8 Logistic models [MeSH] Probability [MeSH] Risk [MeSH] Models, statistical [MeSH] ROC curve [MeSH] #10 OR #11 OR #12 OR #13 OR #14 Medical history taking [MeSH] Physical examination [MeSH] Clinical [title/abstract] #16 OR #17 OR #18 #5 AND #9 AND #15 AND #19 Animals [MeSH] NOT humans [MeSH] Editorial [publication type] Comment [publication type] Letter [publication type] Meta-analysis [publication type] Case reports [publication type]) #22 OR #23 OR #24 OR #25 OR #26 Acute [title/abstract] #20 NOT #21 NOT #27 NOT #28
sec	Appendix Studies were included if they met all of the following criteria: (1) Study population were patients with chest pain suggestive of stable angina pectoris, (2) study reported a new multivariate prediction rule that included clinical variables, (3) outcome was the presence of significant CAD defined as ≥50% stenosis in at least one vessel. Studies were excluded if they met one of the following criteria: (1) Article was a review, guideline, or cost-effectiveness analysis, or (2) study did not report a prediction model, or (3) the outcome of interest was prognostic (e.g. event rate after treatment) or the authors did not use ≥50% stenosis in at least one vessel as their outcome, or (4) study population consisted of unstable patients, asymptomatic patients, patients with known CAD or patients with a specific co-morbidity, or (5) main topic was a diagnostic (imaging) test, or (6) a reference standard other than conventional coronary angiography was used, or (7) authors did not use the traditional classification of typical and atypical chest pain, or (8) the study focused on the association between one particular risk factor and CAD. Articles not accessible were excluded from the analysis. Search strategy Coronary artery disease [MeSH] Coronary heart disease [MeSH] Coronary stenosis [MeSH] Coronary disease #1 OR #2 OR 3# OR 4 Chest pain [MeSH] Angina pectoris [MeSH] Suspected [title/abstract] #6 OR #7 OR #8 Logistic models [MeSH] Probability [MeSH] Risk [MeSH] Models, statistical [MeSH] ROC curve [MeSH] #10 OR #11 OR #12 OR #13 OR #14 Medical history taking [MeSH] Physical examination [MeSH] Clinical [title/abstract] #16 OR #17 OR #18 #5 AND #9 AND #15 AND #19 Animals [MeSH] NOT humans [MeSH] Editorial [publication type] Comment [publication type] Letter [publication type] Meta-analysis [publication type] Case reports [publication type]) #22 OR #23 OR #24 OR #25 OR #26 Acute [title/abstract] #20 NOT #21 NOT #27 NOT #28
title	Appendix
p	Studies were included if they met all of the following criteria: (1) Study population were patients with chest pain suggestive of stable angina pectoris, (2) study reported a new multivariate prediction rule that included clinical variables, (3) outcome was the presence of significant CAD defined as ≥50% stenosis in at least one vessel. Studies were excluded if they met one of the following criteria: (1) Article was a review, guideline, or cost-effectiveness analysis, or (2) study did not report a prediction model, or (3) the outcome of interest was prognostic (e.g. event rate after treatment) or the authors did not use ≥50% stenosis in at least one vessel as their outcome, or (4) study population consisted of unstable patients, asymptomatic patients, patients with known CAD or patients with a specific co-morbidity, or (5) main topic was a diagnostic (imaging) test, or (6) a reference standard other than conventional coronary angiography was used, or (7) authors did not use the traditional classification of typical and atypical chest pain, or (8) the study focused on the association between one particular risk factor and CAD. Articles not accessible were excluded from the analysis.
sec	Search strategy Coronary artery disease [MeSH] Coronary heart disease [MeSH] Coronary stenosis [MeSH] Coronary disease #1 OR #2 OR 3# OR 4 Chest pain [MeSH] Angina pectoris [MeSH] Suspected [title/abstract] #6 OR #7 OR #8 Logistic models [MeSH] Probability [MeSH] Risk [MeSH] Models, statistical [MeSH] ROC curve [MeSH] #10 OR #11 OR #12 OR #13 OR #14 Medical history taking [MeSH] Physical examination [MeSH] Clinical [title/abstract] #16 OR #17 OR #18 #5 AND #9 AND #15 AND #19 Animals [MeSH] NOT humans [MeSH] Editorial [publication type] Comment [publication type] Letter [publication type] Meta-analysis [publication type] Case reports [publication type]) #22 OR #23 OR #24 OR #25 OR #26 Acute [title/abstract] #20 NOT #21 NOT #27 NOT #28
title	Search strategy
p	Coronary artery disease [MeSH] Coronary heart disease [MeSH] Coronary stenosis [MeSH] Coronary disease #1 OR #2 OR 3# OR 4 Chest pain [MeSH] Angina pectoris [MeSH] Suspected [title/abstract] #6 OR #7 OR #8 Logistic models [MeSH] Probability [MeSH] Risk [MeSH] Models, statistical [MeSH] ROC curve [MeSH] #10 OR #11 OR #12 OR #13 OR #14 Medical history taking [MeSH] Physical examination [MeSH] Clinical [title/abstract] #16 OR #17 OR #18 #5 AND #9 AND #15 AND #19 Animals [MeSH] NOT humans [MeSH] Editorial [publication type] Comment [publication type] Letter [publication type] Meta-analysis [publication type] Case reports [publication type]) #22 OR #23 OR #24 OR #25 OR #26 Acute [title/abstract] #20 NOT #21 NOT #27 NOT #28
p	Coronary artery disease [MeSH]
p	Coronary heart disease [MeSH]
p	Coronary stenosis [MeSH]
p	Coronary disease
p	#1 OR #2 OR 3# OR 4
p	Chest pain [MeSH]
p	Angina pectoris [MeSH]
p	Suspected [title/abstract]
p	#6 OR #7 OR #8
p	Logistic models [MeSH]
p	Probability [MeSH]
p	Risk [MeSH]
p	Models, statistical [MeSH]
p	ROC curve [MeSH]
p	#10 OR #11 OR #12 OR #13 OR #14
p	Medical history taking [MeSH]
p	Physical examination [MeSH]
p	Clinical [title/abstract]
p	#16 OR #17 OR #18
p	#5 AND #9 AND #15 AND #19
p	Animals [MeSH] NOT humans [MeSH]
p	Editorial [publication type]
p	Comment [publication type]
p	Letter [publication type]
p	Meta-analysis [publication type]
p	Case reports [publication type])
p	#22 OR #23 OR #24 OR #25 OR #26
p	Acute [title/abstract]
p	#20 NOT #21 NOT #27 NOT #28
ack	Acknowledgements This study was funded by the Health Care Efficiency Research grant (number 945–04–263) from the Netherlands Organisation for Health Research and Development, a Dutch governmental organisation, and by internal funding through a Health Care Efficiency grant from the Erasmus University Medical Center, Rotterdam. The authors’ work was independent of the funding organisations. The funding organisations had no involvement in the design or conduct of this study; data management and analysis; or manuscript preparation and review or authorisation for submission. Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
title	Acknowledgements
p	This study was funded by the Health Care Efficiency Research grant (number 945–04–263) from the Netherlands Organisation for Health Research and Development, a Dutch governmental organisation, and by internal funding through a Health Care Efficiency grant from the Erasmus University Medical Center, Rotterdam. The authors’ work was independent of the funding organisations. The funding organisations had no involvement in the design or conduct of this study; data management and analysis; or manuscript preparation and review or authorisation for submission.
sec	Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
title	Open Access
p	This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

Annnotations TAB TSV DIC JSON TextAE

last updated at 2021-11-04 12:53:12 UTC

Denotations: 0
Blocks: 0
Relations: 0

PMC:2940023 / 13246-13248 JSONTXT

Document structure show

Annnotations TAB TSV DIC JSON TextAE

PMC:2940023 / 13246-13248 JSON TXT