|
article-title
|
Rhetoric or Rhetoric: Interpreting Cross-Sectional Data When There Are Disparate Control Groups
|
|
body
|
To the Editor—As the inventors and manufacturers of the probiotic Bio-K+ comprising Lactobacillus acidophilus CL1285, Lactobacillus casei LBC80R, and Lactobacillus rhamnosus CLR2, we read with interest the recent Brief Report by Dr. Box and co-authors documenting their practical experience with the product in primary Clostridioides (Clostridium) difficile infection (CDI) prevention [1]. Though this retrospective, cross-sectional study at Scripps Memorial Hospital La Jolla (California) is a useful addition to the body of evidence describing this probiotic [2], it was not designed to determine that the incidence of CDI was reduced.
With proper controls, a cross-sectional study is an acceptable design to compare outcomes between equivalent groups of patients. We read in Box et al. that the hospital aimed to administer Bio-K+ daily to adults taking antibiotics without bias or treatment allocation. Fewer than half of the eligible patients, 41%, received 1 or more doses of the probiotic during their antibiotic treatment. This uncontrolled split of the patients was the basis for the statistical comparisons. However, the comparison groups were statistically different. As indicated by the authors, the probiotic-treated patients were exposed to disproportionately higher doses of antibiotics (24 g vs 10 g; P < .0001) and had a higher burden of illness (Charlson comorbidity index, 4.6 vs 4.1; P = .011). It is concerning to see more cases of nosocomial CDI in these higher-risk patients treated with the probiotic, but that is to be expected. Antibiotic use and frailty are the principal predictors of developing CDI [3]. Without adequate comparison groups, it is not possible to determine from cross-sectional data that there was “no impact of probiotics to reduce Clostridium difficile infection.”
Rather than a cross-sectional study design, the convention to quantify primary prevention of disease is a longitudinal design. Fortunately, the Centers for Medicare and Medicaid Services require all participating hospitals in the United States to submit longitudinal data on nosocomial CDI and compare them with a customized benchmark for the predicted number of cases [4]. Data from many hospitals, including Scripps Memorial Hospital La Jolla, are available in the public domain from 2013 through 2017 [5, 6]. In the years preceding the use of Bio-K+, this hospital’s rate of nosocomial CDI increased, relative to previous years and relative to the predicted benchmark. Though Box et al. inform the reader that probiotics were removed from the hospital’s formulary, this probiotic was administered to adults taking antibiotics from 2016 through 2017. During those years, the rate of hospital-onset C. difficile infection was markedly reduced relative to previous years (Figure 1).
Figure 1. The rates of observed (solid line) and predicted (dashed line) nosocomial Clostridioides (Clostridium) difficile infection (CDI) expressed in cases per 10 000 patient-days at Scripps Memorial Hospital La Jolla, as reported to the Centers for Medicaid and Medicare from 2013 through 2017 [5, 6]. Bio-K+ was used at the hospital in 2016 and 2017, as indicated with a check mark below those years. The authors, which include the Scripps Antimicrobial Stewardship Program (SASP), seem to endorse “strong antimicrobial stewardship practices” instead of a probiotic. CDI prevention is not a simple endeavor, and high-performing hospitals employ a bundle of complimentary approaches to combat this deadly threat [7]. Hand washing, surface disinfection, and patient isolation limit the spread of spores within the hospital. Bio-K+ and other living microbial therapies serve a distinct purpose: to increase a patient’s resilience while taking antibiotics.
Without the proper controls, or statistical plan, quantifying primary disease prevention with cross-sectional data has many limitations, and the null hypothesis becomes a certainty. Even more concerning, an inappropriate study design, like the one described herein, could entirely miss a period of rapidly falling incidence of CDI. It is not possible to surmise which factors led to a markedly reduced number of C. difficile infections at the hospital during the intervention period. We simply disagree that, as the title of the Brief Report suggests, the authors have demonstrated that using this particular probiotic for disease prevention had “no impact.”
|
|
p
|
To the Editor—As the inventors and manufacturers of the probiotic Bio-K+ comprising Lactobacillus acidophilus CL1285, Lactobacillus casei LBC80R, and Lactobacillus rhamnosus CLR2, we read with interest the recent Brief Report by Dr. Box and co-authors documenting their practical experience with the product in primary Clostridioides (Clostridium) difficile infection (CDI) prevention [1]. Though this retrospective, cross-sectional study at Scripps Memorial Hospital La Jolla (California) is a useful addition to the body of evidence describing this probiotic [2], it was not designed to determine that the incidence of CDI was reduced.
|
|
p
|
With proper controls, a cross-sectional study is an acceptable design to compare outcomes between equivalent groups of patients. We read in Box et al. that the hospital aimed to administer Bio-K+ daily to adults taking antibiotics without bias or treatment allocation. Fewer than half of the eligible patients, 41%, received 1 or more doses of the probiotic during their antibiotic treatment. This uncontrolled split of the patients was the basis for the statistical comparisons. However, the comparison groups were statistically different. As indicated by the authors, the probiotic-treated patients were exposed to disproportionately higher doses of antibiotics (24 g vs 10 g; P < .0001) and had a higher burden of illness (Charlson comorbidity index, 4.6 vs 4.1; P = .011). It is concerning to see more cases of nosocomial CDI in these higher-risk patients treated with the probiotic, but that is to be expected. Antibiotic use and frailty are the principal predictors of developing CDI [3]. Without adequate comparison groups, it is not possible to determine from cross-sectional data that there was “no impact of probiotics to reduce Clostridium difficile infection.”
|
|
p
|
Rather than a cross-sectional study design, the convention to quantify primary prevention of disease is a longitudinal design. Fortunately, the Centers for Medicare and Medicaid Services require all participating hospitals in the United States to submit longitudinal data on nosocomial CDI and compare them with a customized benchmark for the predicted number of cases [4]. Data from many hospitals, including Scripps Memorial Hospital La Jolla, are available in the public domain from 2013 through 2017 [5, 6]. In the years preceding the use of Bio-K+, this hospital’s rate of nosocomial CDI increased, relative to previous years and relative to the predicted benchmark. Though Box et al. inform the reader that probiotics were removed from the hospital’s formulary, this probiotic was administered to adults taking antibiotics from 2016 through 2017. During those years, the rate of hospital-onset C. difficile infection was markedly reduced relative to previous years (Figure 1).
|
|
figure
|
Figure 1. The rates of observed (solid line) and predicted (dashed line) nosocomial Clostridioides (Clostridium) difficile infection (CDI) expressed in cases per 10 000 patient-days at Scripps Memorial Hospital La Jolla, as reported to the Centers for Medicaid and Medicare from 2013 through 2017 [5, 6]. Bio-K+ was used at the hospital in 2016 and 2017, as indicated with a check mark below those years.
|
|
label
|
Figure 1.
|
|
caption
|
The rates of observed (solid line) and predicted (dashed line) nosocomial Clostridioides (Clostridium) difficile infection (CDI) expressed in cases per 10 000 patient-days at Scripps Memorial Hospital La Jolla, as reported to the Centers for Medicaid and Medicare from 2013 through 2017 [5, 6]. Bio-K+ was used at the hospital in 2016 and 2017, as indicated with a check mark below those years.
|
|
p
|
The rates of observed (solid line) and predicted (dashed line) nosocomial Clostridioides (Clostridium) difficile infection (CDI) expressed in cases per 10 000 patient-days at Scripps Memorial Hospital La Jolla, as reported to the Centers for Medicaid and Medicare from 2013 through 2017 [5, 6]. Bio-K+ was used at the hospital in 2016 and 2017, as indicated with a check mark below those years.
|
|
p
|
The authors, which include the Scripps Antimicrobial Stewardship Program (SASP), seem to endorse “strong antimicrobial stewardship practices” instead of a probiotic. CDI prevention is not a simple endeavor, and high-performing hospitals employ a bundle of complimentary approaches to combat this deadly threat [7]. Hand washing, surface disinfection, and patient isolation limit the spread of spores within the hospital. Bio-K+ and other living microbial therapies serve a distinct purpose: to increase a patient’s resilience while taking antibiotics.
|
|
p
|
Without the proper controls, or statistical plan, quantifying primary disease prevention with cross-sectional data has many limitations, and the null hypothesis becomes a certainty. Even more concerning, an inappropriate study design, like the one described herein, could entirely miss a period of rapidly falling incidence of CDI. It is not possible to surmise which factors led to a markedly reduced number of C. difficile infections at the hospital during the intervention period. We simply disagree that, as the title of the Brief Report suggests, the authors have demonstrated that using this particular probiotic for disease prevention had “no impact.”
|
|
back
|
Acknowledgments
Potential conflicts of interest. All authors: no reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
|
|
ack
|
Acknowledgments
Potential conflicts of interest. All authors: no reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
|
|
title
|
Acknowledgments
|
|
p
|
Potential conflicts of interest. All authors: no reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
|
