Prospective Evaluation of a Machine Learning Approach to Predicting Absence of Serious Bacterial Infection
Blake Martin, MD
BACKGROUND: Over a third of children admitted to the PICU are diagnosed with serious bacterial infections (SBIs). It is thus unsurprising that PICU providers often broadly prescribe antibiotics to their critically ill pediatric patients. Unfortunately, this leads to many children receiving broad-spectrum antibiotics while microbiologic studies are pending and puts them at risk for antibiotic-associated adverse effects such as nephrotoxicity, clostridium difficile colitis, and prolonged hospital length of stay. GAP: There are no clinically validated predictive models which utilize readily-available electronic health record (EHR) data to predict which children admitted to the PICU are at low risk for SBI and for whom unnecessary antibiotic exposure could be minimized. HYPOTHESIS: We hypothesize that silent, prospective validation of our two previously developed machine learning-based SBI predictive models (which utilize readily-available EHR inputs such as vital signs and lab values) will demonstrate negative predictive values ≥95% for ruling out SBI, will correctly identify ≥20% of the SBI-negative children who are subsequently given unnecessary antibiotics, and will demonstrate the potential to spare these children a median of ≥3.0 antibiotic-days per patient. We also hypothesize that a pilot qualitative needs assessment using PICU clinician focus groups will identify key themes able to inform future development of a trusted and actionable, model-based antibiotic decision-making clinical decision support (CDS) tool. METHODS: We will perform a prospective cohort study of children 3-months-to-18-years-old admitted to our PICU over a 1-year period during which we prospectively evaluate in silent fashion (model results not surfaced to clinicians) the performance of two previously developed ML-based predictive models in identifying SBI-negative children for whom antibiotics could potentially have been reduced. We will also perform two focus groups of multidisciplinary PICU care providers involved in antibiotic decision-making (nurses, physicians, and advanced practice providers) to discuss the decisional needs of end-users of a future CDS tool based on these SBI predictive models. RESULTS: Pending. IMPACT: This project represents a paradigm shift in personalized antibiotic decision-making. If successful, we will have laid the groundwork for subsequent use of machine learning-based predictive models able to aid PICU clinicians in minimizing unnecessary antibiotic exposure and associated adverse effects. Optional/Additional Comments: Our definition of serious bacterial infection (SBI) is adapted from the National Healthcare Safety Network and includes positive culture from any normally sterile body fluid/site (e.g. bacteremia, meningitis, pyelonephritis) as well as certain culture-negative conditions (culture negative septic shock, and pneumonia diagnosed via chest radiograph). The machine learning predictive models we will be prospectively evaluating include a Random Forest model and a ridge penalized logistic regression model. Each uses 100+ EHR data element inputs with imputation of missing values. Predictive models will produce SBI-status predictions every 6 hours throughout the entirety of a child’s PICU stay.