Reducing Alert Burden in Pediatric Inpatient Care: Transitioning a Medication on Hold Alert from Interruptive to Non-interruptive Format

Background: Clinical decision support (CDS) alerts are integral to modern pediatric care, yet excessive interruptive alerts contribute to alert fatigue and may reduce alert effectiveness. The alert demonstrated minimal clinical value while contributing significantly to alert fatigue.

Objectives: To evaluate if transitioning a high-firing medication on hold alert from an interruptive (Figure 1) to a non-interruptive storyboard (Figure 2) format would affect provider behavior or increase safety risks.

Design/Methods: We conducted a pre-post cohort study comparing provider practices six months before and after the transition. The alert fired when ≥2 medications or ≥1 anticoagulant were held for >48 hours. A comparison was made to evaluate provider practices in resuming medications during these periods. Data was extracted from the EHR (Epic) and institutional risk reporting system (Riskonnect). A large language model (Microsoft Co-pilot) was used to analyze free-text reports in Riskonnect for medication-related adverse events. All identified events were manually reviewed for relevance and independent keyword-based free-text queries were conducted to identify any missed events.

Results: Following the transition, the number of alert interactions decreased from 33,632 (3.0 clicks per hospital encounter) to 305 (0.02 clicks per hospital encounter) in a six-month period (Figure 3). There was no significant difference in the median hold duration of medications (81.5 hours v. 85.6 hours p-value 0.22) or in the proportion of medications never resumed before discharge (43% in both cohorts). No increase in reported safety events related to medications on hold was identified.

Conclusions: Transitioning the alert to a non-interruptive format substantially reduced alert burden without compromising patient safety. These findings highlight the importance to routinely evaluate CDS tools and suggest that alerts lacking demonstrated benefit may warrant redesign or retirement to reduce clinician fatigue and preserve alert effectiveness. Additionally, this study demonstrates the utility of large language models as a support tool for efficient, scalable review of safety report narratives.