Evaluation of an Ambient Artificial Intelligence Documentation Platform for Clinicians

Abstract

Importance: The increase of electronic health record (EHR) work negatively impacts clinician well-being. One potential solution is incorporating an ambient artificial intelligence (AI) documentation platform. Objective: To understand clinician experience before and after implementing ambient AI. Design, Setting, and Participants: This quality improvement study was a pilot evaluation with before and after survey and EHR metrics conducted at a large health care organization in Northern and Central California. Clinicians were purposively sampled to be representative of region and specialty. Ambient AI was implemented in April 2024 with EHR data from 3 months before and after implementation. Data were analyzed from May to September 2024. Exposure: Ambient AI access. Main Outcomes and Measures: Metrics of time were examined in notes per appointment, off-hour EHR activities (5:30 pm to 7:00 am on weekdays and nonscheduled weekends and holidays), documentation note length, progress note length, NASA Task Load Index (NASA-TLX) score, mini-Z burnout question, and overall experience. It was hypothesized that time in notes per appointment would decrease and clinical well-being would improve. Logistic regression and linear mixed-effect models were used. Results: Among 100 clinicians (53 male [53.0%]; mean [SD] age, 48.9 [11.0] years), 58 clinicians (58.0%) were in primary care and 92 clinicians had EHR metrics. Among 57 clinicians who completed both preimplementation and postimplementation surveys, there was a decrease in burnout from 24 clinicians (42.1%) to 20 clinicians (35.1%), although this was not a significant difference (P = .12). Mean (SD) NASA-TLX scores all decreased after using ambient AI: mental demand of note writing (12.2 [4.0] to 6.3 [3.7]), hurried or rushed pace (13.2 [4.0] to 6.4 [4.2]), and effort to accomplish note writing (12.5 [4.1] to 7.4 [4.3]) (all P < .001). Mean (SD) time in notes per appointment significantly decreased from 6.2 (4.0) to 5.3 (3.5) minutes (P < .001), with a bigger decrease for female vs male clinicians (8.1 [3.9] to 6.7 [3.6] minutes vs 4.7 [3.5] to 4.2 [3.1] minutes; P = .001). More primary care clinicians (33 of 38 clinicians [85.8%]) reported that ambient AI improved overall satisfaction at work compared with clinicians in medical (4 of 11 clinicians [36.4%]) and surgical (4 of 8 clinicians [50.0%]) subspecialties (P < .001). After adjusting for participant characteristics, model results suggested that mean scores for NASA-TLX decreased for mental demand (-6.12 [95% CI, -7.52 to -4.72]), hurried or rushed pace (-6.96 [95% CI, -8.42 to -5.50]), and effort to accomplish note writing (-5.57 [95% CI, -6.93 to -4.21]), while mean time in note taking decreased by less than 1 minute per appointment (0.91 minutes [95% CI, -1.20 to -0.62 minutes]) (all P < .001). Conclusions and Relevance: This study found that ambient AI was associated with improved overall experience and time in notes for clinicians but with varying outcomes by sex and specialty. Future research should investigate outcomes after widescale expansion of this rapidly evolving technology.

Ähnliche Arbeiten