6ER-008 Reducing workload in systematic reviews on medication management

Abstract

<h3>Background and Importance</h3> Systematic reviews are essential for evidence-based decision making in hospital pharmacy but require labour intensive manual screening prone to error. Large language models have enabled artificial intelligence (AI) tools such as active learning (AL)-assisted screening. AL-assisted screening ranks abstracts by inclusion probability, updating as decisions are made, and allowing reviewers to focus on likely relevant abstracts. This may reduce workload, yet AL-assisted screening has not been validated for medication management systematic reviews. <h3>Aim and Objectives</h3> We aimed to validate AL-assisted screening compared with manual screening in reducing the number of abstracts screened. Secondary objectives included detecting relevant abstracts missed by human error. <h3>Material and Methods</h3> We simulated screening of four large systematic review datasets (3475–16,218 abstracts; 0.08–1% included) using ASReview, an open-source AL tool. The systematic reviews focused on prescribing cascades, adverse drug reactions/events, hospital pharmacy interventions on length of stay, and pharmacy technician interventions on wards. ASReview parameters were systematically varied, including labelling strategies. In title/abstract labelling, inclusion for full-text decisions are based on the title/abstract, which can be error-prone (‘noisier’). In full-text labelling, inclusion is determined after reading the complete article. Since ASReview improves with each new labelled record, accurate labelling is critical for model learning. In addition, ASReview was trained with 1% of random abstracts, and including at least one relevant article. Project leads reassessed the top 100 records to detect missed inclusions. Primary outcome was screening efficiency; secondary outcome was error detection. Descriptive analysis was used. <h3>Results</h3> ASReview reduced records to screen by ~90% while maintaining sensitivity. For three datasets, optimal performance (100% full-text includes, 89–90% screening reduction) was achieved with recommended ASReview parameters and full-text labelling. For the largest dataset (16,218 abstracts, 0.08% includes), 87% of full-text includes were detected with this setup, but 100% detection was achieved using simpler models with title/abstract labelling. In two datasets, ASReview identified an additional article inclusion missed during manual screening. <h3>Conclusion and Relevance</h3> ASReview markedly reduced workload in medication management systematic reviews without loss of accuracy and while detecting some human errors. Optimal parameters depend on dataset characteristics: full-text labelling is best for most, but title/abstract labelling with simpler models may be preferable when the overall proportion of includes is very low. <h3>Conflict of Interest</h3> No conflict of interest

Ähnliche Arbeiten