Evaluating Model Interpretability in Speech-Based Clinical Artificial Intelligence Systems

Abstract

Purpose: Model interpretability is a critical requirement for deploying artificial intelligence (AI) applications in clinical speech settings, yet existing evaluation methods often overlook this aspect. The complex nature of speech features and the opaque decision-making processes of AI models underscore the need for a tailored framework to assess interpretability in speech-based clinical AI systems. This article proposes an initial framework composed of eight key factors to consider when evaluating the interpretability of clinical speech AI models. These factors are categorized into two groups: functional factors and clinician-centered factors. The functional factors, which can be assessed independently without user involvement, include faithfulness and computational efficiency. The clinician-centered factors include four established ones from existing literature (cognitive load, human–AI task performance, mental model, and user trust) and two new factors tailored to the unique demands of clinical speech applications: clinical understandability and decision relevance. We further suggest evaluation methods for each of the identified factors and propose modifying existing instruments such as the System Usability Scale and the Healthcare Systems Usability Scale to evaluate the two newly introduced factors. Conclusions: Our identified factors form an initial framework for evaluating the interpretability of speech-based clinical AI systems, supporting the effective integration of AI into clinical workflows. Future works include conducting an evaluation experiment using the proposed framework and refining it further based on the findings from the experiment.

Ähnliche Arbeiten