Comparisons of machine learning models to logistic regression in orthopedic sports medicine are confounded by methodological heterogeneity: a systematic review and meta-analysis

Abstract

IMPORTANCE: The modeling methodology and reporting of performance metrics in the development of machine learning (ML) models in orthopedic sports medicine have not yet been systematically assessed. OBJECTIVE: The purpose of this study was to systematically review this literature for clinical prediction models utilizing ML and to evaluate the methodological quality of modeling and performance reporting, as well as to compare the performance of ML with logistic regression (LR) predictions where applicable. EVIDENCE REVIEW: A systematic search was conducted of the MEDLINE, Scopus, and Embase databases in September 2025 for articles pertaining to clinical prediction models using ML in orthopedic sports medicine. Study demographics, outcomes, modeling workflow, and risk-of-bias information were collected. A random-effects meta-regression controlling for article and sample size was performed to compare the differences, where applicable, in performance benefit, measured by area under the curve (AUC), of utilizing ML models over LR. FINDINGS: A total of 1033 articles were screened, resulting in the inclusion of 52 articles in the final analysis. The most frequently utilized ML algorithm was random forest, followed by boosted trees and support vector machines. Most noteworthy sources of potential bias were encountered in outcome imbalance and the management of continuous predictors. A total of 25 studies performed a total of 168 pairwise comparisons between ML and LR. For 43 comparisons at high risk of bias, logit-transformed AUC regression (logit(AUC)) was 0.08 (-0.22-0.48) higher for ML, for 125 comparisons at low risk of bias, logit(AUC) was 0.00 (-0.18-0.18) lower for ML. Overall, random forest (RF) models demonstrated superior performance with a logit(AUC) of 0.11 (0.00-0.21). CONCLUSION AND RELEVANCE: While RF algorithms were associated with higher performance relative to traditional methods in well-constructed prediction problems (i.e. adequately powered datasets with appropriate feature handling, class balance considerations, and rigorous validation), no conclusive recommendations can be made regarding the superiority of ML over LR, given the current evidence. Improvements in methodology and standardized reporting of performance metrics are required for the useful interpretation of future comparisons. LEVEL OF EVIDENCE: IV, Systematic Review and Meta-Analysis with more than 2 negative criteria.

Ähnliche Arbeiten