Random Forest Machine Learning Analysis of Generative AI’s Impact on Learning Effectiveness in Indonesian Higher Education

Abstract

Generative Artificial Intelligence (GenAI) has rapidly penetrated Indonesian higher education, creating opportunities for learning innovation while raising concerns about effectiveness and academic integrity. This study develops a machine learning–based quantitative model to analyze the impact of GenAI usage on learning effectiveness, with a particular focus on Informatics students as key digital literacy stakeholders. Data were collected from a simulated survey of 300 students, covering demographics, GPA, exam scores, GenAI usage patterns, digital literacy, motivation, self-efficacy, academic integrity, and institutional support. Preprocessing steps included normalization of continuous variables, one-hot encoding of categorical variables, and feature selection using Recursive Feature Elimination (RFE). Six machine learning algorithms—Logistic Regression, Decision Tree, Random Forest, Support Vector Machine (SVM), XGBoost, and Artificial Neural Network—were compared to identify the best predictive model. Results show that Random Forest achieved the highest performance, with 87% accuracy and an AUC greater than 0.90, significantly outperforming other algorithms. The most influential predictors were digital literacy, institutional policies, and frequency of GenAI usage, while demographic variables contributed minimally. These findings suggest that GenAI can enhance learning effectiveness in Informatics education when supported by critical digital literacy and ethical awareness. The novelty of this study lies in integrating survey-based educational data with Random Forest machine learning to empirically model GenAI’s role in Indonesian higher education. The results provide practical implications for policymakers, educators, and institutions to design AI-integrated learning strategies that maximize innovation while safeguarding academic integrity.

Ähnliche Arbeiten