TutorTrace: A Dataset and Taxonomy for Classifying Learner Behavioral States during AI-Assisted Programming Education

AI programming tutors provide scalable support, yet lack the behavioral context human tutors rely on to adapt support to learners' needs. We present TutorTrace, a dataset and behavioral abstraction pipeline that makes learners' behavioral context visible and computable in real time from low-level IDE telemetry. Across four deployments in two introductory Python courses (N=480), TutorTrace captures approximately 180K telemetry events, 13,633 behavioral segments, and 27 continuously computed metrics. From this foundation, we derive a taxonomy of learner activity before the first AI query, between consecutive queries, and across the full session, enabling systems to respond not just to what learners say, but to what they have done leading up to the help-seeking moment. In a preliminary classroom evaluation, behavior-aware prompts were associated with a decrease in intervals between queries with no independent work from 50.0% to 20.7%. As an additional demonstration of downstream utility, we evaluate TutorTrace on two held-out prediction tasks: whether a learner will query within the next 60 seconds (AUROC=.726) and whether an upcoming query reflects guided or dependent help-seeking (AUROC=.717). Together, these findings show how behavioral context can enable adaptive AI tutoring at scale.

Which CS1 Students Will Fail? Identifying Digital Markers from Learning Analytics in Computer Systems and Architecture Using Weighted Academic Momentum and Interaction Logs

Digital learning platforms generate rich behavioural traces (digital markers) that offer the potential to identify struggling students early. This paper investigates whether a combination of traditional and digital markers can predict failure in a first-year CS1 course (Computer Systems and Architecture) with sufficient recall to enable timely intervention. Using data from four cohorts (2017-2021, N=284) at a large public university in sub-Saharan Africa, we conducted a mixed-methods stakeholder elicitation to identify ten candidate factors. These were operationalised into a comprehensive feature set spanning demographics, self-reported surveys, Moodle interaction logs, and continuous assessment scores. A systematic ablation study using logistic regression with 5-fold cross-validation and SMOTE+ENN resampling revealed that the most predictive feature subset was Base + Demo + LMS: weighted academic momentum (M = 0.1Q1 + 0.15Q2 + 0.2Q3 + 0.55T1), basic demographics (gender, sponsorship, COVID-19 cohort), and a binary indicator of any LMS activity. On a held-out test set, logistic regression achieved 74.7% accuracy, 0.742 macro F1, and an AUC of 0.800. At the default threshold of 0.5, the model identified 87% of failing students (recall = 0.87) with a 41% false positive rate. SHAP analysis confirmed that weighted academic momentum is the strongest predictor, followed by its interaction with LMS engagement. These results demonstrate that simple digital markers can power a practical early-warning system by the fifth week of the semester. Our main contributions are: (1) a multi-source dataset and a stakeholder-guided methodology; (2) an ablation study quantifying feature group contributions; and (3) an interpretable, high-recall model ready for deployment.

Archetypes or ability? Clustering for modelling student mathematical competence

Personalised learning systems often assume that mathematical ability is combined of discrete abilities, acquired sequentially and dependent upon first acquiring foundational abilities, and students often report different strengths. In this work, we explore the validity of these assumptions by applying clustering methods to a large dataset of 119,034 students, spanning 13 national-level exams sat in the United Kingdom and collected by the platform. Classifying question results as pass or fail, we use a Bernoulli Mixture Model to search for latent populations which would be indicative of discrete skill-sets. We find that few distinct clusters are present in the data and that the dominant factor is overall student ability, which is further supported by the high degree of linear correlation between the probability distributions of the resulting clusters. Our best performing model achieves an accuracy of 78 percent, competitive with more complicated models in the literature whilst being more explainable. Comparing this models performance with logistic regression baselines and with k-nearest neighbours, we find a small improvement when using performance on each individual question as features. This suggests that whilst overall ability level is the dominant factor for predicting performance, small further personalisation improvements can be made by tailoring to a students exact strengths, but that students do not appear to develop strongly differing ability across topics. Our work offers a national scale test of machine learning in education and offers a new benchmark for the field, demonstrating how explainable models can reach competitive performance

Apriori-based Analysis of Learned Helplessness in Mathematics Tutoring: Behavioral Patterns by Level, Intervention, and Outcome

This study applied the Apriori algorithm to analyze behavioral interaction patterns associated with learned helplessness (LH) in mathematics tutoring system logs. Interaction data were examined across three dimensions: LH level (low vs. high), system-based intervention (with vs. without), and problem-solving outcomes (solved vs. unsolved). The analysis of the complete dataset showed that skipping problems without using hints was the most frequent pattern linked to unsolved outcomes, while persistence behaviors such as not skipping were less dominant overall. Comparisons by LH level showed that low-LH students had stronger links between problem solving and not skipping, as well as positive associations between hint use and solved outcomes. High-LH students showed more avoidance patterns, with skipping strongly tied to unsolved outcomes. In the comparison of system-based intervention conditions, students without intervention had the highest lift for persistence-success links, while the with-intervention group had stronger patterns involving skipping behaviors leading to unsolved outcomes. Outcome-specific analysis showed that not skipping was consistently associated with solved problems across all groups, while skipping without hints predicted unsolved outcomes. Practical implications and recommendations are discussed.