Causal Modelling of Support Interventions for Student Competency Assessment

Accurate assessment of student competencies is essential for enabling educators to identify individual needs, design targeted interventions, and evaluate the effectiveness of educational strategies. Empirical assessment procedures are typically grounded in psychometric models, such as item response theory, which relate student competence levels to performance on assessment tasks. In this paper, we advocate adopting a structural causal modelling approach to educational assessment, moving beyond probabilistic belief updating toward a framework that explicitly supports interventional and counterfactual reasoning. We propose a corresponding protocol for its construction and analyse the practical relevance of forms of reasoning that remain inaccessible to standard associative models, including the explicit modelling of interventions such as hints and the related counterfactual scenario analysis. Although our protocol requires the structural equations to be elicited from experts, the necessary information is purely logical and does not rely on probabilistic, less tenable assumptions. We illustrate the approach using data from an assessment that employs complex tasks designed to measure compulsory school student algorithmic skills.

Archetypes or ability? Clustering for modelling student mathematical competence

Personalised learning systems often assume that mathematical ability is combined of discrete abilities, acquired sequentially and dependent upon first acquiring foundational abilities, and students often report different strengths. In this work, we explore the validity of these assumptions by applying clustering methods to a large dataset of 119,034 students, spanning 13 national-level exams sat in the United Kingdom and collected by the platform. Classifying question results as pass or fail, we use a Bernoulli Mixture Model to search for latent populations which would be indicative of discrete skill-sets. We find that few distinct clusters are present in the data and that the dominant factor is overall student ability, which is further supported by the high degree of linear correlation between the probability distributions of the resulting clusters. Our best performing model achieves an accuracy of 78 percent, competitive with more complicated models in the literature whilst being more explainable. Comparing this models performance with logistic regression baselines and with k-nearest neighbours, we find a small improvement when using performance on each individual question as features. This suggests that whilst overall ability level is the dominant factor for predicting performance, small further personalisation improvements can be made by tailoring to a students exact strengths, but that students do not appear to develop strongly differing ability across topics. Our work offers a national scale test of machine learning in education and offers a new benchmark for the field, demonstrating how explainable models can reach competitive performance