Assessing the Accuracy of Outcome-Based Assessment Decisions Using the Two-Parameter IRT Model (2PL): An Applied Framework for Concentrating Test Information at Critical Cut Scores

Abstract

  This study aimed to examine the precision of assessment decisions in an Outcome-Based Education (OBE) context by calibrating a 40-item True/False test designed to measure five specific learning outcomes using the 2PL-IRT model. The study employed a descriptive-analytical approach using a census sample of 275 students at a private university in Jordan. The selection of the 2PL model was intended to ensure stable parameter estimation given the sample size, with a focus on the parameters of discrimination (a) and difficulty (b). The results indicated that the items had discrimination indices ranging from 1.18 to 2.10 and difficulty parameters from -1.10 to 1.60, reflecting their effectiveness in differentiating students with varying ability levels. However, the TIF function revealed that maximum measurement precision was centered at the average ability level (θ ≈ 0), rather than at the cut score required in OBE contexts. Fit analyses showed that most items conformed to the model assumptions, although items q10, q20, and q24 displayed significant p-value deviations. The study concludes that traditional tests, despite possessing sound psychometric properties, may functionally fail to support critical “mastery” decisions and therefore recommends redesigning the item difficulty distribution to align with decision thresholds rather than with the mean ability level