The validity and reliability of adaptive comparative judgements in the assessment of graphical capability.

The valid and reliable assessment of capability is of paramount importance in education.
Operationalizing assessment practices of divergent problems can be particularly challenging due
to the variety of potential responses. This paper investigates the use of Adaptive Comparative
Judgements (ACJ) in the assessment of graphical capability. A cohort of undergraduate Initial
Technology Teacher Education (ITTE) students (N=128) participated in this study which involved
completing a design task and subsequently assessing the work of their peers through ACJ and
criterion referenced assessment. The performance from both methods was analysed and identified
a high level of reliability for ACJ. Correlations between the criteria scores and ACJ parameter
values suggest its validity as an assessment mechanism, however they also present the potential
for additional variable to be influencing holistic judgements.