Undergraduate students’ reasoning about the quality of experimental measurements of covarying secondary data

Abstract

We have investigated how first-year and second-year university students judge the quality of secondary experimental data consisting of measurements of covarying quantities, and to what extent they consider multiple measurements of covarying quantities as a single data set characterised by its mean and spread. Five cohorts of students at two universities participated in the study. They offered written responses to three open-ended questions. Individual follow-up interviews were conducted with fourteen students that were based on their previous written answers to understand their reasoning in more depth. A fine-grained analysis was undertaken to unpack students’ reasoning by grouping the criteria they used to judge the quality of the data into three broad categories: criteria independent of the data, criteria that based on the variation of the raw data, and criteria based on the variation of the derived quantity. We argue that responses in each category can be linked to different learning objectives. The students proposed various actions to increase the trustworthiness of the covarying data or a conclusion based on it. We have investigated to what extent decisions and proposed actions were consistent. Students generally showed a fragmented and primarily qualitative understanding of the concepts of mean, uncertainty, and line of best fit. Many students proposed to determine the value of a derived quantity by applying the equation relating it to the measurands to individual data points, all data points, or all data points except for outliers. Students appear to consider the best fit line as a tool for eliminating outliers, but much less frequently as a way to determine a derived quantity. We discuss implications for instructional practice and further research.