TECHNOLOGICAL CHANGE IN ASSESSING ECONOMICS: A CAUTIONARY WELCOME

The use of computer-based automated assignment systems in economics has expanded significantly in recent years. The most widely used system is Aplia which was developed by Paul Romer in 2000. Aplia is a computer application designed to replace traditional paper-based assignments in economics. The main features of Aplia are: (1) interactive content including problem sets, experiments and news analysis; (2) digital editions of a textbook; (3) assignment sets that are customised to specific textbooks; and (4) immediate feedback for both students and instructors. Its ability to present the dynamics of diagrams and graphs is critical to its use in economics. This paper analyses the effectiveness of Aplia and traditional paper-based assignments and tutorials using summative assessment results. The analysis is based on a managerial economics course that was taught to over 380 students at NUI Galway in the first semester of 2008-09. The course was designed so that each student was required to complete eight assignments for 25% m arks available for the course. They completed six of the eight assignments by Aplia and the remaining two by paper. The final exam was organised into eight sections with each section corresponding to a particular assignment. Our basic test is to examine whether a student's performance in a particular section of the exam is affected by whether the student completed the corresponding assignment on paper or online. We also examined if how the student performed on a particular assignment, regardless of the type, predicted how well they did on the corresponding examination question. We found little statistical evidence in support of either hypothesis