Formative and Summative Electronic Assessment in Economics Using Aplia

The use of computer-based automated assignment systems in economics has expanded significantly in recent years. The most widely used system is Aplia which was developed by Paul Romer in 2000. Aplia is a computer application designed to replace traditional paper-based assignments in economics. The main features of Aplia are (1) interactive content including problem sets, experiments and news analysis, (2) digital editions of a textbook, (3) assignment sets that are customized to specific textbooks and (4) immediate feedback for both students and instructors. Its ability to present the dynamics of diagrams and graphs is critical to its use in economics. This paper analyses the effectiveness of Aplia and traditional paper-based assignments and tutorials using summative assessment results. The analysis is based on a managerial economics course that was taught to over 380 students at NUI Galway in the first semester of 2008-09. The course was designed so that each student had to complete 8 assignments for 25% of the total marks available for the course. They completed 6 of the 8 assignments by Aplia and the remaining 2 by paper. Each student was assigned to one of four groups. For any given assignment one groups (1/4) of the students completed the assignment in written form while the remaining (3/4) completed the assignment by Aplia. The pattern of coursework type looked as follows (where W is written and A is A: Group A: W A A A W A A A  Group B: A W A A A W A A  Group C: A A W A A A W A Group D: A A A W A A A W The final exam was organized into 8 sections with each section corresponding to a particular assignment. Our basic test is to examine whether a student’s performance in a particular section of the exam is affected by whether the student completed the corresponding assignment on paper or on-line. For example, did the students from Group A do better/worse on assignment 1, which the completed in written form, that the Groups B, C, and D who completed the assignments on Aplia. (We also examined if how the student performed on a particular assignment, regardless of the type, predicted how well they did on the corresponding examination question.) We found little statistical evidence in support of either hypothesis.