Development And Pilot Of A Novel, Descriptive Student Assessment For The PBL Component Of A Therapeutics Course

Background : Summative assessment of participation in the PBL component of a therapeutics course was initiated at the students’ request. The key consideration was that assessment would be valid, feasible and acceptable and promote learning aligned to Barrows' four educational objectives. Initial pilot of a 3 domain numerical instrument derived from the 44-item validated Tutotest showed that while scores correlated strongly with those on Tutotest, its validity and educational impact were poor and it was decided to develop a novel instrument with high educational impact. Summary of Work: Using Pangaro’s descriptive RIME synthesis for in-training clerkship as a model, characteristics of four levels of student participation in PBL were debated by experienced PBL tutors until consensus was reached. Detailed descriptors wereprovided for each, characterized as: Observer; Participant; Hypothesier and Synthesiser in the discussion phase and Bare Reporter; Reliable Reporter; Interpreter and Educator for the report phase. The OPHS: BRIE instrument was used alongside the original numerical instrument for 800+ PBL assessments and the psychometrics, feasibility, acceptability and educational impact of each were compared. Summary of Results: We report that psychometrics for the RIME based assessment were similar to those for the numerical score. In terms of stakeholder acceptability, feasibility and educational impact, the descriptive assessment scored better. Furthermore, the model enabled faculty to identify and address specific weaknesses in student participation and to document meaningful progression of student participation in PBL over the semester. Conclusions: Participation in PBL can be reliably, validly and feasibly assessed using a novel descriptive score derived from Pangaro’s RIME synthesis which additionally provides rich feedback to faculty and students.