Development and pilot of a novel, developmental student assessment for the PBL component of a therapeutics course

Introduction “Modified” PBL is a component of therapeutics modules in Stages I & II of the GEM programme. PBL was deployed to promote learning in clinical context; to develop clinical reasoning skills and to facillitate the development of team working & self direction . It was recently decided to award marks for ‘participation” in each session. To do so , we developed & piloted a novel PBL assessment instrument, based on the descriptive, developmental RIME scheme of Pangaro . AIM The purpose of this study was to examine the utility of a novel assessment of participation in PBL using an adaptation of the RIME framework relevant to PBL performnce. METHODS Approximately 900 episodes of student participation were assessed by tutors of Stage II GEM in semester one 2012-13. Discussion and Report phases of each session were scored separately according to the behavioural descriptors below. Students were invited to self assess using the instrument and record this in an anonymised online log. Tutors and students gave feedback on the instrument. RESULTS Tutors prefered the new instrument to a previous multidomain assessment of participation and they considered it to have face validity. They reported confident identification of each “level” of performance in both discussion and report, and this is supported by reasonable interassessor reliability for 42 double scored episodes. All scale levels were used by assessors, while the “old” multidomain likert scores clustered around 3/5. Tutors also reported that the novel assessment enriched formative feedback. DISCUSSION & CONCLUSIONS This pilot study supports the further development and validation of a novel, holistic and developmentally relevant assesment of student participation in PBL. We acknowledge that it may not be considered theoretically “sound” by purists to assess performance in PBL at all , nonetheless we consider that in mixed curricula such an assessment may be educationally necessary. This novel scale was developed in consultation with three experienced PBL tutors and draws heavily on the RIME work of Pangaro. We report high face validity, good use of all elements of the scale and reasonable reliability in this pilot. Pending larger scale validation, we propose that this instrument may be most useful in the assessment of PBL participation in mixed methods curricula where there is an emphasis on formative, development feedback.