Validation of a novel Problem Based Learning assessment tool

Background : Previously we have described a novel feedback tool for students participating in PBL:BRIE/OPHS with each letter describing the role students undertook during the reporting(BRIE) and discussion(OPHS) phase of PBL. We sought to develop and validate a numeric tool based on BRIE/OPHS that can be used in the summative assessment of PBL. Summary of Work: Each letter on the BRIE/OPHS was assigned a numeric value. Differences between the numeric scores generated between our currently used validated tool(CT) and the novel BRIE/OPHS tool(NT) were assessed by Mann Whitney whilst correlation was assessed using Spearmans test. The relationship between the component parts of both tools was also analysed. Summary of Results: 372 PBL episodes were analysed. A strong correlation was observed between the novel tool(NT) and currently used(CT) assessment format(r=0.85, p<0.0001). The report and discussion sections when analysed alone revealed similar correlations(Report:r =0.87, p<0.001; Discussion:r = 0.81, p<0.001). When total scores were assessed a significant difference was observed between tools such that students scored lower when assessed using the new instrument (Median CT 0.6 v NT 0.5, p<0.001). Analysis of the components revealed that this was due to a lower median mark in the discussion phase of the PBL session(NT 0.4 vs CT 0.6,p<0.001) with no differences observed in scores between tools in the report phase(NT 0.6 vs CT 0.6,p=0.22). Conclusions: The BRIE/OPHS, a novel feedback tool, can be successfully adapted to provide a numeric score that demonstrates strong correlation with currently available PBL assessment tools.