To what extent does the Health Professions Admission Test-Ireland predict performance in early undergraduate tests of communication and clinical skills? - An observational cohort study

Kelly, M. E. and Regan, D. and Dunne, F. and Henn, P. and Newell, J. and O'Flynn, S. (2013) To what extent does the Health Professions Admission Test-Ireland predict performance in early undergraduate tests of communication and clinical skills? - An observational cohort study. BMC Medical Education, 13 (1). ISSN 14726920 (ISSN)

[img] Text
Kelly et al 2013 To what extent....pdf

Download (430kB)
Official URL: http://www.scopus.com/inward/record.url?eid=2-s2.0...

Abstract

Background: Internationally, tests of general mental ability are used in the selection of medical students. Examples include the Medical College Admission Test, Undergraduate Medicine and Health Sciences Admission Test and the UK Clinical Aptitude Test. The most widely used measure of their efficacy is predictive validity.A new tool, the Health Professions Admission Test- Ireland (HPAT-Ireland), was introduced in 2009. Traditionally, selection to Irish undergraduate medical schools relied on academic achievement. Since 2009, Irish and EU applicants are selected on a combination of their secondary school academic record (measured predominately by the Leaving Certificate Examination) and HPAT-Ireland score. This is the first study to report on the predictive validity of the HPAT-Ireland for early undergraduate assessments of communication and clinical skills. Method. Students enrolled at two Irish medical schools in 2009 were followed up for two years. Data collected were gender, HPAT-Ireland total and subsection scores; Leaving Certificate Examination plus HPAT-Ireland combined score, Year 1 Objective Structured Clinical Examination (OSCE) scores (Total score, communication and clinical subtest scores), Year 1 Multiple Choice Questions and Year 2 OSCE and subset scores. We report descriptive statistics, Pearson correlation coefficients and Multiple linear regression models. Results: Data were available for 312 students. In Year 1 none of the selection criteria were significantly related to student OSCE performance. The Leaving Certificate Examination and Leaving Certificate plus HPAT-Ireland combined scores correlated with MCQ marks.In Year 2 a series of significant correlations emerged between the HPAT-Ireland and subsections thereof with OSCE Communication Z-scores; OSCE Clinical Z-scores; and Total OSCE Z-scores. However on multiple regression only the relationship between Total OSCE Score and the Total HPAT-Ireland score remained significant; albeit the predictive power was modest. Conclusion: We found that none of our selection criteria strongly predict clinical and communication skills. The HPAT- Ireland appears to measures ability in domains different to those assessed by the Leaving Certificate Examination. While some significant associations did emerge in Year 2 between HPAT Ireland and total OSCE scores further evaluation is required to establish if this pattern continues during the senior years of the medical course. © 2013 Kelly et al.; licensee BioMed Central Ltd.

Item Type: Article
Additional Information: Cited By :2; Export Date: 23 February 2015; Correspondence Address: Kelly, M.E.; Department of General Practice, Clinical Science Institute, National University of Ireland, Galway, Ireland; email: maureen.kelly@nuigalway.ie; References: Patterson, F., Ferguson, E., (2010) Selection for Medical Education and Training, , Oxford, UK: Wiley Online Library Swanwick T; (2012) Medical College Admission Test, , http://www.aamc.org/mcat, Association of American Medical Colleges (cited 17/11/2012); (2012) BioMedical Admission Test, , http://www.admissionstests.cambridgeassessment.org.uk/adt/bmat, Cambridge Assessment Group [cited 23/10/2012]; (2012) Undergraduate Medicine and Health Sciences Admission Test, , http://umat.acer.edu.au, Australian Council for Educational Research [cited17/02/2012]; (2012) The UK Clinical Aptitude Test, , http://www.ukcat.ac.uk, UKCAT Consortium (cited 17/10/2012); Prideaux, D., (2011) Assessment for Selection for the Health Care Professions and Specialty Training: Consensus Statement and Recommendations from the Ottawa 2010., , Conference Medical Teacher; Donnon, T., Paolucci, E.O., Violato, C., The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research (2007) Academic Medicine, 82 (1), pp. 100-106. , DOI 10.1097/01.ACM.0000249878.25186.b7, PII 0000188820070100000015; Julian, E.R., Validity of the medical college admission test for predicting medical school performance (2005) Academic Medicine, 80 (10), pp. 910-917. , DOI 10.1097/00001888-200510000-00010; Lynch, B., Mackenzie, R., Dowell, J., Cleland, J., Prescott, G., Does the UKCAT predict Year 1 performance in medical school? (2009) Med Educ, 43 (12), pp. 1203-1209. , 10.1111/j.1365-2923.2009.03535.x 19930512; Yates, J., James, D., The value of the UK clinical aptitude test in predicting pre-clinical performance: A prospective cohort study at Nottingham medical school (2010) BMC Med Educ, 10, p. 55. , 10.1186/1472-6920-10-55 20667093; Yates, J., James, D., The UK clinical aptitude test and clinical course performance at Nottingham: A prospective cohort study (2013) BMC Med Educ, 13, p. 32. , 10.1186/1472-6920-13-32 23442227; Wright, S.R., Bradley, P.M., Has the UK clinical aptitude test improved medical student selection? (2010) Med Educ, 44 (11), pp. 1069-1076. , 10.1111/j.1365-2923.2010.03792.x 20946477; Poole, P., Shulruf, B., Rudland, J., Wilkinson, T., Comparison of UMAT scores and GPA in prediction of performance in medical school: A national study (2012) Med Educ, 46 (2), pp. 163-171. , 10.1111/j.1365-2923.2011.04078.x 22239330; Wilkinson, D.Z.J., Parker, M., Predictive validity of the undergraduate medicine and health sciences admission test for medical students' academic performance (2011) Med J Aust, 194 (7), pp. 341-344. , 21470083; Emery, J., The predictive validity of the BioMedical admissions test for pre-clinical examination performance (2009) Med Educ, 43 (6), pp. 557-564. , 10.1111/j.1365-2923.2009.03367.x 19493180; McManus, I., Wakeford, R., Powis, D., James, D., Predictive validity of the Biomedical admissions test: An evaluation and case study (2011) Med Teach, 33 (1), pp. 53-57. , 10.3109/0142159X.2010.525267 21182383; Ferguson, E., James, D., Madeley, L., Factors associated with success in medical school: Systematic review of the literature (2002) British Medical Journal, 324 (7343), pp. 952-957; (2012) Health Professions Admission Test - Ireland, , http://www.hpat-ireland.acer.edu.au, HPAT-Ireland(cited 17/11/2012); Fottrell, P., (2006) Medical Education in Ireland: A New Direction. Report of the Working Group on Undergraduate Education and Training (the Fottrell Report), , http://www.dohc.ie/publications/fottrell.html?lang=en, (cited 30/04/2013) Available from; Flynn S, O., Mills, A., Fitzgerald, T., (2012) National Research Group Evaluating Revised Entry Mechanisms to Medicine Interim Report School Leaver Entrants, , (cited 7/03/2013) available from: http://www.ucd.ie/t4cms/HPAT-report- July2012.pdf; (2012) Australian Council for Educational Research, , (cited 26/10/2012]; Available from: http://www.acer.edu.au/; Medical Council, I., (2010) Eight Domains of Good Professional Practice, , http://www.medicalcouncil.ie/Education-and-Training/Good-Professional- Practice/Eight-Domains-of-Good-Professional-Practice-as-devised-by-Medical- Council.pdf, (cited 20/11/2012); Available from; Frank, J.R., (2005) The CanMEDS 2005 Physician Competency Framework: Better Standards, Better Physicians, Better Care, , Ottawa, Ontario, Canada: Royal College of Physicians and Surgeons of Canada; Medical Council, I., Number and Type of Complaints Received by Preliminary Proceedings Committee in 2010, , https://medicalcouncil.ie/News-and-Publications/Publications/ Annual-Reports-Statistics-/Statistics/Statistics-2010/Statistics-2010.html, cited 18/02/2012; Papadakis, M.A., Teherani, A., Banach, M.A., Knettler, T.R., Rattner, S.L., Stern, D.T., Veloski, J.J., Hodgson, C.S., Disciplinary action by medical boards and prior behavior in medical school (2005) New England Journal of Medicine, 353 (25), pp. 2673-2682. , http://content.nejm.org/cgi/reprint/353/25/2673.pdf, DOI 10.1056/NEJMsa052596; Siu, E., Reiter, H.I., Overview: What's worked and what hasn't as a guide towards predictive Admissions tool development (2009) Adv Health Sci Educ, 14 (5), pp. 759-775. , 10.1007/s10459-009-9160-8; (2012) Selection Criteria for Undergraduate Entry to Medicine for EU Applicants 2012, , http://www.tcd.ie/Admissions/undergraduate/assets/pdf/ HPAT-Brochure-2012-lr.pdf, Central Applications Office [cited 02/06/2012]; Available from; Harden, R., Stevenson, M., Downie, W.W., Wilson, G., Assessment of clinical competence using objective structured examination (1975) Br Med J, 1 (5955), pp. 447-451. , 10.1136/bmj.1.5955.447 1115966; Hamdy, H., Prasad, K., Anderson, B., Scherpbier, A., Williams, R., Zwierstra, R., Cuddihy, H., BEME systematic review: Predictive values of measurements obtained in medical schools and future performance in medical practice (2006) Medical Teacher, 28 (2), pp. 103-116. , DOI 10.1080/01421590600622723, PII J08731265184; Coates, H., Establishing the criterion validity of the Graduate Medical School Admissions Test (GAMSAT) (2008) Med Educ, 42, pp. 999-1006. , 10.1111/j.1365-2923.2008.03154.x 18823519; McManus, C., Powis, D., Testing medical school selection tests (2007) Medical Journal of Australia, 186 (3), pp. 118-119; McManus, I.C., Smithers, E., Partridge, P., Keeling, A., Fleming, P.R., A levels and intelligence as predictors of medical careers in UK doctors: 20 Year prospective study (2003) British Medical Journal, 327 (7407), pp. 139-142; Sackett, P.R., Yang, H., Correction for range restriction: An expanded typology (2000) J Appl Psychol, 85 (1), p. 112. , 10740961; Halpenny, D.C.K., Halpenny, M., Burke, J., Torreggiani, W.C., The Health Professions Admission Test (HPAT) score and leaving certificate results can independently predict academic performance in medical school: Do we need both tests? (2010) Ir Med J, 103 (10), pp. 300-302. , 21560500; Quinn, A., Corrigan, M.A., Broderick, J., McEntee, G., Hill, A.D., A comparison of performances of consultant surgeons, NCHDs and medical students in a modified HPAT examination (2010) Ir Med J, 103 (6), pp. 172-173. , Epub 2010/07/31 20669599; O'Flynn, S., Mills, A., Fitzgerald, T., Entry to medical school-the gender question. What has happened? (2013) Ir Med J, , in press ref:S5443; James, D., Yates, J., Nicholson, S., Comparison of A level and UKCAT performance in students applying to UK medical and dental schools in 2006: Cohort study (2010) BMJ, 340, p. 3478. , Epub 2010/02/18 10.1136/bmj.c478 20160316; Walshe, J., Scrap unfair aptitude testing for entry into medicine say doctors (2009) Irish Independent, , Ed Tuesday 25-08-09 page 5; Murray, N., More students with lower leaving cert points get medicine (2011) Irish Examiner, , http://www.irishexaminer.com/ireland/education/more-students-with-lower- leaving-cert-points-get-medicine-165133, Ed. Tuesday, August 23, 2011 (cited 30/04/2013) Available from; Koczwara, A., Patterson, F., Zibarras, L., Kerrin, M., Irish, B., Wilkinson, M., Evaluating cognitive ability, knowledge tests and situational judgement tests for postgraduate selection (2012) Med Educ, 46 (4), pp. 399-408. , 10.1111/j.1365-2923.2011.04195.x 22429176
Uncontrolled Keywords: Ability; Assessment; Cognitive; HPAT-Ireland; Medical; Predictive; Selection; Student; Validity; adolescent; aptitude test; article; clinical competence; education; female; human; interpersonal communication; Ireland; male; medical education; predictive value; standard; statistics; young adult; Aptitude Tests; College Admission Test; Communication; Education, Medical, Undergraduate; Humans; Predictive Value of Tests
Depositing User: National Forum
Date Deposited: 03 Dec 2015 20:37
Last Modified: 03 Dec 2015 20:41
URI: http://eprints.teachingandlearning.ie/id/eprint/2651

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year