By Jean Ling Tan
Did you know that self-reported coaching does not significantly improve MMI scores? In a study, it was actually found that coaching significantly reduced MMI scores, while repeating an MMI at the same institute during two application cycles did not improve scores unless stations were very similar/the same! (1)
I didn’t know this myself until I started doing research this year for my Year 2 PEP. (We’ve previously covered what PEP is briefly in this blog post.) My PEP title is: “Developing automated MMI interview tools for student selection in Medicine”, which is a huge area of interest for me since I do a lot of admissions-related content here on International Medics.
The MMI Interview Format
The MMI interview is a specific interview format that was derived from OSCE (Objective Structured Clinical Examination) methodology. For those of you who might not have taken an OSCE before, you rotate around a number of stations where you would be tested on a range of different skills such as carrying out a physical examination and taking a patient history. OSCEs are not only done in Medicine but also in other healthcare courses such as Nursing and Midwifery. An examinee’s performance in one station is independent of their performance in other stations, meaning that one’s score in a previous OSCE station will not affect scores on subsequent stations.
MMI methodology is very similar to the OSCEs but the skills assessed are not clinical, nor are they objective. (We have previously described what an MMI interview involves in a blog post, click here to read it.) Most UK medical schools now use the MMI, but did you know it only came about in 2004 at McMaster University, Canada? At that point, it was called the “admissions OSCE”. (2)
Note: All referenced literature, including the original paper describing the first-ever MMI in 2004 have been cited so that you can check them out for yourselves!
The MMI is now used in different parts of the world, not only for entry into Medicine courses but also for medical residency/specialty applications and other healthcare courses such as Pharmacy and Dentistry.
Student Selection in Medicine
We thought the Medicine application was complicated enough, but actually, there’s a lot more to consider from the medical school’s point of view! Any selection tool that is used has to be valid, reliable, acceptable and from a more practical point of view, feasible and cost-effective. If tools were not fair, that would be an easy way to get sued. Medical schools have an obligation to ensure that they select people who will become good doctors and ensure no one is unfairly disadvantaged.
There are multiple ways to describe validity (shown in the image below), which contribute to the overall view of the test.
Selection tools are basically anything that can be used by a medical school to decide whether you’re a suitable candidate for entry onto the course. Examples of selection tools would be UCAS application information, personal statements, references, aptitude tests such as the UCAT, and interviews.
Automating MMIs
Advancements in technology and current circumstances (yes COVID, I’m talking about you) provide a prime opportunity to explore automated MMI tools. An example of how MMIs have already been automated would be the use of tablets to score them.
The argument for automated MMI tools to augment human decision-making is that this could potentially reduce bias and allow institutions to increase the number of stations in their MMIs, thus increasing the reliability of MMIs.
There has only been one study looking at fully automated MMIs so far (4) but automated MMIs could be coming soon. To what extent can they be automated? Can face-to-face interviews be replaced entirely? This is currently unclear.
What are your thoughts on automating MMIs? Do you think MMIs can be fully automated in the future? Do let me know your thoughts!
I hope you enjoyed reading this snapshot of the research I have done for my PEP! Make sure to check out our other blog posts and follow us on Instagram to stay up to date with our content!
-Jean.
References
1. Griffin B, Harding D, Wilson I, Yeomans N. Does practice make perfect? The effect of coaching and retesting on selection tests used for admission to an Australian medical school. Medical Journal of Australia. 2008;189(5):270-273.
2. Eva K, Rosenfeld J, Reiter H, Norman G. An admissions OSCE: the multiple mini-interview. Medical Education. 2004;38(3):314-326.
3. Rees E, Hawarden A, Dent G, Hays R, Bates J, Hassell A. Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37. Medical Teacher. 2016;38(5):443-455.
4. Callwood A, Gillam L, Christidis A, Doulton J, Harris J, Coleman M et al. Evaluating a first fully automated interview grounded in MMI methodology: results from a feasibility study. 2021;.