|Year : 2021 | Volume
| Issue : 2 | Page : 154-158
The multiple mini-interviews: The experience from a saudi postgraduate residency program toward a more objective selection process
Khalid Mohammed Alayed1, Walid Alkeridy2, Musa Alzahrani1
1 Department of Medicine, King Saud University, Riyadh, Saudi Arabia
2 Department of Medicine, King Saud University, Riyadh, Saudi Arabia; Department of Medicine, Geriatric Division, University of British Columbia, Vancouver, Canada
|Date of Submission||05-Jul-2020|
|Date of Decision||27-Sep-2020|
|Date of Acceptance||05-Oct-2020|
|Date of Web Publication||03-Dec-2020|
Khalid Mohammed Alayed
Department of Medicine, King Saud University, West College of Medicine Building, 3rd Floor, Riyadh 12372
Source of Support: None, Conflict of Interest: None
Purpose: The multiple mini-interview (MMI) is a validated technique used in the admissions process in some undergraduate and postgraduate schools and is reported to reduce subjectivity in selecting postgraduate applicants. No studies have been conducted in Saudi Arabia concerning the MMI. The authors report their experience of transitioning from traditional interviews to the MMI and the results of a post-MMI survey undertaken by participating applicants and interviewers. Materials and Methods: The authors did retrospective analysis after MMI had been conducted at the College of Medicine, King Saud University, Riyadh, Saudi Arabia, in 2019, in coordination with an internal medicine residency program. They implemented MMIs totaling four stations of 10 min each that focused on the domains of communication, attitude, knowledge, and a mini-interview personalized for each candidate. Ten questioners interviewed 99 applicants, of whom 68 undertook a post-MMI survey. Results: In terms of their perceptions and experience, the applicants and interviewers responded positively to the transition from traditional interviews to the MMI. MMI was seen to be more objective, 75% of applicants felt it was associated with less anxiety, and 79% believed it provided a better portrayal of their abilities. Conclusion: The use of the MMI in selecting postgraduate applicants in Saudi Arabia is feasible and acceptable. Furthermore, it may give an improved objective portrayal of applicants' abilities and reduce their interview-associated anxiety.
Keywords: Interview selection, multiple mini-interview, Saudi postgraduate
|How to cite this article:|
Alayed KM, Alkeridy W, Alzahrani M. The multiple mini-interviews: The experience from a saudi postgraduate residency program toward a more objective selection process. J Nat Sci Med 2021;4:154-8
|How to cite this URL:|
Alayed KM, Alkeridy W, Alzahrani M. The multiple mini-interviews: The experience from a saudi postgraduate residency program toward a more objective selection process. J Nat Sci Med [serial online] 2021 [cited 2021 May 16];4:154-8. Available from: https://www.jnsmonline.org/text.asp?2021/4/2/154/302439
| Introduction|| |
Postgraduate trainee selection for a residency program is a complex process. Some program directors consider the personal interview to be the most critical aspect of an applicant's potential selection.,, During a traditional interview, more information about the applicant's interpersonal skills and other nonacademic domains is usually gathered, and red flags can be carefully addressed through the traditional interview that could be overlooked on a routine application., In addition, applicants place significant value on the interview as a means of portraying their best skills and often consider it an essential determinant of program selection. However, many aspects of the traditional interview depend on subjectivity,, which might challenge a fair and unbiased interview process., Moreover, the current literature questions the capability of interviews to predict important outcomes, such as resident performance during the program period., To address some of the traditional interview's limitations, investigators such as Eva et al. have developed and adopted an approach called the multiple mini-interview (MMI). The MMI includes 10 stations that assess multiple domains, with emphasis on nonacademic skills, considered to be essential for the success of trainees during their residency program., It has been claimed that the MMI is more objective, effective, enjoyable, and cost-effective than the traditional selection process.,,,
The Saudi Commission for Health Specialties (SCFHSs) regulates all medical training programs as well as health-care-related practices and accreditation at all levels of health care in Saudi Arabia. After completing medical school education, candidates typically apply for a position in one of the residency programs through the SCFHS and enter a Phase I matching system that coordinates the applicant's choice in terms of their preferred specialty and training region/city. After Phase I matching results are determined, the names of residents who have been assigned to internal medicine (usually around 200–250 applicants) are sent to the internal medicine training centers in their preferred region/city, so the centers can offer interviews to these candidates.
The King Saud University internal medicine program had traditionally used a filtering system to select approximately 100 candidates for interviews, based on their grade point average, Saudi medical licensing examination results, and a detailed review of their curricula vitae (seeking an interest in internal medicine and future goals) and letters of recommendation. The interview structure standardly included the program director and a chief resident (with or without an additional consultant), all sitting in one room, with candidates entering the room one at a time to answer open-ended questions in the conventional interview manner.
Aiming toward a more objective process for the internal medicine residency training program, we adopted the MMI model in our selection criteria, after obtaining approval from the Residency Program Committee. We sought responses to a questionnaire to understand perceptions of the change. Herewith, we aim to report our positive experience in converting from a traditional interview process to the MMI model.
| Materials and Methods|| |
We used the previously described filtering system to select 100 applicants for the MMI but with a different weight for each component compared to previous years. These new emphases were approved by the Residency Program Committee at King Saud University.
The structured MMI contained four stations that were determined in advance by the residency training committee aimed to address the nonacademic skills – domains of communication, attitude, and knowledge – chosen by the residency training committee. We incorporated a mini-interview (conducted by the program director and a chief resident) with prepared tailored questions, based on each candidate's application, into the process. The aim of the mini-interview was to underscore exceptional achievement and identify areas of concerns.
We conducted a literature review before constructing the MMI to ensure that the stations were assessing the selected domains accurately. We adopted an approach used in a study by Dore, Kreuger, and Ladhani after obtaining permission from the authors. Each station presented a maximum possible score of 10, and examiners could notate comments. Applicants were stationed in separate rooms, and interviewers moved from one room to another, questioning applicants for a strictly regulated time of 10 min per station.
After the MMI was concluded, the applicants and the interviewers completed a questionnaire adopted from a study by Eva et al. Subsequently, we analyzed the questionnaire based on simple frequency statistics. The questionnaire was conducted during the MMI process for feedback purposes and then the data were retrospectively collected.
Ethical approval of this study was granted by the Institutional Review Board (IRB) of the College of Medicine Research Center, King Saud University, Saudi Arabia. Ref. No. 20/0206/IRB obtained on March 1, 2020, given that the study does not disclose the participant's identity and poses no risk to the participants; in addition, the analysis was done retrospectively to describe our experience for the residents' selection. Permission was granted by IRB for the finding to be published.
| Results|| |
A total of 10 interviewers conducted the MMI. The interviewers are all academics who hold at least the rank of Assistant Professor at King Saud University. Each group of interviewers was required to meet for at least one preinterview session to discuss scoring and reduce variability. All the interviewers indicated that using the MMI was a positive experience.
Although most interviewers held the view that the MMI was more difficult to conduct than a more traditional interview, they all affirmed that the MMI had helped them develop an accurate portrayal of the candidates [Table 1].
Although only 99 interviewees attended, we had mailed invitations to 104 applicants. In addition to the date, time, and location, the e-mail outlined the new interview style, communicated the ultimate goals for adapting this style, and defined the interview length. A post-MMI survey was completed by 68 of the 99 applicants, resulting in a response rate of 67%. A total of 67 applicants completed the entire survey, and only one applicant's survey was incomplete.
For the first question, “Do you believe that you were able to present an accurate portrayal of your ability?” 54 of the 68 applicants (79%) responded that they were able to present an accurate portrayal of their ability, answering either “definitely” (28 responses, 41%) or “somewhat” (26 responses, 38%). Three applicants (4.4%) answered “definitely not,” and 11 (16%) answered “not really.” These responses are presented in [Figure 1].
The second question was, “Compared to a standard interview, do you think the MMI causes candidates more or less anxiety?” For this question, 52 of the 68 applicants (75%) responded that the MMI causes less anxiety, with 33 (48.5%) answering “a lot less” and 18 (26.5%) answering “a little less.” Four applicants (5.9%) answered “a lot more,” and 13 (19%) answered “a little more.” These responses are presented in [Figure 2].
|Figure 2: Anxiety experienced at multiple mini-interview compared with traditional interviews|
Click here to view
Regarding whether the instructions given before each station were clear enough, 54 of the 67 applicants (79%) responded “definitely,” 11 (16%) responded “somewhat,” 2 (3%) responded “not really,” and 1 (1.5%) responded “definitely not.”
The majority believed that all stations were well timed. [Table 2] presents the detailed responses regarding whether the time available for each station was appropriate. Regarding the difficulty of the stations, applicants gave overall ratings of easy or somewhat easy. [Table 3] presents the responses concerning the perceived difficulty of each interview section. In response to a question about whether the candidates believed the use of the MMI would stop them from applying to an institution, 63 of 67 respondents (94%) answered “definitely not,” 2 (2.99%) answered “not really,” and 2 (2.99%) answered “somewhat.”
|Table 2: Candidate responses to postmultiple mini-interview survey related to timing of each station|
Click here to view
|Table 3: Candidate responses to postmultiple mini-interview survey related to difficulty of each station|
Click here to view
| Discussion|| |
The adoption of the MMI model for selecting trainees for admission to the internal medicine postgraduate residency training program was supported by our administration throughout the process of design, approval, and implementation. The positive acceptance of the proposal to undertake such a significant transition to a different selection model was enabled through the rigorous efforts of the program's committee members and their clarifying communication with the stakeholders. Further, we were privileged to have examples of prior successful implementation of the MMI model from different disciplines at multiple sites around the world, which was greatly beneficial toward influencing the administration to grant their approval.,,,
The participants in the MMI process considered the stations to be well timed and effectively organized. Many applicants expressed positive views, noting much appreciation and satisfaction concerning how the interviews were conducted. We explained the stations to the interviewers in advance of the MMI to minimize variability and presented a 15-min orientation for applicants about the process, which 97% of applicants reported provided clear instruction.
From the interviewers' perspectives, although most considered the process to be more difficult to conduct than a traditional interview, the MMI had helped them to develop an accurate portrayal of the candidates. Furthermore, the interviewers considered that the MMI was enjoyable and functioned efficiently. A large number of the applicants expressed that the MMI facilitated accurate assessments, with 79% of respondents indicating that they were able to present an accurate portrayal of their abilities.
Moreover, most of the applicants to our program did not indicate experiencing more anxiety with the MMI selection model in comparison with the traditional interview, which contrasted the findings of Eva et al. In fact, 75% of our responding applicants reported experiencing less anxiety with the MMI, compared to traditional interviews. This relatively lower anxiety level in relation to Eva et al.'s MMI model could be attributed to multiple factors. First, we carefully prepared the setting to ensure the applicant's comfort. The applicants remained in their stations, while the interviewers rotated between stations. Second, we designed the stations to allow some rest for applicants to prepare for the next interview. Third, we trained the interviewers to introduce themselves in a calm and accommodating manner while maintaining the highest degree of professionalism and objectivity.
Overall, the interviewers' general and positive acceptance of the MMI accorded with the findings of previous studies of organizations implementing the MMI model., Most of our interviewers were faculty members in the College of Medicine and were familiar with the objective structured clinical examination style used to conduct periodic assessments of undergraduate medical students, which likely explains their high degree of acceptance and ease in implementing the MMI model. The core principle of the MMI selection process is providing a standardized assessment method of the noncognitive domains of applicant's performance which cannot be reliably assessed by the classic interview., The performance of applicants on MMI correlates well with important outcomes of residency training programs such as performance on national licensing examination. However, some authors raised concerns on the internal consistency and reliability of MMI in assessing some of the important noncognitive domains such as professionalism. Understanding the limitations of assessment methods such as the MMI should help training programs design the MMI stations thoughtfully.
Our study has several strengths. First, the King Saud University internal medicine residency program is one of the largest, if not the largest, postgraduate training program within the Kingdom of Saudi Arabia. Second, the selection process was viewed objectively by both the applicants and the interviewers, which provided supportive evidence of the reliability and effectiveness of the process. Third, the number of candidates and the response rate were reasonably high, relative to other local postgraduate training programs. Finally, we believe that the academic experience of the interviewers and the tradition and reputation of King Saud University increased the level of strength.
The limitations of our study include the inability to assess the association between the MMI selection model and the accepted trainees' performance, which is a significant deprivation of valuable data. However, since February 2020, our program has experienced numerous interruptions to its academic and clinical services due to the severity of acute respiratory syndrome coronavirus 2, COVID-19 pandemic. Nevertheless, we intend to report this outcome when our training, routine academic activities, and clinical services commence, and data can be accumulated in future. Further limitations of this study include a smaller sample size compared to more recent studies, and conducting the study in a single academic center limits generalization of the findings to the broader community of internal medicine program applicants, within and outside Saudi Arabia. Despite these limitations, we believe that this is the first study in Saudi Arabia to report the experience of applicants and interviewers regarding the MMI selection model. We believe that the results of our study could assist program directors throughout Saudi Arabia and the world in choosing the most appropriate method for assessing their applicants. Thus, our study may enhance the relationship for collaboration with other national and international programs to conduct larger studies across diverse disciplines.
| Conclusion|| |
In a post-MMI survey, interviewers and responding applicants compared a more traditional selection process to the MMI and considered the MMI to be more objective and organized; specifically, it provided a better assessment of applicant abilities, significantly reduced applicant anxiety, instilled greater confidence among applicants, enabled applicants to express their abilities more effectively, and generated a greater sense of satisfaction among all the participants.
The authors thank the interviewers, the Residency Program Committee, and Director of Postgraduate Medical Education, Dr. Riyadh Al Sehli, for their useful contributions. In addition, we express our gratitude to the College of Medicine Research Center, the Deanship of Scientific Research, and King Saud University.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Provan JL, Cuttress L. Preferences of program directors for evaluation of candidates for postgraduate training. CMAJ 1995;153:919-23.
Wagoner NE, Suriano JR, Stoner JA. Factors used by program directors to select residents. J Med Educ 1986;61:10-21.
Taylor CA, Weinstein L, Mayhew HE. The process of resident selection: A view from the residency director's desk. Obstet Gynecol 1995;85:299-303.
Puryear JB, Lewis LA. Description of the interview process in selecting students for admission to U.S. Medical Schools. J Med Educ 1981;56:881-5.
Burkhardt JC. What can we learn from resident selection interviews? J Grad Med Educ 2015;7:673-5.
Makdisi G, Takeuchi T, Rodriguez J, Rucinski J, Wise L. How we select our residents-A survey of selection criteria in general surgery residents. J Surg Educ 2011;68:67-72.
Love JN, Howell JM, Hegarty CB, McLaughlin SA, Coates WC, Hopson LR, et al
. Factors that influence medical student selection of an emergency medicine residency program: Implications for training programs. Acad Emerg Med 2012;19:455-60.
Gordon MJ, Lincoln JA. Family practice resident selection: Value of the interview. J Fam Pract 1976;3:175-7.
Lauren AR. Hiring as cultural matching: The case of elite professional service firms. Am Sociol Rev 2012;77:999-1022.
Stephenson-Famy A, Houmard BS, Oberoi S, Manyak A, Chiang S, Kim S. Use of the interview in resident candidate selection: A review of the literature. J Grad Med Educ 2015;7:539-48.
Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: The multiple mini-interview. Med Educ 2004;38:314-26.
Oyler DR, Smith KM, Elson EC, Bush H, Cook AM. Incorporating multiple mini-interviews in the postgraduate year 1 pharmacy residency program selection process. Am J Health Syst Pharm 2014;71:297-304.
Dore KL, Kreuger S, Ladhani M, Rolfson D, Kurtz D, Kulasegaram K, et al
. The reliability and acceptability of the multiple mini-interview as a selection instrument for postgraduate admissions. Acad Med 2010;85:S60-3.
Razack S, Faremo S, Drolet F, Snell L, Wiseman J, Pickering J. Multiple mini-interviews versus traditional interviews: Stakeholder acceptability comparison. Med Educ 2009;43:993-1000.
Hofmeister M, Lockyer J, Crutcher R. The acceptability of the multiple mini interview for resident selection. Fam Med 2008;40:734-40.
kumar, Narendra, bhardwaj, shailaja, rahman, Eqram. Multiple mini-interview as a predictor of performance in the objective structured clinical examination among Physician Associates in the United Kingdom: a cohort study. Advances in Medical Education and Practice, 2018;9:239.
Haider SI, Bari MF, Ijaz S. Using multiple mini-interviews for students' admissions in Pakistan: A pilot study. Adv Med Educ Pract 2020;11:179-85.
Reiter HI, Eva KW, Rosenfeld J, Norman GR. Multiple mini-interviews predict clerkship and licensing examination performance. Med Educ 2007;41:378-84.
Sebok SS, Luu K, Klinger DA. Psychometric properties of the multiple mini-interview used for medical admissions: Findings from generalizability and Rasch analyses. Adv Health Sci Educ Theory Pract 2014;19:71-84.
Li Q, Guan X, Wu P, Wang X, Zhou L, Tong Y, et al
. Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia. N Engl J Med 2020;382:1199-207.
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3]