Submission ID 77327

Code P2
At the end of this workshop, participants will be able to:
Category Medical Education
Type Poster
Will the presenter be a: Other
Presenter Other MD
Title Implementing A New Canadian Resident Matching Service Dossier Tool at the Department of Family Medicine at the University of Ottawa
Background/Purpose The Department of Family Medicine (DFM), University of Ottawa receives 850-1000 Canadian Resident Matching Service (CaRMS) applications each year. The DFM scores each dossier to determine which candidates receive an interview offer. The previous method for reviewing dossiers lacked interrater reliability. We created a standardized dossier review tool to achieve interrater reliability for 2022.
Methods A national survey and literature review allowed the creation of a survey that focused on dossier evaluators' perceived important dossier elements. The results were assessed for themes. Themes were used to create a second survey that asked dossier evaluators to rank thematic elements using a Likert scale. Using the Delphi method each element with >70% ranking agreement was considered to reach consensus. These were used to create a draft review tool. This was piloted in 2021, parallel to the actual dossier reviews process. Elements with 70% reliability were used to create a final dossier review tool. This tool was implemented for all dossier reviews in the 2022 selection process. 50 dossiers were reviewed three times, each by three distinct reviewers. The process was evaluated with a separate survey to evaluators.
Results Interrater reliability was established for 87% of the tool elements. Evaluators felt the tool was simple, focused, and reliable. This met our desired outcomes.
Discussion The tool we created is reliable between evaluators. We will review the 2022 match data, to determine if the most appropriate candidates were selected. We will follow this cohort to determine possible outcomes such as lower rates of remediation or transfer requests.
Keyword 1 Selection
Keyword 2 Carms
Keyword 3 Dossier
Abstract content most relevant to: (check all that apply) Residency Education
Abstract Track - First Choice Admissions
Authors Edward Seale
Sohil Rangwala
Ed Seale
Chandra Landry
Kim Rozon

Loading . . .
please wait . . . loading