Submission ID 77327
| Code | P2 |
|---|---|
| At the end of this workshop, participants will be able to: | |
| Category | Medical Education |
| Type | Poster |
| Will the presenter be a: | Other |
| Presenter Other | MD |
| Title | Implementing A New Canadian Resident Matching Service Dossier Tool at the Department of Family Medicine at the University of Ottawa |
| Background/Purpose | The Department of Family Medicine (DFM), University of Ottawa receives 850-1000 Canadian Resident Matching Service (CaRMS) applications each year. The DFM scores each dossier to determine which candidates receive an interview offer. The previous method for reviewing dossiers lacked interrater reliability. We created a standardized dossier review tool to achieve interrater reliability for 2022. |
| Methods | A national survey and literature review allowed the creation of a survey that focused on dossier evaluators' perceived important dossier elements. The results were assessed for themes. Themes were used to create a second survey that asked dossier evaluators to rank thematic elements using a Likert scale. Using the Delphi method each element with >70% ranking agreement was considered to reach consensus. These were used to create a draft review tool. This was piloted in 2021, parallel to the actual dossier reviews process. Elements with 70% reliability were used to create a final dossier review tool. This tool was implemented for all dossier reviews in the 2022 selection process. 50 dossiers were reviewed three times, each by three distinct reviewers. The process was evaluated with a separate survey to evaluators. |
| Results | Interrater reliability was established for 87% of the tool elements. Evaluators felt the tool was simple, focused, and reliable. This met our desired outcomes. |
| Discussion | The tool we created is reliable between evaluators. We will review the 2022 match data, to determine if the most appropriate candidates were selected. We will follow this cohort to determine possible outcomes such as lower rates of remediation or transfer requests. |
| Keyword 1 | Selection |
| Keyword 2 | Carms |
| Keyword 3 | Dossier |
| Abstract content most relevant to: (check all that apply) | Residency Education |
| Abstract Track - First Choice | Admissions |
| Authors | Edward Seale Sohil Rangwala Ed Seale Chandra Landry Kim Rozon |
