Accepted Type
Oral
Code
OA1-2-4
Acceptance Declaration
Accept
Additional Information
I declare I have no actual or potential conflict of interest in relation to this program.
Was this work accepted for CCME 2020?
no
Category
General Call (Workshop, Oral Presentation, Poster Presentation)
Type
Oral
Sub Type
Education Innovation
Will the presenter be a:
Other
Presenter Other
Faculty
Affiliation
Considered for Poster
yes
Title
Structured File Review: Inter-rater reliability and impact on socioeconomic diversity
Length of Presentation
Background/Purpose
In medical school admissions, autobiographical sketches and statements have been fraught with low reliability and concerns about an adverse impact on applicants with socioeconomic challenges. This study evaluated a recently implemented structured abbreviated autobiographical sketch (AABS) in Schulich Medicine Admissions in terms of reliability and effect on socioeconomic diversity of the incoming medical school class.
Summary of the Innovation
In 2018, the AABS was created to assess applicants' values and life experiences in relation to Schulich Medicine's four core values: 1. Teamwork and leadership; 2. Social accountability and social responsibility; 3. Respect for diversity, equity, and inclusion; and 4. Higher learning skills. Applicant AABS files (n=730) randomly distributed to physician and community member raters were scored using a structured tool. No other information (e.g. GPA, MCAT scores, full name, demographics) was provided to raters. In addition to implicit bias training, all raters received an unidentified test file to establish standardized statistical criteria for the need for an additional tie-breaker rater. Socioeconomic status using the AAMC parental education-occupation tool was captured through a voluntary diversity survey and available for 571 (78%) of the applicant files reviewed.
Conclusion
The reliability was high with a Generalizability coefficient of 0.93 and the variance of scores was primarily attributed to applicants (74%). Socioeconomic diversity of the applicant pool vs. admitted students was not statistically different (27.2% vs. 23.1%, p=0.31). Using a structured AABS tool in appropriately trained raters allows for high reliability and no significant impact on applicants with socioeconomic challenges.
Keyword 1
file review
Keyword 2
socioeconomic status
Keyword 3
medical school admissions
Level of Training
General
Abstract Themes
Admissions
Additional Theme (First choice)
Additional Theme (Second Choice)
Additional Theme (Third Choice)
Authors
Presenter
Saad Chahine
Term 1
Yes
Term 2
Yes
Term 3
Yes
Term 4
Yes
Term 5
Yes
