Accepted Type
Oral
Code
OH1-2-5
Acceptance Declaration
Accept
Additional Information
I declare I have no actual or potential conflict of interest in relation to this program.
Was this work accepted for CCME 2020?
no
Category
General Call (Workshop, Oral Presentation, Poster Presentation)
Type
Oral
Sub Type
Education Research
Will the presenter be a:
Student
Affiliation
Considered for Poster
yes
Title
A Target Population Derived Method for Establishing Radiograph Interpretation Competency
Length of Presentation
Background/Purpose
Clinical skills competency standards are often developed in an ad hoc method, with a poorly delineated connection to the target clinical population. Using pediatric musculoskeletal (MSK) radiograph interpretation as an example, we inserted the Ebel criterion-referenced method into an end-to-end standard setting procedure to derive a performance-based standard.
Methods
This was a multi-center prospective mixed-methods study. Emergency physicians interpreted 1,835 pediatric MSK extremity radiographs using an on-line platform. This generated 195,321 data points, which were used to determine case difficulty terciles (easy, intermediate, hard). Cases were also ranked by clinical significance (low, medium, high) and each case was plotted in a three-by-three matrix of difficulty and significance. A multidisciplinary panel of physician and parent stakeholders then determined acceptable accuracy for each of the nine cells. An overall competency standard was derived from the weighted sum.
Results
The median difficulty rating was -1.8 logits (IQR -4.1, 3.2), with significant differences between body regions (p<0.0001). There were 1,055 (57.8%) low, 424 (23.1%) medium, and 356 (19.1%) high risk cases. The panel's acceptable scores across the nine cells ranged from 76% to 95% and the sum of equal weighted scores yielded an overall performance-based competency score of 85.5% accuracy.
Conclusion
Using a purposeful sample of radiographs and the Ebel method of standard setting, we derived a performance-based competency standard for the skill of radiograph interpretation. This method could be generalized to drive evidence-based competency standard-setting for other image sets and can be tailored to specific levels of professional accountability.
Keyword 1
Clinical Education
Keyword 2
Assessment
Keyword 3
Simulation
Level of Training
General
Abstract Themes
Teaching and learning
Teaching and Learning
- Clinical Skills
- Competency-Based Education
- E-Learning/Technology
- Feedback
- Integrated instruction & learning
- Simulation
Additional Theme (First choice)
Assessment
Additional Theme (Second Choice)
Continuing Medical Education
Additional Theme (Third Choice)
Authors
Presenter
Joshua E. Herman
Term 1
Yes
Term 2
Yes
Term 3
Yes
Term 4
Yes
Term 5
Yes
