Submission ID 109382
| Code | OI-4-5 |
|---|---|
| Category | Medical Education |
| Type | Oral |
| Abstract Title | Using Chatgpt in Psychiatry to Design Script Concordance Tests in Undergraduate Medical Education: Mixed Methods Study |
| Will the presenter be a: | Early Career Faculty - (Less than 7 years of practice) |
| Methods | This mixed method study evaluated 3 ChatGPT-generated SCTs with 3 expert-created SCTs using a predefined framework. Clinician-educators as well as resident doctors in psychiatry involved in undergraduate medical education in Quebec, Canada, evaluated via a web-based survey the 6 SCTs on 3 criteria: the scenario, clinical questions, and expert opinion. They were also asked to describe the strengths and weaknesses of the SCTs. |
| Results | A total of 102 respondents assessed the SCTs. There were no significant distinctions between the 2 types of SCTs concerning the scenario ( P=.84), clinical questions ( P=.99), and expert opinion ( P=.07), as interpretated by the respondents. Indeed, respondents struggled to differentiate between ChatGPT- and expert-generated SCTs. ChatGPT showcased promise in expediting SCT design, aligning well with Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition criteria, albeit with a tendency toward caricatured scenarios and simplistic content. |
| Discussion | This study is the first to concentrate on the design of SCTs supported by AI in a period where medicine is changing swiftly and where technologies generated from AI are expanding much faster. This study suggests that ChatGPT can be a valuable tool in creating educational materials, and further validation is essential to ensure educational efficacy and accuracy. |
| Keyword 1 | artificial intelligence |
| Keyword 2 | concordance scripts |
| Keyword 3 | medical education |
| Abstract content most relevant to: (check all that apply) | Post Graduate Medical Education |
| Abstract Track - First Choice | Teaching and learning |
| Teaching and Learning | E-Learning/Technology |
| Authors | Alexandre Hudon (Lead Author) Alexandre Hudon (Presenter) Véronique Phan (Presenter) Barnabé Kiepura (Co Author) Myriam Pelletier (Co Author) Véronique Phan (Co Author) |
