SubmissionId 29992

PCW - 5


  • Maria Palacios
  • Elizabeth Oddone Paolucci


Reg Fee?


Length of Presentation
Half Day (4 hours)


Abstract (Rationale / Background / Purpose)

Rationale/Background: Medical educators seem to struggle with the concept of reliability of assessment scores. Reliability issues in medical education have been largely addressed using Classical Test Theory (CTT), even in situations where Generalizability Theory (G-theory) is more appropriate, mainly due to educators’ lack of understanding of G-theory and its practical applications. In an increasingly evidence-based oriented medical education world, medical educators need to have a clear understanding of appropriate analytic methods in order to critically assess the scientific literature that will inform their practice. G-theory is a statistical method that allows educators to assess the dependability (or reliability) of behavioral measurements. In other words, the ability to generalize from a person’s observed score on a test or another assessment, to the score that person may obtain under a universe of possible test conditions. Unlike CTT, G-theory allows us to estimate the influence of multiple sources of measurement error in one single analysis. The purpose of this workshop is to introduce learners to the foundations of G-theory and its practical application in medical education.

Instructional Methods: Short presentations of workshop content will be followed by hands-on exercises. No statistical software will be used. The learners will have ample opportunity to apply theoretical content carrying out practical exercises throughout the workshop.

Target Audience: Educators with an interest in measurement in medical education. No previous experience with statistical methods used in reliability studies is required.

Purpose: To increase participants understanding of G-theory and its practical applications.

Learning objectives: At the end of the workshop the learners will be able to:

  • Describe the main differences between Classical Test Theory and G-theory
  • Describe the practical uses of G-theory
  • Identify the object of measurement and facets of generalization in a G-study
  • Identify different sources of error that contribute to inaccuracy of measurements
  • Identify different G-study designs
  • Describe the three steps in G-analysis
  • Design a basic G-study
  • Describe the difference between relative and absolute error
  • Provide a basic interpretation of G-coefficients
  • Describe the practical uses of a Decision-study (D-study)

Term 1

Term 2

Term 3

Term 4

Term 5

Loading . . .
please wait . . . loading