Skip to content
    Accreditation & Policy

    Evaluating Assessment Methods in Psychiatry Didactic Education: A Structured Narrative Review

    Coelho et alHarvard review of psychiatry

    Evaluating Assessment Methods in Psychiatry Didactic Education: A Structured Narrative Review
    A structured narrative review synthesized evidence on didactic educational interventions and structured assessment strategies used across psychiatry residency and continuing medical education (CME) programs. A literature search across PubMed, ERIC, and PsycInfo conducted in August 2024 identified 13 studies that met the eligibility criteria of assessing didactic or curriculum-based interventions in psychiatry residents or practicing psychiatrists using validated or replicable residency-developed instruments.
    The didactic formats identified across the included literature encompassed short-format lectures, simulation-based learning, role-play, team-based learning, CME, and longitudinal quality improvement curricula. Five of the 13 studies reported formal psychometric metrics for their evaluation tools, including the Harvard Program in Refugee Trauma Confidence Questionnaire (Cronbach's α = 0.97), the CME Teaching Effectiveness Instrument (Cronbach's α = 0.94), and the Modified Course Impact Questionnaire (Cronbach's α = 0.76–0.88). Additionally, 3 studies integrated objective performance assessments, such as standardized patient examinations, milestone ratings, and structured vignettes.
    The authors noted that while several individual studies demonstrated strong psychometric properties, the broader literature was constrained by inconsistent reporting of reliability and validity, as well as an absence of long-term follow-up. The review concludes that evaluation methodology variability restricts comparability and replicability across psychiatry training curricula, underscoring the necessity for validated instruments, performance-based endpoints, and longitudinal outcome tracking in future didactic program development.

    Key Takeaways

    • Across 13 included studies evaluating didactic interventions in psychiatry training, only 5 utilized instruments with reported psychometric data, including the Harvard Program in Refugee Trauma Confidence Questionnaire (Cronbach's α = 0.97), the CME Teaching Effectiveness Instrument (Cronbach's α = 0.94), and the Modified Course Impact Questionnaire (Cronbach's α = 0.76–0.88).
    • Objective performance measures were incorporated in only 3 of the 13 studies—specifically utilizing standardized patient assessments, milestone ratings, and structured vignettes—while many studies lacked long-term follow-up and demonstrated inconsistent reporting of validity and reliability.
    Read the original sourceHarvard review of psychiatry