Journal of Continuing Education in the Health Professions



HOME \| SITE MAP \| CONTACT US
Your Location: Home > Volume 29, Issue 4

CURRENT ISSUE BACK ISSUES SUBSCRIBE ADVERTISE ABOUT JCEHP FOR AUTHORS JCEHP AWARD SEARCH		Volume 29 (4) Volume 29, Issue 4, Fall 2009 J Contin Educ Health Prof 2009; 29(4): 242 ORIGINAL RESEARCH Interrater reliability to assure valid content in peer review of CME-accredited presentations Mark Quigg, Fred A. Lado A b s t r a c t Introduction: The Accreditation Council for Continuing Medical Education (ACCME) provides guidelines for continuing medical education (CME) materials to mitigate problems in the independence or validity of content in certified activities; however, the process of peer review of materials appears largely unstudied and the reproducibility of peer-review audits for ACCME accreditation and designation of American Medical Association Category 1 Credit^TM is unknown. Methods: Categories of presentation defects were constructed from discussions of the CME committee of the American Epilepsy Society: (1) insufficient citation, (2) poor formatting, (3) nonacknowledgment of non-FDA– approved use, (4) misapplied data, (5) 1-sided data, (6) self- or institutional promotion, (7) conflict of interest/ commercial bias, (8) other, or (9) no defect. A PowerPoint lecture (n = 29 slides) suitable for presentation to general neurologists was purposefully created with the above defects. A multirater, multilevel kappa statistic was determined from the number and category of defects. Results: Of 14 reviewers, 12 returned completed surveys (86%) identifying a mean6 standard deviation 1.661.1 defects / slide. The interrater kappa equaled 0.115 (poor reliability) for number of defects / slides. No individual categories achieved kappa > 0.38. Discussion: Interrater reliability on the rating of durable materials used in subspecialty CME was poor. Guidelines for CME appropriate content are too subjective to be applied reliably by raters knowledgeable in their specialty field but relatively untrained in the specifics of CME requirements. The process of peer review of CME materials would be aided by education of physicians on validation of materials appropriate for CME. Lessons For Practice Physician peer-reviewers of materials for CME-accredited presentations have poor interrater reliability in designating the numbers and types of defects in durable teaching materials. Although reliability was poor, identification of potential commercial bias had the best interrater reliability. Physicians untrained in the requirements of CME accreditation may venture outside the scope of CME review.
HOME \| SITE MAP \| CONTACT US Copyright © 1996-2009 JCEHP.com & The Journal of Continuing Education in the Health Professions All rights reserved Disclaimer · About This Site · Web Editor · Make JCEHP Your Homepage Information on this site was last updated: 24 January 2010