Self-Study from Web-Based and Printed Guideline Materials

A Randomized, Controlled Trial among Resident Physicians

20 June 2000 | Volume 132 Issue 12 | Pages 938-946

Background: On-line physician education is increasing, but its efficacy in comparison with existing self-study methods is unknown.

Objective: To compare knowledge, learning efficiency, and learner satisfaction produced by self-study of World Wide Web–based and print-based guidelines for care after acute myocardial infarction.

Design: Randomized, controlled trial.

Setting: 12 family medicine and internal medicine residency programs at four universities.

Participants: 162 residents.

Interventions: In proctored sessions, participants were randomly assigned to study from printed materials or from SAGE (Self-Study Acceleration with Graphic Evidence), a Web-based tutorial system. Both methods used identical self-assessment questions and answers and guideline text, but SAGE featured hyperlinks to specific guideline passages and graphic evidence animations.

Measurements: Scores on multiple-choice knowledge tests, score gain per unit of study time, and ratings on a learner satisfaction scale.

Results: Immediate post-test scores on a 20-point scale were similar in the SAGE and control groups (median score, 15.0 compared with 14.5; P > 0.2), but SAGE users spent less time studying (median, 27.0 compared with 38.5 minutes; P < 0.001) and therefore had greater learning efficiency (median score gain, 8.6 compared with 6.7 points per hour; P = 0.04). On a scale of 5 to 20, SAGE users were more satisfied with learning (median rating, 17.0 compared with 15.0; P < 0.001). After 4 to 6 months, knowledge had decreased to the same extent in the SAGE and control groups (median score, 12.0 compared with 11.0; P = 0.12).

Conclusions: On-line tutorials may produce greater learning efficiency and satisfaction than print materials do, but one self-study exposure may be insufficient for long-term knowledge retention. Further research is needed to identify instructional features that motivate greater final learning and retention.

Author and Article Information

From University of California, Los Angeles, School of Medicine, Los Angeles, California.

Presented in part at the Society for General Internal Medicine Annual Meeting, 30 April–1 May 1999, San Francisco, California.

Acknowledgments: The authors thank the SAGE trial participants and the following chief residents, program directors, and administrators who were instrumental in their residents' participation: David Graham, MD, Oregon Health Sciences University, Portland, Oregon; Jodi Friedman, MD, and Roland Sakiyama, MD, University of California, Los Angeles, Los Angeles, California; Harry Hollander, MD, and Jill Thomas, University of California, San Francisco, San Francisco, California; Stephanie Silas, MD, H. James Williams, MD, Ita M. Killeen, MD, and Judi Weston, University of Utah, Salt Lake City, Utah. They also thank LuAnn Wilkerson, PhD, Anju Relan, PhD, and Martin Shapiro, MD, PhD, for insightful critical review of the instructional design and the trial design.

Grant Support: In part by a National Research Service Award (T32 PE19001-09) from the Health Resources and Services Administration of the U.S. Department of Health and Human Services. Additional project support was provided by the University of California, Los Angeles, Stein–Oppenheimer Fund and by the GTE Foundation through the University of California, Los Angeles, Center for Digital Innovation. Dr. Mangione was partially supported by the Robert Wood Johnson Foundation as a Generalist Faculty Scholar (award no. 029250).

Requests for Single Reprints: Douglas S. Bell, MD, University of California, Los Angeles, Division of General Internal Medicine and Health Services Research, 911 Broxton Plaza, Room 218, Los Angeles, CA 90095-1736; e-mail, sagequery{at}gim.med.ucla.edu.

Requests To Purchase Bulk Reprints (minimum, 100 copies): the Reprints Coordinator; phone, 215-351-2657; e-mail, reprints{at}mail.acponline.org.

Current Author Addresses: Drs. Bell, Hays, and Mangione: Department of Medicine, Division of General Internal Medicine and Health Services Research, University of California, Los Angeles, 911 Broxton Plaza, Los Angeles, CA 90095-1736.

Dr. Fonarow: Department of Medicine, Division of Cardiology, University of California, Los Angeles, CHS 67-130A, Los Angeles, CA 90095-1679.

Author Contributions: Conception and design: D.S. Bell, G.C. Fonarow, R.D. Hays, C.M. Mangione.

Analysis and interpretation of the data: D.S. Bell, G.C. Fonarow, R.D. Hays, C.M. Mangione.

Drafting of the article: D.S. Bell, G.C. Fonarow, R.D. Hays, C.M. Mangione.

Critical revision of the article for important intellectual content: D.S. Bell, G.C. Fonarow, R.D. Hays, C.M. Mangione.

Final approval of the article: D.S. Bell, G.C. Fonarow, R.D. Hays, C.M. Mangione.

Provision of study materials or patients: D.S. Bell, G.C. Fonarow, C.M. Mangione.

Statistical expertise: D.S. Bell, R.D. Hays, C.M. Mangione.

Obtaining of funding: D.S. Bell, C.M. Mangione.

Administrative, technical, or logistic support: D.S. Bell, C.M. Mangione.

Collection and assembly of data: D.S. Bell.