Using Artificial Intelligence to Evaluate the Quality of Narrative Feedback in Surgical Entrustable Professional Activities.
Listen to this summary
This study asked whether fine-tuned large language models could reproduce expert ratings of narrative feedback quality in surgical EPA micro-assessments, and whether a simplified rubric (mQuAL) would improve consistency. Across 1,509 comments from a general surgery residency, encoder and generative models closely matched expert scores for suggestion and connection, with encoder models outperforming generative ones for evidence (accuracy 0.93 vs 0.77). Simplifying evidence scoring to a binary rubric reduced rater disagreement from 27.8% to 8.5% and improved model performance, supporting AI-assisted feedback quality assessment in surgical training.
This is one of 33,000+ journals available on OSLR. Try it free for 14 days.
Free 14-day trial. 33,000+ journals. Cancel anytime.

More from Journal of the American College of Surgeons
View all →Nov 20, 2026 · Journal of the American College of Surgeons
Gastrointestinal and Hepatopancreatobiliary Cancer Surgical Deserts and Population Cancer Mortality: A National County-Level Analysis.
Nov 19, 2026 · Journal of the American College of Surgeons
Locum Tenens Surgery: What We Know, What We Need to Know, and What We Should Do.
Nov 19, 2026 · Journal of the American College of Surgeons
Trauma Clinician Perspectives on Geriatric-Centered Trauma Care: A Qualitative Study.
Nov 18, 2026 · Journal of the American College of Surgeons
Preoperative Advanced Therapy Use and Anastomotic Complications After Elective Surgery for Crohn's Disease.
Nov 18, 2026 · Journal of the American College of Surgeons
Role of Surgeon-Model Interaction in the Real-World Performance of Surgical Risk Prediction Tools.
Nov 18, 2026 · Journal of the American College of Surgeons
State Variation in Crime Victim Compensation Denials after Violent Injury in the United States.
More in Surgery
View all →Nov 20, 2026 · Journal of pain and symptom management
ACCOMPANI: A Psychosocial Nurse Intervention for African American Families in Pediatric Oncology.
Nov 20, 2026 · Journal of pain and symptom management
Perceived Clinical Utility of a Routine Family Experience Feedback Loop in Pediatric Palliative Care.
Nov 20, 2026 · Journal of pain and symptom management
Predictors of Clinician Success, Quality, and Resilience in Outpatient Pediatric Palliative Care.
Nov 20, 2026 · Surgery
Midterm outcomes with extended follow-up of open and endovascular repair of visceral artery aneurysms using a target-artery preservation strategy.
Nov 20, 2026 · Surgery
Associations of smoking and obesity with chronic pain after incisional ventral hernia repair: a register- and survey-based nationwide study.
Nov 20, 2026 · Surgery
Racial disparities in minimally invasive pancreatectomy: A systematic review and meta-analysis.
"Oslr has become part of my weekly routine on my day off. The clinical relevance of the summaries is outstanding — I'd rate it 9/10. Being able to consume research hands-free is a huge advantage for busy physicians."
Dr. Jennifer Thompson
Portland, OR


