Assessing the clinical competence of large language models for tobacco use disorder: A multi-domain expert evaluation

The authors aimed to systematically evaluate the clinical competence of five large language models (LLMs) in providing tobacco cessation support, focusing on their accuracy, safety, guideline adherence, and communication quality. Their findings indicated that while all LLMs showed some competence, GPT-4.5 and Claude 3.5 Sonnet performed best, achieving scores suitable for supervised clinical use, whereas open-weight models required further validation before clinical implementation. The study underscores the importance of clinician oversight in medication-based interventions for tobacco use disorder.