Quality of AI chatbot-generated information on hypersensitivity pneumonitis for clinical and patient use
Sarcoidosis Vasculitis and Diffuse Lung Diseases, cilt.43, sa.2, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 43 Sayı: 2
- Basım Tarihi: 2026
- Doi Numarası: 10.36141/svdld.2026.18399
- Dergi Adı: Sarcoidosis Vasculitis and Diffuse Lung Diseases
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, EMBASE
- Anahtar Kelimeler: Hypersensitivity pneumonitis, artificial intelligence, chatbots.
- Sağlık Bilimleri Üniversitesi Adresli: Evet
Özet
Background and Aim: Hypersensitivity pneumonitis (HP) is a complex, immune mediated interstitial lung disease in which accurate diagnosis and long term management require integration of clinical, radiologic, and exposure-related information. Patients increasingly use artificial intelligence (AI) based chatbots to obtain disease related information; however, the quality, readability, and patient usability of such content remain unclear. This study aimed to evaluate the quality, reliability, readability, and patient-centered usability of AI chatbot generated information on HP. Materials and Methods: Using Google Trends, we identified four of the most frequently searched patient-oriented questions regarding HP: (1) What is HP and what causes it? (2) What are the clinical features of HP? (3) How is HP treated? (4) How is HP diagnosed? These questions were submitted verbatim to eight AI chatbots (ChatGPT-5.1, Claude 3, Microsoft Copilot, DeepSeek V3, Gemini Pro, Grok 4, Kimi K2, Perplexity AI). A total of 32 responses were independently evaluated in a blinded fashion by four pulmonology professors specializing in interstitial lung diseases. Content quality and reliability were assessed using DISCERN; understandability and actionability with PEMAT-P; global written readability with the Written Readability Rating (WRR); and structural readability with the Flesch-Kincaid Grade Level (FKGL). Results: All chatbot outputs required advanced literacy, with FKGL scores ranging from 20.17 to 29.07 and a mean of approximately 24–25, indicating college or postgraduate reading level. No chatbot produced content within the recommended patient-appropriate range (FKGL ≤ 8). WRR scores declined with increasing clinical complexity, from 67.85 for definitional content (Q1) to 51.227 for diagnostic explanations (Q4). DISCERN scores varied substantially across models (35.001–57.103), with most chatbots falling into the “fair-good” range, reflecting partially reliable but incomplete information. PEMAT-P understandability scores were moderate (highest: 66.302% for DeepSeek V3), whereas actionability was consistently low, indicating a lack of clear, concrete patient instructions. Conclusion: AI chatbots can generate clinically rich explanations of HP but currently produce content that is too complex and insufficiently actionable for most patients. The observed “readability-quality gap” suggests that chatbot-generated information should not yet replace professionally curated patient education materials. A hybrid model in which AI-generated drafts are refined by clinical experts may represent the most appropriate strategy for safe integration of AI tools into patient education on complex interstitial lung diseases such as HP.