Evaluation of ChatGPT-4o® responses on pediatric urolithiasis: Is it useful?
Urolithiasis, cilt.53, sa.1, 2025 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 53 Sayı: 1
- Basım Tarihi: 2025
- Doi Numarası: 10.1007/s00240-025-01880-4
- Dergi Adı: Urolithiasis
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, BIOSIS, EMBASE, MEDLINE
- Anahtar Kelimeler: ChatGPT-4o, GQS, mDISCERN, Pediatric, Urolithiasis
- Sağlık Bilimleri Üniversitesi Adresli: Evet
Özet
This study aims to evaluate the reliability, quality and readability of ChatGPT-4o responses regarding pediatric urolithiasis. Forty frequently asked questions about pediatric urinary stones were posed to ChatGPT-4o twice, one week apart. The reliability of ChatGPT-4o’s responses was assessed using the five-point DISCERN tool (mDISCERN). The overall quality of the responses was evaluated using the Global Quality Scale (GQS). To assess the readability of ChatGPT-4o’s responses, multiple metrics were employed, including the Flesch Reading Ease (FRE) score, the Flesch-Kincaid Grade Level (FKGL), the Gunning Fog Index (GFI), the Coleman-Liau Index (CLI), and the Simple Measure of Gobbledygook (SMOG). The median mDISCERN score was 5 (range: 4–5), and the median GQS score was 5 (range: 3–5), indicating high reliability and quality. However, readability metrics suggested a high level of difficulty: FRE (27.98 ± 13.65), FKGL (11.46 ± 1.88), SMOG (14.96 ± 1.64), GFI (17.27 ± 2.37), and CLI (15.60 ± 1.95). Only 2.5% of responses were understandable to individuals with reading skills at a 10–12-year-old level, 37.5% were suitable for college-level readers, and 60% required professional-level comprehension. A moderate correlation was observed between mDISCERN and GQS scores (r = 0.42, p = 0.007), but neither correlated significantly with readability metrics. ChatGPT-4o may provide reliable and high-quality information about pediatric urinary stones; however, the advanced reading level of its responses presents a significant barrier to accessibility for patients and caregivers. Therefore, despite its potential utility, the readability challenge must be addressed to ensure equitable patient education.