THE PERFORMANCE OF LARGE LANGUAGE MODELS IN BONE TUMOUR IMAGING: COMPARATIVE ANALYSIS WITH RADIOLOGISTS USING TEXT AND IMAGE-BASED EVALUATION


Çamur E., Cesur T., Güneş Y. C., DURAN S.

Comptes Rendus de L'Academie Bulgare des Sciences, cilt.79, sa.1, ss.95-102, 2026 (SCI-Expanded, Scopus)

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 79 Sayı: 1
  • Basım Tarihi: 2026
  • Doi Numarası: 10.7546/crabs.2026.01.12
  • Dergi Adı: Comptes Rendus de L'Academie Bulgare des Sciences
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, BIOSIS, MathSciNet, zbMATH
  • Sayfa Sayıları: ss.95-102
  • Anahtar Kelimeler: ChatGPT, artificial intelligence, bone tumours, large language models, radiology
  • Sağlık Bilimleri Üniversitesi Adresli: Hayır

Özet

Large language models (LLMs) are emerging as transformative tools in radiology, with potential to enhance diagnostic workflows. However, their performance in bone tumour imaging – a domain requiring both knowledge-based reasoning and visual interpretation – remains unclear. This study compares the diagnostic performance of LLMs with radiologists across text and image-based tasks. In this cross-sectional study, two LLMs and two radiologists (a junior and a senior) were evaluated using fifty text-based multiple-choice questions (MCQs) and fifty radiographs with clinical vignettes from a public dataset. Participants classified lesions as benign or malignant, identified “don’t-touch” lesions, and provided the most likely diagnosis. Responses were benchmarked against a reference standard using McNemar’s tests. In MCQs, ChatGPT-5 (92.0%) and Gemini 2.5 Pro (90.0%) achieved accuracies comparable to SR (88.0%) and JR (84.0%) (p > 0.05). For benign–malignant classification, LLMs (50.0%, 48.0%) were similar to JR (66.0%) but inferior to SR (94.0%) (p < 0.05). In identifying “don’t-touch” lesions, LLMs (46.0%) matched JR (64.0%) yet underperformed compared to SR (92.0%) (p < 0.05). For specific diagnosis, LLMs showed low accuracy (38.0%, 30.0%) versus JR (60.0%) and SR (86.0%) (p < 0.01). LLMs may serve as useful adjuncts for clinicians and radiologists in text-based tasks and in distinguishing between benign and malignant bone tumours. However, their diagnostic accuracy remains limited.