Diagnostic accuracy of artificial intelligence technologies in the analysis of magnetic resonance imaging results for the detection and assessment of demyelinating lesions: a retrospective diagnostic study on benchmark datasets
https://doi.org/10.33667/2078-5631-2025-21-45-49
Abstract
Objective. To evaluate the diagnostic accuracy of artificial intelligence (AI) technologies for automated detection of magnetic resonance imaging (MRI) signs of multiple sclerosis (MS) and differentiation from other pathologies using benchmark datasets.
Materials and methods. A retrospective diagnostic study was conducted in accordance with the STARD 2015 methodology. Two AI services integrated into the Unified Radiological Information Service of the Moscow EMIAS were tested. The benchmark dataset (n=100) included results of brain MRI with and without contrast. Diagnostic accuracy metrics were assessed: area under the ROC curve (AUROC), sensitivity, specificity, and accuracy.
Results. AI service 1 demonstrated an accuracy of 0.86 (95% CI 0.79–0.93), sensitivity of 0.73 (95% CI 0.6–0.85), and specificity of 0.98 (95% CI 0.94–1.0). AI service 2 showed superior results: accuracy of 0.99 (95% CI 0.97–1.0), sensitivity of 1.0 (95% CI 1.0–1.0), and specificity of 0.98 (95% CI 0.94–1.0).
Conclusions. AI technologies show high potential for automating MRI analysis in MS diagnosis. However, significant differences in accuracy and reproducibility between AI services highlight the need for further research in real-world clinical settings.
About the Authors
N. D. Adamiia
Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department
Russian Federation
Russian Federation
Adamiia Naala D., radiologist, postgraduate student
Moscov
A. V. Vladzymyrskyy
Research and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Moscow Health Care Department
Russian Federation
Russian Federation
Vladzymyrskyy Anton V., DM Sci (habil.), deputy director for research
Moscov
References
1. Yagudina R. I., Kulikov A. Yu., & Litvinenko M. M. The economic burden of multiple sclerosis. Farmakoekonomika: Teoriya i Praktika. 2013; 1 (1): 3–34. (In Russ.). https://doi.org/10.30809/phe.1.2013.1
2. Boiko A. N., Kukel T. M., Lysenko M. A., Vdovichenko T. V., Gusev E. I. (Clinical epidemiology of multiple sclerosis in Moscow: Descriptive epidemiology based on the population of one city district. Zhurnal Nevrologii i Psikhiatrii Imeni S. S. Korsakova. Spetsvypuski. 2013; 113 (10–2): 8–14. (In Russ.).
3. Thompson A. J., Banwell B. L., Barkhof F. et al. Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. The Lancet Neurology. 2018; 17 (2): 162–173.
4. Filippi M., Rocca M. A., Ciccarelli O. et al. MRI criteria for the diagnosis of multiple sclerosis: MAGNIMS consensus guidelines. Multiple Sclerosis Journal 2020; 26 (4): 502–512.
5. Morozov S. P., Pavlov N. A., Petryaykin A. V. et al. (2022). MosMedData: A set of diagnostic magnetic resonance images of the brain with and without signs of multiple sclerosis [Database]. Certificate of State Registration of Database No. 2022621197, Russian Federation. (In Russ.).
6. Vladzymyrskyy A. V., Shulkin I. M., Omelyanskaya O. V. et al. (2023). MosMedData: MRI with signs of multiple sclerosis [Database]. Certificate of State Registration of Database No. 2023621236, Russian Federation. (In Russ.).
7. Thompson A. J. et al. (2018). Диагностика рассеянного склероза: пересмотр критериев Макдональда за 2017 год. Lancet Neurol. 17: 162–173. DOI: 10.1016/S1474–4422(17)30470-2
8. Federau C., Hainc N., Edjlali M., Zhu G., Mastilovic M., Nierobisch N., Uhlemann J. P., Paganucci S., Granziera C., Heinzlef O., Kipp L. B., Wintermark M. Quality and performance assessment of neuro-radiological follow-up MRI reading in multiple sclerosis using intelligent software automation. Neuroradiology. 2024; 66 (3): 361–369. (In Russ.). https://doi.org/10.1007/s00234-024-03293-3
9. Sieber V, Rusche T, Yang S, Stieltjes B, Fischer U, Trebeschi S, Cattin P, Nguyen-Kim DL, Psychogios MN, Lieb JM, Sporns PB. Automated assessment of brain MRIs in multiple sclerosis patients significantly reduces reading time. Neuroradiology. 2024 Dec; 66 (12): 2171–2176. DOI: 10.1007/s00234-024-03497-7. Epub 2024 Nov 8. PMID: 39514032; PMCID: PMC 11611969.
10. Mendelsohn Z., Pemberton H. G., Gray J., Goodkin O., Carrasco F. P., Scheel M. et al. Commercial volumetric MRI reporting tools in multiple sclerosis: a systematic review of the evidence. Neuroradiology. 2023 Jan; 65 (1): 5–24.
11. Sonke Peters, Lars Schmill, Carl Alexander Gless, Klarissa Sturner, Olav Jansen, Svea Seehafer. AI-based assessment of longitudinal multiple sclerosis MRI: Strengths and weaknesses in clinical practice. Eur J Radiol. 2025 Feb; 183: 111941. https://doi.org/10.1016/j.ejrad.2025.111941
12. Bobrovskaya T.M., Vasilyev Yu.A., Nikitin N. Yu., Arzamasov, K. M. Approaches to the formation of datasets in radiology. Vrach i Informatsionnye Tekhnologii. 2023; 4: 14–23. (In Russ.). https://doi.org/10.25881/18110193_2023_4_14
13. Vasilyev Yu.A., Vladzymyrskyy A. V., Omelyanskaya O. V. et al. (2023). Methodology for testing and monitoring artificial intelligence-based software for medical diagnostics. Digital Diagnostics. 2023; 4 (3): 252–267. (In Russ.). https://doi.org/10.17816/DD321971
14. Scientific-practical clinical center for diagnostics and telemedicine technologies of the Moscow healthcare department. (2023). Clinical trials of artificial intelligence systems (radiological diagnostics). Moscow. (In Russ.).
15. Nahm FS. Receiver operating characteristic curve: overview and practical use for clinicians. Korean J Anesthesiol. 2022 Feb; 75 (1): 25–36. DOI: 10.4097/kja.21209
16. Tuchinov B. N., Suvorov V., Motorin K. O., Pavlovskiy E. N., Vasylkiv L. M., Stankevich Yu.A., Tulupov A. A. Application of a computer vision algorithm for detecting demyelination foci in multiple sclerosis on MRI images. Siberian Scientific Medical Journal. 2024; 44 (1): 107–115. (In Russ.).
17. Shpakovsky, A. Yu., Mulitsa, A. V., & Blagochinnaya, K. V. Potential of artificial intelligence in characterizing demyelinating lesions in patients with multiple sclerosis. Comorbidnaya Neurologiya. 2024; 1 (4): 38–43. (In Russ.).
18. Fedulov A., Karapetyan G., Kosik I., Borisov A., Blagochinnaya K., Volkova N. Artificial intelligence technologies in monitoring pathomorphological changes of the central nervous system in multiple sclerosis. Nauka i Innovatsii. 2023;2 (240): 75–83. (In Russ.).
19. Collorone S, Coll L, Lorenzi M, Llado X, Sastre-Garriga J, Tintore M, Montalban X, Rovira A, Pareto D, Tur C. Artificial intelligence applied to MRI data to tackle key challenges in multiple sclerosis. Mult Scler. 2024 Jun; 30 (7): 767–784. DOI: 10.1177/13524585241249422
20. Amin M, Martinez-Heras E, Ontaneda D, Prados Carrasco F. Artificial Intelligence and Multiple Sclerosis. Curr Neurol Neurosci Rep. 2024 Aug; 24 (8): 233–243. DOI: 10.1007/s11910-024-01354-x
Review
For citations:
Adamiia N.D., Vladzymyrskyy A.V. Diagnostic accuracy of artificial intelligence technologies in the analysis of magnetic resonance imaging results for the detection and assessment of demyelinating lesions: a retrospective diagnostic study on benchmark datasets. Medical alphabet. 2025;(21):45-49. (In Russ.) https://doi.org/10.33667/2078-5631-2025-21-45-49
Views:
38
Email this article 