VALIDATION OF DIAGNOSTIC ACCURACY OF ANARTIFICIAL INTELLIGENCE ALGORITHM FOR DETECTING MULTIPLE SCLEROSIS IN A CITY POLYCLINIC SETTING

The objective of the study is to evaluate the diagnostic accuracy of an original artificial intelligence (AI) algorithm for detecting MS in the radiology department of primary (outpatient) hospital.

Materials and methods. Depersonalized results of brain magnetic resonance imaging (MRI) studies performed in the period from August 22, 2019 to September 26, 2019 in 93 patients (42 men (mean age 47,5±15,9 years) and 51 women (mean age 52,3±16,8 years)) were analyzed. All patients signed a voluntary informed consent form. Brain MRIwere carried out on the VANTAGE Atlas 1,5T MRI scanner (Toshiba, Japan) under a standard protocol.

Results. All MRI studies were analyzed by AI-algorithm (index-test). It decisions were compared with a  reference test (groundtruth). The sensitivity of the index-test is 100%, specificity — 75,3%, accuracy —  76,3%, negative predictive value — 100%, area under ROC-curve — 0,861. The algorithm reliably sorts out the studies without signs of MS. The algorithmshows sufficient quality and excellent reproducibility of the results on independent data.

Conclusion. The developed AI algorithm ensures effective triage of MRI studies in primary care settings, maintaining an optimal index of suspicion in MS.

1. Hanoh E.V., Rozhdestvenskij A.S., Kudryavceva E.A. [Research on hereditary factors of multiple sclerosis susceptibility and peculiarities of its course in russian ethnic group. The Bulletin of Siberian Branch of Russian Academy of Medical Science, 2011, Vol. 31, No. 1, рр. 113–118 (In Russ.).

2. Atlas of MS 2013. Multiple Sclerosis International Federation, 2013. 28 р.

3. Shmidt T.E., Yahno N.N. Multiple sclerosis: inflammation, degeneration: guidelines for clinicians. 3rd ed. Мoscow: MEDpress-inform, 2012, 272 р. (In Russ.).

4. Karussis D. The diagnosis of multiple sclerosis and the various related demyelinating syndromes: a critical review // J. Autoimmun. 2014. Feb-Mar. Vol. 48–49. P. 134–142. doi: 10.1016/j.jaut.2014.01.022.

5. Guillemin F., Baumann C., Epstein J. LORSEP Group. Older Age at Multiple Sclerosis Onset Is an Independent Factor of Poor Prognosis: A PopulationBased Cohort Study // Neuroepidemiology. 2017. No. 48 (3–4). P. 179–187. doi: 10.1159/000479516.

6. Gelfand J.M. Multiple sclerosis: diagnosis, differential diagnosis, and clinical presentation // Handb Clin Neurol. 2014. No 122. P. 269–290. doi: 10.1016/B978-0-444-52001-2.00011-X.

7. Bryuhov V.V., Krotenkova I.A., Morozova S.N. A current view on the MRI diagnosis of multiple sclerosis: an update of 2016 revised MRI criteria. S.S. Korsakov journal of neurology and psychiatry, 2017, Vol. 117, No. 2–2, рp. 66–73 (In Russ.).

8. Gombolevskij V.A., Lajpan A.Sh., Shapiev A.N. MAGNIMS diagnosis and control criteria use in multiple sclerosis. Мoscow, 2018; Issue 11. 12 р. (Series «Best practices in imaging and instrumental diagnostics») (In Russ.).

9. Zaharova M.N., Abramova A.A., Askarova L.Sh. et al. Multiple sclerosis: diagnosis and treatment issues. Мoscow: MediaMente, 2018, 240 р. (In Russ.).

10. Gusev A.V., Pliss M.A. The basic recommendations for the creation and development of information systems in health care based on artificial intelligence. Information technologies for the physician, 2018, No. 3, рp. 45–60 (In Russ.).

11. Morozov S.P., Vladzimirskij A.V., Gombolevskij V.A., Kuz’mina E.S., Ledihova N.V. Artificial intelligence: natural language processing for peerreview in radiology. Journal of radiology and nuclear medicine, 2018, Vol. 99, No. 5, рp. 253–258 (In Russ.).

12. Arani L.A., Hosseini A., Asadi F. Intelligent Computer Systems for Multiple Sclerosis Diagnosis: a Systematic Review of Reasoning Techniques and Methods // Acta Inform Med. 2018. Dec. 26 (4). P. 258–264. doi: 10.5455/aim.2018.26:258-264.

13. Ranschaert E.R., Morozov S.P., Algra P.R. Artificial intelligence in medical imaging. Springer International Publishing, 2019. 369 р.

14. Solomon A.J., Naismith R.T., Cross A.H. Misdiagnosis of multiple sclerosis: Impact of the 2017 McDonald criteria on clinical practice // Neurology. 2019. Jan 1. Vol. 92, No. 1. P. 26–33. doi: 10.1212/WNL.0000000000006583.

15. Morozov S.P., Vladzymyrskyy A.V., Klyashtornyj V.G., Andrejchenko A.E., Kul’berg N.S., Gombolevskij V.A., Serguniva K.A. Clinical studies of intelligence technology-based software (radiology).]. Мoscow, 2019, Issue 57, 33 р. (Series «Best practices in imaging and instrumental diagnostics») (In Russ.).

16. Morozov S.P., Vladzymyrskyy A.V., Klyashtornyy V.G. Clinical acceptance of software based on artificial intelligence technologies (radiology). M., 2019, Issue 57, 51 р. (Series «Best practices in medical imaging»). arXiv: 1908.00381.

17. Avants B.B., Tustison N., Song G. Advanced normalization tools (ANTS) // Insight j. 2009. Jun 4. No. 2. P. 1–35.

18. He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition // Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. P. 770–778.

19. Gros C., De Leener B., Badji A. Automatic segmentation of the spinal cord and intramedullary multiple sclerosis lesions with convolutional neural networks // Neuroimage. 2019. Jan 1. No. 184. P. 901–915. doi: 10.1016/j.neuroimage.2018.09.081.

20. Wang S.H., Tang C., Sun J. Multiple Sclerosis Identification by 14-Layer Convolutional Neural Network With Batch Normalization, Dropout, and Stochastic Pooling // Front Neurosci. 2018. Nov. 8. P. 812–818. doi: 10.3389/fnins.2018.00818.

21. Yoo Y., Tang L.Y.W., Brosch T. Deep learning of joint myelin and T1w MRI features in normal-appearing brain tissue to distinguish between multiple sclerosis patients and healthy controls // Neuroimage Clin. 2017. Oct. 14. No. 17. P. 169–178. doi: 10.1016/j.nicl.2017.10.015.

22. Valverde S., Cabezas M., Roura E. Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach // Neuroimage. 2017. Jul 15. No. 155. P. 159–168. doi: 10.1016/j.neuroimage.2017.04.034.

23. Hackmack K., Paul F., Weygandt M. Alzheimer’s Disease Neuroimaging Initiative. Multi-scale classification of disease using structural MRI and wavelet transform // Neuroimage. 2012. Aug 1. No. 62 (1). P. 48–58. doi: 10.1016/j.neuroimage.2012.05.022.