The capabilities of artificial intelligence in segmentation and detection of spinal images at the current stage of development: a systematic review

INTRODUCTION: From the very beginning of the theoretical substantiation of the creation of artificial intelligence, there were ideas about creating an electronic assistant for a doctor. The most effective for solving such a problem in medical diagnostics are complex systems based on one of the most advanced methods of artificial intelligence — neural networks. Study design: a systematic review.

OBJECTIVE: Determining the capabilities of neural networks at the current stage of their development in the field of such tasks as segmentation and detection of spine images.

MATERIALS AND METHODS: Using the PRISMA protocol, a search was performed in the Pubmed database for the period from January 2017 to December 31, 2022 using keywords «deep learning» OR «neural network» OR «artificial Intelligence», AND («spine») AND («detection» OR «segmentation»).

RESULTS: For the systematic review, 30 articles were selected that described such functions of neural networks as segmentation and detection in the analysis of spinal images. DISCUSSION: Based on the analysis of literature sources, conclusions were made about the usefulness of using artificial intelligence at the current stage of development in vertebrology.

CONCLUSION: Neural networks cope quite well with the tasks of segmentation and detection in vertebrology. In segmentation tasks, there is a clear leader — U-Net and its various modifications. In detection, the leading architectures are: SCRL, 3D FCN, CNN of our own design and a combination of Mask R-CNN+ResNet101 networks. Software based on the work of artificial intelligence can help both a radiologist and a vertebrologist reduce the load and simplify the work by automating and semi-automating the diagnostic process.

1. Sarki R., Ahmed K., Wang H., Zhang Y., Wang K. Automated detection of COVID-19 through convolutional neural network using chest x-ray images // PloS ONE. 2022. Vol. 17, No. 1. e0262052. doi: 10.1371/journal.pone.0262052

2. Kann B.H., Hosny A., Aerts H. Artificial intelligence for clinical oncology // Cancer Cell. 2021. Vol. 39, No. 12. P. 916–927. doi: 10.1016/j.ccell.2021.04.002.

3. Gerussi A., Scaravaglio M., Cristoferi L. et al. Artificial intelligence for precision medicine in autoimmune liver disease // Front Immunol. 2022. Vol. 13, No. 11. P. 966329. doi: 10.3389/fimmu.2022.966329.

4. Kröner P.T., Engels M.M., Glicksberg B.S. et al. Artificial intelligence in gastroenterology: A state-of-the-art review // World J. Gastroenterol. 2021. Vol. 40, No. 27. P. 6794–6824. doi: 10.3748/wjg.v27.i40.6794.

5. Tricco A.C., Lillie E., Zarin W., O’Brien K.K., Colquhoun H., Levac. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation // Annals of Internal Medicine. 2018. Vol. 169, No. 7. P. 467–473. doi: 10.7326/M18-0850.

6. Vrtovec T., Ibragimov B. Spinopelvic measurements of sagittal balance with deep learning: systematic review and critical evaluation // European Spine Journal. 2022. Vol. 31, No. 8. P. 2031–2045. doi: 10.1007/s00586-022-07155-5.

7. Qu B., Cao J., Qian C. et al. Current development and prospects of deep learning in spine image analysis: a literature review // Quant. Imaging Med. Surg. 2022. Vol. 12. P. 3454–3479. doi: 10.21037/qims-21-939.

8. Lundervold A. S. An overview of deep learning in medical imaging focusing on MRI // Zeitschrift für Medizinische Physik. 2019. Vol. 29, No. 2. P. 102–127. doi: 10.1016/j.zemedi.2018.11.002.

9. Goedmakers C.M.W., Pereboom L.M., Schoones J.W., de Leeuw den Bouter M.L, Remis R.F. et al. Machine learning for image analysis in the cervical spine: Systematic review of the available models and methods // Brain & Spine. 2022. Vol. 2, No. 14. P. 101666. doi: 10.1016/j.bas.2022.101666.

10. Wang R., Lei T., Cui R., Zhang B., Meng H., Nandi A. Medical image segmentation using deep learning: A survey // IET Image Processing. 2022. Vol. 16, No. 5. P. 1243–1267. doi: 10.1049/ipr2.12419

11. Ashraf R., Habib M.A., Akram M. et al. Deep convolution neural network for big data medical image classification // IEEE Access. 2020. Vol. 8. P. 105659–105670. doi: 10.1109/ACCESS.2020.2998808.

12. Dalechina A.V., Beljaev M.G., Tjurina A.N., Zolotova S.V., Pronin I.N., Golanov A.V. Machine learning methods in glioma segmentation for planning stereotactic radiotherapy. Diagnostic radiology and radiotherapy, 2019, No. 2, рр. 24–31 (In Russ.) doi: 10.22328/2079-5343-2019-10-2-24-31.

13. Litjens G., Kooi T., Bejnordi B.E., Setio A.A.A., Ciompi F. et al. A survey on deep learning in medical image analysis // Medical image analysis. 2017. Vol. 42. P. 60– 88. doi: 10.1016/j.media.2017.07.005.

14. Al Arif S. M. R., Knapp K., Slabaugh G. Fully automatic cervical vertebrae segmentation framework for X-ray images // Computer methods and programs in biomedicine. 2018. Vol. 157. P. 95–111. doi: 10.1016/j.cmpb.2018.01.006.

15. Kim S., Bae W.C., Masuda K., Chung C.B., Hwang D. Fine-grain segmentation of the intervertebral discs from MR spine images using deep convolutional neural networks: BSU-Net // Applied sciences. 2018. Vol. 8, No. 9. P. 1656. doi: 10.3390/app8091656.

16. Fan G., Liu H., Wu Z. et al. Deep learning–based automatic segmentation of lumbosacral nerves on CT for spinal Intervention: a translational Study // American Journal of Neuroradiology. 2019. Vol. 40, No. 6. P. 1074–1081. doi: 10.3174/ajnr.A6070.

17. Huang J., Shen H., Wu J. et al. Spine Explorer: a deep learning based fully automated program for efficient and reliable quantifications of the vertebrae and discs on sagittal lumbar spine MR images // The Spine Journal. 2020. Vol. 20, No. 4. P. 590–599. doi: 10.1016/j.spinee.2019.11.010.

18. Kolařík M., Burget R., Uher V., Říha K., Dutta M.K. Optimized high resolution 3D dense-U-Net network for brain and spine segmentation // Applied Sciences. 2019. Vol. 9, No. 3. P. 404. doi: 10.3390/app9030404.

19. Rehman F., Ali Shah F.I., Riaz N., Gilani S.O. A robust scheme of vertebrae segmentation for medical diagnosis // IEEE Access. 2019. Vol. 7. P. 120387–120398. doi: 10.1109/ACCESS.2019.2936492.

20. Fang Y., Li W., Chen X. et al. Opportunistic osteoporosis screening in multi-detector CT images using deep convolutional neural networks // European Radiology. 2021. Vol. 31, No. 4. P. 1831–1842. doi: 10.1007/s00330-020-07312-8.

21. Zhang Q., Du Y., Wei Z., Liu H., Yang X., Zhao D. Spine medical image segmentation based on deep learning // Journal of Healthcare Engineering. 2021. Vol. 2021. P. 1917946. doi: 10.1155/2021/1917946.

22. Rak M., Steffen J., Meyer A., Hansen C., Tönnies K.D. Combining convolutional neural networks and star convex cuts for fast whole spine vertebra segmentation in MRI // Computer Methods and Programs in Biomedicine. 2019. Vol. 177. P. 47–56. doi: 10.1016/j.cmpb.2019.05.003.

23. Masood R. F., Taj I. A., Khan M. A., Qureshi M. A., Hassan T. Deep learning based vertebral body segmentation with extraction of spinal measurements and disorder disease classification // Biomedical Signal Processing and Control. 2022. Vol. 71. P. 103230. doi: 10.1016/j.bspc.2021.103230.

24. Lavdas I., Glocker B., Kamnitsas K. et al. Fully automatic, multiorgan segmentation in normal whole body magnetic resonance imaging (MRI), using classification forests (CF s), convolutional neural networks (CNN s), and a multi‐atlas (MA) approach // Medical physics. 2017. Vol. 44, No. 10. P. 5210–5220. doi: 10.1002/mp.12492.

25. Han Z., Wei B., Mercado A., Leung S., Li S. Spine-GAN: Semantic segmentation of multiple spinal structures // Medical image analysis. 2018. Vol. 50. P. 23–35. doi: 10.1016/j.media.2018.08.005.

26. Li X., Dou Q., Chen H. et al. 3D multi-scale FCN with random modality voxel dropout learning for intervertebral disc localization and segmentation from multi-modality MR images // Medical image analysis. 2018. Vol. 45. P. 41–54. doi: 10.1016/j.media.2018.01.004.

27. Das P., Pal C., Acharyya A., Chakrabarti A., Basu S. Deep neural network for automated simultaneous intervertebral disc (IVDs) identification and segmentation of multi-modal MR images // Computer Methods and Programs in Biomedicine. 2021. Vol. 205. P. 106074. doi: 10.1016/j.cmpb.2021.106074.

28. Huo X., Cui G., Tan J., Shao K. Automatic measurement of axial vertebral rotation in 3D vertebral models // Biomedical Physics and Engineering Express. 2021. Vol. 7, No. 6. P. 065034. doi: 10.1088/2057-1976/ac2c55.

29. Tran V.L., Lin H.Y., Liu H.W. MBNet: A multi-task deep neural network for semantic segmentation and lumbar vertebra inspection on X-ray images // Proceedings of the Asian Conference on Computer Vision. 2020. doi: 10.1007/978-3-030-69541-5_38.

30. Zhang Q., Du Y., Wei Z. et al. Spine medical image segmentation based on deep learning // Journal of Healthcare Engineering. 2021. Vol. 2021. P. 1917946. doi: 10.1155/2021/1917946.

31. Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for biomedical image segmentation // Medical Image Computing and Computer-Assisted Intervention. 2015. P. 234–241. doi: 10.1007/978-3-319-24574-4_28.

32. Abdullina K.M., Spivak A.I. Application of a neural network of the U-Net architecture for the task of localizing violations of the integrity of digital images. Scientific and technical bulletin of information technologies, mechanics and optics, 2020, Vol. 20, No. 3, рр. 425–431 (In Russ.). doi: 10.17586/2226-1494-2020-20-3-425-431.

33. Hamad Yu.A., Simonov K.V., Kenc A.S. Algorithms for segmentation and recognition of objects in medical images based on shearlet transformation and neural networks. Informatization and communication, 2020, No. 2, рр. 35–45 (In Russ.). doi: 10.34219/2078-8320-2020-11-2-35-45.

34. Charng J., Xiao D., Mehdizadeh M. et al. Deep learning segmentation of hyperautofluorescent fleck lesions in Stargardt disease // Scientific Reports. 2020. Vol. 10, No. 1. P. 16491. doi: 10.1038/s41598-020-73339-y.

35. Sezer A. Mask Region-Based Convolutional Neural Network segmentation of the humerus and scapula from proton density-weighted axial shoulder magnetic resonance images // Jt. Dis. Relat. Surg. 2023. Vol. 20, No. 34. P. 583–589. doi: 10.52312/jdrs.2023.1291.

36. Heller N., Athianathen N., Kalapara A. et al. The kits19 challenge data: 300 kidney tumor cases with clinical context, ct semantic segmentations, and surgical outcomes // arXiv preprint. arXiv: 201904.00445.2019.

37. Zhao S., Wu X., Chen B., Li S. Automatic vertebrae recognition from arbitrary spine MRI images by a category-Consistent self-calibration detection framework // Medical Image Analysis. 2021. Vol. 67. P. 101826. doi: 10.1016/j.media.2020.101826.

38. Hallinan J.T.P.D., Zhu L., Yang K. et al. Deep learning model for automated detection and classification of central canal, lateral recess, and neural foraminal stenosis at lumbar spine MRI // Radiology. 2021. Vol. 300, No. 1. P. 130–138. doi: 10.1148/radiol.2021204289.

39. Huang Y., Uneri A., Jones C.K. et al. 3D vertebrae labeling in spine CT an accurate, memory-efficient (Ortho2D) framework // Physics in Medicine & Biology. 2021. Vol. 66, No. 12. P. 125020. doi: 10.1088/1361-6560/ac07c7.

40. Forsberg D., Sjöblom E., Sunshine J.L. Detection and Labeling of Vertebrae in MR Images Using Deep Learning with Clinical Annotations as Training Data // Journal of Digital Imaging. 2017. Vol. 30. P. 406–412. doi: 10.1007/s10278-017-9945-x.

41. Kurochka K.S., Tsalka I.M. Vertebrae detection in X-ray images based on deep convolutional neural networks // 2017 IEEE 14th International Scientific Conference on Informatics. IEEE. 2017. P. 194–196.

42. Yang D., Xiong T., Xu D., Huang Q., Liu D., Zhou S.K. et al. Automatic vertebra labeling in large-scale 3D CT using deep image-to-image network with message passing and sparsity regularization // International Conference on Information Processing in Medical Imaging. 2017. P. 633–644. doi: 10.1007/978-3-319-59050-9_50.

43. Liao H., Mesfin A., Luo J. Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long Range Contextual Information // IEEE transactions on medical imaging. 2018. Vol. 37. P. 1266–1275. doi: 10.1109/TMI.2018.2798293

44. Jakubicek R., Chmelik J., Ourednicek P., Jan J. Deep-learning-based fully automatic spine centerline detection in CT data // 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 2019. P. 2407–2410. doi: 10.1109/EMBC.2019.8856528.

45. Zhou Y., Liu Y., Chen Q., Gu G., Sui X. Automatic lumbar MRI detection and identification based on deep learning // Journal of digital imaging. 2019. Vol. 32, No. 3. P. 513–520. doi: 10.1007/s10278-018-0130-7.

46. Chen Y., Gao Y., Li K., Zhao L., Zhao J. Vertebrae identification and localization utilizing fully convolutional networks and a hidden Markov model // IEEE Transactions on Medical Imaging. 2019. Vol. 39, No. 2. P. 387–399. doi: 10.1109/TMI.2019.2927289.

47. Roggen T., Bobic M., Givenchi N., Scheib S.G. Deep Learning model for markerless tracking in spinal SBRT // Physica Medica. 2020. Vol. 74. P. 66–73. doi: 10.1016/j.ejmp.2020.04.029.

48. Zhang D., Chen B., Li S. Sequential conditional reinforcement learning for simultaneous vertebral body detection and segmentation with modeling the spine anatomy // Medical Image Analysis. 2021. Vol. 67. P. 101861. doi: 10.1016/j.media.2020.101861.

49. Kim K.C., Cho H.C., Jang T.J., Choi J.M., Seo J.K. Automatic detection and segmentation of lumbar vertebrae from X-ray images for compression fracture evaluation // Computer Methods and Programs in Biomedicine. 2021. Vol. 200. P. 105833. doi: 10.1016/j.cmpb.2020.105833.

50. Wang Yu., Huang L., Wu M. et al. Multi-input adaptive neural network for automatic detection of cervical vertebral landmarks on X-rays // Computers in Biology and Medicine. 2022. Vol. 146. P. 105576. doi: 10.1016/j.compbiomed.2022.105576.