{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T11:36:46Z","timestamp":1782301006428,"version":"3.54.5"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T00:00:00Z","timestamp":1710979200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T00:00:00Z","timestamp":1710979200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Benign paroxysmal positional vertigo (BPPV) is a prevalent form of vertigo that necessitates a skilled physician to diagnose by observing the nystagmus and vertigo resulting from specific changes in the patient\u2019s position. In this study, we aim to explore the integration of eye movement video and position information for BPPV diagnosis and apply artificial intelligence (AI) methods to improve the accuracy of BPPV diagnosis.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>We collected eye movement video and diagnostic data from 518 patients with BPPV who visited the hospital for examination from January to March 2021 and developed a BPPV dataset. Based on the characteristics of the dataset, we propose a multimodal deep learning diagnostic model, which combines a video understanding model, self-encoder, and cross-attention mechanism structure.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Result<\/jats:title>\n                <jats:p>Our validation test on the test set showed that the average accuracy of the model reached 81.7%, demonstrating the effectiveness of the proposed multimodal deep learning method for BPPV diagnosis. Furthermore, our study highlights the significance of combining head position information and eye movement information in BPPV diagnosis. We also found that postural and eye movement information plays a critical role in the diagnosis of BPPV, as demonstrated by exploring the necessity of postural information for the diagnostic model and the contribution of cross-attention mechanisms to the fusion of postural and oculomotor information. Our results underscore the potential of AI-based methods for improving the accuracy of BPPV diagnosis and the importance of considering both postural and oculomotor information in BPPV diagnosis.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-024-02438-x","type":"journal-article","created":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T11:02:30Z","timestamp":1711018950000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Multimodal deep learning-based diagnostic model for BPPV"],"prefix":"10.1186","volume":"24","author":[{"given":"Hang","family":"Lu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuxing","family":"Mao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinsen","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,3,21]]},"reference":[{"key":"2438_CR1","doi-asserted-by":"crossref","unstructured":"Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention\u2013MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18 (pp. 234-241). Berlin: Springer International Publishing; 2015.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"2438_CR2","doi-asserted-by":"crossref","unstructured":"Wu N, Phang J, Park J, Shen Y, Huang Z, Zorin M, Geras KJ. Deep neural networks improve radiologists\u2019 performance in breast cancer screening. IEEE transactions on medical imaging. 2019;39(4):1184-94.","DOI":"10.1109\/TMI.2019.2945514"},{"key":"2438_CR3","doi-asserted-by":"crossref","unstructured":"Manna S, Bhattacharya S, Pal U. SSLM: Self-supervised learning for medical diagnosis from MR video. arXiv e-prints, arXiv-2104. 2021.","DOI":"10.36227\/techrxiv.21382737"},{"key":"2438_CR4","unstructured":"Azad R, Aghdam EK, Rauland A, Jia Y, Avval AH, Bozorgpour A, Merhof D. Medical image segmentation review: The success of u-net. arXiv preprint, arXiv:2211.14830. 2022."},{"key":"2438_CR5","unstructured":"Myronenko A, Siddiquee MMR, Yang D, He Y, Xu D. Automated head and neck tumor segmentation from 3D PET\/CT. arXiv preprint, arXiv:2209.10809. 2022."},{"issue":"2","key":"2438_CR6","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1109\/TPAMI.2018.2798607","volume":"41","author":"T Baltru\u0161aitis","year":"2018","unstructured":"Baltru\u0161aitis T, Ahuja C, Morency LP. Multimodal machine learning: A survey and taxonomy. IEEE Trans Pattern Anal Mach Intell. 2018;41(2):423\u201343.","journal-title":"IEEE Trans Pattern Anal Mach Intell."},{"key":"2438_CR7","first-page":"24206","volume":"34","author":"H Akbari","year":"2021","unstructured":"Akbari H, Yuan L, Qian R, Chuang WH, Chang SF, Cui Y, et al. Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text. Adv Neural Inf Process Syst. 2021;34:24206\u201321.","journal-title":"Adv Neural Inf Process Syst."},{"issue":"12","key":"2438_CR8","doi-asserted-by":"publisher","first-page":"2014","DOI":"10.1093\/jamia\/ocac168","volume":"29","author":"M Golovanevsky","year":"2022","unstructured":"Golovanevsky M, Eickhoff C, Singh R. Multimodal attention-based deep learning for Alzheimer\u2019s disease diagnosis. J Am Med Inform Assoc. 2022;29(12):2014\u201322.","journal-title":"J Am Med Inform Assoc."},{"key":"2438_CR9","doi-asserted-by":"crossref","unstructured":"Huang SC, Shen L, Lungren MP, Yeung S. Gloria: A multimodal global-local representation learning framework for label-efficient medical image recognition. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision. Virtual: IEEE; 2021. p. 3942\u201351.","DOI":"10.1109\/ICCV48922.2021.00391"},{"issue":"7","key":"2438_CR10","doi-asserted-by":"publisher","first-page":"710","DOI":"10.1136\/jnnp.2006.100420","volume":"78","author":"M Von Brevern","year":"2007","unstructured":"Von Brevern M, Radtke A, Lezius F, Feldmann M, Ziese T, Lempert T, et al. Epidemiology of benign paroxysmal positional vertigo: a population based study. J Neurol Neurosurg Psychiatry. 2007;78(7):710\u20135.","journal-title":"J Neurol Neurosurg Psychiatry."},{"issue":"5","key":"2438_CR11","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1016\/S0194-5998(00)70190-2","volume":"122","author":"RA Nunez","year":"2000","unstructured":"Nunez RA, Cass SP, Furman JM. Short-and long-term outcomes of canalith repositioning for benign paroxysmal positional vertigo. Otolaryngol Head Neck Surg. 2000;122(5):647\u201352.","journal-title":"Otolaryngol Head Neck Surg."},{"issue":"22","key":"2438_CR12","doi-asserted-by":"publisher","first-page":"2067","DOI":"10.1212\/01.wnl.0000313378.77444.ac","volume":"70","author":"TD Fife","year":"2008","unstructured":"Fife TD, Iverson D, Lempert T, Furman J, Baloh R, Tusa R, et al. Practice Parameter: Therapies for benign paroxysmal positional vertigo (an evidence-based review):[RETIRED]: Report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology. 2008;70(22):2067\u201374.","journal-title":"Neurology."},{"issue":"3\u20134","key":"2438_CR13","doi-asserted-by":"publisher","first-page":"105","DOI":"10.3233\/VES-150553","volume":"25","author":"M Von Brevern","year":"2015","unstructured":"Von Brevern M, Bertholon P, Brandt T, Fife T, Imai T, Nuti D, et al. Benign paroxysmal positional vertigo: diagnostic criteria. J Vestib Res. 2015;25(3\u20134):105\u201317.","journal-title":"J Vestib Res."},{"key":"2438_CR14","doi-asserted-by":"crossref","first-page":"S1","DOI":"10.1177\/0194599816679931","volume":"156","author":"N Bhattacharyya","year":"2017","unstructured":"Bhattacharyya N, Gubbels SP, Schwartz SR, Edlow JA, El-Kashlan H, Fife T, et al. Clinical practice guideline: benign paroxysmal positional vertigo (update). Otolaryngol-Head Neck Surg. 2017;156:S1\u201347.","journal-title":"Otolaryngol-Head Neck Surg."},{"issue":"1","key":"2438_CR15","doi-asserted-by":"publisher","first-page":"236","DOI":"10.4081\/audiores.2020.236","volume":"10","author":"P Malara","year":"2020","unstructured":"Malara P, Castellucci A, Martellucci S. Upright head roll test: A new contribution for the diagnosis of lateral semicircular canal benign paroxysmal positional vertigo. Audiol Res. 2020;10(1):236.","journal-title":"Audiol Res."},{"key":"2438_CR16","unstructured":"Cui X, Feng Y, Mei L, He C, Lu X, Zhang H, et\u00a0al. The analysis of nystagmus in patients with posterior canal benign paroxysmal positional vertigoin positioning test. Lin Chuang er bi yan hou tou Jing wai ke za zhi= J Clin Otorhinolaryngol Head Neck Surg. 2015;29(1):27\u201330."},{"issue":"4","key":"2438_CR17","first-page":"701","volume":"19","author":"L Na","year":"2021","unstructured":"Na L, Junliang H, Di Z, Wei F, Chen X, Yunsong Z, et al. The Development and Validation of an Artificial Intelligence-Assisted Diagnosis System for BPPV. Chin J Otol. 2021;19(4):701.","journal-title":"Chin J Otol."},{"key":"2438_CR18","unstructured":"Simonyan K, Zisserman A. Two-stream convolutional networks for action recognition in videos. Adv Neural Inf Process Syst. 2014;27."},{"key":"2438_CR19","doi-asserted-by":"crossref","unstructured":"Wang L, Tong Z, Ji B, Wu G. Tdn: Temporal difference networks for efficient action recognition. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. Virtual: IEEE; 2021. p. 1895-1904.","DOI":"10.1109\/CVPR46437.2021.00193"},{"key":"2438_CR20","doi-asserted-by":"crossref","unstructured":"Tran D, Bourdev L, Fergus R, Torresani L, Paluri M. Learning spatiotemporal features with 3d convolutional networks. In: Proceedings of the IEEE international conference on computer vision. Santiago: IEEE; 2015. p. 4489\u201397.","DOI":"10.1109\/ICCV.2015.510"},{"key":"2438_CR21","doi-asserted-by":"crossref","unstructured":"Feichtenhofer C, Fan H, Malik J, He K. Slowfast networks for video recognition. In: Proceedings of the IEEE\/CVF international conference on computer vision. Seoul: IEEE; 2019. p. 6202\u201311.","DOI":"10.1109\/ICCV.2019.00630"},{"key":"2438_CR22","doi-asserted-by":"crossref","unstructured":"Feichtenhofer C. X3d: Expanding architectures for efficient video recognition. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. Virtual: IEEE; 2020. p. 203\u201313.","DOI":"10.1109\/CVPR42600.2020.00028"},{"key":"2438_CR23","unstructured":"Ryoo MS, Piergiovanni AJ, Tan M, Angelova A. Assemblenet: Searching for multi-stream neural connectivity in video architectures. arXiv preprint arXiv:1905.13209. 2019."},{"key":"2438_CR24","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Houlsby N. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. 2020."},{"key":"2438_CR25","doi-asserted-by":"crossref","unstructured":"Neimark D, Bar O, Zohar M, Asselmann D. Video transformer network. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision. Virtual: IEEE; 2021. p. 3163\u201372.","DOI":"10.1109\/ICCVW54120.2021.00355"},{"key":"2438_CR26","doi-asserted-by":"crossref","unstructured":"Ding X, Zhang X, Han J, Ding G. Scaling up your kernels to 31x31: Revisiting large kernel design in cnns. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE; 2022. p. 11963\u201375.","DOI":"10.1109\/CVPR52688.2022.01166"},{"key":"2438_CR27","doi-asserted-by":"crossref","unstructured":"Liu Z, Mao H, Wu CY, Feichtenhofer C, Darrell T, Xie S. A convnet for the 2020s. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE; 2022. p. 11976\u201386.","DOI":"10.1109\/CVPR52688.2022.01167"},{"key":"2438_CR28","doi-asserted-by":"crossref","unstructured":"Chen YC, Li L, Yu L, El Kholy A, Ahmed F, Gan Z, Liu J. Uniter: Universal image-text representation learning. In European conference on computer vision (pp. 104-120). Cham: Springer International Publishing. 2020, August.","DOI":"10.1007\/978-3-030-58577-8_7"},{"key":"2438_CR29","unstructured":"Joze HRV, Shaban A, Iuzzolino ML, Koishida K. MMTM: Multimodal transfer module for CNN fusion. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. Virtual: IEEE; 2020. p. 13289\u201399."},{"key":"2438_CR30","doi-asserted-by":"crossref","unstructured":"He K, Chen X, Xie S, Li Y, Doll\u00e1r P, Girshick R. Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE; 2022. p. 16000\u20139.","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"2438_CR31","doi-asserted-by":"crossref","unstructured":"Tan H, Bansal M. Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490. 2019.","DOI":"10.18653\/v1\/D19-1514"},{"key":"2438_CR32","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. p. 770\u20138.","DOI":"10.1109\/CVPR.2016.90"},{"issue":"1","key":"2438_CR33","first-page":"30","volume":"14","author":"J McClure","year":"1985","unstructured":"McClure J. Horizontal canal BPV. J Otolaryngol. 1985;14(1):30\u20135.","journal-title":"J Otolaryngol."},{"key":"2438_CR34","doi-asserted-by":"crossref","unstructured":"Dix MoR CS. Hallpike. \"The pathology, symptomatology and diagnosis of certain common disorders of the vestibular system.\" California: SAGE Publications; 1952. p 341-354.","DOI":"10.1177\/003591575204500604"},{"key":"2438_CR35","doi-asserted-by":"crossref","unstructured":"Wang Z, Yan W, Oates T. Time series classification from scratch with deep neural networks: A strong baseline. In 2017 International joint conference on neural networks (IJCNN) (pp. 1578-1585). Alaska: IEEE; 2017, May.","DOI":"10.1109\/IJCNN.2017.7966039"},{"key":"2438_CR36","doi-asserted-by":"crossref","unstructured":"Cai Z, Fan Q, Feris RS, Vasconcelos N. A unified multi-scale deep convolutional neural network for fast object detection. In Computer Vision\u2013ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11\u201314, 2016, Proceedings, Part IV 14 (pp. 354-370). Berlin: Springer International Publishing; 2016.","DOI":"10.1007\/978-3-319-46493-0_22"},{"key":"2438_CR37","doi-asserted-by":"crossref","unstructured":"Karim F, Majumdar S, Darabi H, Harford S. Multivariate LSTM-FCNs for time series classification. Neural Networks. 2019;116:237-45.","DOI":"10.1016\/j.neunet.2019.04.014"},{"key":"2438_CR38","doi-asserted-by":"crossref","unstructured":"Ismail Fawaz H, Lucas B, Forestier G, Pelletier C, Schmidt DF, Weber J, Petitjean F. Inceptiontime: Finding alexnet for time series classification. Data Mining and Knowledge Discovery. 2020;34(6):1936-62.","DOI":"10.1007\/s10618-020-00710-y"},{"key":"2438_CR39","doi-asserted-by":"crossref","unstructured":"Dempster A, Petitjean F, Webb GI. ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery. 2020;34(5):1454-95.","DOI":"10.1007\/s10618-020-00701-z"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-024-02438-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-024-02438-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-024-02438-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T11:05:04Z","timestamp":1711019104000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-024-02438-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,21]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["2438"],"URL":"https:\/\/doi.org\/10.1186\/s12911-024-02438-x","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,21]]},"assertion":[{"value":"5 March 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 January 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The study design was reviewed and approved by the Ethics Committee of the Second Affiliated Hospital of Army Medical University. This study used de-identified data. All experimental protocols were approved by the Ethics Committee of the Second Affiliated Hospital of the Army Medical University. And The Ethics Review Committee of the Second Affiliated Hospital of Army Medical University reviewed and approved the waiver of informed consent. All methods were performed in accordance with relevant guidelines and regulations.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"82"}}