{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T19:45:05Z","timestamp":1771703105008,"version":"3.50.1"},"reference-count":55,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,6,16]],"date-time":"2021-06-16T00:00:00Z","timestamp":1623801600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"UK Engineering and Physical Sciences Research Council (EPSRC)","award":["EP\/R005273\/1"],"award-info":[{"award-number":["EP\/R005273\/1"]}]},{"name":"Elizabeth Blackwell Institute for Health Research, University of Bristol and the Wellcome Trust Institutional Strategic Support Fund","award":["204813\/Z\/16\/Z"],"award-info":[{"award-number":["204813\/Z\/16\/Z"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Parkinson\u2019s disease (PD) is a chronic neurodegenerative condition that affects a patient\u2019s everyday life. Authors have proposed that a machine learning and sensor-based approach that continuously monitors patients in naturalistic settings can provide constant evaluation of PD and objectively analyse its progression. In this paper, we make progress toward such PD evaluation by presenting a multimodal deep learning approach for discriminating between people with PD and without PD. Specifically, our proposed architecture, named MCPD-Net, uses two data modalities, acquired from vision and accelerometer sensors in a home environment to train variational autoencoder (VAE) models. These are modality-specific VAEs that predict effective representations of human movements to be fused and given to a classification module. During our end-to-end training, we minimise the difference between the latent spaces corresponding to the two data modalities. This makes our method capable of dealing with missing modalities during inference. We show that our proposed multimodal method outperforms unimodal and other multimodal approaches by an average increase in F1-score of 0.25 and 0.09, respectively, on a data set with real patients. We also show that our method still outperforms other approaches by an average increase in F1-score of 0.17 when a modality is missing during inference, demonstrating the benefit of training on multiple modalities.<\/jats:p>","DOI":"10.3390\/s21124133","type":"journal-article","created":{"date-parts":[[2021,6,16]],"date-time":"2021-06-16T21:58:32Z","timestamp":1623880712000},"page":"4133","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Multimodal Classification of Parkinson\u2019s Disease in Home Environments with Resiliency to Missing Modalities"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6358-2261","authenticated-orcid":false,"given":"Farnoosh","family":"Heidarivincheh","sequence":"first","affiliation":[{"name":"School of Computer Science, Electrical and Electronic Engineering, and Engineering Maths, University of Bristol, Bristol BS8 1UB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7708-3110","authenticated-orcid":false,"given":"Ryan","family":"McConville","sequence":"additional","affiliation":[{"name":"School of Computer Science, Electrical and Electronic Engineering, and Engineering Maths, University of Bristol, Bristol BS8 1UB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0333-2417","authenticated-orcid":false,"given":"Catherine","family":"Morgan","sequence":"additional","affiliation":[{"name":"Translational Health Sciences, University of Bristol Medical School, Bristol BS8 1UD, UK"},{"name":"Movement Disorders Group, Bristol Brain Centre, North Bristol NHS Trust, Bristol BS10 5PN, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3761-296X","authenticated-orcid":false,"given":"Roisin","family":"McNaney","sequence":"additional","affiliation":[{"name":"Department of Human Centred Computing, Monash University, Melbourne, VIC 3000, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6510-835X","authenticated-orcid":false,"given":"Alessandro","family":"Masullo","sequence":"additional","affiliation":[{"name":"School of Computer Science, Electrical and Electronic Engineering, and Engineering Maths, University of Bristol, Bristol BS8 1UB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6478-1403","authenticated-orcid":false,"given":"Majid","family":"Mirmehdi","sequence":"additional","affiliation":[{"name":"School of Computer Science, Electrical and Electronic Engineering, and Engineering Maths, University of Bristol, Bristol BS8 1UB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8931-4422","authenticated-orcid":false,"given":"Alan L.","family":"Whone","sequence":"additional","affiliation":[{"name":"Translational Health Sciences, University of Bristol Medical School, Bristol BS8 1UD, UK"},{"name":"Movement Disorders Group, Bristol Brain Centre, North Bristol NHS Trust, Bristol BS10 5PN, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6552-8541","authenticated-orcid":false,"given":"Ian","family":"Craddock","sequence":"additional","affiliation":[{"name":"School of Computer Science, Electrical and Electronic Engineering, and Engineering Maths, University of Bristol, Bristol BS8 1UB, UK"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1136\/jnnp.2007.131045","article-title":"Parkinson\u2019s disease: Clinical features and diagnosis","volume":"79","author":"Jankovic","year":"2008","journal-title":"J. Neurol. Neurosurg. Psychiatry"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"555","DOI":"10.3389\/fnins.2017.00555","article-title":"How wearable sensors can support Parkinson\u2019s disease diagnosis and treatment: A systematic review","volume":"11","author":"Rovini","year":"2017","journal-title":"Front. Neurosci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.artmed.2018.08.007","article-title":"A survey on computer-assisted Parkinson\u2019s disease diagnosis","volume":"95","author":"Pereira","year":"2019","journal-title":"Artif. Intell. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"429","DOI":"10.3233\/JPD-191781","article-title":"Systematic review looking at the use of technology to measure free-living symptom and activity outcomes in Parkinson\u2019s disease in the home or a home-like environment","volume":"10","author":"Morgan","year":"2020","journal-title":"J. Parkinson\u2019s Dis."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/MIS.2015.57","article-title":"Bridging e-Health and the Internet of Things: The SPHERE Project","volume":"30","author":"Zhu","year":"2015","journal-title":"IEEE Intell. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Woznowski, P., Burrows, A., Diethe, T., Fafoutis, X., Hall, J., Hannuna, S., Camplani, M., Twomey, N., Kozlowski, M., and Tan, B. (2017). SPHERE: A sensor platform for healthcare in a residential environment. Designing, Developing, and Facilitating Smart Cities, Springer.","DOI":"10.1007\/978-3-319-44924-1_14"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12910-017-0183-z","article-title":"Smart homes, private homes? An empirical study of technology researchers\u2019 perceptions of ethical issues in developing smart-home health technologies","volume":"18","author":"Birchley","year":"2017","journal-title":"BMC Med. Ethics"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ziefle, M., Rocker, C., and Holzinger, A. (2011, January 18\u201322). Medical technology in smart homes: Exploring the user\u2019s perspective on privacy, intimacy and trust. Proceedings of the IEEE Computer Software and Applications Conference, Munich, Germany.","DOI":"10.1109\/COMPSACW.2011.75"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1136\/jnnp-2016-314524","article-title":"Subtle motor disturbances in PREDICT-PD participants","volume":"88","author":"Noyce","year":"2017","journal-title":"J. Neurol. Neurosurg. Psychiatry"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1111\/ejn.14094","article-title":"The clinical heterogeneity of Parkinson\u2019s disease and its therapeutic implications","volume":"49","author":"Greenland","year":"2019","journal-title":"Eur. J. Neurosci."},{"key":"ref_11","unstructured":"Kingma, D.P., and Welling, M. (2014, January 14\u201316). Auto-Encoding Variational Bayes. Proceedings of the International Conference on Learning Representations (ICLR), Banff, AL, Canada."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"127","DOI":"10.3109\/03091902.2016.1148792","article-title":"Parkinson\u2019s disease hand tremor detection system for mobile application","volume":"40","author":"Fraiwan","year":"2016","journal-title":"J. Med. Eng. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Um, T.T., Pfister, F.M., Pichler, D., Endo, S., Lang, M., Hirche, S., Fietzek, U., and Kuli\u0107, D. (2017, January 13\u201317). Data augmentation of wearable sensor data for parkinson\u2019s disease monitoring using convolutional neural networks. Proceedings of the ACM International Conference on Multimodal Interaction, Glasgow, UK.","DOI":"10.1145\/3136755.3136817"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12984-018-0446-z","article-title":"Vision-based assessment of parkinsonism and levodopa-induced dyskinesia with pose estimation","volume":"15","author":"Li","year":"2018","journal-title":"J. Neuroeng. Rehabil."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.parkreldis.2019.02.028","article-title":"Upper limb motor pre-clinical assessment in Parkinson\u2019s disease using machine learning","volume":"63","author":"Cavallo","year":"2019","journal-title":"Parkinsonism Relat. Disord."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-020-61789-3","article-title":"High-Resolution Motor State Detection in parkinson\u2019s Disease Using convolutional neural networks","volume":"10","author":"Pfister","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Pintea, S.L., Zheng, J., Li, X., Bank, P.J., van Hilten, J.J., and van Gemert, J.C. (2018, January 8\u201314). Hand-tremor frequency estimation in videos. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany.","DOI":"10.1007\/978-3-030-11024-6_14"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Dadashzadeh, A., Whone, A., Rolinski, M., and Mirmehdi, M. (2021, January 4\u20136). Exploring Motion Boundaries in an End-to-End Network for Vision-based Parkinson\u2019s Severity Assessment. Proceedings of the International Conference on Pattern Recognition Applications and Methods (ICPRAM), Virtual Event.","DOI":"10.5220\/0010309200890097"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"650","DOI":"10.1016\/j.parkreldis.2015.02.026","article-title":"Detecting and monitoring the symptoms of Parkinson\u2019s disease using smartphones: A pilot study","volume":"21","author":"Arora","year":"2015","journal-title":"Parkinsonism Relat. Disord."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hammerla, N., Fisher, J., Andras, P., Rochester, L., Walker, R., and Pl\u00f6tz, T. (2015, January 25\u201330). PD disease state assessment in naturalistic environments using deep learning. Proceedings of the AAAI Conference on Artificial Intelligence, Austin, TX, USA.","DOI":"10.1609\/aaai.v29i1.9484"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1016\/j.parkreldis.2016.09.009","article-title":"Unsupervised home monitoring of Parkinson\u2019s disease motor symptoms using body-worn accelerometers","volume":"33","author":"Fisher","year":"2016","journal-title":"Parkinsonism Relat. Disord."},{"key":"ref_22","first-page":"e8335","article-title":"A kinematic sensor and algorithm to detect motor fluctuations in Parkinson disease: Validation study under real conditions of use","volume":"5","author":"Moral","year":"2018","journal-title":"JMIR Rehabil. Assist. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"101984","DOI":"10.1016\/j.artmed.2020.101984","article-title":"Cartesian genetic programming for diagnosis of Parkinson disease through handwriting analysis: Performance vs. interpretability issues","volume":"111","author":"Parziale","year":"2021","journal-title":"Artif. Intell. Med."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Taleb, C., Likforman-Sulem, L., Mokbel, C., and Khachab, M. (2020). Detection of Parkinson\u2019s disease from handwriting using deep learning: A comparative study. Evol. Intell., 1\u201312.","DOI":"10.1007\/s12065-020-00470-0"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Gazda, M., Hire\u0161, M., and Drot\u00e1r, P. (2021). Multiple-Fine-Tuned Convolutional Neural Networks for Parkinson\u2019s Disease Diagnosis From Offline Handwriting. IEEE Trans. Syst. Man Cybern. Syst., 1\u201312.","DOI":"10.1109\/TSMC.2020.3048892"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Lamba, R., Gulati, T., Alharbi, H.F., and Jain, A. (2021). A hybrid system for Parkinson\u2019s disease diagnosis using machine learning techniques. Int. J. Speech Technol., 1\u201311.","DOI":"10.1007\/s10772-021-09837-9"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Miao, Y., Lou, X., and Wu, H. (2021, January 22\u201324). The Diagnosis of Parkinson\u2019s Disease Based on Gait, Speech Analysis and Machine Learning Techniques. Proceedings of the 2021 International Conference on Bioinformatics and Intelligent Computing, Harbin, China.","DOI":"10.1145\/3448748.3448804"},{"key":"ref_28","unstructured":"Masullo, A., Burghardt, T., Damen, D., Hannuna, S., Ponce-L\u00f3pez, V., and Mirmehdi, M. (2018, January 3\u20136). CaloriNet: From silhouettes to calorie estimation in private environments. Proceedings of the British Machine Vision Conference (BMVC), Newcastle upon Tyne, UK."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Masullo, A., Burghardt, T., Damen, D., Perrett, T., and Mirmehdi, M. (2020). Person Re-ID by Fusion of Video Silhouettes and Wearable Signals for Home Monitoring Applications. Sensors, 20.","DOI":"10.3390\/s20092576"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1109\/JSTSP.2020.2987728","article-title":"Multimodal intelligence: Representation learning, information fusion, and applications","volume":"14","author":"Zhang","year":"2020","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_31","first-page":"423","article-title":"Multimodal machine learning: A survey and taxonomy","volume":"41","author":"Ahuja","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"63373","DOI":"10.1109\/ACCESS.2019.2916887","article-title":"Deep multimodal representation learning: A survey","volume":"7","author":"Guo","year":"2019","journal-title":"IEEE Access"},{"key":"ref_33","unstructured":"Lu, J., Batra, D., Parikh, D., and Lee, S. (2019, January 8\u201314). ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada."},{"key":"ref_34","unstructured":"Li, L.H., Yatskar, M., Yin, D., Hsieh, C.J., and Chang, K.W. (2019). Visualbert: A simple and performant baseline for vision and language. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Nguyen, D.K., and Okatani, T. (2019, January 16\u201321). Multi-task learning of hierarchical vision-language representation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01074"},{"key":"ref_36","unstructured":"Wang, Y., Huang, W., Sun, F., Xu, T., Rong, Y., and Huang, J. (2020, January 6\u201312). Deep multimodal fusion by channel exchanging. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Virtual Event."},{"key":"ref_37","unstructured":"Hou, M., Tang, J., Zhang, J., Kong, W., and Zhao, Q. (2019, January 8\u201314). Deep multimodal multilinear fusion with high-order polynomial pooling. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"P\u00e9rez-R\u00faa, J.M., Vielzeuf, V., Pateux, S., Baccouche, M., and Jurie, F. (2019, January 16\u201321). MFAS: Multimodal fusion architecture search. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00713"},{"key":"ref_39","unstructured":"Afouras, T., Chung, J.S., Senior, A., Vinyals, O., and Zisserman, A. (2018). Deep audio-visual speech recognition. IEEE Trans. Pattern Anal. Mach. Intell., 1\u201311."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Gan, C., Huang, D., Zhao, H., Tenenbaum, J.B., and Torralba, A. (2020, January 13\u201319). Music gesture for visual sound separation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01049"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1049\/iet-cvi.2017.0112","article-title":"Energy expenditure estimation using visual and inertial sensors","volume":"12","author":"Tao","year":"2017","journal-title":"IET Comput. Vis."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Henschel, R., von Marcard, T., and Rosenhahn, B. (2019, January 16\u201321). Simultaneous identification and tracking of multiple people using video and IMUs. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00106"},{"key":"ref_43","unstructured":"Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., and Ng, A.Y. (July, January 28). Multimodal deep learning. Proceedings of the International Conference on Machine Learning (ICML), Bellevue, WA, USA."},{"key":"ref_44","unstructured":"Suzuki, M., Nakayama, K., and Matsuo, Y. (2016). Joint multimodal learning with deep generative models. arXiv."},{"key":"ref_45","unstructured":"Wu, M., and Goodman, N. (2018, January 3\u20138). Multimodal generative models for scalable weakly-supervised learning. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Montreal, QC, Canada."},{"key":"ref_46","unstructured":"Vedantam, R., Fischer, I., Huang, J., and Murphy, K. (May, January 30). Generative models of visually grounded imagination. Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_47","unstructured":"Tsai, Y.H.H., Liang, P.P., Zadeh, A., Morency, L.P., and Salakhutdinov, R. (2019, January 6\u20139). Learning factorized multimodal representations. Proceedings of the International Conference on Learning Representations (ICLR), New Orleans, LA, USA."},{"key":"ref_48","unstructured":"Shi, Y., Siddharth, N., Paige, B., and Torr, P.H. (2019, January 8\u201314). Variational mixture-of-experts autoencoders for multi-modal deep generative models. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Vancouver, BC, Canada."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Hall, J., Hannuna, S., Camplani, M., Mirmehdi, M., Damen, D., Burghardt, T., Tao, L., Paiement, A., and Craddock, I. (2016, January 24\u201325). Designing a Video Monitoring System for AAL applications: The SPHERE Case Study. Proceedings of the 2nd IET International Conference on Technologies for Active and Assisted Living (TechAAL 2016), London, UK.","DOI":"10.1049\/ic.2016.0061"},{"key":"ref_50","unstructured":"(2021, May 25). OpenNI. Available online: https:\/\/structure.io\/openni."},{"key":"ref_51","unstructured":"(2021, May 25). Axivity-AX3. Available online: https:\/\/axivity.com\/product\/ax3."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Twomey, N., Diethe, T., Fafoutis, X., Elsts, A., McConville, R., Flach, P., and Craddock, I. (2018). A comprehensive study of activity recognition using accelerometers. Informatics, 5.","DOI":"10.20944\/preprints201803.0147.v1"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"102770","DOI":"10.1016\/j.jnca.2020.102770","article-title":"Energy-efficient activity recognition framework using wearable accelerometers","volume":"168","author":"Elsts","year":"2020","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1123\/jab.2013-0319","article-title":"Spatiotemporal gait patterns during overt and covert evaluation in patients with Parkinson\u2019s disease and healthy subjects: Is there a Hawthorne effect?","volume":"31","author":"Espinosa","year":"2015","journal-title":"J. Appl. Biomech."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"e041303","DOI":"10.1136\/bmjopen-2020-041303","article-title":"Protocol for PD SENSORS: Parkinson\u2019s Disease Symptom Evaluation in a Naturalistic Setting producing Outcome measuRes using SPHERE technology. An observational feasibility study of multi-modal multi-sensor technology to measure symptoms and activities of daily living in Parkinson\u2019s disease","volume":"10","author":"Morgan","year":"2020","journal-title":"BMJ Open"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/12\/4133\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:17:04Z","timestamp":1760163424000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/12\/4133"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,16]]},"references-count":55,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["s21124133"],"URL":"https:\/\/doi.org\/10.3390\/s21124133","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,16]]}}}