{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T07:24:21Z","timestamp":1790666661977,"version":"4.1.0"},"reference-count":28,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T00:00:00Z","timestamp":1760659200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100009633","name":"Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health","doi-asserted-by":"publisher","award":["R01HD092239"],"award-info":[{"award-number":["R01HD092239"]}],"id":[{"id":"10.13039\/100009633","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009633","name":"Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health","doi-asserted-by":"publisher","award":["R01HD074819"],"award-info":[{"award-number":["R01HD074819"]}],"id":[{"id":"10.13039\/100009633","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Bolus tracking is a critical component of swallowing analysis, as the speed, course, and integrity of bolus movement from the mouth to the stomach, along with the presence of residue, serve as key indicators of potential abnormalities. Existing machine learning approaches for videofluoroscopic swallowing study (VFSS) analysis heavily rely on annotated data and often struggle to detect residue, which is visually subtle and underrepresented. This study proposes an unsupervised architecture to segment both bolus and residue, marking the first successful machine learning-based residue segmentation in swallowing analysis with quantitative evaluation. We introduce an unsupervised convolutional autoencoder that segments bolus and residue without requiring pixel-level annotations. To address the locality bias inherent in convolutional architectures, we incorporate positional encoding into the input representation, enabling the model to capture global spatial context. The proposed model was validated on a diverse set of VFSS images annotated by certified raters. Our method achieves an intersection over union (IoU) of 61% for bolus segmentation\u2014comparable to state-of-the-art supervised methods\u2014and 52% for residue detection. Despite not using pixel-wise labels for training, our model significantly outperforms top-performing supervised baselines in residue detection, as confirmed by statistical testing. These findings suggest that learning from negative space provides a robust and generalizable pathway for detecting clinically significant but sparsely represented features like residue.<\/jats:p>","DOI":"10.3390\/jimaging11100368","type":"journal-article","created":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T10:28:36Z","timestamp":1760696916000},"page":"368","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Unsupervised Segmentation of Bolus and Residue in Videofluoroscopy Swallowing Studies"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-8239-2397","authenticated-orcid":false,"given":"Farnaz","family":"Khodami","sequence":"first","affiliation":[{"name":"Faculty of Electrical and Computer Engineering, University of Toronto, 10 King\u2019s College Rd, Toronto, ON M5S 1A1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6738-1651","authenticated-orcid":false,"given":"Mehdy","family":"Dousty","sequence":"additional","affiliation":[{"name":"Faculty of Electrical and Computer Engineering, University of Toronto, 10 King\u2019s College Rd, Toronto, ON M5S 1A1, Canada"},{"name":"Vector Institute, 108 College St, Toronto, ON M5G 0C6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James L.","family":"Coyle","sequence":"additional","affiliation":[{"name":"School of Health and Rehabilitation Sciences, University of Pittsburgh, Forbes Tower, 3600 Atwood St, Pittsburgh, PA 15260, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ervin","family":"Sejdi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Electrical and Computer Engineering, University of Toronto, 10 King\u2019s College Rd, Toronto, ON M5S 1A1, Canada"},{"name":"North York General Hospital, 4001 Leslie St, North York, ON M2K 1E1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2643","DOI":"10.1111\/jgs.16137","article-title":"Swallowing disorders in the older population","volume":"67","author":"Christmas","year":"2019","journal-title":"J. Am. Geriatr. Soc."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1007\/s00455-014-9579-9","article-title":"A systematic review of self-reported swallowing assessments in progressive neurological disorders","volume":"30","author":"Keage","year":"2015","journal-title":"Dysphagia"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"225","DOI":"10.2478\/raon-2019-0028","article-title":"Swallowing disorders after treatment for head and neck cancer","volume":"53","author":"Pezdirec","year":"2019","journal-title":"Radiol. Oncol."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"de Sire, A., Ferrillo, M., Lippi, L., Agostini, F., de Sire, R., Ferrara, P.E., Raguso, G., Riso, S., Roccuzzo, A., and Ronconi, G. (2022). Sarcopenic dysphagia, malnutrition, and oral frailty in elderly: A comprehensive review. Nutrients, 14.","DOI":"10.3390\/nu14050982"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1080\/17549507.2023.2293633","article-title":"Safety-related outcomes for patients with a tracheostomy and the use of flexible endoscopic evaluation of swallowing (FEES) for assessment and management of swallowing: A systematic review","volume":"27","author":"Morris","year":"2024","journal-title":"Int. J. Speech Lang. Pathol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/2050571X.2020.1826109","article-title":"Multiple swallow behaviour during high resolution pharyngeal manometry: Prevalence and sub-typing in healthy adults","volume":"25","author":"Omari","year":"2022","journal-title":"Speech Lang. Hear."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"103815","DOI":"10.1016\/j.dsp.2022.103815","article-title":"Swallowing disorders analysis using surface EMG biomarkers and classification models","volume":"133","year":"2023","journal-title":"Digit. Signal Process."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1080\/17549507.2020.1861327","article-title":"Training for videofluoroscopic swallowing analysis: A systematic review","volume":"23","author":"Edwards","year":"2021","journal-title":"Int. J. Speech-Lang. Pathol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1007\/s00455-021-10293-5","article-title":"Visual Analysis of Swallowing Efficiency and Safety (VASES): A standardized approach to rating pharyngeal residue, penetration, and aspiration during FEES","volume":"37","author":"Curtis","year":"2022","journal-title":"Dysphagia"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Bhatt, D., Patel, C., Talsania, H., Patel, J., Vaghela, R., Pandya, S., Modi, K., and Ghayvat, H. (2021). CNN variants for computer vision: History, architecture, application, challenges and future scope. Electronics, 10.","DOI":"10.3390\/electronics10202470"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1109\/JBHI.2023.3269692","article-title":"Hand grasp classification in egocentric video after cervical spinal cord injury","volume":"28","author":"Dousty","year":"2023","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"6214","DOI":"10.1109\/JBHI.2024.3495699","article-title":"Personalized Video-Based Hand Taxonomy Using Egocentric Video in the Wild","volume":"29","author":"Dousty","year":"2024","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Caliskan, H., Mahoney, A.S., Coyle, J.L., and Sejdi\u0107, E. (2020, January 20\u201324). Automated bolus detection in videofluoroscopic images of swallowing using mask-RCNN. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Virtual.","DOI":"10.1109\/EMBC44109.2020.9176664"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1007\/s11554-023-01398-1","article-title":"Deep learning models for bolus segmentation in videofluoroscopic swallow studies","volume":"21","author":"Li","year":"2024","journal-title":"J. Real-Time Image Process."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2013MICCAI 2015: 18th International Conference, Munich, Germany. proceedings, part III 18.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_16","unstructured":"Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., and Liang, J. (2018, January 20). Unet++: A nested u-net architecture for medical image segmentation. Proceedings of the Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain. Proceedings 4."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18\u201323). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_18","unstructured":"Simonyan, K. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. (2017, January 4\u20136). Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_22","unstructured":"Tan, M., and Le, Q. (2019, January 9\u201315). Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the International Conference on Machine Learning. PMLR, Long Beach, CA, USA."},{"key":"ref_23","first-page":"275","article-title":"Effect of label noise in semantic segmentation of high resolution aerial images and height data","volume":"2","author":"Maiti","year":"2022","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Bandini, A., Smaoui, S., and Steele, C.M. (2022). Automated pharyngeal phase detection and bolus localization in videofluoroscopic swallowing study: Killing two birds with one stone?. Comput. Methods Programs Biomed., 225.","DOI":"10.1016\/j.cmpb.2022.107058"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Xiao, H., Li, L., Liu, Q., Zhu, X., and Zhang, Q. (2023). Transformers in medical image segmentation: A review. Biomed. Signal Process. Control, 84.","DOI":"10.1016\/j.bspc.2023.104791"},{"key":"ref_26","unstructured":"Wu, J., Fu, R., Fang, H., Zhang, Y., Yang, Y., Xiong, H., Liu, H., and Xu, Y. (2024, January 3\u20135). Medsegdiff: Medical image segmentation with diffusion probabilistic model. Proceedings of the Medical Imaging with Deep Learning. PMLR, Paris, France."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Rahman, A., Valanarasu, J.M.J., Hacihaliloglu, I., and Patel, V.M. (2023, January 18\u201322). Ambiguous medical image segmentation using diffusion models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01110"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3570","DOI":"10.1109\/TMI.2024.3424471","article-title":"HiDiff: Hybrid diffusion framework for medical image segmentation","volume":"43","author":"Chen","year":"2024","journal-title":"IEEE Trans. Med. Imaging"}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/10\/368\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T11:14:12Z","timestamp":1760699652000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/11\/10\/368"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,17]]},"references-count":28,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2025,10]]}},"alternative-id":["jimaging11100368"],"URL":"https:\/\/doi.org\/10.3390\/jimaging11100368","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,17]]}}}