{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,12]],"date-time":"2026-04-12T13:43:28Z","timestamp":1776001408443,"version":"3.50.1"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2024,8,8]],"date-time":"2024-08-08T00:00:00Z","timestamp":1723075200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,8,8]],"date-time":"2024-08-08T00:00:00Z","timestamp":1723075200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100023890","name":"Technische Universit\u00e4t Hamburg","doi-asserted-by":"publisher","award":["i3 initiative"],"award-info":[{"award-number":["i3 initiative"]}],"id":[{"id":"10.13039\/501100023890","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J CARS"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>Commonly employed in polyp segmentation, single-image UNet architectures lack the temporal insight clinicians gain from video data in diagnosing polyps. To mirror clinical practices more faithfully, our proposed solution, <jats:italic>PolypNextLSTM<\/jats:italic>, leverages video-based deep learning, harnessing temporal information for superior segmentation performance with least parameter overhead, making it possibly suitable for edge devices.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p><jats:italic>PolypNextLSTM<\/jats:italic> employs a UNet-like structure with ConvNext-Tiny as its backbone, strategically omitting the last two layers to reduce parameter overhead. Our temporal fusion module, a Convolutional Long Short Term Memory (ConvLSTM), effectively exploits temporal features. Our primary novelty lies in <jats:italic>PolypNextLSTM<\/jats:italic>, which stands out as the leanest in parameters and the fastest model, surpassing the performance of five state-of-the-art image and video-based deep learning models. The evaluation of the SUN-SEG dataset spans easy-to-detect and hard-to-detect polyp scenarios, along with videos containing challenging artefacts like fast motion and occlusion.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Comparison against 5 image-based and 5 video-based models demonstrates <jats:italic>PolypNextLSTM<\/jats:italic>\u2019s superiority, achieving a Dice score of 0.7898 on the hard-to-detect polyp test set, surpassing image-based PraNet (0.7519) and video-based PNS+ (0.7486). Notably, our model excels in videos featuring complex artefacts such as ghosting and occlusion.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p><jats:italic>PolypNextLSTM<\/jats:italic>, integrating pruned ConvNext-Tiny with ConvLSTM for temporal fusion, not only exhibits superior segmentation performance but also maintains the highest frames per speed among evaluated models. Code can be found here: <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/mtec-tuhh\/PolypNextLSTM\">https:\/\/github.com\/mtec-tuhh\/PolypNextLSTM<\/jats:ext-link>.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s11548-024-03244-6","type":"journal-article","created":{"date-parts":[[2024,8,8]],"date-time":"2024-08-08T11:03:25Z","timestamp":1723115005000},"page":"2111-2119","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["PolypNextLSTM: a lightweight and fast polyp video segmentation network using ConvNext and ConvLSTM"],"prefix":"10.1007","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8552-2227","authenticated-orcid":false,"given":"Debayan","family":"Bhattacharya","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Konrad","family":"Reuter","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Finn","family":"Behrendt","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lennart","family":"Maack","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sarah","family":"Grube","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexander","family":"Schlaefer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,8,8]]},"reference":[{"issue":"3","key":"3244_CR1","doi-asserted-by":"publisher","first-page":"515","DOI":"10.1016\/j.gtc.2018.04.004","volume":"47","author":"PJ Pickhardt","year":"2018","unstructured":"Pickhardt PJ, Pooler BD, Kim DH, Hassan C, Matkowskyj KA, Halberg RB (2018) The natural history of colorectal polyps: overview of predictive static and dynamic features. Gastroenterol Clin North Am 47(3):515\u2013536","journal-title":"Gastroenterol Clin North Am"},{"key":"3244_CR2","doi-asserted-by":"publisher","first-page":"g2467","DOI":"10.1136\/bmj.g2467","volume":"348","author":"H Brenner","year":"2014","unstructured":"Brenner H, Stock C, Hoffmeister M (2014) Effect of screening sigmoidoscopy and screening colonoscopy on colorectal cancer incidence and mortality: systematic review and meta-analysis of randomised controlled trials and observational studies. BMJ 348:g2467. https:\/\/doi.org\/10.1136\/bmj.g2467","journal-title":"BMJ"},{"issue":"16","key":"3244_CR3","doi-asserted-by":"publisher","first-page":"281","DOI":"10.3238\/arztebl.m2021.0111","volume":"118","author":"R Cardoso","year":"2021","unstructured":"Cardoso R, Zhu A, Guo F, Heisser T, Hoffmeister M, Brenner H (2021) Incidence and mortality of proximal and distal colorectal cancer in Germany. Dtsch Arztebl Int 118(16):281\u2013287. https:\/\/doi.org\/10.3238\/arztebl.m2021.0111","journal-title":"Dtsch Arztebl Int"},{"key":"3244_CR4","doi-asserted-by":"publisher","first-page":"893","DOI":"10.1055\/A-1306-7590\/ID\/JR19561-7\/BIB","volume":"53","author":"OF Ahmad","year":"2021","unstructured":"Ahmad OF (2021) Establishing key research questions for the implementation of artificial intelligence in colonoscopy: a modified Delphi method. Endoscopy 53:893\u2013901. https:\/\/doi.org\/10.1055\/A-1306-7590\/ID\/JR19561-7\/BIB","journal-title":"Endoscopy"},{"key":"3244_CR5","doi-asserted-by":"publisher","first-page":"4037190","DOI":"10.1155\/2017\/4037190","volume":"2017","author":"D V\u00e1zquez","year":"2017","unstructured":"V\u00e1zquez D, Bernal J, S\u00e1nchez FJ, Fern\u00e1ndez-Esparrach G, L\u00f3pez AM, Romero A, Drozdzal M, Courville A (2017) A benchmark for endoluminal scene segmentation of colonoscopy images. J Healthc Eng 2017:4037190. https:\/\/doi.org\/10.1155\/2017\/4037190","journal-title":"J Healthc Eng"},{"key":"3244_CR6","doi-asserted-by":"publisher","unstructured":"Jha D, Smedsrud PH, Riegler MA, Halvorsen P, Lange T, Johansen D, Johansen HD (2020) Kvasir-seg: a segmented polyp dataset. In: MultiMedia modeling: 26th international conference, MMM 2020, Daejeon, South Korea, January 5-8, 2020, Proceedings, Part II. Springer, Berlin, Heidelberg, pp 451\u2013462. https:\/\/doi.org\/10.1007\/978-3-030-37734-2_37","DOI":"10.1007\/978-3-030-37734-2_37"},{"key":"3244_CR7","doi-asserted-by":"crossref","unstructured":"Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention \u2013 MICCAI 2015, Springer, Cham, pp 234\u2013241","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"3244_CR8","doi-asserted-by":"crossref","unstructured":"Zhou Z, Rahman\u00a0Siddiquee MM, Tajbakhsh N, Liang J (2018) UNet++: a nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, Springer, Cham, pp 3\u201311","DOI":"10.1007\/978-3-030-00889-5_1"},{"key":"3244_CR9","doi-asserted-by":"publisher","unstructured":"Jha D, Smedsrud PH, Riegler MA, Johansen D, Lange TD, Halvorsen P D, Johansen H (2019) ResuNet++: an advanced architecture for medical image segmentation. In: 2019 IEEE international symposium on multimedia (ISM), pp 225\u20132255. https:\/\/doi.org\/10.1109\/ISM46123.2019.00049","DOI":"10.1109\/ISM46123.2019.00049"},{"key":"3244_CR10","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2021.104815","volume":"137","author":"M Yeung","year":"2021","unstructured":"Yeung M, Sala E, Sch\u00f6nlieb C-B, Rundo L (2021) Focus U-Net: a novel dual attention-gated CNN for polyp segmentation during colonoscopy. Comput Biol Med 137:104815. https:\/\/doi.org\/10.1016\/j.compbiomed.2021.104815","journal-title":"Comput Biol Med"},{"key":"3244_CR11","first-page":"181","volume":"2021","author":"K Patel","year":"2021","unstructured":"Patel K, Bur AM, Wang G (2021) Enhanced U-Net: a feature enhancement network for polyp segmentation. Proc Int Robot Vis Conf 2021:181\u2013188","journal-title":"Proc Int Robot Vis Conf"},{"key":"3244_CR12","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N (2021) An image is worth 16x16 words: transformers for image recognition at scale. In: International conference on learning representations. https:\/\/openreview.net\/forum?id=YicbFdNTTy"},{"key":"3244_CR13","doi-asserted-by":"publisher","unstructured":"Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: 2021 IEEE\/CVF international conference on computer vision (ICCV), pp 9992\u201310002. https:\/\/doi.org\/10.1109\/ICCV48922.2021.00986","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"3244_CR14","doi-asserted-by":"crossref","unstructured":"Zhang Y, Liu H, Hu Q (2021) Transfuse: Fusing transformers and CNNs for medical image segmentation. In: Medical image computing and computer assisted intervention - MICCAI 2021. Springer, Cham, pp 14\u201324","DOI":"10.1007\/978-3-030-87193-2_2"},{"key":"3244_CR15","doi-asserted-by":"publisher","unstructured":"Dong B, Wang W, Fan D-P, Li J, Fu H, Shao L (2023) Polyp-PVT: polyp segmentation with pyramid vision transformers. In: CAAI artificial intelligence research 2, 9150015. https:\/\/doi.org\/10.26599\/AIR.2023.9150015","DOI":"10.26599\/AIR.2023.9150015"},{"key":"3244_CR16","doi-asserted-by":"crossref","unstructured":"He K, Chen X, Xie S, Li Y, Doll\u00e1r P, Girshick RB (2021) Masked autoencoders are scalable vision learners. CoRR. arXiv:2111.06377","DOI":"10.1109\/CVPR52688.2022.01553"},{"key":"3244_CR17","unstructured":"Shaharabany T, Dahan A, Giryes R, Wolf L (2023) AutoSAM: adapting SAM to medical images by overloading the prompt encoder"},{"key":"3244_CR18","doi-asserted-by":"crossref","unstructured":"Puyal JG-B, Bhatia KK, Brandao P, Ahmad OF, Toth D, Kader R, Lovat L, Mountney P, Stoyanov D (2020) Endoscopic polyp segmentation using a hybrid 2d\/3d CNN. In: Medical image computing and computer assisted intervention - MICCAI 2020. Springer, Cham, pp 295\u2013305","DOI":"10.1007\/978-3-030-59725-2_29"},{"key":"3244_CR19","doi-asserted-by":"crossref","unstructured":"Ji G-P, Chou Y-C, Fan D-P, Chen G, Fu H, Jha D, Shao L (2021) Progressively normalized self-attention network for video polyp segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 142\u2013152","DOI":"10.1007\/978-3-030-87193-2_14"},{"issue":"6","key":"3244_CR20","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1007\/s11633-022-1371-y","volume":"19","author":"G-P Ji","year":"2022","unstructured":"Ji G-P, Xiao G, Chou Y-C, Fan D-P, Zhao K, Chen G, Gool LV (2022) Video polyp segmentation: a deep learning perspective. Mach Intell Res 19(6):531\u2013549. https:\/\/doi.org\/10.1007\/s11633-022-1371-y","journal-title":"Mach Intell Res"},{"key":"3244_CR21","doi-asserted-by":"crossref","unstructured":"Zhao X, Wu Z, Tan S, Fan D-J, Li Z, Wan X, Li G (2022) Semi-supervised spatial temporal attention network for video polyp segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 456\u2013466","DOI":"10.1007\/978-3-031-16440-8_44"},{"key":"3244_CR22","doi-asserted-by":"crossref","unstructured":"Lin J, Dai Q, Zhu L, Fu H, Wang Q, Li W, Rao W, Huang X, Wang L (2023) Shifting more attention to breast lesion segmentation in ultrasound videos","DOI":"10.1007\/978-3-031-43898-1_48"},{"key":"3244_CR23","doi-asserted-by":"crossref","unstructured":"Ma Y, Chen X, Cheng K, Li Y, Sun B (2021) LDpolypvideo benchmark: a large-scale colonoscopy video dataset of diverse polyps. In: Medical image computing and computer assisted intervention \u2013 MICCAI 2021. Springer, Cham, pp 387\u2013396","DOI":"10.1007\/978-3-030-87240-3_37"},{"issue":"4","key":"3244_CR24","doi-asserted-by":"publisher","first-page":"960","DOI":"10.1016\/j.gie.2020.07.060","volume":"93","author":"M Misawa","year":"2021","unstructured":"Misawa M, Kudo S-E, Mori Y, Hotta K, Ohtsuka K, Matsuda T, Saito S, Kudo T, Baba T, Ishida F, Itoh H, Oda M, Mori K (2021) Development of a computer-aided detection system for colonoscopy and a publicly accessible large colonoscopy video database (with video). Gastrointest Endosc 93(4):960\u20139673. https:\/\/doi.org\/10.1016\/j.gie.2020.07.060","journal-title":"Gastrointest Endosc"},{"key":"3244_CR25","doi-asserted-by":"publisher","unstructured":"Liu Z, Mao H, Wu C-Y, Feichtenhofer C, Darrell T, Xie S (2022) A convnet for the 2020s. In: 2022 IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp 11966\u201311976. https:\/\/doi.org\/10.1109\/CVPR52688.2022.01167","DOI":"10.1109\/CVPR52688.2022.01167"},{"issue":"4","key":"3244_CR26","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","volume":"40","author":"L-C Chen","year":"2018","unstructured":"Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834\u2013848. https:\/\/doi.org\/10.1109\/TPAMI.2017.2699184","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"3244_CR27","doi-asserted-by":"crossref","unstructured":"Fan D-P, Ji G-P, Zhou T, Chen G, Fu H, Shen J, Shao L (2020) PraNet: parallel reverse attention network for polyp segmentation. In: Medical image computing and computer assisted intervention \u2013 MICCAI 2020. Springer, Cham, pp 263\u2013273","DOI":"10.1007\/978-3-030-59725-2_26"},{"key":"3244_CR28","doi-asserted-by":"crossref","unstructured":"Wei J, Hu Y, Zhang R, Li Z, Zhou SK, Cui S (2021) Shallow attention network for polyp segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 699\u2013708","DOI":"10.1007\/978-3-030-87193-2_66"},{"key":"3244_CR29","doi-asserted-by":"crossref","unstructured":"Rahman MM, Marculescu R (2023) Medical image segmentation via cascaded attention decoding. In: Proceedings of the IEEE\/CVF winter conference on applications of computer vision (WACV), pp 6222\u20136231","DOI":"10.1109\/WACV56688.2023.00616"},{"key":"3244_CR30","doi-asserted-by":"publisher","unstructured":"Lu X, Wang W, Ma C, Shen J, Shao L, Porikli F (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. In: 2019 IEEE\/CVF conference on computer vision and pattern recognition (CVPR), pp 3618\u20133627. https:\/\/doi.org\/10.1109\/CVPR.2019.00374","DOI":"10.1109\/CVPR.2019.00374"},{"issue":"1","key":"3244_CR31","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1038\/s41597-023-01981-y","volume":"10","author":"S Ali","year":"2023","unstructured":"Ali S, Jha D, Ghatwary N, Realdon S, Cannizzaro R, Salem OE, Lamarque D, Daul C, Riegler MA, Anonsen KV, Petlund A, Halvorsen P, Rittscher J, Lange T, East JE (2023) A multi-centre polyp detection and segmentation dataset for generalisability assessment. Scientific Data 10(1):75. https:\/\/doi.org\/10.1038\/s41597-023-01981-y","journal-title":"Scientific Data"}],"container-title":["International Journal of Computer Assisted Radiology and Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03244-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11548-024-03244-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11548-024-03244-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,30]],"date-time":"2024-09-30T17:09:58Z","timestamp":1727716198000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11548-024-03244-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,8]]},"references-count":31,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["3244"],"URL":"https:\/\/doi.org\/10.1007\/s11548-024-03244-6","relation":{},"ISSN":["1861-6429"],"issn-type":[{"value":"1861-6429","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,8]]},"assertion":[{"value":"17 January 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 July 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 August 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors state no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The research conducted for this paper adheres to ethical principles and guidelines concerning the utilization of publicly available datasets. The datasets employed in this study, SUN-SEG, and PolypGen are publicly accessible resources without individual identifiers, thus obviating the need for specific consent from individuals.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}