{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T16:14:47Z","timestamp":1777047287102,"version":"3.51.4"},"reference-count":57,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2022,9,23]],"date-time":"2022-09-23T00:00:00Z","timestamp":1663891200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The National Natural Science Foundation of China","award":["62033001"],"award-info":[{"award-number":["62033001"]}]},{"name":"The National Natural Science Foundation of China","award":["52175075"],"award-info":[{"award-number":["52175075"]}]},{"name":"The National Natural Science Foundation of China","award":["cstc2021ycjh-bgzxm0157"],"award-info":[{"award-number":["cstc2021ycjh-bgzxm0157"]}]},{"name":"Chongqing Research Program of Basic Research, and Frontier Exploration","award":["62033001"],"award-info":[{"award-number":["62033001"]}]},{"name":"Chongqing Research Program of Basic Research, and Frontier Exploration","award":["52175075"],"award-info":[{"award-number":["52175075"]}]},{"name":"Chongqing Research Program of Basic Research, and Frontier Exploration","award":["cstc2021ycjh-bgzxm0157"],"award-info":[{"award-number":["cstc2021ycjh-bgzxm0157"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Image segmentation plays an important role in the sensing systems of autonomous underwater vehicles for fishing. Via accurately perceiving the marine organisms and surrounding environment, the automatic catch of marine products can be implemented. However, existing segmentation methods cannot precisely segment marine animals due to the low quality and complex shapes of collected marine images in the underwater situation. A novel multi-scale transformer network (MulTNet) is proposed for improving the segmentation accuracy of marine animals, and it simultaneously possesses the merits of a convolutional neural network (CNN) and a transformer. To alleviate the computational burden of the proposed network, a dimensionality reduction CNN module (DRCM) based on progressive downsampling is first designed to fully extract the low-level features, and then they are fed into a proposed multi-scale transformer module (MTM). For capturing the rich contextural information from different subregions and scales, four parallel small-scale encoder layers with different heads are constructed, and then they are combined with a large-scale transformer layer to form a multi-scale transformer module. The comparative results demonstrate MulTNet outperforms the existing advanced image segmentation networks, with MIOU improvements of 0.76% in the marine animal dataset and 0.29% in the ISIC 2018 dataset. Consequently, the proposed method has important application value for segmenting underwater images.<\/jats:p>","DOI":"10.3390\/s22197224","type":"journal-article","created":{"date-parts":[[2022,9,26]],"date-time":"2022-09-26T03:34:17Z","timestamp":1664163257000},"page":"7224","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["MulTNet: A Multi-Scale Transformer Network for Marine Image Segmentation toward Fishing"],"prefix":"10.3390","volume":"22","author":[{"given":"Xi","family":"Xu","sequence":"first","affiliation":[{"name":"State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing 400044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2160-4300","authenticated-orcid":false,"given":"Yi","family":"Qin","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing 400044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dejun","family":"Xi","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing 400044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ruotong","family":"Ming","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing 400044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Xia","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing 400044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3937","DOI":"10.1155\/2020\/3937580","article-title":"Marine organism detection and classification from underwater vision based on the deep CNN method","volume":"2020","author":"Han","year":"2020","journal-title":"Math. Probl. Eng."},{"key":"ref_2","unstructured":"Zhuang, P., Xing, L., Liu, Y., Guo, S., and Qiao, Y. (2017). Marine Animal Detection and Recognition with Advanced Deep Learning Models. CLEF, Shenzhen Institutes of Advanced Technology Chinese Academy of Sciences. Working Note."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Cao, Z., Principe, J.C., Ouyang, B., Dalgleish, F., and Vuorenkoski, A. (2015, January 19\u201322). Marine animal classification using combined CNN and hand-designed image features. Proceedings of the OCEANS 2015\u2014MTS\/IEEE Washington, Washington, DC, USA.","DOI":"10.23919\/OCEANS.2015.7404375"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5006011","DOI":"10.1109\/TIM.2021.3049276","article-title":"Multipath fusion Mask R-CNN with double attention and its application into gear pitting detection","volume":"70","author":"Xi","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the MICCAI 2015: Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"108130","DOI":"10.1016\/j.asoc.2021.108130","article-title":"Tree CycleGAN with maximum diversity loss for image augmentation and its application into gear pitting detection","volume":"114","author":"Qin","year":"2022","journal-title":"Appl. Soft Comput."},{"key":"ref_8","unstructured":"Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., and Kainz, B. (2018). Attention u-net: Learning where to look for the pancreas. arXiv, preprint."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lin, G., Milan, A., Shen, C., and Reid, I. (2017, January 21\u201326). Refinenet: Multi-path refine-ment networks for high-resolution semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.549"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1109\/LGRS.2018.2802944","article-title":"Road extraction by deep residual u-net","volume":"15","author":"Zhang","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"075402","DOI":"10.1088\/1361-6501\/ac5439","article-title":"Coal petrography extraction approach based on multiscale mixed-attention-based residual U-net","volume":"33","author":"Jin","year":"2022","journal-title":"Meas. Sci. Technol."},{"key":"ref_12","first-page":"095003","article-title":"High-resolution remote sensing image semantic segmentation based on a deep feature aggregation network","volume":"32","author":"Wang","year":"2021","journal-title":"Meas. Sci. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"103997","DOI":"10.1016\/j.imavis.2020.103997","article-title":"PCANet: Pyramid convolutional attention network for semantic segmentation","volume":"103","author":"Sang","year":"2020","journal-title":"Image Vis. Comput."},{"key":"ref_14","unstructured":"Yu, F., and Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv, preprint."},{"key":"ref_15","unstructured":"Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2014). Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv, preprint."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8\u201314). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"106881","DOI":"10.1016\/j.asoc.2020.106881","article-title":"Cascade knowledge diffusion network for skin lesion diagnosis and segmentation","volume":"99","author":"Jin","year":"2021","journal-title":"Appl. Soft. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"108488","DOI":"10.1016\/j.oceaneng.2020.108488","article-title":"An artificial intelligence segmentation method for recognizing the free surface in a sloshing tank","volume":"220","author":"Wei","year":"2021","journal-title":"Ocean Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1016\/j.mcm.2012.12.025","article-title":"An improved K-means clustering algorithm for fish image segmentation","volume":"58","author":"Yao","year":"2013","journal-title":"Math. Comp. Modell."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Martin-Abadal, M., Riutort-Ozcariz, I., Oliver-Codina, G., and Gonzalez-Cid, Y. (2019, January 17\u201320). A deep learning solution for Posidonia oceanica seafloor habitat multiclass recognition. Proceedings of the OCEANS 2019-Marseille, Marseille, France.","DOI":"10.1109\/OCEANSE.2019.8867304"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"60956","DOI":"10.1109\/ACCESS.2018.2875412","article-title":"Deep semantic segmentation in an AUV for online posidonia oceanica meadows identification","volume":"6","year":"2018","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"101083","DOI":"10.1016\/j.ecoinf.2020.101083","article-title":"SeaGrassDetect: A novel method for the detection of seagrass from unlabelled underwater videos","volume":"57","author":"Sengupta","year":"2020","journal-title":"Ecol. Inform."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"095021","DOI":"10.1088\/1361-6501\/ac7438","article-title":"An image processing method for an explosion field fireball based on edge recursion","volume":"33","author":"Wang","year":"2022","journal-title":"Meas. Sci. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1109\/TIP.2017.2759252","article-title":"Color balance and fusion for underwater image enhancement","volume":"27","author":"Ancuti","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_26","first-page":"239","article-title":"Underwater image enhancement using an integrated colour model","volume":"34","author":"Iqbal","year":"2007","journal-title":"Int. J. Comput. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/j.oceaneng.2014.11.036","article-title":"Deriving inherent optical properties from background color and underwater image enhancement","volume":"94","author":"Zhao","year":"2015","journal-title":"Ocean Eng."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, Y., Zhang, J., Cao, Y., and Wang, Z. (2017, January 17\u201320). A deep CNN method for underwater image enhancement. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.","DOI":"10.1109\/ICIP.2017.8296508"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"48719","DOI":"10.1109\/ACCESS.2022.3170893","article-title":"Fast Computation of Hahn Polynomials for High Order Moments","volume":"10","author":"Mahmmod","year":"2022","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Al-Utaibi, K.A., Abdulhussain, S.H., Mahmmod, B.M., Naser, M.A., Alsabah, M., and Sait, S.M. (2021). Reliable recurrence algorithm for high-order Krawtchouk polynomials. Entropy, 23.","DOI":"10.3390\/e23091162"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Skinner, K.A., and Matthew, J.-R. (2017, January 21\u201326). Underwater image dehazing with a light field camera. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.224"},{"key":"ref_32","first-page":"65","article-title":"Imaging systems for advanced underwater vehicles","volume":"8","author":"Bonin","year":"2011","journal-title":"J. Marit. Res."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Eleftherakis, D., and Vicen-Bueno, R. (2020). Sensors to increase the security of underwater communication cables: A review of underwater monitoring sensors. Sensors, 20.","DOI":"10.3390\/s20030737"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4376","DOI":"10.1109\/TIP.2019.2955241","article-title":"An underwater image enhancement benchmark dataset and beyond","volume":"29","author":"Li","year":"2019","journal-title":"IEEE T. Image Process."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Duarte, A., Codevilla, F., Gaya, J.D.O., and Botelho, S.S. (2016, January 10\u2013123). A dataset to evaluate underwater image restoration methods. Proceedings of the OCEANS 2016-Shanghai, Shanghai, China.","DOI":"10.1109\/OCEANSAP.2016.7485524"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Radolko, M., Farhadifard, F., and Von Lukas, U.F. (2016, January 19\u201323). Dataset on underwater change detection. Proceedings of the OCEANS 2016 MTS\/IEEE Monterey, Monterey, CA, USA.","DOI":"10.1109\/OCEANS.2016.7761129"},{"key":"ref_37","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"ref_38","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv, preprint."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., and Zettlemoyer, L. (2019). Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv, preprint.","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"ref_40","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_41","unstructured":"Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., and Liu, P.J. (2019). Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv, preprint."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Wang, Y., Xu, Z., Wang, X., Shen, C., Cheng, B., Shen, H., and Xia, H. (2021, January 19\u201325). End-to-end video instance segmentation with transformers. Proceedings of the Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Virtual.","DOI":"10.1109\/CVPR46437.2021.00863"},{"key":"ref_43","unstructured":"Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., and Zhou, Y. (2021). Transunet: Transformers make strong encoders for medical image segmentation. arXiv, preprint."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23\u201328). End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"ref_45","unstructured":"Beal, J., Kim, E., Tzeng, E., Park, D.H., Zhai, A., and Kislyuk, D. (2020). Toward transformer-based object detection. arXiv, preprint."},{"key":"ref_46","unstructured":"Zhang, Q., and Yang, Y. (2021). ResT: An efficient transformer for visual recognition. arXiv, preprint."},{"key":"ref_47","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., and Houlsby, N. (2020). An image is worth 16 \u00d7 16 words: Transformers for image recognition at scale. arXiv, preprint."},{"key":"ref_48","unstructured":"Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and J\u00e9gou, H. (2021, January 18\u201324). Training data-efficient image transformers & distillation through attention. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., and Guo, B. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. arXiv, preprint.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Xie, Y., Zhang, J., Shen, C., and Xia, Y. (2021). CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer.","DOI":"10.1007\/978-3-030-87199-4_16"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.patrec.2021.04.024","article-title":"TrSeg: Transformer for semantic segmentation","volume":"148","author":"Jin","year":"2021","journal-title":"Pattern Recogn. Lett."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"102327","DOI":"10.1016\/j.media.2021.102327","article-title":"FAT-Net: Feature adaptive transformers for automated skin lesion segmentation","volume":"76","author":"Wu","year":"2022","journal-title":"Med. Image Anal."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"109352","DOI":"10.1016\/j.measurement.2021.109352","article-title":"A new deep transfer learning network based on convolutional auto-encoder for mechanical fault diagnosis","volume":"178","author":"Qian","year":"2021","journal-title":"Measurement"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"107927","DOI":"10.1016\/j.ress.2021.107927","article-title":"Multicellular LSTM-based deep learning model for aero-engine remaining useful life prediction","volume":"216","author":"Xiang","year":"2021","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"4861","DOI":"10.1109\/TCSVT.2019.2963772","article-title":"Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light","volume":"30","author":"Liu","year":"2020","journal-title":"IEEE Trans. Circ. Syst. Vid."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Xi, D., Qin, Y., and Wang, S. (2021). YDRSNet: An integrated Yolov5-Deeplabv3+ real-time segmentation network for gear pitting measurement. J. Intell. Manuf., 1\u201315.","DOI":"10.1007\/s10845-021-01876-y"},{"key":"ref_57","first-page":"12077","article-title":"SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers","volume":"34","author":"Xie","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7224\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:38:23Z","timestamp":1760143103000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7224"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,23]]},"references-count":57,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["s22197224"],"URL":"https:\/\/doi.org\/10.3390\/s22197224","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,23]]}}}