{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T17:11:03Z","timestamp":1780333863277,"version":"3.54.1"},"reference-count":53,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2023,9,22]],"date-time":"2023-09-22T00:00:00Z","timestamp":1695340800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2022YFC2602202"],"award-info":[{"award-number":["2022YFC2602202"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004761","name":"Hainan Natural Science Foundation","doi-asserted-by":"crossref","award":["621QN216"],"award-info":[{"award-number":["621QN216"]}],"id":[{"id":"10.13039\/501100004761","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100010834","name":"Education Department of Hainan Province","doi-asserted-by":"publisher","award":["Hnky2021-13"],"award-info":[{"award-number":["Hnky2021-13"]}],"id":[{"id":"10.13039\/501100010834","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Science Foundation for the State Key Laboratory for Infectious Disease Prevention and Control of China","award":["2022SKLID203"],"award-info":[{"award-number":["2022SKLID203"]}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities of China","doi-asserted-by":"publisher","award":["2662020LXQD002"],"award-info":[{"award-number":["2662020LXQD002"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,9,22]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The identification of viruses from negative staining transmission electron microscopy (TEM) images has mainly depended on experienced experts. Recent advances in artificial intelligence have enabled virus recognition using deep learning techniques. However, most of the existing methods only perform virus classification or semantic segmentation, and few studies have addressed the challenge of virus instance segmentation in TEM images. In this paper, we focus on the instance segmentation of severe acute respiratory syndrome coronavirus type 2 (SARS-CoV-2) and other respiratory viruses and provide experts with more effective information about viruses. We propose an effective virus instance segmentation network based on the You Only Look At CoefficienTs backbone, which integrates the Swin Transformer, dense connections and the coordinate-spatial attention mechanism, to identify SARS-CoV-2, H1N1 influenza virus, respiratory syncytial virus, Herpes simplex virus-1, Human adenovirus type 5 and Vaccinia virus. We also provide a public TEM virus dataset and conduct extensive comparative experiments. Our method achieves a mean average precision score of 83.8 and F1 score of 0.920, outperforming other state-of-the-art instance segmentation algorithms. The proposed automated method provides virologists with an effective approach for recognizing and identifying SARS-CoV-2 and assisting in the diagnosis of viruses. Our dataset and code are accessible at https:\/\/github.com\/xiaochiHNU\/Virus-Instance-Segmentation-Transformer-Network.<\/jats:p>","DOI":"10.1093\/bib\/bbad373","type":"journal-article","created":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T23:28:31Z","timestamp":1698708511000},"source":"Crossref","is-referenced-by-count":6,"title":["VISN: virus instance segmentation network for TEM images using deep attention transformer"],"prefix":"10.1093","volume":"24","author":[{"given":"Chi","family":"Xiao","sequence":"first","affiliation":[{"name":"State key laboratory of digital medical engineering , School of Biomedical Engineering, , 570228, Haikou , China"},{"name":"Hainan University , School of Biomedical Engineering, , 570228, Haikou , China"},{"name":"Key Laboratory of Biomedical Engineering of Hainan Province , One Health Institute, , 570228, Haikou , China"},{"name":"Hainan University , One Health Institute, , 570228, Haikou , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun","family":"Wang","sequence":"additional","affiliation":[{"name":"Laboratory of Microelectronic Devices & Integrated Technology, Institute of Microelectronics, Chinese Academy of Sciences , 100029, Beijing , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shenrong","family":"Yang","sequence":"additional","affiliation":[{"name":"State key laboratory of digital medical engineering , School of Biomedical Engineering, , 570228, Haikou , China"},{"name":"Hainan University , School of Biomedical Engineering, , 570228, Haikou , China"},{"name":"Key Laboratory of Biomedical Engineering of Hainan Province , One Health Institute, , 570228, Haikou , China"},{"name":"Hainan University , One Health Institute, , 570228, Haikou , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Minxin","family":"Heng","sequence":"additional","affiliation":[{"name":"State key laboratory of digital medical engineering , School of Biomedical Engineering, , 570228, Haikou , China"},{"name":"Hainan University , School of Biomedical Engineering, , 570228, Haikou , China"},{"name":"Key Laboratory of Biomedical Engineering of Hainan Province , One Health Institute, , 570228, Haikou , China"},{"name":"Hainan University , One Health Institute, , 570228, Haikou , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junyi","family":"Su","sequence":"additional","affiliation":[{"name":"State key laboratory of digital medical engineering , School of Biomedical Engineering, , 570228, Haikou , China"},{"name":"Hainan University , School of Biomedical Engineering, , 570228, Haikou , China"},{"name":"Key Laboratory of Biomedical Engineering of Hainan Province , One Health Institute, , 570228, Haikou , China"},{"name":"Hainan University , One Health Institute, , 570228, Haikou , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Xiao","sequence":"additional","affiliation":[{"name":"Key Laboratory for Matter Microstructure and Function of Hunan Province , Key Laboratory of Low-dimensional Quantum Structures and Quantum Control, School of Physics and Electronics, , 410081, Changsha , China"},{"name":"Hunan Normal University , Key Laboratory of Low-dimensional Quantum Structures and Quantum Control, School of Physics and Electronics, , 410081, Changsha , China"},{"name":"National Key Laboratory of Intelligent Tracking and Forecasting for Infectious Diseases , National Institute for Viral Disease Control and Prevention, , 102206, Beijing , China"},{"name":"Chinese Center for Disease Control and Prevention , National Institute for Viral Disease Control and Prevention, , 102206, Beijing , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jingdong","family":"Song","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Intelligent Tracking and Forecasting for Infectious Diseases , National Institute for Viral Disease Control and Prevention, , 102206, Beijing , China"},{"name":"Chinese Center for Disease Control and Prevention , National Institute for Viral Disease Control and Prevention, , 102206, Beijing , China"},{"name":"NHC Key Laboratory of Biosafety , National Institute for Viral Disease Control and Prevention, , 102206, Beijing , China"},{"name":"Chinese Center for Disease Control and Prevention , National Institute for Viral Disease Control and Prevention, , 102206, Beijing , China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weifu","family":"Li","sequence":"additional","affiliation":[{"name":"College of Informatics, Huazhong Agricultural University , 430070, Wuhan , China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2023,10,30]]},"reference":[{"issue":"3","key":"2023103020153912000_ref1","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1111\/his.14264","article-title":"Hunting coronavirus by transmission electron microscopy\u2013a guide to sars-cov-2-associated ultrastructural pathology in covid-19 tissues","volume":"78","author":"Hopfer","year":"2021","journal-title":"Histopathology"},{"issue":"1","key":"2023103020153912000_ref2","doi-asserted-by":"crossref","DOI":"10.1002\/rmv.2019","article-title":"Virus detection by transmission electron microscopy: still useful for diagnosis and a plus for biosafety","volume":"29","author":"Roingeard","year":"2019","journal-title":"Rev Med Virol"},{"key":"2023103020153912000_ref3","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1007\/s11427-013-4476-2","article-title":"Electron microscopy: essentials for viral structure, morphogenesis and rapid diagnosis","volume":"56","author":"Zhang","year":"2013","journal-title":"Sci China Life Sci"},{"key":"2023103020153912000_ref4","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1128\/CMR.00027-09","article-title":"Modern uses of electron microscopy for detection of viruses","volume":"22","author":"Goldsmith","year":"2009","journal-title":"Clin Microbiol Rev"},{"issue":"7798","key":"2023103020153912000_ref5","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1038\/s41586-020-2012-7","article-title":"A pneumonia outbreak associated with a new coronavirus of probable bat origin","volume":"579","author":"Zhou","year":"2020","journal-title":"Nature"},{"issue":"2","key":"2023103020153912000_ref6","doi-asserted-by":"crossref","first-page":"1297","DOI":"10.1093\/bib\/bbaa334","article-title":"Identification and characterization of circrnas encoded by mers-cov, sars-cov-1 and sars-cov-2","volume":"22","author":"Cai","year":"2021","journal-title":"Brief Bioinform"},{"issue":"1","key":"2023103020153912000_ref7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-020-17796-z","article-title":"Morphogenesis and cytopathic effect of sars-cov-2 infection in human airway epithelial cells","volume":"11","author":"Zhu","year":"2020","journal-title":"Nat Commun"},{"key":"2023103020153912000_ref8","doi-asserted-by":"crossref","DOI":"10.1093\/bib\/bbab114","article-title":"Multi-omics data integration and network-based analysis drives a multiplex drug repurposing approach to a shortlist of candidate drugs against covid-19","volume":"22","author":"Tomazou","year":"2021","journal-title":"Brief Bioinform"},{"issue":"10260","key":"2023103020153912000_ref9","doi-asserted-by":"crossref","first-page":"e64","DOI":"10.1016\/S0140-6736(20)32079-1","article-title":"Why misinterpretation of electron micrographs in sars-cov-2-infected tissue goes viral","volume":"396","author":"Dittmayer","year":"2020","journal-title":"Lancet"},{"key":"2023103020153912000_ref10","doi-asserted-by":"crossref","DOI":"10.1016\/j.bspc.2021.102588","article-title":"A fully automated deep learning-based network for detecting covid-19 from a new and large lung ct scan dataset","volume":"68","author":"Rahimzadeh","year":"2021","journal-title":"Biomed Signal Process Control"},{"key":"2023103020153912000_ref11","first-page":"3227","article-title":"Virus recognition based on local texture","volume-title":"International Conference on Pattern Recognition","author":"Sintorn","year":"2014"},{"key":"2023103020153912000_ref12","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.biosystemseng.2015.01.005","article-title":"Computer vision for virus image classification","volume":"138","author":"Santos","year":"2015","journal-title":"Biosyst Eng"},{"key":"2023103020153912000_ref13","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.patrec.2016.04.022","article-title":"Virus image classification using multi-scale completed local binary pattern features extracted from filtered images by multi-scale principal component analysis","volume":"79","author":"Wen","year":"2016","journal-title":"Pattern Recogn Lett"},{"key":"2023103020153912000_ref14","first-page":"352","article-title":"Contrast virus microscopy images recognition via k-nn classifiers","volume-title":"IEEE International Conference on Control System, Computing and Engineering","author":"Shakri","year":"2017"},{"key":"2023103020153912000_ref15","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1007\/978-3-030-34872-4_3","article-title":"An efficient method for automatic recognition of virus particles in tem images","volume-title":"International Conference on Pattern Recognition and Machine Intelligence","author":"Kumar","year":"2019"},{"issue":"1","key":"2023103020153912000_ref16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1743-422X-3-57","article-title":"Identification and classification of human cytomegalovirus capsids in textured electron micrographs using deformed template matching","volume":"3","author":"Ryner","year":"2006","journal-title":"Virol J"},{"issue":"5","key":"2023103020153912000_ref17","doi-asserted-by":"crossref","first-page":"1996","DOI":"10.1109\/TIP.2013.2244216","article-title":"Automatic virus particle selection\u2014the entropy approach","volume":"22","author":"da Concei\u00e7\u00e3o","year":"2013","journal-title":"IEEE Trans Image Process"},{"issue":"2","key":"2023103020153912000_ref18","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1111\/j.1365-2818.2011.03556.x","article-title":"Segmentation of virus particle candidates in transmission electron microscopy images","volume":"245","author":"Kylberg","year":"2012","journal-title":"J Microsc"},{"key":"2023103020153912000_ref19","first-page":"348","article-title":"Entropy virus microscopy images recognition via neural network classifiers","volume-title":"IEEE International Conference on Control System, Computing and Engineering","author":"Shakri","year":"2017"},{"key":"2023103020153912000_ref20","doi-asserted-by":"crossref","DOI":"10.1016\/j.cmpb.2020.105766","article-title":"Virus identification in electron microscopy images by residual mixed attention network","volume":"198","author":"Xiao","year":"2021","journal-title":"Comput Methods Programs Biomed"},{"key":"2023103020153912000_ref21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/9781119786122.ch1","article-title":"Machine learning-based virus type classification using transmission electron microscopy virus images","volume-title":"Machine Vision Inspection Systems, Volume 2: Machine Learning-Based Approaches","author":"Jena","year":"2021"},{"key":"2023103020153912000_ref22","doi-asserted-by":"crossref","DOI":"10.1016\/j.cmpb.2021.106318","article-title":"Tem virus images: benchmark dataset and deep learning classification","volume":"209","author":"Matuszewski","year":"2021","journal-title":"Comput Methods Programs Biomed"},{"issue":"2","key":"2023103020153912000_ref23","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1007\/s00418-018-1759-5","article-title":"Detection of herpesvirus capsids in transmission electron microscopy images using transfer learning","volume":"151","author":"Shaga Devan","year":"2019","journal-title":"Histochem Cell Biol"},{"key":"2023103020153912000_ref24","first-page":"1","article-title":"Deep learning methods for virus identification from digital images","volume-title":"International Conference on Image and Vision Computing New Zealand","author":"Zhang L and Yan W-Q","year":"2020"},{"key":"2023103020153912000_ref25","article-title":"Faster r-cnn: towards real-time object detection with region proposal networks","volume":"28","author":"Ren","year":"2015","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023103020153912000_ref26","first-page":"779","article-title":"You only look once: Unified, real-time object detection","volume-title":"Proceedings of the Conference on Computer Vision and Pattern Recognition","author":"Redmon","year":"2016"},{"key":"2023103020153912000_ref27","first-page":"21","article-title":"Ssd: Single shot multibox detector","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Liu","year":"2016"},{"issue":"2","key":"2023103020153912000_ref28","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1007\/s12560-018-9335-7","article-title":"Virus particle detection by convolutional neural network in transmission electron microscopy images","volume":"10","author":"Ito","year":"2018","journal-title":"Food Environ Virol"},{"key":"2023103020153912000_ref29","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.cmpb.2019.05.026","article-title":"Reducing the u-net size for practical scenarios: virus recognition in electron microscopy images","volume":"178","author":"Matuszewski","year":"2019","journal-title":"Comput Methods Programs Biomed"},{"key":"2023103020153912000_ref30","first-page":"234","article-title":"U-net: Convolutional networks for biomedical image segmentation","volume-title":"International Conference on Medical image computing and computer-assisted intervention","author":"Ronneberger","year":"2015"},{"key":"2023103020153912000_ref31","first-page":"9157","article-title":"Yolact: Real-time instance segmentation","volume-title":"Proceedings of the International Conference on Computer Vision","author":"Bolya","year":"2019"},{"key":"2023103020153912000_ref32","article-title":"Swin transformer: hierarchical vision transformer using shifted windows","author":"Liu","year":"2021","journal-title":"arXiv preprint arXiv:210314030"},{"key":"2023103020153912000_ref33","article-title":"An image is worth 16x16 words: transformers for image recognition at scale","author":"Dosovitskiy","year":"2020","journal-title":"arXiv preprint arXiv:201011929"},{"issue":"6","key":"2023103020153912000_ref34","doi-asserted-by":"crossref","first-page":"1245","DOI":"10.1109\/TMM.2017.2648498","article-title":"Diversified visual attention networks for fine-grained object classification","volume":"19","author":"Zhao","year":"2017","journal-title":"IEEE Trans Multimed"},{"key":"2023103020153912000_ref35","first-page":"7132","article-title":"Squeeze-and-excitation networks","volume-title":"Proceedings of the Conference on Computer Vision and Pattern Recognition","author":"Hu","year":"2018"},{"key":"2023103020153912000_ref36","first-page":"3","article-title":"Cbam: Convolutional block attention module","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Woo","year":"2018"},{"key":"2023103020153912000_ref37","first-page":"13713","article-title":"Coordinate attention for efficient mobile network design","volume-title":"Proceedings of the Conference on Computer Vision and Pattern Recognition","author":"Hou","year":"2021"},{"key":"2023103020153912000_ref38","article-title":"Pytorch: an imperative style, high-performance deep learning library","volume":"32","author":"Paszke","year":"2019","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023103020153912000_ref39","article-title":"Fixing weight decay regularization in Adam","author":"Loshchilov","year":"2017","journal-title":"arXiv preprint arXiv:171105101"},{"key":"2023103020153912000_ref40","first-page":"740","article-title":"Microsoft coco: Common objects in context","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Lin","year":"2014"},{"key":"2023103020153912000_ref41","first-page":"17721","article-title":"Solov2: dynamic and fast instance segmentation","volume":"33","author":"Wang","year":"2020","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023103020153912000_ref42","article-title":"Yolact++: better real-time instance segmentation","author":"Bolya","year":"2020","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2023103020153912000_ref43","first-page":"4433","article-title":"Sparse instance activation for real-time instance segmentation","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Cheng","year":"2022"},{"key":"2023103020153912000_ref44","article-title":"Rtmdet: an empirical study of designing real-time object detectors","author":"Lyu","year":"2022","journal-title":"arXiv preprint arXiv:221207784"},{"key":"2023103020153912000_ref45","first-page":"2359","article-title":"Fully convolutional instance-aware semantic segmentation","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Li","year":"2017"},{"key":"2023103020153912000_ref46","article-title":"Retinamask: learning to predict masks improves state-of-the-art single-shot detection for free","author":"Cheng-Yang","year":"2019","journal-title":"arXiv preprint arXiv:190103353"},{"key":"2023103020153912000_ref47","article-title":"Mask r-cnn","author":"He","year":"2017","journal-title":"Proceedings of the International Conference on Computer Vision"},{"key":"2023103020153912000_ref48","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.ins.2021.02.066","article-title":"PA-Net: Learning local features using by pose attention for short-term person re-identification","volume":"565","author":"Wang","year":"2021","journal-title":"Inform Sciences"},{"key":"2023103020153912000_ref49","doi-asserted-by":"crossref","first-page":"110495","DOI":"10.1016\/j.chaos.2020.110495","article-title":"Corodet: a deep learning based classification for covid-19 detection using chest x-ray images","volume":"142","author":"Hussain","year":"2021","journal-title":"Chaos, Solitons & Fractals"},{"issue":"1","key":"2023103020153912000_ref50","first-page":"1","article-title":"Covid-net: a tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images","volume":"10","author":"Wang","year":"2020","journal-title":"Sci Rep"},{"issue":"5","key":"2023103020153912000_ref51","first-page":"821","article-title":"Identification of covid-19 samples from chest x-ray images using deep learning: a comparison of transfer learning approaches","volume":"28","author":"Rahaman","year":"2020","journal-title":"J Xray Sci Technol"},{"key":"2023103020153912000_ref52","first-page":"244","article-title":"A sars-cov-2 microscopic image dataset with ground truth images and visual features","volume-title":"Chinese conference on pattern recognition and computer vision (PRCV)","author":"Li","year":"2020"},{"key":"2023103020153912000_ref53","article-title":"Learning to estimate shapley values with vision transformers","author":"Covert","year":"2022","journal-title":"arXiv preprint arXiv:220605282"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/24\/6\/bbad373\/52690001\/bbad373.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/24\/6\/bbad373\/52690001\/bbad373.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T23:29:17Z","timestamp":1698708557000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbad373\/7333675"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,22]]},"references-count":53,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,9,22]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbad373","relation":{},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,11,1]]},"published":{"date-parts":[[2023,9,22]]},"article-number":"bbad373"}}