{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T16:16:51Z","timestamp":1775578611341,"version":"3.50.1"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,2,1]],"date-time":"2021-02-01T00:00:00Z","timestamp":1612137600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2021,2,1]],"date-time":"2021-02-01T00:00:00Z","timestamp":1612137600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61701497"],"award-info":[{"award-number":["61701497"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Strategic Priority Research Program of Chinese Academy of Science","award":["XDB32030200"],"award-info":[{"award-number":["XDB32030200"]}]},{"name":"Instrument function development innovation program of Chinese Academy of Sciences","award":["282019000057"],"award-info":[{"award-number":["282019000057"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BioData Mining"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Microscopic imaging is a crucial technology for visualizing neural and tissue structures. Large-area defects inevitably occur during the imaging process of electron microscope (EM) serial slices, which lead to reduced registration and semantic segmentation, and affect the accuracy of 3D reconstruction. The continuity of biological tissue among serial EM images makes it possible to recover missing tissues utilizing inter-slice interpolation. However, large deformation, noise, and blur among EM images remain the task challenging. Existing flow-based and kernel-based methods have to perform frame interpolation on images with little noise and low blur. They also cannot effectively deal with large deformations on EM images.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>In this paper, we propose a sparse self-attention aggregation network to synthesize pixels following the continuity of biological tissue. First, we develop an attention-aware layer for consecutive EM images interpolation that implicitly adopts global perceptual deformation. Second, we present an adaptive style-balance loss taking the style differences of serial EM images such as blur and noise into consideration. Guided by the attention-aware module, adaptively synthesizing each pixel aggregated from the global domain further improves the performance of pixel synthesis. Quantitative and qualitative experiments show that the proposed method is superior to the state-of-the-art approaches.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>The proposed method can be considered as an effective strategy to model the relationship between each pixel and other pixels from the global domain. This approach improves the algorithm\u2019s robustness to noise and large deformation, and can accurately predict the effective information of the missing region, which will greatly promote the data analysis of neurobiological research.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s13040-021-00236-z","type":"journal-article","created":{"date-parts":[[2021,2,1]],"date-time":"2021-02-01T13:04:02Z","timestamp":1612184642000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Sparse self-attention aggregation networks for neural sequence slice interpolation"],"prefix":"10.1186","volume":"14","author":[{"given":"Zejin","family":"Wang","sequence":"first","affiliation":[]},{"given":"Jing","family":"Liu","sequence":"additional","affiliation":[]},{"given":"Xi","family":"Chen","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2836-6413","authenticated-orcid":false,"given":"Guoqing","family":"Li","sequence":"additional","affiliation":[]},{"given":"Hua","family":"Han","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,2,1]]},"reference":[{"key":"236_CR1","volume-title":"2018 25th IEEE International Conference on Image Processing (ICIP)","author":"P Afshar","year":"2018","unstructured":"Afshar P, Shahroudnejad A, Mohammadi A, Plataniotis KN. Carisi: Convolutional autoencoder-based inter-slice interpolation of brain tumor volumetric images. In: 2018 25th IEEE International Conference on Image Processing (ICIP). Athens: IEEE: 2018. p. 1458\u201362."},{"key":"236_CR2","unstructured":"Wu Z, Wei J, Yuan W, Wang J, Tasdizen T. Inter-slice image augmentation based on frame interpolation for boosting medical image segmentation accuracy. arXiv preprint arXiv:2001.11698. 2020."},{"key":"236_CR3","volume-title":"2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019)","author":"T Nguyen-Duc","year":"2019","unstructured":"Nguyen-Duc T, Yoo I, Thomas L, Kuan A, Lee W-C, Jeong W-K. Weakly supervised learning in deformable em image registration using slice interpolation. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). Venezia: IEEE: 2019. p. 670\u20133."},{"issue":"1","key":"236_CR4","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1186\/s13040-017-0161-5","volume":"10","author":"Q Xie","year":"2017","unstructured":"Xie Q, Chen X, Deng H, Liu D, Sun Y, Zhou X, Yang Y, Han H. An automated pipeline for bouton, spine, and synapse detection of in vivo two-photon images. BioData Min. 2017; 10(1):40.","journal-title":"BioData Min"},{"issue":"1","key":"236_CR5","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1186\/s13040-018-0183-7","volume":"11","author":"W Li","year":"2018","unstructured":"Li W, Liu J, Xiao C, Deng H, Xie Q, Han H. A fast forward 3d connection algorithm for mitochondria and synapse segmentations from serial em images. BioData Min. 2018; 11(1):24.","journal-title":"BioData Min"},{"key":"236_CR6","volume-title":"European Conference on Computer Vision","author":"G Long","year":"2016","unstructured":"Long G, Kneip L, Alvarez JM, Li H, Zhang X, Yu Q. Learning image matching by simply watching video. In: European Conference on Computer Vision. Amsterdam: Springer: 2016. p. 434\u201350."},{"key":"236_CR7","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Z Liu","year":"2017","unstructured":"Liu Z, Yeh RA, Tang X, Liu Y, Agarwala A. Video frame synthesis using deep voxel flow. In: Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE: 2017. p. 4463\u201371."},{"key":"236_CR8","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"S Niklaus","year":"2017","unstructured":"Niklaus S, Mai L, Liu F. Video frame interpolation via adaptive convolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 2017. p. 670\u20139."},{"key":"236_CR9","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"S Niklaus","year":"2017","unstructured":"Niklaus S, Mai L, Liu F. Video frame interpolation via adaptive separable convolution. In: Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE: 2017. p. 261\u2013270."},{"key":"236_CR10","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"S Niklaus","year":"2018","unstructured":"Niklaus S, Liu F. Context-aware synthesis for video frame interpolation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 2018. p. 1701\u201310."},{"key":"236_CR11","volume-title":"IEEE transactions on pattern analysis and machine intelligence","author":"W Bao","year":"2019","unstructured":"Bao W, Lai W-S, Zhang X, Gao Z, Yang M-H. Memc-net: Motion estimation and motion compensation driven neural network for video interpolation and enhancement. In: IEEE transactions on pattern analysis and machine intelligence. Washington, D.C.: IEEE: 2019."},{"key":"236_CR12","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"W Bao","year":"2019","unstructured":"Bao W, Lai W-S, Ma C, Zhang X, Gao Z, Yang M-H. Depth-aware video frame interpolation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 2019. p. 3703\u201312."},{"key":"236_CR13","volume-title":"Advances in Neural Information Processing Systems","author":"D Eigen","year":"2014","unstructured":"Eigen D, Puhrsch C, Fergus R. Depth map prediction from a single image using a multi-scale deep network. In: Advances in Neural Information Processing Systems. Montreal: MIT press: 2014. p. 2366\u201374."},{"key":"236_CR14","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"D Eigen","year":"2015","unstructured":"Eigen D, Fergus R. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In: Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE: 2015. p. 2650\u20138."},{"key":"236_CR15","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"P Wang","year":"2015","unstructured":"Wang P, Shen X, Lin Z, Cohen S, Price B, Yuille AL. Towards unified depth and semantic prediction from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE: 2015. p. 2800\u20139."},{"key":"236_CR16","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"A Dosovitskiy","year":"2015","unstructured":"Dosovitskiy A, Fischer P, Ilg E, Hausser P, Hazirbas C, Golkov V, Van Der Smagt P, Cremers D, Brox T. Flownet: Learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision. Santiago: IEEE: 2015. p. 2758\u201366."},{"key":"236_CR17","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"E Ilg","year":"2017","unstructured":"Ilg E, Mayer N, Saikia T, Keuper M, Dosovitskiy A, Brox T. Flownet 2.0: Evolution of optical flow estimation with deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 2017. p. 2462\u201370."},{"key":"236_CR18","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"A Ranjan","year":"2017","unstructured":"Ranjan A, Black MJ. Optical flow estimation using a spatial pyramid network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE: 2017. p. 4161\u201370."},{"issue":"6","key":"236_CR19","doi-asserted-by":"publisher","first-page":"881","DOI":"10.1109\/42.544506","volume":"15","author":"GJ Grevera","year":"1996","unstructured":"Grevera GJ, Udupa JK. Shape-based interpolation of multidimensional grey-level images. IEEE Trans Med Imaging. 1996; 15(6):881\u201392.","journal-title":"IEEE Trans Med Imaging"},{"issue":"7","key":"236_CR20","doi-asserted-by":"publisher","first-page":"711","DOI":"10.1109\/42.875193","volume":"19","author":"T-Y Lee","year":"2000","unstructured":"Lee T-Y, Wang W-H. Morphology-based three-dimensional interpolation. IEEE Trans Med Imaging. 2000; 19(7):711\u201321.","journal-title":"IEEE Trans Med Imaging"},{"issue":"7","key":"236_CR21","doi-asserted-by":"publisher","first-page":"922","DOI":"10.1109\/TMI.2004.828352","volume":"23","author":"GP Penney","year":"2004","unstructured":"Penney GP, Schnabel JA, Rueckert D, Viergever MA, Niessen WJ. Registration-based interpolation. IEEE Trans Med Imaging. 2004; 23(7):922\u20136.","journal-title":"IEEE Trans Med Imaging"},{"issue":"6","key":"236_CR22","doi-asserted-by":"publisher","first-page":"2089","DOI":"10.1109\/TCBB.2018.2822803","volume":"16","author":"RK Sevakula","year":"2018","unstructured":"Sevakula RK, Singh V, Verma NK, Kumar C, Cui Y. Transfer learning for molecular cancer classification using deep neural networks. IEEE\/ACM Trans Comput Biol Bioinforma. 2018; 16(6):2089\u2013100.","journal-title":"IEEE\/ACM Trans Comput Biol Bioinforma"},{"issue":"2","key":"236_CR23","doi-asserted-by":"publisher","first-page":"432","DOI":"10.1109\/TCBB.2016.2535233","volume":"15","author":"M Shin","year":"2016","unstructured":"Shin M, Jang D, Nam H, Lee KH, Lee D. Predicting the absorption potential of chemical compounds through a deep learning approach. IEEE\/ACM Trans Comput Biol Bioinforma. 2016; 15(2):432\u201340.","journal-title":"IEEE\/ACM Trans Comput Biol Bioinforma"},{"issue":"5","key":"236_CR24","doi-asserted-by":"crossref","first-page":"1669","DOI":"10.1109\/TCBB.2017.2740381","volume":"15","author":"V Vijayan","year":"2017","unstructured":"Vijayan V, Milenkovi\u0107 T. Multiple network alignment via multimagna++. IEEE\/ACM Trans Comput Biol Bioinforma. 2017; 15(5):1669\u201382.","journal-title":"IEEE\/ACM Trans Comput Biol Bioinforma"},{"key":"236_CR25","volume-title":"Advances in Neural Information Processing Systems","author":"A Vaswani","year":"2017","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141., Polosukhin I. Attention is all you need. In: Advances in Neural Information Processing Systems. Long Beach: MIT press: 2017. p. 5998\u20136008."},{"key":"236_CR26","unstructured":"Parmar N, Vaswani A, Uszkoreit J, Kaiser L, Shazeer N, Ku A, Tran D. Image Transformer. In: ICML: 2018. p. 4052\u20134061. http:\/\/proceedings.mlr.press\/v80\/parmar18a.html."},{"key":"236_CR27","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"X Wang","year":"2018","unstructured":"Wang X, Girshick R, Gupta A, He K. Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE: 2018. p. 7794\u2013803."},{"key":"236_CR28","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"J Fu","year":"2019","unstructured":"Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H. Dual attention network for scene segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE: 2019. p. 3146\u201354."},{"key":"236_CR29","unstructured":"Yuan Y, Wang J. Ocnet: Object context network for scene parsing. arXiv preprint arXiv:1809.00916. 2018."},{"key":"236_CR30","unstructured":"Huang L, Yuan Y, Guo J, Zhang C, Chen X, Wang J. Interlaced sparse self-attention for semantic segmentation. arXiv preprint arXiv:1907.12273. 2019."},{"key":"236_CR31","unstructured":"Child R, Gray S, Radford A, Sutskever I. Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509. 2019."},{"key":"236_CR32","doi-asserted-by":"crossref","unstructured":"Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y. Residual dense network for image restoration. IEEE Trans Pattern Anal Mach Intell. 2020. https:\/\/ieeexplore.ieee.org\/abstract\/document\/8964437.","DOI":"10.1109\/TPAMI.2020.2968521"},{"key":"236_CR33","unstructured":"Zhang H, Goodfellow I, Metaxas D, Odena A. Self-attention generative adversarial networks. In: International conference on machine learning. PMLR: 2019. p. 7354\u20137363."},{"key":"236_CR34","unstructured":"Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition. In: International Conference on Learning Representations: 2015."},{"key":"236_CR35","volume-title":"European Conference on Computer Vision","author":"J Johnson","year":"2016","unstructured":"Johnson J, Alahi A, Fei-Fei L. Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision. Amsterdam: Springer: 2016. p. 694\u2013711."},{"key":"236_CR36","doi-asserted-by":"crossref","unstructured":"Gatys LA, Ecker AS, Bethge M. Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition: 2016. p. 2414\u201323.","DOI":"10.1109\/CVPR.2016.265"},{"key":"236_CR37","volume-title":"Proceedings of 1st International Conference on Image Processing","author":"P Charbonnier","year":"1994","unstructured":"Charbonnier P, Blanc-Feraud L, Aubert G, Barlaud M. Two deterministic half-quadratic regularization algorithms for computed imaging. In: Proceedings of 1st International Conference on Image Processing. Texas: IEEE: 1994. p. 168\u201372."},{"key":"236_CR38","unstructured":"Kingma DP, Ba J. Adam (2014), A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR), vol 1412.2015."},{"issue":"3","key":"236_CR39","doi-asserted-by":"publisher","first-page":"297","DOI":"10.2307\/1932409","volume":"26","author":"LR Dice","year":"1945","unstructured":"Dice LR. Measures of the amount of ecologic association between species. Ecology. 1945; 26(3):297\u2013302.","journal-title":"Ecology"},{"key":"236_CR40","volume-title":"International Conference on Medical Image Computing and Computer-assisted Intervention","author":"O Ronneberger","year":"2015","unstructured":"Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention. Munich: Springer: 2015. p. 234\u201341."}],"container-title":["BioData Mining"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s13040-021-00236-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s13040-021-00236-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s13040-021-00236-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T09:59:29Z","timestamp":1724407169000},"score":1,"resource":{"primary":{"URL":"https:\/\/biodatamining.biomedcentral.com\/articles\/10.1186\/s13040-021-00236-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,1]]},"references-count":40,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["236"],"URL":"https:\/\/doi.org\/10.1186\/s13040-021-00236-z","relation":{},"ISSN":["1756-0381"],"issn-type":[{"value":"1756-0381","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,1]]},"assertion":[{"value":"31 August 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 January 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 February 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"10"}}