{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T18:08:02Z","timestamp":1767204482378,"version":"build-2238731810"},"reference-count":38,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,6,8]],"date-time":"2023-06-08T00:00:00Z","timestamp":1686182400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neuroinform."],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>The exorbitant cost of accurately annotating the large-scale serial scanning electron microscope (SEM) images as the ground truth for training has always been a great challenge for brain map reconstruction by deep learning methods in neural connectome studies. The representation ability of the model is strongly correlated with the number of such high-quality labels. Recently, the masked autoencoder (MAE) has been shown to effectively pre-train Vision Transformers (ViT) to improve their representational capabilities.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In this paper, we investigated a self-pre-training paradigm for serial SEM images with MAE to implement downstream segmentation tasks. We randomly masked voxels in three-dimensional brain image patches and trained an autoencoder to reconstruct the neuronal structures.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results and discussion<\/jats:title>\n                    <jats:p>We tested different pre-training and fine-tuning configurations on three different serial SEM datasets of mouse brains, including two public ones, SNEMI3D and MitoEM-R, and one acquired in our lab. A series of masking ratios were examined and the optimal ratio for pre-training efficiency was spotted for 3D segmentation. The MAE pre-training strategy significantly outperformed the supervised learning from scratch. Our work shows that the general framework of can be a unified approach for effective learning of the representation of heterogeneous neural structural features in serial SEM images to greatly facilitate brain connectome reconstruction.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fninf.2023.1118419","type":"journal-article","created":{"date-parts":[[2023,6,8]],"date-time":"2023-06-08T01:41:11Z","timestamp":1686188471000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Learning the heterogeneous representation of brain's structure from serial SEM images using a masked autoencoder"],"prefix":"10.3389","volume":"17","author":[{"given":"Ao","family":"Cheng","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiahao","family":"Shi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lirong","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ruobing","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,6,8]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2106.08254","article-title":"Beit: Bert pre-training of image transformers","author":"Bao","year":"2021","journal-title":"arXiv [Preprint]"},{"key":"B2","article-title":"\u201cLearned versus hand-designed feature representations for 3d agglomeration,\u201d","author":"Bogovic","year":"2013","journal-title":"CVPR"},{"key":"B3","first-page":"4171","article-title":"\u201cBERT: Pre-training of deep bidirectional transformers for language understanding,\u201d","author":"Devlin","year":"2019","journal-title":"Proceedings of NAACL-HLT"},{"key":"B4","article-title":"\u201cAn image is worth 16x16 words: transformers for image recognition at scale,\u201d","author":"Dosovitskiy","year":"2021","journal-title":"ICLR"},{"key":"B5","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1111\/jmi.12224","article-title":"High-resolution, high-throughput imaging with a multibeam scanning electron microscope","volume":"259","author":"Eberle","year":"2018","journal-title":"J. Microsc"},{"key":"B6","article-title":"\u201cAccurate, large minibatch SGD: training imagenet in 1 hour,\u201d","author":"Goyal","year":"2017","journal-title":"CVPR"},{"key":"B7","first-page":"574","article-title":"\u201cUNETR: transformers for 3d medical image segmentation,\u201d","author":"Hatamizadeh","year":"2022","journal-title":"Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision"},{"key":"B8","article-title":"Masked autoencoders are scalable vision learners","author":"He","year":"2021","journal-title":"arXiv preprint arXiv:2111.06377"},{"key":"B9","first-page":"646","article-title":"\u201cDeep networks with stochastic depth,\u201d","author":"Huang","year":"2016","journal-title":"European Conference on Computer Vision"},{"key":"B10","article-title":"\u201cAdversarial learning for semi-supervised semantic segmentation,\u201d","author":"Hung","year":"2018","journal-title":"Proceedings of the British Machine Vision Conference (BMVC)"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1101\/548081v1","article-title":"Segmentation-enhanced cyclegan","author":"Januszewski","year":"2019","journal-title":"bioRxiv [Preprint]"},{"key":"B12","doi-asserted-by":"publisher","first-page":"648","DOI":"10.1016\/j.cell.2015.06.054","article-title":"Saturated reconstruction of a volume of neocortex","volume":"162","author":"Kasthuri","year":"2015","journal-title":"Cell"},{"key":"B13","article-title":"\u201cAdam: a method for stochastic optimization,\u201d","author":"Kingma","year":"2014","journal-title":"ICLR"},{"key":"B14","article-title":"Instance segmentation of unlabeled modalities via cyclic segmentation GAN","author":"Lauenburg","year":"2022","journal-title":"arXiv preprint arXiv:2204.03082"},{"key":"B15","article-title":"Superhuman accuracy on the SNEMI3D connectomics challenge","author":"Lee","year":"2017","journal-title":"arXiv preprint arXiv:1706.00120"},{"key":"B16","first-page":"103","article-title":"\u201cTwo stream active query suggestion for active learning in connectomics,\u201d","author":"Lin","year":"2020","journal-title":"European Conference on Computer Vision"},{"key":"B17","article-title":"PyTorch connectomics: a scalable and flexible segmentation framework for em connectomics","author":"Lin","year":"2021","journal-title":"arXiv preprint arXiv:2112.05754"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1608.03983","article-title":"Sgdr: Stochastic gradient descent with warm restarts","author":"Loshchilov","year":"2016","journal-title":"arXiv [Preprint]"},{"key":"B19","article-title":"Decoupled weight decay regularization","author":"Loshchilov","year":"2017","journal-title":"arXiv preprint arXiv:1711.05101"},{"key":"B20","article-title":"\u201cGenerative pretraining from pixels,\u201d","author":"Mark","year":"2020","journal-title":"ICML"},{"key":"B21","doi-asserted-by":"publisher","first-page":"1369","DOI":"10.1109\/TPAMI.2019.2960224","article-title":"Semi-supervised semantic segmentation with high- and low-level consistency","volume":"43","author":"Mittal","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B22","doi-asserted-by":"publisher","first-page":"e71715","DOI":"10.1371\/journal.pone.0071715","article-title":"Machine learning of hierarchical clustering to segment 2D and 3D images","volume":"8","author":"Nunez-Iglesias","year":"2013","journal-title":"PLoS ONE"},{"key":"B23","first-page":"2536","article-title":"\u201cContext encoders: feature learning by inpainting,\u201d","author":"Pathak","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B24","author":"Radford","year":"2018","journal-title":"Improving Language Understanding by Generative Pre-Training"},{"key":"B25","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI blog"},{"key":"B26","first-page":"3347","article-title":"\u201cTransfusion: understanding transfer learning for medical imaging,\u201d","author":"Raghu","year":"","journal-title":"Proceedings of the 33rd International Conference on Neural Information Processing Systems"},{"key":"B27","article-title":"\u201cTransfusion: Understanding transfer learning for medical imaging,\u201d","author":"Raghu","year":"","journal-title":"Advances in Neural Information Processing Systems, Vol. 32"},{"key":"B28","doi-asserted-by":"crossref","first-page":"846","DOI":"10.1080\/01621459.1971.10482356","article-title":"Objective criteria for the evaluation of clustering methods","volume":"66","author":"Rand","year":"1971","journal-title":"J. Am. Stat. Assoc"},{"key":"B29","first-page":"234","article-title":"\u201cU-Net: convolutional networks for biomedical image segmentation,\u201d","author":"Ronneberger","year":"2015","journal-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention"},{"key":"B30","unstructured":"Dropout: A simple way to prevent neural networks from overfitting19291958\n            SrivasatavaN.\n            HintonG.\n            KrizhevskyA.\n            SutskeverI.\n            SalakhutdinovR.\n          33259321J. Mach. Learn. Res152014"},{"key":"B31","article-title":"\u201cAttention is all you need,\u201d","author":"Vaswani","year":"2017","journal-title":"NeurIPS"},{"key":"B32","first-page":"1096","article-title":"\u201cExtracting and composing robust features with denoising autoencoders,\u201d","author":"Vincent","year":"2008","journal-title":"Proceedings of the 25th International Conference on Machine Learning"},{"key":"B33","doi-asserted-by":"publisher","first-page":"3371","DOI":"10.5555\/1756006.1953039","article-title":"Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion","volume":"11","author":"Vincent","year":"2010","journal-title":"J. Mach. Learn. Res"},{"key":"B34","first-page":"14668","article-title":"\u201cMasked feature prediction for self-supervised visual pre-training,\u201d","author":"Wei","year":"2022","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B35","first-page":"66","article-title":"\u201cMitoEM dataset: large-scale 3D mitochondria instance segmentation from EM images,\u201d","author":"Wei","year":"2020","journal-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention"},{"key":"B36","doi-asserted-by":"publisher","first-page":"103145","DOI":"10.1016\/j.jvcir.2021.103145","article-title":"Rethinking pre-training on medical imaging","volume":"78","author":"Wen","year":"2021","journal-title":"J. Visual Commun. Image Represent."},{"key":"B37","first-page":"9653","article-title":"\u201cSimMIM: a simple framework for masked image modeling,\u201d","author":"Xie","year":"2022","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B38","doi-asserted-by":"publisher","first-page":"188","DOI":"10.1109\/TMI.2019.2922960","article-title":"CT super-resolution GAN constrained by the identical, residual, and cycle learning ensemble (GAN-circle)","volume":"39","author":"You","year":"2020","journal-title":"IEEE Trans. Med. Imaging"}],"updated-by":[{"DOI":"10.3389\/fninf.2023.1337766","type":"corrigendum","label":"Corrigendum","source":"publisher","updated":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T00:00:00Z","timestamp":1701043200000}}],"container-title":["Frontiers in Neuroinformatics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2023.1118419\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T09:17:46Z","timestamp":1701076666000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2023.1118419\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,8]]},"references-count":38,"alternative-id":["10.3389\/fninf.2023.1118419"],"URL":"https:\/\/doi.org\/10.3389\/fninf.2023.1118419","relation":{},"ISSN":["1662-5196"],"issn-type":[{"value":"1662-5196","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,8]]},"article-number":"1118419"}}