{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T02:56:04Z","timestamp":1787021764953,"version":"build-2736575974"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1013115","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2025,6,6]],"date-time":"2025-06-06T00:00:00Z","timestamp":1749168000000}}],"reference-count":52,"publisher":"Public Library of Science (PLoS)","issue":"5","license":[{"start":{"date-parts":[[2025,5,28]],"date-time":"2025-05-28T00:00:00Z","timestamp":1748390400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000024","name":"Canadian Institutes for Health Research","doi-asserted-by":"crossref","award":["180542"],"award-info":[{"award-number":["180542"]}],"id":[{"id":"10.13039\/501100000024","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Electron microscopy (EM) has revolutionized our understanding of cellular structures at the nanoscale. Accurate image segmentation is required for analyzing EM images. While manual segmentation is reliable, it is labor-intensive, incentivizing the development of automated segmentation methods. Although deep learning-based segmentation has demonstrated expert-level performance, it lacks generalizable performance across diverse EM datasets. Current approaches usually use either convolutional or transformer-based neural networks for image feature extraction. We developed the RETINA method, which combines pre-training on the large, unlabeled CEM500K EM image dataset with a hybrid neural-network model architecture that integrates both local (convolutional layer) and global (transformer layer) image processing to learn from manual image annotations. RETINA outperformed existing models on cellular structure segmentation on five public EM datasets. This improvement works toward automated cellular structure segmentation for the EM community.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1013115","type":"journal-article","created":{"date-parts":[[2025,5,28]],"date-time":"2025-05-28T14:15:02Z","timestamp":1748441702000},"page":"e1013115","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":7,"title":["RETINA: Reconstruction-based pre-trained enhanced TransUNet for electron microscopy segmentation on the CEM500K dataset"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7959-6403","authenticated-orcid":true,"given":"Cheng","family":"Xing","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronald","family":"Xie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0185-8861","authenticated-orcid":true,"given":"Gary D.","family":"Bader","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,5,28]]},"reference":[{"key":"pcbi.1013115.ref001","first-page":"1","article-title":"Modular segmentation, spatial analysis and visualization of volume electron microscopy datasets","author":"A M\u00fcller","year":"2024","journal-title":"Nat Protoc"},{"key":"pcbi.1013115.ref002","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1038\/s43586-022-00131-9","article-title":"Volume electron microscopy","volume":"2","author":"CJ Peddie","year":"2022","journal-title":"Nat Rev Methods Primers"},{"issue":"6475","key":"pcbi.1013115.ref003","doi-asserted-by":"crossref","DOI":"10.1126\/science.aaz5357","article-title":"Correlative three-dimensional super-resolution and block-face electron microscopy of whole vitreously frozen cells","volume":"367","author":"DP Hoffman","year":"2020","journal-title":"Science"},{"issue":"18","key":"pcbi.1013115.ref004","doi-asserted-by":"crossref","DOI":"10.1016\/j.cell.2021.07.017","article-title":"Whole-body integration of gene expression and single-cell morphology","volume":"184","author":"HM Vergara","year":"2021","journal-title":"Cell"},{"issue":"7871","key":"pcbi.1013115.ref005","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1038\/s41586-021-03778-8","article-title":"Connectomes across development reveal principles of brain maturation","volume":"596","author":"D Witvliet","year":"2021","journal-title":"Nature"},{"issue":"4","key":"pcbi.1013115.ref006","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1038\/s41587-021-01094-0","article-title":"Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning","volume":"40","author":"NF Greenwald","year":"2022","journal-title":"Nat Biotechnol"},{"issue":"1","key":"pcbi.1013115.ref007","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1093\/bioinformatics\/btaa1094","article-title":"EM-stellar: benchmarking deep learning for electron microscopy image segmentation","volume":"37","author":"A Khadangi","year":"2021","journal-title":"Bioinformatics"},{"key":"pcbi.1013115.ref008","doi-asserted-by":"crossref","first-page":"102920","DOI":"10.1016\/j.media.2023.102920","article-title":"Segmentation in large-scale cellular electron microscopy with deep learning: A literature survey","volume":"89","author":"A Aswath","year":"2023","journal-title":"Med Image Anal"},{"issue":"16","key":"pcbi.1013115.ref009","doi-asserted-by":"crossref","first-page":"2555","DOI":"10.1093\/bioinformatics\/btx188","article-title":"DeepEM3D: approaching human-level performance on 3D anisotropic EM image segmentation","volume":"33","author":"T Zeng","year":"2017","journal-title":"Bioinformatics"},{"issue":"1","key":"pcbi.1013115.ref010","doi-asserted-by":"crossref","first-page":"15462","DOI":"10.1038\/s41598-022-19356-5","article-title":"Deep neural networks allow expert-level brain meningioma segmentation and present potential for improvement of clinical practice","volume":"12","author":"A Boaro","year":"2022","journal-title":"Sci Rep"},{"issue":"7902","key":"pcbi.1013115.ref011","doi-asserted-by":"crossref","first-page":"736","DOI":"10.1038\/s41586-022-04488-5","article-title":"Regulation of liver subcellular architecture controls metabolic homeostasis","volume":"603","author":"G Parlakg\u00fcl","year":"2022","journal-title":"Nature"},{"issue":"1","key":"pcbi.1013115.ref012","doi-asserted-by":"crossref","first-page":"2561","DOI":"10.1038\/s41598-021-81590-0","article-title":"Dense cellular segmentation for EM using 2D-3D neural network ensembles","volume":"11","author":"MD Guay","year":"2021","journal-title":"Sci Rep"},{"issue":"7953","key":"pcbi.1013115.ref013","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1038\/s41586-023-05793-3","article-title":"Spatial mapping of mitochondrial networks and bioenergetics in lung cancer","volume":"615","author":"M Han","year":"2023","journal-title":"Nature"},{"key":"pcbi.1013115.ref014","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1109\/CVPR.2009.5206848","article-title":"Imagenet: a large-scale hierarchical image database.","volume-title":"2009 IEEE Conference on Computer Vision and Pattern Recognition","author":"J Deng","year":"2009"},{"issue":"7","key":"pcbi.1013115.ref015","first-page":"3523","article-title":"Image segmentation using deep learning: a survey","volume":"44","author":"S Minaee","year":"2022","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"1","key":"pcbi.1013115.ref016","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1038\/s42003-023-04608-5","article-title":"Efficient end-to-end learning for cell segmentation with machine generated weak annotations","volume":"6","author":"P Shrestha","year":"2023","journal-title":"Commun Biol"},{"key":"pcbi.1013115.ref017","first-page":"4918","article-title":"Rethinking imagenet pre-training.","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"K He","year":"2019"},{"key":"pcbi.1013115.ref018","first-page":"2959","article-title":"Unsupervised pre-training of image features on non-curated data.","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"M Caron","year":"2019"},{"issue":"5","key":"pcbi.1013115.ref019","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1038\/nmeth.3806","article-title":"EMPIAR: a public archive for raw electron microscopy image data","volume":"13","author":"A Iudin","year":"2016","journal-title":"Nat Methods"},{"issue":"7883","key":"pcbi.1013115.ref020","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1038\/s41586-021-03992-4","article-title":"An open-access volume electron microscopy atlas of whole cells and tissues","volume":"599","author":"CS Xu","year":"2021","journal-title":"Nature"},{"key":"pcbi.1013115.ref021","doi-asserted-by":"crossref","DOI":"10.7554\/eLife.65894","article-title":"CEM500K, a large-scale heterogeneous unlabeled cellular electron microscopy image dataset for deep learning","volume":"10","author":"R Conrad","year":"2021","journal-title":"Elife"},{"key":"pcbi.1013115.ref022","first-page":"1597","article-title":"A simple framework for contrastive learning of visual representations.","volume-title":"International Conference on Machine Learning","author":"T Chen","year":"2020"},{"key":"pcbi.1013115.ref023","first-page":"9729","article-title":"Momentum contrast for unsupervised visual representation learning.","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"K He","year":"2020"},{"key":"pcbi.1013115.ref024","first-page":"69","article-title":"Unsupervised learning of visual representations by solving jigsaw puzzles.","volume-title":"European conference on computer vision","author":"M Noroozi","year":"2016"},{"key":"pcbi.1013115.ref025","first-page":"1995","article-title":"Convolutional networks for images, speech, and time series","volume-title":"The handbook of brain theory and neural networks","author":"Y LeCun","year":"1995"},{"key":"pcbi.1013115.ref026","article-title":"Attention is all you need","volume":"30","author":"A Vaswani","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"pcbi.1013115.ref027","article-title":"An image is worth 16x16 words: transformers for image recognition at scale","author":"A Dosovitskiy","year":"2020","journal-title":"arXiv preprint"},{"key":"pcbi.1013115.ref028","first-page":"234","article-title":"U-net: convolutional networks for biomedical image segmentation.","volume-title":"Medical image computing and computer-assisted intervention\u2013MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III","author":"O Ronneberger","year":"2015"},{"issue":"12","key":"pcbi.1013115.ref029","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"SegNet: a deep convolutional encoder-decoder architecture for image segmentation","volume":"39","author":"V Badrinarayanan","year":"2017","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"pcbi.1013115.ref030","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1007\/978-3-319-10590-1_53","article-title":"Visualizing and understanding convolutional networks.","volume-title":"Computer Vision\u2013ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I","author":"MD Zeiler","year":"2014"},{"key":"pcbi.1013115.ref031","article-title":"Understanding the effective receptive field in deep convolutional neural networks","volume":"29","author":"W Luo","year":"2016","journal-title":"Adv Neural Inf Process Syst"},{"key":"pcbi.1013115.ref032","first-page":"272","article-title":"Swin unetr: swin transformers for semantic segmentation of brain tumors in mri images.","volume-title":"International MICCAI Brainlesion Workshop","author":"A Hatamizadeh","year":"2021"},{"key":"pcbi.1013115.ref033","article-title":"Transunet: transformers make strong encoders for medical image segmentation","author":"J Chen","year":"2021","journal-title":"arXiv preprint"},{"key":"pcbi.1013115.ref034","article-title":"Improved baselines with momentum contrastive learning","author":"X Chen","year":"2020","journal-title":"arXiv preprint"},{"key":"pcbi.1013115.ref035","author":"CREMI","year":"2016"},{"issue":"3","key":"pcbi.1013115.ref036","doi-asserted-by":"crossref","first-page":"648","DOI":"10.1016\/j.cell.2015.06.054","article-title":"Saturated reconstruction of a volume of neocortex","volume":"162","author":"N Kasthuri","year":"2015","journal-title":"Cell"},{"key":"pcbi.1013115.ref037","doi-asserted-by":"crossref","first-page":"126","DOI":"10.3389\/fnana.2014.00126","article-title":"A workflow for the automatic segmentation of organelles in electron microscopy image stacks","volume":"8","author":"AJ Perez","year":"2014","journal-title":"Front Neuroanat"},{"key":"pcbi.1013115.ref038","doi-asserted-by":"crossref","first-page":"103693","DOI":"10.1016\/j.compbiomed.2020.103693","article-title":"Automatic segmentation of mitochondria and endolysosomes in volumetric electron microscopy data","volume":"119","author":"M \u017derovnik Meku\u010d","year":"2020","journal-title":"Comput Biol Med"},{"key":"pcbi.1013115.ref039","article-title":"3D transunet: Advancing medical image segmentation through vision transformers","author":"J Chen","year":"2023","journal-title":"arXiv preprint"},{"key":"pcbi.1013115.ref040","first-page":"770","article-title":"Deep residual learning for image recognition.","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"K He","year":"2016"},{"issue":"2","key":"pcbi.1013115.ref041","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1038\/s41592-020-01008-z","article-title":"nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation","volume":"18","author":"F Isensee","year":"2021","journal-title":"Nat Methods"},{"key":"pcbi.1013115.ref042","first-page":"488","article-title":"nnu-net revisited: a call for rigorous validation in 3d medical image segmentation.","volume-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention","author":"F Isensee","year":"2024"},{"key":"pcbi.1013115.ref043","first-page":"16000","article-title":"Masked autoencoders are scalable vision learners.","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"K He","year":"2022"},{"key":"pcbi.1013115.ref044","article-title":"ScribFormer: transformer makes CNN work better for scribble-based medical image segmentation","author":"Z Li","year":"2024","journal-title":"IEEE Trans Med Imaging"},{"key":"pcbi.1013115.ref045","unstructured":"Ma J, Li F, Wang B. U-mamba: enhancing long-range dependency for biomedical image segmentation. arXiv preprint 2024. https:\/\/arxiv.org\/abs\/2401.04722"},{"key":"pcbi.1013115.ref046","article-title":"Autoaugment: learning augmentation policies from data","author":"ED Cubuk","year":"2018","journal-title":"arXiv preprint"},{"issue":"1","key":"pcbi.1013115.ref047","article-title":"Instance segmentation of mitochondria in electron microscopy images with a generalist deep learning model trained on a diverse dataset","volume":"14","author":"R Conrad","year":"2023","journal-title":"Cell Syst"},{"key":"pcbi.1013115.ref048","article-title":"Downstream datasets make surprisingly good pretraining corpora","author":"K Krishna","year":"2022","journal-title":"arXiv preprint"},{"issue":"1","key":"pcbi.1013115.ref049","first-page":"12","article-title":"Total internal reflection fluorescence (TIRF) microscopy","volume":"50","author":"KN Fish","year":"2009","journal-title":"Curr Protoc. Cytomet"},{"issue":"7883","key":"pcbi.1013115.ref050","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1038\/s41586-021-03977-3","article-title":"Whole-cell organelle segmentation in volume electron microscopy","volume":"599","author":"L Heinrich","year":"2021","journal-title":"Nature"},{"issue":"5","key":"pcbi.1013115.ref051","doi-asserted-by":"crossref","first-page":"127","DOI":"10.3390\/jimaging8050127","article-title":"A review of watershed implementations for segmentation of volumetric images","volume":"8","author":"A Kornilov","year":"2022","journal-title":"J Imaging"},{"key":"pcbi.1013115.ref052","first-page":"2961","article-title":"Mask r-cnn.","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"K He","year":"2017"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1013115","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2025,6,6]],"date-time":"2025-06-06T00:00:00Z","timestamp":1749168000000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1013115","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,6]],"date-time":"2025-06-06T13:51:08Z","timestamp":1749217868000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1013115"}},"subtitle":[],"editor":[{"given":"Jason A.","family":"Papin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,5,28]]},"references-count":52,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5,28]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1013115","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2025.01.08.632030","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,28]]}}}