{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,26]],"date-time":"2026-09-26T22:34:52Z","timestamp":1790462092835,"version":"4.1.0"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,6,15]],"date-time":"2021-06-15T00:00:00Z","timestamp":1623715200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,15]],"date-time":"2021-06-15T00:00:00Z","timestamp":1623715200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Automated segmentation of nuclei in microscopic images has been conducted to enhance throughput in pathological diagnostics and biological research. Segmentation accuracy and speed has been significantly enhanced with the advent of convolutional neural networks. A barrier in the broad application of neural networks to nuclei segmentation is the necessity to train the network using a set of application specific images and image labels. Previous works have attempted to create broadly trained networks for universal nuclei segmentation; however, such networks do not work on all imaging modalities, and best results are still commonly found when the network is retrained on user specific data. Stochastic optical reconstruction microscopy (STORM) based super-resolution fluorescence microscopy has opened a new avenue to image nuclear architecture at nanoscale resolutions. Due to the large size and discontinuous features typical of super-resolution images, automatic nuclei segmentation can be difficult. In this study, we apply commonly used networks (Mask R-CNN and UNet architectures) towards the task of segmenting super-resolution images of nuclei. First, we assess whether networks broadly trained on conventional fluorescence microscopy datasets can accurately segment super-resolution images. Then, we compare the resultant segmentations with results obtained using networks trained directly on our super-resolution data. We next attempt to optimize and compare segmentation accuracy using three different neural network architectures.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Results indicate that super-resolution images are not broadly compatible with neural networks trained on conventional bright-field or fluorescence microscopy images.  When the networks were trained on super-resolution data, however, we attained nuclei segmentation accuracies (F1-Score) in excess of 0.8, comparable to past results found when conducting nuclei segmentation on conventional fluorescence microscopy images.  Overall, we achieved the best results utilizing the Mask R-CNN architecture.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>We found that convolutional neural networks are powerful tools capable of accurately and quickly segmenting localization-based super-resolution microscopy images of nuclei. While broadly trained and widely applicable segmentation algorithms are desirable for quick use with minimal input, optimal results are still found when the network is both trained and tested on visually similar images. We provide a set of Colab notebooks to disseminate the software into the broad scientific community (<jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/YangLiuLab\/Super-Resolution-Nuclei-Segmentation\">https:\/\/github.com\/YangLiuLab\/Super-Resolution-Nuclei-Segmentation<\/jats:ext-link>).<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12859-021-04245-x","type":"journal-article","created":{"date-parts":[[2021,6,15]],"date-time":"2021-06-15T07:02:56Z","timestamp":1623740576000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Application of convolutional neural networks towards nuclei segmentation in localization-based super-resolution fluorescence microscopy images"],"prefix":"10.1186","volume":"22","author":[{"given":"Christopher A.","family":"Mela","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yang","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,6,15]]},"reference":[{"key":"4245_CR1","doi-asserted-by":"publisher","first-page":"1247","DOI":"10.1038\/s41592-019-0612-7","volume":"16","author":"JC Caicedo","year":"2019","unstructured":"Caicedo JC, Goodman A, Haghighi M, Heng C, Singh S, Karhohs KW, Becker T, Carpenter AE. Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nat Methods. 2019;16:1247\u201353.","journal-title":"Nat Methods"},{"key":"4245_CR2","unstructured":"Kromp F, Fischer L, Bozsaky E, Ambros I, Doerr W, Taschner-Mandl S, Ambros P, Hanbury A. Deep Learning architectures for generalized immunofluorescence based nuclear image segmentation. 2019."},{"issue":"11","key":"4245_CR3","doi-asserted-by":"publisher","first-page":"e1005177","DOI":"10.1371\/journal.pcbi.1005177","volume":"12","author":"DA Van Valen","year":"2016","unstructured":"Van Valen DA, Kudo T, Lane KM, Macklin DN, Quach NT, DeFelice MM, Maayan I, Tanouchi Y, Ashley EA, Covert MW. Deep learning automates the quantitative analysis of individual cells in live-cell imaging experiments. PLoS Comput Biol. 2016;12(11):e1005177.","journal-title":"PLoS Comput Biol"},{"key":"4245_CR4","doi-asserted-by":"publisher","first-page":"418","DOI":"10.1038\/nature16496","volume":"529","author":"AN Boettiger","year":"2016","unstructured":"Boettiger AN, Bintu B, Moffitt JR, Wang S, Beliveau BJ, Fudenberg G, Imakaev M, Mirny LA. Wu C-t, Zhuang X: Super-resolution imaging reveals distinct chromatin folding for different epigenetic states. Nature. 2016;529:418\u201322.","journal-title":"Nature"},{"issue":"4","key":"4245_CR5","doi-asserted-by":"publisher","first-page":"873","DOI":"10.1016\/j.celrep.2018.06.085","volume":"24","author":"J Xu","year":"2018","unstructured":"Xu J, Ma H, Jin J, Uttam S, Fu R, Huang Y, Liu Y. Super-resolution imaging of higher-order chromatin structures at different epigenomic states in single mammalian cells. Cell Rep. 2018;24(4):873\u201382.","journal-title":"Cell Rep"},{"issue":"6","key":"4245_CR6","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.1016\/j.cell.2015.01.054","volume":"160","author":"MA Ricci","year":"2015","unstructured":"Ricci MA, Manzo C, Garc\u00eda-Parajo MF, Lakadamyali M, Cosma MP. Chromatin fibers are formed by heterogeneous groups of nucleosomes in vivo. Cell. 2015;160(6):1145\u201358.","journal-title":"Cell"},{"issue":"1","key":"4245_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-019-13993-7","volume":"11","author":"J Xu","year":"2020","unstructured":"Xu J, Ma H, Ma H, Jiang W, Mela CA, Duan M, Zhao S, Gao C, Hahm E-R, Lardo SM, et al. Super-resolution imaging reveals the evolution of higher-order chromatin folding in early carcinogenesis. Nat Commun. 2020;11(1):1\u201317.","journal-title":"Nat Commun"},{"issue":"4","key":"4245_CR8","doi-asserted-by":"publisher","first-page":"566","DOI":"10.1016\/j.molcel.2017.07.013","volume":"67","author":"KR Kieffer-Kwon","year":"2017","unstructured":"Kieffer-Kwon KR, Nimura K, Rao SSP, Xu J, Jung S, Pekowska A, Dose M, Stevens E, Mathe E, Dong P, et al. Myc regulates chromatin decompaction and nuclear architecture during B cell activation. Mol Cell. 2017;67(4):566\u201378.","journal-title":"Mol Cell"},{"issue":"20 Pt A","key":"4245_CR9","doi-asserted-by":"publisher","first-page":"3023","DOI":"10.1016\/j.febslet.2015.04.012","volume":"589","author":"M Lakadamyali","year":"2015","unstructured":"Lakadamyali M, Cosma MP. Advanced microscopy methods for visualizing chromatin structure. FEBS Lett. 2015;589(20 Pt A):3023\u201330.","journal-title":"FEBS Lett"},{"key":"4245_CR10","first-page":"686","volume-title":"Automated object tracing for biomedical image segmentation using a deep convolutional neural network","author":"EM Rutter","year":"2018","unstructured":"Rutter EM, Lagergren JH, Flores KB. Automated object tracing for biomedical image segmentation using a deep convolutional neural network. Cham: Springer; 2018. p. 686\u201394."},{"key":"4245_CR11","doi-asserted-by":"crossref","unstructured":"Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention\u2014MICCAI 2015: 2015; Munich, DE. Cham: Springer. p. 234\u201341.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"4245_CR12","doi-asserted-by":"crossref","unstructured":"He K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. In: IEEE international conference on computer vision (ICCV), Venice, IT. IEEE; 2017. p. 2980\u20138.","DOI":"10.1109\/ICCV.2017.322"},{"key":"4245_CR13","first-page":"3","volume-title":"UNet++: a nested U-Net architecture for medical image segmentation","author":"Z Zhou","year":"2018","unstructured":"Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J. UNet++: a nested U-Net architecture for medical image segmentation. Cham: Springer; 2018. p. 3\u201311."},{"key":"4245_CR14","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1016\/j.media.2019.05.004","volume":"55","author":"J Yi","year":"2019","unstructured":"Yi J, Wu P, Jiang M, Huang Q, Hoeppner DJ, Metaxas DN. Attentive neural cell instance segmentation. Med Image Anal. 2019;55:228\u201340.","journal-title":"Med Image Anal"},{"key":"4245_CR15","doi-asserted-by":"crossref","unstructured":"Hollandi R, Szkalisity A, Toth T, Tasnadi E, Molnar C, Mathe B, Grexa I, Molnar J, Balind A, Gorbe M et al. A deep learning framework for nucleus segmentation using image style transfer. bioRxiv. 2019.","DOI":"10.1101\/580605"},{"key":"4245_CR16","first-page":"640","volume-title":"IRNet: .ntation","author":"Y Zhou","year":"2019","unstructured":"Zhou Y, Chen H, Xu J, Dou Q, Heng P-A. IRNet: .ntation. Cham: Springer; 2019. p. 640\u20138."},{"key":"4245_CR17","doi-asserted-by":"crossref","unstructured":"Vuola AO, Akram SU, Kannala J. Mask-RCNN and U-Net ensembled for nuclei segmentation. In: International symposium on biomedical imaging, Venice, ITA. IEEE; 2019. p. 208\u201312.","DOI":"10.1109\/ISBI.2019.8759574"},{"key":"4245_CR18","doi-asserted-by":"crossref","unstructured":"Fan R, Cheng M-M, Hou Q, Mu T-J, Wang J, Hu S-M. S4Net: single stage salient-instance segmentation. In: Computer vision and pattern recognition (CVPR), Long Beach, CA, USA. IEEE; 2019. p. 6096\u2013105.","DOI":"10.1109\/CVPR.2019.00626"},{"key":"4245_CR19","first-page":"53","volume-title":"Segmentation of Cell nuclei in fluorescence microscopy images using deep learning","author":"H Narotamo","year":"2019","unstructured":"Narotamo H, Sanches JM, Silveira M. Segmentation of Cell nuclei in fluorescence microscopy images using deep learning. Cham: Springer; 2019. p. 53\u201364."},{"key":"4245_CR20","doi-asserted-by":"crossref","unstructured":"Schmidt U, Weigert M, Broaddus C, Myers G. Cell detection with star-convex polygons. In: medical image computing and computer assisted intervention\u2014MICCAI 2018, Granada, ES. Cham: Springer; 2018. p. 265\u201373.","DOI":"10.1007\/978-3-030-00934-2_30"},{"key":"4245_CR21","doi-asserted-by":"publisher","first-page":"952","DOI":"10.1002\/cyto.a.23863","volume":"95A","author":"JC Caicedo","year":"2019","unstructured":"Caicedo JC, Roth J, Goodman A, Becker T, Karhohs KW, Broisin M, Molnar C, McQuin C, Singh S, Theis FJ, et al. Evaluation of deep learning strategies for nucleus segmentation in fluorescence images. Cytometry A. 2019;95A:952\u201365.","journal-title":"Cytometry A"},{"key":"4245_CR22","doi-asserted-by":"crossref","unstructured":"Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: conference on computer vision and pattern recognition (CVPR), Boston, MA, USA. IEEE; 2015. p. 3431\u201340.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"4245_CR23","doi-asserted-by":"crossref","unstructured":"Johnson JW. Adapting Mask-Rcnn for automatic nucleus segmentation. In: Computer vision conference: 25\u201326 April 2019, Las Vegas, NV, USA. Springer; 2019. p. 399\u2013407.","DOI":"10.1007\/978-3-030-17798-0_32"},{"key":"4245_CR24","unstructured":"Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. 2015. arXiv e-prints, arXiv:1506.01497."},{"key":"4245_CR25","first-page":"21","volume-title":"SSD: Single Shot MultiBox Detector","author":"W Liu","year":"2016","unstructured":"Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC. SSD: Single Shot MultiBox Detector. Cham: Springer; 2016. p. 21\u201337."},{"key":"4245_CR26","doi-asserted-by":"crossref","unstructured":"Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: unified, real-time object detection. In: IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA. IEEE; 2016.","DOI":"10.1109\/CVPR.2016.91"},{"key":"4245_CR27","doi-asserted-by":"crossref","unstructured":"Javad Shafiee M, Chywl B, Li F, Wong A. Fast YOLO: a fast you only look once system for real-time embedded object detection in video. 2017. arXiv e-prints, arXiv:1709.05943.","DOI":"10.15353\/vsnl.v3i1.171"},{"key":"4245_CR28","first-page":"622","volume-title":"Nuclear instance segmentation using a proposal-free spatially aware deep learning framework","author":"N Alemi Koohbanani","year":"2019","unstructured":"Alemi Koohbanani N, Jahanifar M, Gooya A, Rajpoot N. Nuclear instance segmentation using a proposal-free spatially aware deep learning framework. Cham: Springer; 2019. p. 622\u201330."},{"key":"4245_CR29","first-page":"237","volume-title":"Panoptic segmentation with an end-to-end cell R-CNN for pathology image analysis","author":"D Zhang","year":"2018","unstructured":"Zhang D, Song Y, Liu D, Jia H, Liu S, Xia Y, Huang H, Cai W. Panoptic segmentation with an end-to-end cell R-CNN for pathology image analysis. Cham: Springer; 2018. p. 237\u201344."},{"key":"4245_CR30","doi-asserted-by":"crossref","unstructured":"Guerrero-Pena FA, Fernandez PDM, Ren TI, Yui M, Rothenberg E, Cunha A. Multiclass weighted loss for instance segmentation of cluttered cells. In: International conference on image processing, Athens, GRC. IEEE; 2018. p. 2451\u201355.","DOI":"10.1109\/ICIP.2018.8451187"},{"issue":"7","key":"4245_CR31","first-page":"1550","volume":"36","author":"N Kumar","year":"2017","unstructured":"Kumar N, Verma R, Sharma S, Bhargava S, Vahadane A, Sethi A. A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans Biomed Eng. 2017;36(7):1550\u201360.","journal-title":"IEEE Trans Biomed Eng"},{"issue":"Suppl 2","key":"4245_CR32","doi-asserted-by":"publisher","first-page":"1376","DOI":"10.1017\/S143192761900761X","volume":"25","author":"PR Gudla","year":"2019","unstructured":"Gudla PR, Zaki G, Shachar S, Misteli T, Pegoraro G. Deep learning based segmentation of nuclei from fluorescence microscopy images. Microsc Microanal. 2019;25(Suppl 2):1376\u20137.","journal-title":"Microsc Microanal"},{"key":"4245_CR33","first-page":"428","volume-title":"Robust Segmentation of Nucleus in Histopathology Images via Mask R-CNN","author":"X Xie","year":"2019","unstructured":"Xie X, Li Y, Zhang M, Shen L. Robust Segmentation of Nucleus in Histopathology Images via Mask R-CNN. Cham: Springer; 2019. p. 428\u201336."},{"issue":"2","key":"4245_CR34","doi-asserted-by":"publisher","first-page":"448","DOI":"10.1109\/TMI.2018.2865709","volume":"38","author":"P Naylor","year":"2018","unstructured":"Naylor P, La\u00e9 M, Reyal F, Walter T. Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Trans Med Imaging. 2018;38(2):448\u201359.","journal-title":"IEEE Trans Med Imaging"},{"key":"4245_CR35","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1016\/j.compbiomed.2019.04.006","volume":"108","author":"W Wanga","year":"2019","unstructured":"Wanga W, Tafta DA, Chena Y-J, Zhanga J, Wallaceb CT, Xuc M, Watkinsb SC, Xing J. Learn to segment single cells with deep distance estimator and deep cell detector. Comput Biol Med. 2019;108:133\u201341.","journal-title":"Comput Biol Med"},{"key":"4245_CR36","doi-asserted-by":"publisher","first-page":"101563","DOI":"10.1016\/j.media.2019.101563","volume":"58","author":"S Graham","year":"2019","unstructured":"Graham S, Vu QD, Raza SEA, Azam A, Tsang YW, Kwak JT, Rajpoot N. Hover-Net: simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med Image Anal. 2019;58:101563.","journal-title":"Med Image Anal"},{"key":"4245_CR37","doi-asserted-by":"crossref","unstructured":"Lin T-Y, Doll\u00e1r P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: IEEE conference on computer vision and pattern recognition; Honolulu, HI, USA. IEEE; 2017.","DOI":"10.1109\/CVPR.2017.106"},{"key":"4245_CR38","first-page":"677","volume-title":"Pyramid-based fully convolutional networks for cell segmentation","author":"T Zhao","year":"2018","unstructured":"Zhao T, Yin Z. Pyramid-based fully convolutional networks for cell segmentation. Cham: Springer; 2018. p. 677\u201385."},{"issue":"7","key":"4245_CR39","doi-asserted-by":"publisher","first-page":"676","DOI":"10.1038\/nmeth.2019","volume":"9","author":"J Schindelin","year":"2012","unstructured":"Schindelin J, Arganda-Carreras I, Frise E, Kaynig V, Longair M, Pietzsch T, Preibisch S, Rueden C, Saalfeld S, Schmid B, et al. Fiji: an open-source platform for biological-image analysis. Nat Methods. 2012;9(7):676\u201382.","journal-title":"Nat Methods"},{"key":"4245_CR40","doi-asserted-by":"publisher","DOI":"10.1101\/2020.03.20.000133","author":"L Von Chamier","year":"2020","unstructured":"Von Chamier L, Laine RF, Jukkala J, Spahn C, Krentzel D, Nehme E, Lerche M, Hern\u00e1ndez-P\u00e9rez S, Mattila PK, Karinou E, et al. ZeroCostDL4Mic: an open platform to use deep-learning in microscopy. bioRxiv. 2020. https:\/\/doi.org\/10.1101\/2020.03.20.000133.","journal-title":"bioRxiv"},{"key":"4245_CR41","unstructured":"Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow. https:\/\/github.com\/matterport\/Mask_RCNN."}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-021-04245-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12859-021-04245-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-021-04245-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,15]],"date-time":"2021-06-15T07:03:31Z","timestamp":1623740611000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-021-04245-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,15]]},"references-count":41,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["4245"],"URL":"https:\/\/doi.org\/10.1186\/s12859-021-04245-x","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,15]]},"assertion":[{"value":"10 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 May 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 June 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not Applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not Applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"I declare that the authors have no competing interests as defined by BMC, or other interests that might be perceived to influence the results and\/or discussion reported in this paper.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"325"}}