{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T17:25:41Z","timestamp":1785432341836,"version":"3.56.0"},"reference-count":27,"publisher":"Optica Publishing Group","issue":"4","license":[{"start":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T00:00:00Z","timestamp":1680220800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/doi.org\/10.1364\/OA_License_v2#VOR-OA"},{"start":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T00:00:00Z","timestamp":1680220800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/opg.optica.org\/policies\/opg-tdm-policy.json"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["Grant No. 82160345"],"award-info":[{"award-number":["Grant No. 82160345"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"the Major Science and technology plan of Hainan","award":["ZDKJ2021016"],"award-info":[{"award-number":["ZDKJ2021016"]}]},{"name":"Key Research and Development Program of Hainan province","award":["No. ZDYF2021GXJS017"],"award-info":[{"award-number":["No. ZDYF2021GXJS017"]}]},{"name":"Key science and technology plan project of Haikou","award":["2021-016"],"award-info":[{"award-number":["2021-016"]}]},{"name":"Start-up Fund from Hainan University","award":["KYQD(ZR)20022"],"award-info":[{"award-number":["KYQD(ZR)20022"]}]},{"name":"Start-up Fund from Hainan University","award":["KYQD(ZR)-20077"],"award-info":[{"award-number":["KYQD(ZR)-20077"]}]}],"content-domain":{"domain":["opg.optica.org"],"crossmark-restriction":false},"short-container-title":["Biomed. Opt. Express"],"published-print":{"date-parts":[[2023,4,1]]},"abstract":"<jats:p>High-density localization based on deep learning is a very effective method to accelerate single molecule localization microscopy (SMLM). Compared with traditional high-density localization methods, deep learning-based methods enable a faster data processing speed and a higher localization accuracy. However, the reported high-density localization methods based on deep learning are still not fast enough to enable real time data processing for large batches of raw images, which is probably due to the heavy computational burden and computation complexity in the U-shape architecture used in these models. Here we propose a high-density localization method called FID-STORM, which is based on an improved residual deconvolutional network for the real-time processing of raw images. In FID-STORM, we use a residual network to extract the features directly from low-resolution raw images rather than the U-shape network from interpolated images. We also use a model fusion from TensorRT to further accelerate the inference of the model. In addition, we process the sum of the localization images directly on GPU to obtain an additional speed gain. Using simulated and experimental data, we verified that the FID-STORM method achieves a processing speed of 7.31\u2005ms\/frame at 256\u2009\u00d7\u2009256 pixels @ Nvidia RTX 2080 Ti graphic card, which is shorter than the typical exposure time of 10\u223c30\u2005ms, thus enabling real-time data processing in high-density SMLM. Moreover, compared with a popular interpolated image-based method called Deep-STORM, FID-STORM enables a speed gain of \u223c26 times, without loss of reconstruction accuracy. We also provided an ImageJ plugin for our new method.<\/jats:p>","DOI":"10.1364\/boe.484540","type":"journal-article","created":{"date-parts":[[2023,3,23]],"date-time":"2023-03-23T03:00:09Z","timestamp":1679540409000},"page":"1833","update-policy":"https:\/\/doi.org\/10.1364\/crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Deep learning using a residual deconvolutional network enables real-time high-density single-molecule localization microscopy"],"prefix":"10.1364","volume":"14","author":[{"given":"Zhiwei","family":"Zhou","sequence":"first","affiliation":[{"name":"Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junnan","family":"Wu","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/03q648j11","id-type":"ROR","asserted-by":"publisher"}]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9366-6586","authenticated-orcid":true,"given":"Zhengxia","family":"Wang","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/03q648j11","id-type":"ROR","asserted-by":"publisher"}]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2400-966X","authenticated-orcid":true,"given":"Zhen-Li","family":"Huang","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/03q648j11","id-type":"ROR","asserted-by":"publisher"}]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"285","published-online":{"date-parts":[[2023,3,31]]},"reference":[{"key":"boe-14-4-1833-R1","doi-asserted-by":"publisher","first-page":"460","DOI":"10.1038\/nbt.4106","volume":"36","author":"Ouyang","year":"2018","journal-title":"Nat. Biotechnol."},{"key":"boe-14-4-1833-R2","doi-asserted-by":"publisher","first-page":"31766","DOI":"10.1364\/OE.467574","volume":"30","author":"Zhou","year":"2022","journal-title":"Opt. Express"},{"key":"boe-14-4-1833-R3","doi-asserted-by":"publisher","first-page":"499","DOI":"10.1038\/nmeth.1605","volume":"8","author":"Jones","year":"2011","journal-title":"Nat. Methods"},{"key":"boe-14-4-1833-R4","doi-asserted-by":"publisher","first-page":"909","DOI":"10.1038\/s41592-020-0918-5","volume":"17","author":"Diekmann","year":"2020","journal-title":"Nat. Methods"},{"key":"boe-14-4-1833-R5","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1364\/OPTICA.5.000458","volume":"5","author":"Nehme","year":"2018","journal-title":"Optica"},{"key":"boe-14-4-1833-R6","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1109\/TPAMI.2015.2439281","volume":"38","author":"Dong","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"boe-14-4-1833-R7","doi-asserted-by":"publisher","first-page":"15432","DOI":"10.1364\/OE.392358","volume":"28","author":"Yao","year":"2020","journal-title":"Opt. Express"},{"key":"boe-14-4-1833-R8","doi-asserted-by":"publisher","first-page":"734","DOI":"10.1038\/s41592-020-0853-5","volume":"17","author":"Nehme","year":"2020","journal-title":"Nat. Methods"},{"key":"boe-14-4-1833-R9","doi-asserted-by":"publisher","first-page":"1082","DOI":"10.1038\/s41592-021-01236-x","volume":"18","author":"Speiser","year":"2021","journal-title":"Nat. Methods"},{"key":"boe-14-4-1833-R10","doi-asserted-by":"publisher","first-page":"913","DOI":"10.1038\/s41592-018-0153-5","volume":"15","author":"Zhang","year":"2018","journal-title":"Nat. Methods"},{"key":"boe-14-4-1833-R11","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1073\/pnas.1916219117","volume":"117","author":"M\u00f6ckl","year":"2020","journal-title":"Proc. Natl. Acad. Sci."},{"key":"boe-14-4-1833-R12","doi-asserted-by":"publisher","first-page":"6158","DOI":"10.1364\/OE.27.006158","volume":"27","author":"Hershko","year":"2019","journal-title":"Opt. Express"},{"key":"boe-14-4-1833-R13","first-page":"234","article-title":"U-Net: Convolutional Networks for Biomedical Image Segmentation","author":"Ronneberger","year":"2015"},{"key":"boe-14-4-1833-R14","first-page":"391","article-title":"Accelerating the Super-Resolution Convolutional Neural Network","author":"Dong","year":"2016"},{"key":"boe-14-4-1833-R16","doi-asserted-by":"publisher","first-page":"3064","DOI":"10.1109\/TMI.2020.2986781","volume":"39","author":"Liu","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"boe-14-4-1833-R17","doi-asserted-by":"publisher","first-page":"39298","DOI":"10.1109\/ACCESS.2018.2853194","volume":"6","author":"Shu","year":"2018","journal-title":"IEEE Access"},{"key":"boe-14-4-1833-R19","doi-asserted-by":"publisher","first-page":"21029","DOI":"10.1364\/OE.27.021029","volume":"27","author":"Li","year":"2019","journal-title":"Opt. Express"},{"key":"boe-14-4-1833-R21","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1016\/j.cogsys.2019.10.004","volume":"59","author":"McNeely-White","year":"2020","journal-title":"Cognitive Systems Research"},{"key":"boe-14-4-1833-R22","doi-asserted-by":"publisher","first-page":"1277","DOI":"10.1364\/OPTICA.4.001277","volume":"4","author":"Wang","year":"2017","journal-title":"Optica"},{"key":"boe-14-4-1833-R24","doi-asserted-by":"publisher","first-page":"2389","DOI":"10.1093\/bioinformatics\/btu202","volume":"30","author":"Ovesn\u00fd","year":"2014","journal-title":"Bioinformatics"},{"key":"boe-14-4-1833-R25","doi-asserted-by":"publisher","first-page":"24329","DOI":"10.1364\/OE.26.024329","volume":"26","author":"Li","year":"2018","journal-title":"Opt. Express"},{"key":"boe-14-4-1833-R26","doi-asserted-by":"publisher","first-page":"7945","DOI":"10.1364\/OE.25.007945","volume":"25","author":"Petrov","year":"2017","journal-title":"Opt. Express"},{"key":"boe-14-4-1833-R27","first-page":"105","volume":"1","author":"Hameed","year":"2015","journal-title":"Physics Journal"},{"key":"boe-14-4-1833-R28","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1007\/s12043-014-0828-0","volume":"84","author":"Kumar Reddy","year":"2015","journal-title":"Pramana"},{"key":"boe-14-4-1833-R29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2020\/8273173","volume":"2020","author":"Lee","year":"2020","journal-title":"Math. Probl. Eng."},{"key":"boe-14-4-1833-R30","doi-asserted-by":"publisher","first-page":"4360","DOI":"10.1016\/j.csbj.2022.08.023","volume":"20","author":"Luo","year":"2022","journal-title":"Comput. Struct. Biotechnol. J."},{"key":"boe-14-4-1833-R31","unstructured":"Super-resolution Localization Imaging Group , \u201c Plug in ,\u201d Github , 2023 , https:\/\/github.com\/SRMLabHUST\/FID-STORM ."}],"container-title":["Biomedical Optics Express"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/opg.optica.org\/viewmedia.cfm?URI=boe-14-4-1833&seq=0","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T15:31:17Z","timestamp":1722958277000},"score":1,"resource":{"primary":{"URL":"https:\/\/opg.optica.org\/abstract.cfm?URI=boe-14-4-1833"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,31]]},"references-count":27,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023]]},"published-print":{"date-parts":[[2023]]}},"URL":"https:\/\/doi.org\/10.1364\/boe.484540","relation":{},"ISSN":["2156-7085","2156-7085"],"issn-type":[{"value":"2156-7085","type":"print"},{"value":"2156-7085","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,31]]},"assertion":[{"value":"Optica Publishing Group","name":"publisher","label":"This article is maintained by"},{"value":"https:\/\/doi.org\/10.1364\/BOE.484540","name":"articlelink","label":"Crossref DOI link to publisher maintained version"},{"value":"research-article","name":"content_type","label":"Article type"},{"value":"Screened by Similarity Check","name":"cross_check","label":"Similarity check"},{"value":"Yes","order":0,"name":"peer_reviewed","label":"Peer reviewed","group":{"name":"peer_review","label":"Peer review"}},{"value":"Single blind","order":1,"name":"review_process","label":"Review process","group":{"name":"peer_review","label":"Peer review"}},{"value":"28 December 2022","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"17 March 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"31 March 2023","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}},{"value":"\u00a9 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement","name":"copyright","label":"Copyright"}]}}