{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T09:27:09Z","timestamp":1785403629381,"version":"3.56.0"},"reference-count":22,"publisher":"Optica Publishing Group","issue":"9","license":[{"start":{"date-parts":[[2024,8,28]],"date-time":"2024-08-28T00:00:00Z","timestamp":1724803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/doi.org\/10.1364\/OA_License_v2#VOR-OA"},{"start":{"date-parts":[[2024,8,28]],"date-time":"2024-08-28T00:00:00Z","timestamp":1724803200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/opg.optica.org\/policies\/opg-tdm-policy.json"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["82260368"],"award-info":[{"award-number":["82260368"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2022YFC3400601"],"award-info":[{"award-number":["2022YFC3400601"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Innovational Fund for Scientific and Technological Personnel of Hainan Province","award":["KJRC2023C03"],"award-info":[{"award-number":["KJRC2023C03"]}]},{"name":"Matching Fund from Collaborative Innovation Center of One Health, Hainan University","award":["XTCX2022JKC08"],"award-info":[{"award-number":["XTCX2022JKC08"]}]}],"content-domain":{"domain":["opg.optica.org"],"crossmark-restriction":false},"short-container-title":["Biomed. Opt. Express"],"published-print":{"date-parts":[[2024,9,1]]},"abstract":"<jats:p>Because conventional low-light cameras used in single-molecule localization microscopy (SMLM) do not have the ability to distinguish colors, it is often necessary to employ a dedicated optical system and\/or a complicated image analysis procedure to realize multi-color SMLM. Recently, researchers explored the potential of a new kind of low-light camera called colorimetry camera as an alternative detector in multi-color SMLM, and achieved two-color SMLM under a simple optical system, with a comparable cross-talk to the best reported values. However, extracting images from all color channels is a necessary but lengthy process in colorimetry camera-based SMLM (called CC-STORM), because this process requires the sequential traversal of a massive number of pixels. By taking advantage of the parallelism and pipeline characteristics of FPGA, in this paper, we report an updated multi-color SMLM method called HCC-STORM, which integrated the data processing tasks in CC-STORM into a home-built CPU-GPU-FPGA heterogeneous computing platform. We show that, without scarifying the original performance of CC-STORM, the execution speed of HCC-STORM was increased by approximately three times. Actually, in HCC-STORM, the total data processing time for each raw image with 1024\u2009\u00d7\u20091024 pixels was 26.9 ms. This improvement enabled real-time data processing for a field of view of 1024\u2009\u00d7\u20091024 pixels and an exposure time of 30 ms (a typical exposure time in CC-STORM). Furthermore, to reduce the difficulty of deploying algorithms into the heterogeneous computing platform, we also report the necessary interfaces for four commonly used high-level programming languages, including C\/C++, Python, Java, and Matlab. This study not only pushes forward the mature of CC-STORM, but also presents a powerful computing platform for tasks with heavy computation load.<\/jats:p>","DOI":"10.1364\/boe.534941","type":"journal-article","created":{"date-parts":[[2024,8,20]],"date-time":"2024-08-20T03:00:15Z","timestamp":1724122815000},"page":"5560","update-policy":"https:\/\/doi.org\/10.1364\/crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Real-time data processing in colorimetry camera-based single-molecule localization microscopy via CPU-GPU-FPGA heterogeneous computation"],"prefix":"10.1364","volume":"15","author":[{"given":"Jiaxun","family":"Lin","sequence":"first","affiliation":[{"id":[{"id":"https:\/\/ror.org\/03q648j11","id-type":"ROR","asserted-by":"publisher"}]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kun","family":"Wang","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/03q648j11","id-type":"ROR","asserted-by":"publisher"}]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2400-966X","authenticated-orcid":true,"given":"Zhen-Li","family":"Huang","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/03q648j11","id-type":"ROR","asserted-by":"publisher"}]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"285","published-online":{"date-parts":[[2024,8,28]]},"reference":[{"key":"boe-15-9-5560-R1","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1038\/s43586-021-00038-x","volume":"1","author":"Lelek","year":"2021","journal-title":"Nat. Rev. Methods Primers"},{"key":"boe-15-9-5560-R2","doi-asserted-by":"publisher","first-page":"3133","DOI":"10.1038\/s41467-022-30719-4","volume":"13","author":"Li","year":"2022","journal-title":"Nat. Commun."},{"key":"boe-15-9-5560-R3","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1146\/annurev-biophys-102521-112912","volume":"51","author":"Liu","year":"2022","journal-title":"Annu. Rev. Biophys."},{"key":"boe-15-9-5560-R4","doi-asserted-by":"publisher","first-page":"1100","DOI":"10.1038\/s42003-022-04040-1","volume":"5","author":"Andronov","year":"2022","journal-title":"Commun. Biol."},{"key":"boe-15-9-5560-R5","doi-asserted-by":"publisher","first-page":"131a","DOI":"10.1016\/j.bpj.2022.11.873","volume":"122","author":"Mau","year":"2023","journal-title":"Biophys. J."},{"key":"boe-15-9-5560-R6","doi-asserted-by":"publisher","first-page":"8618","DOI":"10.1021\/acs.nanolett.2c03140","volume":"22","author":"Martens","year":"2022","journal-title":"Nano Lett."},{"key":"boe-15-9-5560-R7","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1038\/s41377-022-01054-6","volume":"12","author":"Wu","year":"2023","journal-title":"Light: Sci. Appl."},{"key":"boe-15-9-5560-R8","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1186\/s12859-023-05320-1","volume":"24","author":"\u00d8vreb\u00f8","year":"2023","journal-title":"BMC Bioinform."},{"key":"boe-15-9-5560-R9","doi-asserted-by":"publisher","first-page":"2463","DOI":"10.1021\/nl801471d","volume":"8","author":"Bossi","year":"2008","journal-title":"Nano Lett."},{"key":"boe-15-9-5560-R10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fbinf.2022.813494","volume":"2","author":"Butler","year":"2022","journal-title":"Front. Bioinform."},{"key":"boe-15-9-5560-R11","doi-asserted-by":"publisher","first-page":"34797","DOI":"10.1364\/OE.440706","volume":"29","author":"Wang","year":"2021","journal-title":"Opt. Express"},{"key":"boe-15-9-5560-R12","doi-asserted-by":"publisher","first-page":"2514","DOI":"10.1364\/OL.456705","volume":"47","author":"Wang","year":"2022","journal-title":"Opt. Lett."},{"key":"boe-15-9-5560-R13","doi-asserted-by":"publisher","first-page":"33680","DOI":"10.1364\/OE.466074","volume":"30","author":"Zhang","year":"2022","journal-title":"Opt. Express"},{"key":"boe-15-9-5560-R14","doi-asserted-by":"publisher","first-page":"3401","DOI":"10.1364\/BOE.459198","volume":"13","author":"Gui","year":"2022","journal-title":"Biomed. Opt. Express"},{"key":"boe-15-9-5560-R16","doi-asserted-by":"publisher","first-page":"4310","DOI":"10.1364\/BOE.462197","volume":"13","author":"Li","year":"2022","journal-title":"Biomed. Opt. Express"},{"key":"boe-15-9-5560-R17","first-page":"16","article-title":"Accelerating GNN Training on CPU\u2009+\u2009Multi-FPGA Heterogeneous Platform","volume-title":"in","author":"LinNavaux","year":"2022"},{"key":"boe-15-9-5560-R19","doi-asserted-by":"publisher","first-page":"2335","DOI":"10.1109\/LRA.2021.3057845","volume":"6","author":"Plancher","year":"2021","journal-title":"IEEE Rob. Autom. Lett."},{"key":"boe-15-9-5560-R21","doi-asserted-by":"publisher","first-page":"35247","DOI":"10.1364\/OE.439976","volume":"29","author":"Gui","year":"2021","journal-title":"Opt. Express"},{"key":"boe-15-9-5560-R24","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1038\/nmeth.2089","volume":"9","author":"Schneider","year":"2012","journal-title":"Nat. Methods"},{"key":"boe-15-9-5560-R25","doi-asserted-by":"publisher","first-page":"2350006","DOI":"10.1142\/S1793545823500062","volume":"16","author":"Song","year":"2023","journal-title":"J. Innov. Opt. Heal. Sci."},{"key":"boe-15-9-5560-R26","doi-asserted-by":"publisher","first-page":"34641","DOI":"10.1364\/OE.438160","volume":"29","author":"Shang","year":"2021","journal-title":"Opt. Express"},{"key":"boe-15-9-5560-R27","doi-asserted-by":"publisher","first-page":"28079","DOI":"10.1364\/OE.463996","volume":"30","author":"Li","year":"2022","journal-title":"Opt. Express"}],"container-title":["Biomedical Optics Express"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/opg.optica.org\/viewmedia.cfm?URI=boe-15-9-5560&seq=0","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,28]],"date-time":"2024-08-28T11:12:30Z","timestamp":1724843550000},"score":1,"resource":{"primary":{"URL":"https:\/\/opg.optica.org\/abstract.cfm?URI=boe-15-9-5560"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,28]]},"references-count":22,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2024]]},"published-print":{"date-parts":[[2024]]}},"URL":"https:\/\/doi.org\/10.1364\/boe.534941","relation":{},"ISSN":["2156-7085","2156-7085"],"issn-type":[{"value":"2156-7085","type":"print"},{"value":"2156-7085","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,28]]},"assertion":[{"value":"Optica Publishing Group","name":"publisher","label":"This article is maintained by"},{"value":"https:\/\/doi.org\/10.1364\/BOE.534941","name":"articlelink","label":"Crossref DOI link to publisher maintained version"},{"value":"research-article","name":"content_type","label":"Article type"},{"value":"Screened by Similarity Check","name":"cross_check","label":"Similarity check"},{"value":"Yes","order":0,"name":"peer_reviewed","label":"Peer reviewed","group":{"name":"peer_review","label":"Peer review"}},{"value":"Single blind","order":1,"name":"review_process","label":"Review process","group":{"name":"peer_review","label":"Peer review"}},{"value":"8 July 2024","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"16 August 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"28 August 2024","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}},{"value":"\u00a9 2024 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement","name":"copyright","label":"Copyright"}]}}