{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T03:30:51Z","timestamp":1782444651443,"version":"3.54.5"},"reference-count":32,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2024,3,18]],"date-time":"2024-03-18T00:00:00Z","timestamp":1710720000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Technology (MOST)","award":["109-2221-E010-019-MY2"],"award-info":[{"award-number":["109-2221-E010-019-MY2"]}]},{"name":"Ministry of Science and Technology (MOST)","award":["110-2221-E010-019-MY2"],"award-info":[{"award-number":["110-2221-E010-019-MY2"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Graphics processing units (GPUs) facilitate massive parallelism and high-capacity storage, and thus are suitable for the iterative reconstruction of ultrahigh-resolution micro computed tomography (CT) scans by on-the-fly system matrix (OTFSM) calculation using ordered subsets expectation maximization (OSEM). We propose a finite state automaton (FSA) method that facilitates iterative reconstruction using a heterogeneous multi-GPU platform through parallelizing the matrix calculations derived from a ray tracing system of ordered subsets. The FSAs perform flow control for parallel threading of the heterogeneous GPUs, which minimizes the latency of launching ordered-subsets tasks, reduces the data transfer between the main system memory and local GPU memory, and solves the memory-bound of a single GPU. In the experiments, we compared the operation efficiency of OS-MLTR for three reconstruction environments. The heterogeneous multiple GPUs with job queues for high throughput calculation speed is up to five times faster than the single GPU environment, and that speed up is nine times faster than the heterogeneous multiple GPUs with the FIFO queues of the device scheduling control. Eventually, we proposed an event-triggered FSA method for iterative reconstruction using multiple heterogeneous GPUs that solves the memory-bound issue of a single GPU at ultrahigh resolutions, and the routines of the proposed method were successfully executed on each GPU simultaneously.<\/jats:p>","DOI":"10.3390\/s24061947","type":"journal-article","created":{"date-parts":[[2024,3,19]],"date-time":"2024-03-19T04:36:31Z","timestamp":1710822991000},"page":"1947","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Iterative Reconstruction of Micro Computed Tomography Scans Using Multiple Heterogeneous GPUs"],"prefix":"10.3390","volume":"24","author":[{"given":"Wen-Hsiang","family":"Chou","sequence":"first","affiliation":[{"name":"Department of Biomedical Imaging and Radiological Sciences, National Yang Ming Chiao Tung University, Taipei 112304, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cheng-Han","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Biomedical Imaging and Radiological Sciences, National Yang Ming Chiao Tung University, Taipei 112304, Taiwan"},{"name":"Department of Psychiatry, Taichung Veterans General Hospital, Taichung 407219, Taiwan"},{"name":"The Human Brain Research Center, Taichung Veterans General Hospital, Taichung 407219, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7546-1245","authenticated-orcid":false,"given":"Shih-Chun","family":"Jin","sequence":"additional","affiliation":[{"name":"Department of Biomedical Imaging and Radiological Sciences, National Yang Ming Chiao Tung University, Taipei 112304, Taiwan"},{"name":"Department of Electro-Optical Engineering, National Taipei University of Technology, Taipei 106344, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jyh-Cheng","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Biomedical Imaging and Radiological Sciences, National Yang Ming Chiao Tung University, Taipei 112304, Taiwan"},{"name":"School of Medical Imaging, Xuzhou Medical University, Xuzhou 221004, China"},{"name":"Department of Biomedical Imaging and Radiological Science, China Medical University, Taichung 404333, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1016\/0022-5193(70)90109-8","article-title":"Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and X-ray photography","volume":"29","author":"Gordon","year":"1970","journal-title":"J. Theor. Biol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/42.241889","article-title":"Algebraic reconstruction techniques can be made computationally efficient [positron emission tomography application]","volume":"12","author":"Herman","year":"1993","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"41","DOI":"10.3724\/SP.J.1246.2013.01041","article-title":"Application of a simultaneous iterations reconstruction technique for a 3-D water vapor tomography system","volume":"4","author":"Wei","year":"2013","journal-title":"Geod. Geodyn."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1177\/016173468400600107","article-title":"Simultaneous Algebraic Reconstruction Technique (SART): A superior implementation of the ART algorithm","volume":"6","author":"Andersen","year":"1984","journal-title":"Ultrason. Imaging"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1109\/TMI.1982.4307558","article-title":"Maximum likelihood reconstruction for emission tomography","volume":"1","author":"Shepp","year":"1982","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1101","DOI":"10.1109\/42.746730","article-title":"Accelerated iterative transmission CT reconstruction using an ordered subsets convex algorithm","volume":"17","author":"Kamphuis","year":"1998","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1109\/TMI.1987.4307826","article-title":"A maximum a posteriori probability expectation maximization algorithm for image reconstruction in emission tomography","volume":"6","author":"Levitan","year":"1987","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"612","DOI":"10.1364\/JOSAA.1.000612","article-title":"Practical cone-beam algorithm","volume":"1","author":"Feldkamp","year":"1984","journal-title":"J. Opt. Soc. Am. A"},{"key":"ref_9","unstructured":"Klaus, M., Fang, X., and Neophytos, N. (2007). Computational Imaging V, SPIE."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1016\/j.cmpb.2009.09.003","article-title":"On the efficiency of iterative ordered subset reconstruction algorithms for acceleration on GPUs","volume":"98","author":"Xu","year":"2010","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_11","first-page":"94","article-title":"Iterative reconstruction methods in X-ray CT","volume":"28","author":"Beister","year":"2012","journal-title":"Phys. Medica Eur. J. Med. Phys."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/j.jpdc.2012.04.003","article-title":"Graphics processing unit (GPU) programming strategies and trends in GPU computing","volume":"73","author":"Brodtkorb","year":"2013","journal-title":"J. Parallel Distrib. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Leeser, M., Mukherjee, S., and Brock, J. (2014). Fast reconstruction of 3D volumes from 2D CT projection data with GPUs. BMC Res. Notes, 7.","DOI":"10.1186\/1756-0500-7-582"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"709","DOI":"10.7785\/tcrt.2012.500429","article-title":"A Fully GPU-Based Ray-Driven Backprojector via a Ray-Culling Scheme with Voxel-Level Parallelization for Cone-Beam CT Reconstruction","volume":"14","author":"Park","year":"2015","journal-title":"Technol. Cancer Res. Treat."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Xie, L., Hu, Y., Yan, B., Wang, L., Yang, B., Liu, W., Zhang, L., Luo, L., Shu, H., and Chen, Y. (2015). An Effective CUDA Parallelization of Projection in Iterative Tomography Reconstruction. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0142184"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chen, P., Wahib, M., Takizawa, S., Takano, R., and Matsuoka, S. (2019, January 17\u201319). iFDK: A scalable framework for instant high-resolution image reconstruction. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Denver, CO, USA.","DOI":"10.1145\/3295500.3356163"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"468","DOI":"10.1118\/1.3525838","article-title":"Comparing performance of many-core CPUs and GPUs for static and motion compensated reconstruction of C-arm CT data","volume":"38","author":"Hofmann","year":"2011","journal-title":"Med. Phys."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1002\/mp.12714","article-title":"System matrix computation vs. storage on GPU: A comparative study in cone beam CT","volume":"45","author":"Matenine","year":"2018","journal-title":"Med. Phys."},{"key":"ref_19","first-page":"187","article-title":"A multi-thread scheduling method for 3D CT image reconstruction using multi-GPU","volume":"20","author":"Zhu","year":"2012","journal-title":"J. X-ray Sci. Technol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1016\/j.jss.2014.03.083","article-title":"Surfing the optimization space of a multiple-GPU parallel implementation of a X-ray tomography reconstruction algorithm","volume":"95","author":"Blas","year":"2014","journal-title":"J. Syst. Softw."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Orr, L.J., Jimenez, E.S., and Thompson, K.R. (2014, January 8\u201315). Cluster-based approach to a multi-GPU CT reconstruction algorithm. Proceedings of the 2014 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS\/MIC), Seattle, WA, USA.","DOI":"10.1109\/NSSMIC.2014.7431130"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"5334","DOI":"10.1038\/s41598-022-09430-3","article-title":"Scalable and accurate multi-GPU-based image reconstruction of large-scale ptychography data","volume":"12","author":"Yu","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Wang, Q., Li, M., Jiang, M., and Zhang, P. (2019). Image reconstruction by Mumford\u2013Shah regularization for low-dose CT with multi-GPU acceleration. Phys. Med. Biol., 64.","DOI":"10.1088\/1361-6560\/ab2c85"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1428","DOI":"10.1109\/TNS.1976.4328496","article-title":"A Maximum Likelihood Approach to Emission Image Reconstruction from Projections","volume":"23","author":"Rockmore","year":"1976","journal-title":"IEEE Trans. Nucl. Sci."},{"key":"ref_25","first-page":"306","article-title":"EM reconstruction algorithms for emission and transmission tomography","volume":"8","author":"Lange","year":"1984","journal-title":"J. Comput. Assist. Tomogr."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1109\/42.363099","article-title":"Fast gradient-based methods for Bayesian reconstruction of transmission and emission PET images","volume":"13","author":"Mumcuoglu","year":"1994","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"K\u00e1rn\u00fd, M., and Warwick, K. (1997). Computer Intensive Methods in Control and Signal Processing: The Curse of Dimensionality, Birkh\u00e4user Boston.","DOI":"10.1007\/978-1-4612-1996-5"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1088\/0031-9155\/43\/4\/003","article-title":"Iterative reconstruction for helical CT: A simulation study","volume":"43","author":"Nuyts","year":"1998","journal-title":"Phys. Med. Biol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1109\/23.856534","article-title":"Reduction of metal streak artifacts in X-ray computed tomography using a transmission maximum a posteriori algorithm","volume":"47","author":"Man","year":"2000","journal-title":"IEEE Trans. Nucl. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1109\/42.563662","article-title":"Grouped-coordinate ascent algorithms for penalized-likelihood transmission image reconstruction","volume":"16","author":"Fessler","year":"1997","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1109\/TMI.1982.4307572","article-title":"An Improved Algorithm for Reprojecting Rays through Pixel Images","volume":"1","author":"Joseph","year":"1982","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1118\/1.595715","article-title":"Fast calculation of the exact radiological path for a three-dimensional CT array","volume":"12","author":"Siddon","year":"1985","journal-title":"Med. Phys."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/6\/1947\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:15:36Z","timestamp":1760105736000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/6\/1947"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,18]]},"references-count":32,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["s24061947"],"URL":"https:\/\/doi.org\/10.3390\/s24061947","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,18]]}}}