{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,24]],"date-time":"2026-01-24T18:07:27Z","timestamp":1769278047759,"version":"3.49.0"},"reference-count":42,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2023,11,28]],"date-time":"2023-11-28T00:00:00Z","timestamp":1701129600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation","award":["CNS 2007320"],"award-info":[{"award-number":["CNS 2007320"]}]},{"name":"University Of Florida via Dr. Christophe Bobda","award":["CNS 2007320"],"award-info":[{"award-number":["CNS 2007320"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Image stitching involves combining multiple images of the same scene captured from different viewpoints into a single image with an expanded field of view. While this technique has various applications in computer vision, traditional methods rely on the successive stitching of image pairs taken from multiple cameras. While this approach is effective for organized camera arrays, it can pose challenges for unstructured ones, especially when handling scene overlaps. This paper presents a deep learning-based approach for stitching images from large unstructured camera sets covering complex scenes. Our method processes images concurrently by using the SandFall algorithm to transform data from multiple cameras into a reduced fixed array, thereby minimizing data loss. A customized convolutional neural network then processes these data to produce the final image. By stitching images simultaneously, our method avoids the potential cascading errors seen in sequential pairwise stitching while offering improved time efficiency. In addition, we detail an unsupervised training method for the network utilizing metrics from Generative Adversarial Networks supplemented with supervised learning. Our testing revealed that the proposed approach operates in roughly \u223c1\/7th the time of many traditional methods on both CPU and GPU platforms, achieving results consistent with established methods.<\/jats:p>","DOI":"10.3390\/s23239481","type":"journal-article","created":{"date-parts":[[2023,11,28]],"date-time":"2023-11-28T11:43:16Z","timestamp":1701171796000},"page":"9481","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Semi-Supervised Image Stitching from Unstructured Camera Arrays"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6434-8741","authenticated-orcid":false,"given":"Erman","family":"Nghonda Tchinda","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, The Herbert Wertheim College of Engineering, University of Florida, Gainesville, FL 32611-6200, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2849-7197","authenticated-orcid":false,"given":"Maximillian Kealoha","family":"Panoff","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, The Herbert Wertheim College of Engineering, University of Florida, Gainesville, FL 32611-6200, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5487-6049","authenticated-orcid":false,"given":"Danielle","family":"Tchuinkou Kwadjo","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, The Herbert Wertheim College of Engineering, University of Florida, Gainesville, FL 32611-6200, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9042-9470","authenticated-orcid":false,"given":"Christophe","family":"Bobda","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, The Herbert Wertheim College of Engineering, University of Florida, Gainesville, FL 32611-6200, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Szeliski, R. (2011). Computer Vision: Algorithms and Applications, Springer.","DOI":"10.1007\/978-1-84882-935-0"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1007\/s11263-006-0002-3","article-title":"Automatic Panoramic Image Stitching using Invariant Features","volume":"74","author":"Brown","year":"2007","journal-title":"Int. J. Comput. Vis."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1145\/882262.882264","article-title":"Graphcut Textures: Image and Video Synthesis Using Graph Cuts","volume":"22","author":"Kwatra","year":"2003","journal-title":"ACM Trans. Graph."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1145\/882262.882269","article-title":"Poisson Image Editing","volume":"22","author":"Gangnet","year":"2003","journal-title":"ACM Trans. Graph."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/0600000009","article-title":"Image Alignment and Stitching: A Tutorial","volume":"2","author":"Szeliski","year":"2006","journal-title":"Found. Trends. Comput. Graph. Vis."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Tchinda, E.N., Kwadjo, D.T., and Bobda, C. (2019, January 13\u201318). A Distributed Smart Camera Apparatus to Enable Scene Immersion: Work-in-progress. Proceedings of the International Conference on Hardware\/Software Codesign and System Synthesis Companion, New York, NY, USA. CODES\/ISSS \u201919.","DOI":"10.1145\/3349567.3351716"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1111\/cgf.13277","article-title":"High-resolution 360 Video Foveated Stitching for Real-time VR","volume":"36","author":"Lee","year":"2017","journal-title":"Comput. Graph. Forum"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yang, W., Qian, Y., K\u00e4m\u00e4r\u00e4inen, J., Cricri, F., and Fan, L. (2018, January 20\u201324). Object Detection in Equirectangular Panorama. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8546070"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lee, K.Y., and Sim, J.Y. (2020, January 14\u201319). Warping Residual Based Image Stitching for Large Parallax. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual.","DOI":"10.1109\/CVPR42600.2020.00822"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1111\/cgf.12541","article-title":"Panoramic Video from Unstructured Camera Arrays","volume":"34","author":"Perazzi","year":"2015","journal-title":"Comput. Graph. Forum"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"He, B., and Yu, S. (2016). Parallax-Robust Surveillance Video Stitching. Sensors, 16.","DOI":"10.3390\/s16010007"},{"key":"ref_12","unstructured":"Lai, W.S., Gallo, O., Gu, J., Sun, D., Yang, M.H., and Kautz, J. (2019). Video stitching for linear camera arrays. arXiv."},{"key":"ref_13","unstructured":"Jiang, W., and Gu, J. (2015, January 7\u201312). Video stitching with spatial-temporal content-preserving warping. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Boston, MA, USA."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2707","DOI":"10.1109\/TCYB.2014.2381774","article-title":"Efficient Video Stitching Based on Fast Structure Deformation","volume":"45","author":"Li","year":"2015","journal-title":"IEEE Trans. Cybern."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"16761","DOI":"10.1109\/ACCESS.2021.3052474","article-title":"l, r-Stitch Unit: Encoder-Decoder-CNN Based Image-Mosaicing Mechanism for Stitching Non-Homogeneous Image Sequences","volume":"9","author":"Chilukuri","year":"2021","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"763","DOI":"10.1109\/LSP.2021.3070525","article-title":"End-to-End Image Stitching Network via Multi-Homography Estimation","volume":"28","author":"Song","year":"2021","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Nie, L., Lin, C., Liao, K., Liu, S., and Zhao, Y. (2022, January 19\u201324). Deep Rectangling for Image Stitching: A Learning Baseline. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00565"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Song, D.Y., Lee, G., Lee, H., Um, G.M., and Cho, D. (2022). Weakly-Supervised Stitching Network for Real-World Panoramic Image Generation, Springer.","DOI":"10.1007\/978-3-031-19787-1_4"},{"key":"ref_19","unstructured":"Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and Weinberger, K. (2014, January 8\u201313). Generative Adversarial Nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, USA."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Shen, C., Ji, X., and Miao, C. (2019, January 4\u20139). Real-Time Image Stitching with Convolutional Neural Networks. Proceedings of the 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR), Irkutsk, Russia.","DOI":"10.1109\/RCAR47638.2019.9044010"},{"key":"ref_21","unstructured":"Ye, J.C., and Sung, W.K. (2019, January 9\u201315). Understanding geometry of encoder-decoder CNNs. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1109\/TPAMI.2007.70729","article-title":"Image Stitching Using Structure Deformation","volume":"30","author":"Jia","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","unstructured":"Wang, J., Yang, Y., Mao, J., Huang, Z., Huang, C., and Xu, W. (July, January 26). Cnn-rnn: A unified framework for multi-label image classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NA, USA."},{"key":"ref_24","first-page":"91","article-title":"Faster r-cnn: Towards real-time object detection with region proposal networks","volume":"28","author":"Ren","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_25","unstructured":"Chen, L.C., Papandreou, G., Schroff, F., and Adam, H. (2017). Rethinking atrous convolution for semantic image segmentation. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/TPAMI.2019.2929257","article-title":"OpenPose: Realtime multi-person 2D pose estimation using Part Affinity Fields","volume":"43","author":"Cao","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1016\/j.patrec.2011.10.010","article-title":"Fourier implementation of Poisson image editing","volume":"33","author":"Morel","year":"2012","journal-title":"Pattern Recognit. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"300","DOI":"10.5201\/ipol.2016.163","article-title":"Poisson Image Editing","volume":"6","author":"Facciolo","year":"2016","journal-title":"Image Process. Line"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"208","DOI":"10.5201\/ipol.2013.41","article-title":"Selective Contrast Adjustment by Poisson Equation","volume":"3","author":"Petro","year":"2013","journal-title":"Image Process. Line"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"67:1","DOI":"10.1145\/1531326.1531373","article-title":"Coordinates for Instant Image Cloning","volume":"28","author":"Farbman","year":"2009","journal-title":"ACM Trans. Graph."},{"key":"ref_31","first-page":"1","article-title":"Sparse autoencoder","volume":"72","author":"Ng","year":"2011","journal-title":"CS294A Lect. Notes"},{"key":"ref_32","first-page":"1","article-title":"Introduction to semi-supervised learning","volume":"3","author":"Zhu","year":"2009","journal-title":"Synth. Lect. Artif. Intell. Mach. Learn."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18\u201322). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"ref_34","unstructured":"Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., and Chen, X. (2016). Improved Techniques for Training GANs. arXiv."},{"key":"ref_35","unstructured":"Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Klambauer, G., and Hochreiter, S. (2017, January 4\u20139). GANs Trained by a Two Time-Scale Update Rule Converge to a Nash Equilibrium. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_36","unstructured":"Barratt, S., and Sharma, R. (2018). A Note on the Inception Score. arXiv."},{"key":"ref_37","unstructured":"Liu, S., Wei, Y., Lu, J., and Zhou, J. (2018). An Improved Evaluation Framework for Generative Adversarial Networks. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"6184","DOI":"10.1109\/TIP.2021.3092828","article-title":"Unsupervised Deep Image Stitching: Reconstructing Stitched Features to Images","volume":"30","author":"Nie","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Herrmann, C., Wang, C., Bowen, R.S., Keyder, E., Krainin, M., Liu, C., and Zabih, R. (2018, January 8\u201314). Robust image stitching with multiple registrations. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01216-8_4"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Mahendran, A., and Vedaldi, A. (2015, January 7\u201312). Understanding Deep Image Representations by Inverting Them. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299155"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Johnson, J., Alahi, A., and Fei-Fei, L. (2016, January 11\u201314). Perceptual Losses for Real-Time Style Transfer and Super-Resolution. Proceedings of the Computer Vision\u2014ECCV 2016, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46475-6_43"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/23\/9481\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:32:50Z","timestamp":1760131970000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/23\/9481"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,28]]},"references-count":42,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["s23239481"],"URL":"https:\/\/doi.org\/10.3390\/s23239481","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,28]]}}}