{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T20:59:52Z","timestamp":1781211592268,"version":"3.54.1"},"reference-count":37,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T00:00:00Z","timestamp":1713830400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The National Natural Science Foundation of China","award":["42327801"],"award-info":[{"award-number":["42327801"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Unmanned aerial vehicle (UAV) image stitching refers to the process of combining multiple UAV images into a single large-format, wide-field image, and the stitched image often contains large irregular boundaries and multiple stitching seams. Usually, irregular boundaries are addressed using grid-constrained methods, while seams are optimized through the design of energy functions and penalty terms applied to the pixels at the seams. The above-mentioned two solutions can only address one of the two issues individually and are often limited to pairwise stitching of images. To the best of our knowledge, there is no unified approach that can handle both seams and irregular boundaries in the context of multi-image stitching for UAV images. Considering that addressing irregular boundaries involves completing missing information for regularization and that mitigating seams involves generating images near the stitching seams, both of these challenges can be viewed as instances of a mask-based image completion problem. This paper proposes a UAV image stitching method based on a diffusion model. This method uniformly designs masks for irregular boundaries and stitching seams, and the unconditional score function of the diffusion model is then utilized to reverse the process. Additional manifold gradient constraints are applied to restore masked images, eliminating both irregular boundaries and stitching seams and resulting in higher perceptual quality. The restoration maintains high consistency in texture and semantics. This method not only simultaneously addresses irregular boundaries and stitching seams but also is unaffected by factors such as the number of stitched images, the shape of irregular boundaries, and the distribution of stitching seams, demonstrating its robustness.<\/jats:p>","DOI":"10.3390\/rs16091483","type":"journal-article","created":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T08:08:27Z","timestamp":1713859707000},"page":"1483","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Elimination of Irregular Boundaries and Seams for UAV Image Stitching with a Diffusion Model"],"prefix":"10.3390","volume":"16","author":[{"given":"Jun","family":"Chen","sequence":"first","affiliation":[{"name":"Computer Network Information Center, Chinese Academy of Sciences, Beijing 100083, China"},{"name":"School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongxi","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Electronics and Communication Engineering, Guangzhou University, Guangzhou 510006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Electronics and Communication Engineering, Guangzhou University, Guangzhou 510006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Honghua","family":"Tang","sequence":"additional","affiliation":[{"name":"School of Electronics and Communication Engineering, Guangzhou University, Guangzhou 510006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yixian","family":"Tang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianhui","family":"Li","sequence":"additional","affiliation":[{"name":"Computer Network Information Center, Chinese Academy of Sciences, Beijing 100083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"G\u00f3mez-Reyes, J.K., Ben\u00edtez-Rangel, J.P., Morales-Hern\u00e1ndez, L.A., Resendiz-Ochoa, E., and Camarillo-Gomez, K.A. (2022). Image mosaicing applied on UAVs survey. Appl. Sci., 12.","DOI":"10.3390\/app12052729"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chen, Y.S., and Chuang, Y.Y. (2016, January 11\u201314). Natural image stitching with the global similarity prior. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46454-1_12"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lee, K.Y., and Sim, J.Y. (2020, January 13\u201319). Warping residual based image stitching for large parallax. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00822"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lin, W.Y., Liu, S., Matsushita, Y., Ng, T.T., and Cheong, L.F. (2011, January 20\u201325). Smoothly varying affine stitching. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995314"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zaragoza, J., Chin, T.J., Brown, M.S., and Suter, D. (2013, January 23\u201328). As-projective-as-possible image stitching with moving DLT. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.303"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"8016005","DOI":"10.1109\/LGRS.2021.3094977","article-title":"UAV image stitching using shape-preserving warp combined with global alignment","volume":"19","author":"Guo","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"5001305","DOI":"10.1109\/LGRS.2023.3247770","article-title":"UAV image stitching with Transformer and small grid reformation","volume":"20","author":"Cui","year":"2023","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5624717","DOI":"10.1109\/TGRS.2023.3328459","article-title":"VSP-Based Warping for Stitching Many UAV Images","volume":"61","author":"He","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","first-page":"1","article-title":"Rectangling panoramic images via warping","volume":"32","author":"He","year":"2013","journal-title":"ACM Trans. Graph."},{"key":"ref_10","unstructured":"Li, D., He, K., Sun, J., and Zhou, K. (2015, January 7\u201312). A geodesic-preserving method for image warping. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3198","DOI":"10.1109\/TVCG.2020.2965097","article-title":"Content-preserving image stitching with piecewise rectangular boundary constraints","volume":"27","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1145\/882262.882264","article-title":"Graphcut textures: Image and video synthesis using graph cuts","volume":"22","author":"Kwatra","year":"2003","journal-title":"ACM Trans. Graph."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"967","DOI":"10.1007\/s11760-018-1241-9","article-title":"Perception-based seam cutting for image stitching","volume":"12","author":"Li","year":"2018","journal-title":"Signal Image Video Process."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chen, X., Yu, M., and Song, Y. (2022). Optimized seam-driven image stitching method based on scene depth information. Electronics, 11.","DOI":"10.3390\/electronics11121876"},{"key":"ref_15","unstructured":"Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. (2004). ACM SIGGRAPH 2004 Papers, Association for Computing Machinery."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"108019","DOI":"10.1016\/j.patcog.2021.108019","article-title":"Edge-guided composition network for image stitching","volume":"118","author":"Dai","year":"2021","journal-title":"Pattern Recognit."},{"key":"ref_17","unstructured":"Eden, A., Uyttendaele, M., and Szeliski, R. (2006, January 7\u201322). Seamless image stitching of scenes with large motions and exposure differences. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201906), New York, NY, USA."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lin, K., Jiang, N., Cheong, L.F., Do, M., and Lu, J. (2016, January 11\u201314). Seagull: Seam-guided local alignment for parallax-tolerant image stitching. Proceedings of the Computer Vision\u2013ECCV 2016: 14th European Conference, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_23"},{"key":"ref_19","unstructured":"Gao, J., Li, Y., Chin, T.J., and Brown, M.S. (2013, January 6\u201310). Seam-driven image stitching. Proceedings of the Eurographics, Girona, Spain."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, F., and Liu, F. (2014, January 23\u201328). Parallax-tolerant image stitching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.423"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, J., and Zhou, Y. (2022, January 8\u201324). Automatic color image stitching using quaternion rank-1 alignment. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01910"},{"key":"ref_22","unstructured":"Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. (2015, January 6\u201311). Deep unsupervised learning using nonequilibrium thermodynamics. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_23","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"Ho","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_24","unstructured":"Nichol, A.Q., and Dhariwal, P. (2021, January 18\u201324). Improved denoising diffusion probabilistic models. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_25","first-page":"8780","article-title":"Diffusion models beat gans on image synthesis","volume":"34","author":"Dhariwal","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Zhang, K., Liang, J., Cao, J., Wen, B., Timofte, R., and Van Gool, L. (2023, January 17\u201324). Denoising Diffusion Models for Plug-and-Play Image Restoration. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPRW59228.2023.00129"},{"key":"ref_27","first-page":"23593","article-title":"Denoising diffusion restoration models","volume":"35","author":"Kawar","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_28","first-page":"25683","article-title":"Improving diffusion models for inverse problems using manifold constraints","volume":"35","author":"Chung","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chen, Y., Liu, S., and Wang, X. (2021, January 20\u201325). Learning continuous image representation with local implicit image function. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00852"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Nie, L., Lin, C., Liao, K., Liu, S., and Zhao, Y. (2022, January 8\u201324). Deep rectangling for image stitching: A learning baseline. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00565"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Nie, L., Lin, C., Liao, K., Liu, S., and Zhao, Y. (2023, January 2\u20136). Parallax-tolerant unsupervised deep image stitching. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Paris, France.","DOI":"10.1109\/ICCV51070.2023.00680"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3812","DOI":"10.1093\/nar\/gkg509","article-title":"SIFT: Predicting amino acid changes that affect protein function","volume":"31","author":"Ng","year":"2003","journal-title":"Nucleic Acids Res."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"824","DOI":"10.1109\/TPAMI.2018.2889473","article-title":"Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs","volume":"42","author":"Malkov","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","unstructured":"Song, Y., Sohl-Dickstein, J., Kingma, D.P., Kumar, A., Ermon, S., and Poole, B. (2020). Score-based generative modeling through stochastic differential equations. arXiv."},{"key":"ref_35","first-page":"13242","article-title":"Stochastic solutions for linear inverse problems using the prior implicit in a denoiser","volume":"34","author":"Kadkhodaie","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Lin, C.C., Pankanti, S.U., Natesan Ramamurthy, K., and Aravkin, A.Y. (2015, January 7\u201312). Adaptive as-natural-as-possible image stitching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298719"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Ying, Z., Niu, H., Gupta, P., Mahajan, D., Ghadiyaram, D., and Bovik, A. (2020, January 13\u201319). From patches to pictures (PaQ-2-PiQ): Mapping the perceptual space of picture quality. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00363"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/9\/1483\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:32:38Z","timestamp":1760106758000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/9\/1483"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,23]]},"references-count":37,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2024,5]]}},"alternative-id":["rs16091483"],"URL":"https:\/\/doi.org\/10.3390\/rs16091483","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,23]]}}}