{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,22]],"date-time":"2025-11-22T11:08:29Z","timestamp":1763809709720,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2017,7,7]],"date-time":"2017-07-07T00:00:00Z","timestamp":1499385600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 41571436"],"award-info":[{"award-number":["No. 41571436"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Hubei Province Science and Technology Support Program, China","award":["No. 2015BAA027"],"award-info":[{"award-number":["No. 2015BAA027"]}]},{"name":"National Natural Science Foundation of China under Grant","award":["91438203"],"award-info":[{"award-number":["91438203"]}]},{"name":"LIESMARS Special Research Funding"},{"name":"South Wisdom Valley Innovative Research Team Program"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>When mosaicking orthoimages, especially in urban areas with various obvious ground objects like buildings, roads, cars or trees, the detection of optimal seamlines is one of the key technologies for creating seamless and pleasant image mosaics. In this paper, we propose a new approach to detect optimal seamlines for orthoimage mosaicking with the use of deep convolutional neural network (CNN) and graph cuts. Deep CNNs have been widely used in many fields of computer vision and photogrammetry in recent years, and graph cuts is one of the most widely used energy optimization frameworks. We first propose a deep CNN for land cover semantic segmentation in overlap regions between two adjacent images. Then, the energy cost of each pixel in the overlap regions is defined based on the classification probabilities of belonging to each of the specified classes. To find the optimal seamlines globally, we fuse the CNN-classified energy costs of all pixels into the graph cuts energy minimization framework. The main advantage of our proposed method is that the pixel similarity energy costs between two images are defined using the classification results of the CNN based semantic segmentation instead of using the image informations of color, gradient or texture as traditional methods do. Another advantage of our proposed method is that the semantic informations are fully used to guide the process of optimal seamline detection, which is more reasonable than only using the hand designed features defined to represent the image differences. Finally, the experimental results on several groups of challenging orthoimages show that the proposed method is capable of finding high-quality seamlines among urban and non-urban orthoimages, and outperforms the state-of-the-art algorithms and the commercial software based on the visual comparison, statistical evaluation and quantitative evaluation based on the structural similarity (SSIM) index.<\/jats:p>","DOI":"10.3390\/rs9070701","type":"journal-article","created":{"date-parts":[[2017,7,7]],"date-time":"2017-07-07T10:37:17Z","timestamp":1499423837000},"page":"701","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Optimal Seamline Detection for Orthoimage Mosaicking by Combining Deep Convolutional Neural Network and Graph Cuts"],"prefix":"10.3390","volume":"9","author":[{"given":"Li","family":"Li","sequence":"first","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, Hubei, China"}]},{"given":"Jian","family":"Yao","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, Hubei, China"}]},{"given":"Yahui","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, Hubei, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1370-0079","authenticated-orcid":false,"given":"Wei","family":"Yuan","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, Hubei, China"},{"name":"Center for Spatial Information Science, University of Tokyo, Kashiwa 277-8568, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0228-402X","authenticated-orcid":false,"given":"Shuzhu","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, Hubei, China"}]},{"given":"Shenggu","family":"Yuan","sequence":"additional","affiliation":[{"name":"China Transport Telecommunications and Information Center, Beijing 100011, China"}]}],"member":"1968","published-online":{"date-parts":[[2017,7,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.isprsjprs.2009.09.001","article-title":"Seam-line determination for image mosaicking: A technique minimizing the maximum local mismatch and the global cost","volume":"65","author":"Chon","year":"2010","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1000","DOI":"10.1080\/01431161.2010.545083","article-title":"Towards the automatic selection of optimal seam line locations when merging optical remote-sensing images","volume":"33","author":"Yu","year":"2012","journal-title":"Int. J. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.isprsjprs.2012.11.002","article-title":"Automatic determination of seamlines for aerial image mosaicking based on vector roads alone","volume":"76","author":"Wan","year":"2013","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1335","DOI":"10.1109\/LGRS.2013.2293197","article-title":"Seamline determination based on segmentation for urban image mosaicking","volume":"11","author":"Pan","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.isprsjprs.2015.12.007","article-title":"Optimal seamline detection for multiple image mosaicking via graph cuts","volume":"113","author":"Li","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/S0924-2716(01)00033-8","article-title":"Seamline detection in colour orthoimage mosaicking by use of twin snakes","volume":"56","author":"Kerschner","year":"2001","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1109\/TPAMI.2006.99","article-title":"Morphological image compositing","volume":"28","author":"Soille","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, L., Yao, J., Xie, R., Xia, M., and Zhang, W. (2016). A unified framework for street-view panorama stitching. Sensors, 17.","DOI":"10.3390\/s17010001"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1016\/j.imavis.2008.04.014","article-title":"Fast image blending using watersheds and graph cuts","volume":"27","author":"Gracias","year":"2009","journal-title":"Image Vis. Comput."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1145\/1015706.1015718","article-title":"Interactive digital photomontage","volume":"23","author":"Agarwala","year":"2004","journal-title":"ACM Trans. Gr. (TOG)"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"193","DOI":"10.14358\/PERS.75.2.193","article-title":"Radiometric aerial triangulation for the equalization of digital aerial images and orthoimages","volume":"75","author":"Chandelier","year":"2009","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1109\/LGRS.2009.2037442","article-title":"A network-based radiometric equalization approach for digital aerial orthoimages","volume":"7","author":"Pan","year":"2010","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1007\/BF00133570","article-title":"Snakes: Active contour models","volume":"1","author":"Kass","year":"1988","journal-title":"Int. J. Comput. Vis."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1007\/BF01386390","article-title":"A note on two problems in connexion with graphs","volume":"1","author":"Dijkstra","year":"1959","journal-title":"Numer. Math."},{"key":"ref_15","unstructured":"Bellman, R. (1957). Dynamic Programming, Princeton University Press."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1222","DOI":"10.1109\/34.969114","article-title":"Fast approximate energy minimization via graph cuts","volume":"23","author":"Boykov","year":"2001","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wang, L., Ai, H., and Zhang, L. (2010, January 25\u201326). Automated seamline detection in orthophoto mosaicking using improved snakes. Proceedings of the International Conference on Information Engineering and Computer Science (ICIECS), Wuhan, China.","DOI":"10.1109\/ICIECS.2010.5678345"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/j.isprsjprs.2015.04.004","article-title":"Region change rate-driven seamline determination method","volume":"105","author":"Pan","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"121","DOI":"10.14358\/PERS.82.2.121","article-title":"Seamline determination for high resolution orthoimage mosaicking using watershed segmentation","volume":"82","author":"Wang","year":"2016","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"12334","DOI":"10.3390\/rs61212334","article-title":"Automatic seamline network generation for urban orthophoto mosaicking with the use of a digital surface Model","volume":"6","author":"Chen","year":"2014","journal-title":"Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.isprsjprs.2015.11.007","article-title":"SGM-based seamline determination for urban orthophoto mosaicking","volume":"112","author":"Pang","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1145\/367766.368168","article-title":"Algorithm 97: Shortest path","volume":"5","author":"Floyd","year":"1962","journal-title":"Commun. ACM"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1007\/s00138-013-0551-8","article-title":"Dynamic image mosaic via SIFT and dynamic programming","volume":"25","author":"Zeng","year":"2014","journal-title":"Machine Vis. Appl."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1145\/882262.882264","article-title":"Graphcut textures image and video synthesis using graph cuts","volume":"22","author":"Kwatra","year":"2003","journal-title":"ACM Trans. Gr. (TOG)"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1124","DOI":"10.1109\/TPAMI.2004.60","article-title":"An experimental comparison of min-cut\/max-flow algorithms for energy minimization in vision","volume":"26","author":"Boykov","year":"2004","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Cambridge, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_27","unstructured":"Simonyan, K., and Zisserman, A. (arXiv, 2014). Very deep convolutional networks for large-scale image recognition, arXiv."},{"key":"ref_28","unstructured":"Ioffe, S., and Szegedy, C. (arXiv, 2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift, arXiv."},{"key":"ref_29","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the International Conference on Machine Learning (ICML), Haifa, Israel."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3\u20137). Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA.","DOI":"10.1145\/2647868.2654889"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Bottou, L. (2010, January 22\u201327). Large-scale machine learning with stochastic gradient descent. Proceedings of the COMPSTAT, Paris, France.","DOI":"10.1007\/978-3-7908-2604-3_16"},{"key":"ref_32","unstructured":"Badrinarayanan, V., Handa, A., and Cipolla, R. (arXiv, 2015). SegNet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling, arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Eigen, D., and Fergus, R. (2015, January 7\u201313). Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Los Alamitos, CA, USA.","DOI":"10.1109\/ICCV.2015.304"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Xu, W., and Mulligan, J. (2010, January 13\u201318). Performance evaluation of color correction approaches for automatic multi-view image and video stitching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, California, USA.","DOI":"10.1109\/CVPR.2010.5540202"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1348","DOI":"10.1049\/iet-ipr.2011.0641","article-title":"Quantitative quality assessment of stitched panoramic images","volume":"6","author":"Qureshi","year":"2012","journal-title":"IET Image Process."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Dissanayake, V., Herath, S., Rasnayaka, S., Seneviratne, S., Vidanaarachchi, R., and Gamage, C. (2015, January 23\u201325). Quantitative and Qualitative Evaluation of Performance and Robustness of Image Stitching Algorithms. Proceedings of the International Conference on Digital Image Computing: Techniques and Applications (DICTA), Adelaide, Australia.","DOI":"10.1109\/DICTA.2015.7371297"},{"key":"ref_38","unstructured":"(2017, July 07). 2D Semantic Labeling Contest. Available online: http:\/\/www2.isprs.org\/commissions\/comm3\/wg4\/semantic-labeling.html."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1658","DOI":"10.1109\/TGRS.2013.2253110","article-title":"Seamline network refinement based on area Voronoi diagrams with overlap","volume":"52","author":"Pan","year":"2014","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_40","unstructured":"(2017, July 07). Trimble. Available online: http:\/\/www.trimble.com\/."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.isprsjprs.2012.11.003","article-title":"Global seamline networks for orthomosaic generation via local search","volume":"75","author":"Mills","year":"2013","journal-title":"ISPRS J. Photogramm. Remote Sens."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/9\/7\/701\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:41:54Z","timestamp":1760208114000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/9\/7\/701"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,7,7]]},"references-count":41,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2017,7]]}},"alternative-id":["rs9070701"],"URL":"https:\/\/doi.org\/10.3390\/rs9070701","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2017,7,7]]}}}