{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T22:09:24Z","timestamp":1761948564804,"version":"build-2065373602"},"reference-count":65,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2020,3,9]],"date-time":"2020-03-09T00:00:00Z","timestamp":1583712000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["UID\/CEC\/00127\/2019"],"award-info":[{"award-number":["UID\/CEC\/00127\/2019"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Three-dimensional (3D) reconstruction methods generate a 3D textured model from the combination of data from several captures. As such, the geometrical transformations between these captures are required. The process of computing or refining these transformations is referred to as alignment. It is often a difficult problem to handle, in particular due to a lack of accuracy in the matching of features. We propose an optimization framework that takes advantage of fiducial markers placed in the scene. Since these markers are robustly detected, the problem of incorrect matching of features is overcome. The proposed procedure is capable of enhancing the 3D models created using consumer level RGB-D hand-held cameras, reducing visual artefacts caused by misalignments. One problem inherent to this solution is that the scene is polluted by the markers. Therefore, a tool was developed to allow their removal from the texture of the scene. Results show that our optimization framework is able to significantly reduce alignment errors between captures, which results in visually appealing reconstructions. Furthermore, the markers used to enhance the alignment are seamlessly removed from the final model texture.<\/jats:p>","DOI":"10.3390\/s20051497","type":"journal-article","created":{"date-parts":[[2020,3,10]],"date-time":"2020-03-10T11:59:36Z","timestamp":1583841576000},"page":"1497","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Enhancement of RGB-D Image Alignment Using Fiducial Markers"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1335-0803","authenticated-orcid":false,"given":"Tiago","family":"Madeira","sequence":"first","affiliation":[{"name":"Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"given":"Miguel","family":"Oliveira","sequence":"additional","affiliation":[{"name":"Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro, 3810-193 Aveiro, Portugal"},{"name":"Department of Mechanical Engineering, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3754-2749","authenticated-orcid":false,"given":"Paulo","family":"Dias","sequence":"additional","affiliation":[{"name":"Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro, 3810-193 Aveiro, Portugal"},{"name":"Department of Electronics, Telecommunications and Informatics, University of Aveiro, 3810-193 Aveiro, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2020,3,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"15520","DOI":"10.3390\/s150715520","article-title":"UAV-Based Photogrammetry and Integrated Technologies for Architectural Applications\u2014Methodological Strategies for the After-Quake Survey of Vertical Structures in Mantua (Italy)","volume":"15","author":"Achille","year":"2015","journal-title":"Sensors"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"P\u00e9rez, L., Rodr\u00edguez, \u00cd., Rodr\u00edguez, N., Usamentiaga, R., and Garc\u00eda, D.F. (2016). Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review. Sensors, 16.","DOI":"10.3390\/s16030335"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Chen, H., Waslander, S., Yang, T., Zhang, S., Xiong, G., and Liu, K. (2018). Toward a More Complete, Flexible, and Safer Speed Planning for Autonomous Driving via Convex Optimization. Sensors, 18.","DOI":"10.20944\/preprints201805.0164.v2"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Trinidad-Fern\u00e1ndez, M., Beckw\u00e9e, D., Cuesta-Vargas, A., Gonz\u00e1lez-S\u00e1nchez, M., Moreno, F.A., Gonz\u00e1lez-Jim\u00e9nez, J., Joos, E., and Vaes, P. (2020). Validation, Reliability, and Responsiveness Outcomes Of Kinematic Assessment With An RGB-D Camera To Analyze Movement In Subacute And Chronic Low Back Pain. Sensors, 20.","DOI":"10.3390\/s20030689"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"V\u00e1zquez-Arellano, M., Griepentrog, H., Reiser, D., and Paraforos, D. (2016). 3-D Imaging Systems for Agricultural Applications\u2014A Review. Sensors, 16.","DOI":"10.3390\/s16050618"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Di Angelo, L., Di Stefano, P., Guardiani, E., Morabito, A.E., and Pane, C. (2019). 3D Virtual Reconstruction of the Ancient Roman Incile of the Fucino Lake. Sensors, 19.","DOI":"10.3390\/s19163505"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3284","DOI":"10.3390\/rs6043284","article-title":"Segmentation of Sloped Roofs from Airborne LiDAR Point Clouds Using Ridge-Based Hierarchical Decomposition","volume":"6","author":"Fan","year":"2014","journal-title":"Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.isprsjprs.2012.11.004","article-title":"Model driven reconstruction of roofs from sparse LIDAR point clouds","volume":"76","author":"Henn","year":"2013","journal-title":"Int. J. Photogramm. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Newcombe, R.A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A.J., Kohi, P., Shotton, J., Hodges, S., and Fitzgibbon, A. (2011, January 26\u201329). KinectFusion: Real-time dense surface mapping and tracking. Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland.","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1318","DOI":"10.1109\/TCYB.2013.2265378","article-title":"Enhanced Computer Vision With Microsoft Kinect Sensor: A Review","volume":"43","author":"Han","year":"2013","journal-title":"IEEE Trans. Cybern."},{"key":"ref_11","first-page":"591","article-title":"A critical review of automated photogrammetric processing of large datasets","volume":"XLII-2\/W5","author":"Remondino","year":"2017","journal-title":"ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.measurement.2018.01.058","article-title":"The performance evaluation of multi-image 3D reconstruction software with different sensors","volume":"120","author":"Mousavi","year":"2018","journal-title":"Measurement"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1016\/j.geomorph.2012.08.021","article-title":"\u2018Structure-from-Motion\u2019 photogrammetry: A low-cost, effective tool for geoscience applications","volume":"179","author":"Westoby","year":"2012","journal-title":"Geomorphology"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tsai, C.Y., and Huang, C.H. (2017). Indoor Scene Point Cloud Registration Algorithm Based on RGB-D Camera Calibration. Sensors, 17.","DOI":"10.3390\/s17081874"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Liu, H., Li, H., Liu, X., Luo, J., Xie, S., and Sun, Y. (2018). A Novel Method for Extrinsic Calibration of Multiple RGB-D Cameras Using Descriptor-Based Patterns. arXiv.","DOI":"10.3390\/s19020349"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chen, C., Yang, B., Song, S., Tian, M., Li, J., Dai, W., and Fang, L. (2018). Calibrate Multiple Consumer RGB-D Cameras for Low-Cost and Efficient 3D Indoor Mapping. Remote Sens., 10.","DOI":"10.3390\/rs10020328"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"67","DOI":"10.5194\/isprs-annals-III-4-67-2016","article-title":"First experiments with the tango tablet for indoor scanning","volume":"III-4","author":"Zlatanova","year":"2016","journal-title":"ISPRS Anna. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Li, X., and Kesavadas, T. (2018, January 18\u201321). Surgical Robot with Environment Reconstruction and Force Feedback. Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA.","DOI":"10.1109\/EMBC.2018.8512695"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1859","DOI":"10.1109\/ACCESS.2018.2886133","article-title":"Indoor Scene Understanding in 2.5\/3D for Autonomous Agents: A Survey","volume":"7","author":"Naseer","year":"2019","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Li, L., Su, F., Yang, F., Zhu, H., Li, D., Xinkai, Z., Li, F., Liu, Y., and Ying, S. (2018). Reconstruction of Three-Dimensional (3D) Indoor Interiors with Multiple Stories via Comprehensive Segmentation. Remote Sens., 10.","DOI":"10.3390\/rs10081281"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Zheng, X., Chen, R., Hanjiang, X., and Guo, S. (2018). Image-Based Localization Aided Indoor Pedestrian Trajectory Estimation Using Smartphones. Sensors, 18.","DOI":"10.3390\/s18010258"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"8232","DOI":"10.3390\/s150408232","article-title":"A Kinect-Based Real-Time Compressive Tracking Prototype System for Amphibious Spherical Robots","volume":"15","author":"Pan","year":"2015","journal-title":"Sensors"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"75","DOI":"10.5194\/isprs-archives-XLII-2-W1-75-2016","article-title":"A Hybrid 3D Indoor Space Model","volume":"XLII-2\/W1","author":"Jamali","year":"2016","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1111\/cgf.13386","article-title":"State of the Art on 3D Reconstruction with RGB-D Cameras","volume":"37","author":"Theobalt","year":"2018","journal-title":"Comput. Graph. Forum"},{"key":"ref_25","unstructured":"Gokturk, S., Yalcin, H., and Bamji, C. (July, January 27). A Time-Of-Flight Depth Sensor\u2014System Description, Issues and Solutions. Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop, Washington, DC, USA."},{"key":"ref_26","first-page":"521","article-title":"Method of time-coded parallel planes of light for depth measurement","volume":"64","author":"Minou","year":"1981","journal-title":"IEICE Trans."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1016\/0004-3702(71)90015-4","article-title":"Grid coding: A preprocessing technique for robot and machine vision","volume":"2","author":"Will","year":"1971","journal-title":"Artif. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Curless, B., and Levoy, M. (1996, January 4\u20139). A Volumetric Method for Building Complex Models from Range Images. Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, New Orleans, LA, USA.","DOI":"10.1145\/237170.237269"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Rusinkiewicz, S., Hall-holt, O., and Levoy, M. (2002). Real-Time 3D Model Acquisition. ACM Trans. Graph., 21.","DOI":"10.1145\/566654.566600"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1047","DOI":"10.1109\/TRO.2006.878961","article-title":"Metric-based iterative closest point scan matching for sensor displacement estimation","volume":"22","author":"Minguez","year":"2006","journal-title":"IEEE Trans. Rob."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1016\/0031-3203(95)00115-8","article-title":"A New Approach to Image Feature Detection With Applications","volume":"29","author":"Manjunath","year":"1996","journal-title":"Pattern Recognit."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive Image Features from Scale-Invariant Keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vision"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Bay, H., Tuytelaars, T., and Van Gool, L. (2006). SURF: Speeded Up Robust Features. European Conference on Computer Vision, Springer.","DOI":"10.1007\/11744023_32"},{"key":"ref_34","first-page":"20","article-title":"Significant HOG-Histogram of Oriented Gradient Feature Selection for Human Detection","volume":"132","author":"Patwary","year":"2015","journal-title":"Int. J. Comput. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A Computational Approach To Edge Detection","volume":"PAMI-8","author":"Canny","year":"1986","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_36","unstructured":"Sobel, I. An Isotropic 3x3 Image Gradient Operator. Presentation at Stanford A.I. Project 1968, 2014."},{"key":"ref_37","unstructured":"Harris, C., and Stephens, M. (September, January 31). A Combined Corner and Edge Detector. Proceedings of the Fourth Alvey Vision Conference, Manchester, UK."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1023\/A:1007963824710","article-title":"SUSAN\u2014A new approach to low level image processing","volume":"23","author":"Smith","year":"1997","journal-title":"Int. J. Comput. Vis."},{"key":"ref_39","unstructured":"Kenney, C., Zuliani, M., and Manjunath, B. (2005, January 20\u201325). An Axiomatic Approach to Corner Detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1006\/cviu.1996.0510","article-title":"Segmentation and Classification of Edges Using Minimum Description Length Approximation and Complementary Junction Cues","volume":"67","author":"Lindeberg","year":"1997","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Rosten, E., and Drummond, T. (2006). Machine Learning for High-Speed Corner Detection. European Conference on Computer Vision, Springer.","DOI":"10.1007\/11744023_34"},{"key":"ref_42","first-page":"77","article-title":"Feature Detection with Automatic Scale Selection","volume":"30","author":"Lindeberg","year":"1998","journal-title":"Int. J. Comput. Vis."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1016\/j.imavis.2004.02.006","article-title":"Robust Wide Baseline Stereo from Maximally Stable Extremal Regions","volume":"22","author":"Matas","year":"2004","journal-title":"Image Vis. Comput."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Deng, H., Zhang, W., Mortensen, E., Dietterich, T., and Shapiro, L. (2007, January 17\u201322). Principal Curvature-Based Region Detector for Object Recognition. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.","DOI":"10.1109\/CVPR.2007.382972"},{"key":"ref_45","unstructured":"Lindeberg, T. (1991). Discrete Scale-Space Theory and the Scale-Space Primal Sketch. [Ph.D. Thesis, Department of Numerical Analysis and Computing Science, Royal Institute of Technology]."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Jakubovic, A., and Velagic, J. (2018, January 16\u201319). Image Feature Matching and Object Detection Using Brute-Force Matchers. Proceedings of the 2018 International Symposium ELMAR, Zadar, Croatia.","DOI":"10.23919\/ELMAR.2018.8534641"},{"key":"ref_47","first-page":"331","article-title":"Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration","volume":"1","author":"Muja","year":"2009","journal-title":"VISAPP"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Mount, D., Netanyahu, N., and Le Moigne, J. (2003). Efficient Algorithms for Robust Feature Matching. Pattern Recognit., 32.","DOI":"10.1016\/S0031-3203(98)00086-7"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1229","DOI":"10.1109\/34.809117","article-title":"RANSAC-Based DARCES: A new approach to fast automatic registration of partially overlapping range images","volume":"21","author":"Chen","year":"1999","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1002\/rob.20104","article-title":"Mobile robot motion estimation by 2D scan matching with genetic and iterative closest point algorithms","volume":"23","author":"Morales","year":"2006","journal-title":"J. Field Rob."},{"key":"ref_51","unstructured":"Autodesk (2020, February 05). ReCap: Reality Capture and 3D Scanning Software for Intelligent Model Creation. Available online: https:\/\/www.autodesk.com\/products\/recap\/overview."},{"key":"ref_52","unstructured":"Alicevision (2020, February 05). Meshroom: Open Source Photogrammetry Software. Available online: https:\/\/alicevision.org\/#meshroom."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Thrun, S., and Leonard, J.J. (2008). Simultaneous Localization and Mapping. Springer Handb. Rob., 871\u2013889.","DOI":"10.1007\/978-3-540-30301-5_38"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Jorge Nocedal, S.J.W. (2000). Numerical Optimization, Springer.","DOI":"10.1007\/b98874"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Agarwal, S., Snavely, N., M. Seitz, S., and Szeliski, R. (2010). Bundle Adjustment in the Large. Computer Vision\u2014ECCV 2010, Springer.","DOI":"10.1007\/978-3-642-15552-9_3"},{"key":"ref_56","unstructured":"Harltey, A., and Zisserman, A. (2006). Multiple View Geometry in Computer Vision, Cambridge University Press. [2nd ed.]."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Romero Ramirez, F., Mu\u00f1oz-Salinas, R., and Medina-Carnicer, R. (2018). Speeded Up Detection of Squared Fiducial Markers. Image Vision Comput., 76.","DOI":"10.1016\/j.imavis.2018.05.004"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Garrido-Jurado, S., Mu\u00f1oz-Salinas, R., Madrid-Cuevas, F., and Medina-Carnicer, R. (2015). Generation of fiducial marker dictionaries using Mixed Integer Linear Programming. Pattern Recognit., 51.","DOI":"10.1016\/j.patcog.2015.09.023"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Hornegger, J., and Tomasi, C. (1999, January 20\u201327). Representation issues in the ML estimation of camera motion. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece.","DOI":"10.1109\/ICCV.1999.791285"},{"key":"ref_60","unstructured":"Schmidt, J., and Niemann, H. (2001). Using Quaternions for Parametrizing 3-D Rotations in Unconstrained Nonlinear Optimization. Vmv, Aka GmbH."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/MCSE.2007.58","article-title":"Python for Scientific Computing","volume":"9","author":"Oliphant","year":"2007","journal-title":"Comput. Sci. Eng."},{"key":"ref_62","unstructured":"FARO (2020, February 05). Focus Laser Scanner Series. Available online: https:\/\/www.faro.com\/products\/construction-bim\/faro-focus."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Klingensmith, M., Dryanovski, I., Srinivasa, S., and Xiao, J. (2015). Chisel: Real Time Large Scale 3D Reconstruction Onboard a Mobile Device using Spatially Hashed Signed Distance Fields. Robotics: Science and Systems XI, RSS.","DOI":"10.15607\/RSS.2015.XI.040"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/0020-0190(91)90233-8","article-title":"Computing the minimum Hausdorff distance between two point sets on a line under translation","volume":"38","author":"Rote","year":"1991","journal-title":"Inf. Process. Lett."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Oliveira, M., Castro, A., Madeira, T., Dias, P., and Santos, V. (2019). A General Approach to the Extrinsic Calibration of Intelligent Vehicles Using ROS. Iberian Robotics Conference, Springer.","DOI":"10.1007\/978-3-030-35990-4_17"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/5\/1497\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:05:30Z","timestamp":1760173530000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/5\/1497"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,9]]},"references-count":65,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2020,3]]}},"alternative-id":["s20051497"],"URL":"https:\/\/doi.org\/10.3390\/s20051497","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,3,9]]}}}