{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,20]],"date-time":"2026-09-20T14:36:57Z","timestamp":1789915017093,"version":"4.0.1"},"reference-count":40,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2017,7,18]],"date-time":"2017-07-18T00:00:00Z","timestamp":1500336000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Images acquired with a long exposure time using a camera embedded on UAVs (Unmanned Aerial Vehicles) exhibit motion blur due to the erratic movements of the UAV. The aim of the present work is to be able to acquire several images with a short exposure time and use an image processing algorithm to produce a stacked image with an equivalent long exposure time. Our method is based on the feature point image registration technique. The algorithm is implemented on the light-weight IGN (Institut national de l\u2019information g\u00e9ographique) camera, which has an IMU (Inertial Measurement Unit) sensor and an SoC (System on Chip)\/FPGA (Field-Programmable Gate Array). To obtain the correct parameters for the resampling of the images, the proposed method accurately estimates the geometrical transformation between the first and the N-th images. Feature points are detected in the first image using the FAST (Features from Accelerated Segment Test) detector, then homologous points on other images are obtained by template matching using an initial position benefiting greatly from the presence of the IMU sensor. The SoC\/FPGA in the camera is used to speed up some parts of the algorithm in order to achieve real-time performance as our ultimate objective is to exclusively write the resulting image to save bandwidth on the storage device. The paper includes a detailed description of the implemented algorithm, resource usage summary, resulting processing time, resulting images and block diagrams of the described architecture. The resulting stacked image obtained for real surveys does not seem visually impaired. An interesting by-product of this algorithm is the 3D rotation estimated by a photogrammetric method between poses, which can be used to recalibrate in real time the gyrometers of the IMU. Timing results demonstrate that the image resampling part of this algorithm is the most demanding processing task and should also be accelerated in the FPGA in future work.<\/jats:p>","DOI":"10.3390\/s17071646","type":"journal-article","created":{"date-parts":[[2017,7,18]],"date-time":"2017-07-18T10:33:14Z","timestamp":1500373994000},"page":"1646","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Implementation of an IMU Aided Image Stacking Algorithm in a Digital Camera for Unmanned Aerial Vehicles"],"prefix":"10.3390","volume":"17","author":[{"given":"Ahmad","family":"Audi","sequence":"first","affiliation":[{"name":"Universit\u00e9 Paris-Est, IGN, LaSTIG, LOEMI, 73 Avenue de Paris, 94160 Saint-Mand\u00e9, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc","family":"Pierrot-Deseilligny","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris-Est, IGN, LaSTIG, LOEMI, ENSG, 6-8 Avenue Blaise Pascal, 77420 Champs-sur-Marne, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christophe","family":"Meynard","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris-Est, IGN, LaSTIG, LOEMI, 73 Avenue de Paris, 94160 Saint-Mand\u00e9, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0653-9213","authenticated-orcid":false,"given":"Christian","family":"Thom","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris-Est, IGN, LaSTIG, LOEMI, 73 Avenue de Paris, 94160 Saint-Mand\u00e9, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2017,7,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1393","DOI":"10.1007\/s11948-014-9603-3","article-title":"Domesticating the Drone: The Demilitarisation of Unmanned Aircraft for Civil Markets","volume":"21","author":"Boucher","year":"2015","journal-title":"Sci. Eng. Ethics"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.isprsjprs.2014.02.013","article-title":"Unmanned aerial systems for photogrammetry and remote sensing: A review","volume":"92","author":"Colomina","year":"2014","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40327-015-0029-z","article-title":"Visual monitoring of civil infrastructure systems via camera-equipped Unmanned Aerial Vehicles (UAVs): A review of related works","volume":"4","author":"Ham","year":"2016","journal-title":"Vis. Eng."},{"key":"ref_4","first-page":"65","article-title":"Study of lever-arm effect using embedded photogrammetry and on-board gps receiver on UAV for metrological mapping purpose and proposal of a free ground measurements calibration procedure","volume":"XL-3\/W4","author":"Daakir","year":"2016","journal-title":"ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_5","first-page":"327","article-title":"Uav linear photogrammetry","volume":"XL-3\/W3","author":"Tournadre","year":"2015","journal-title":"ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_6","unstructured":"Audi, A., Pierrot Deseilligny, M., Meynard, C., and Thom, C. (2015). Implementation of real-time functionalities in the hardware of an intelligent ultra-light camera specialized for aerial photography. GeoUAV-ISPRS Geospatial Week, ISPRS."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Le Moigne, J., Netanyahu, N.S., and Eastman, R.D. (2010). Image Registration for Remote Sensing, Cambridge University Press.","DOI":"10.1017\/CBO9780511777684"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Lakshmi, K.D., and Vaithiyanathan, V. (2015). Study of Feature based Image Registration Algorithms for Navigation of Unmanned Aerial Vehicles. Indian J. Sci. Technol., 8.","DOI":"10.17485\/ijst\/2015\/v8i22\/79315"},{"key":"ref_9","unstructured":"Patel, P., and Shah, V. (2014). Image Registration Techniques: A Comprehensive Survey. Int. J. Innov. Res. Dev., 3, Available online: http:\/\/www.ijird.com\/index.php\/ijird\/article\/view\/47537."},{"key":"ref_10","unstructured":"Mann, S., and Picard, R.W. (1995, January 7\u201311). On Being \u2019undigital\u2019 with Digital Cameras: Extending Dynamic Range By Combining Differently Exposed Pictures. Proceedings of the IS&T\u2019s 48th Annual Conference, Washington, DC, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Debevec, P.E., and Malik, J. (1997, January 3\u20138). Recovering high dynamic range radiance maps from photographs. Proceedings of the Siggraph: 24th International ACM Conference on Computer Graphics & INteractive Techniques: Visual Proceedings, Los Angeles, CA, USA.","DOI":"10.1145\/258734.258884"},{"key":"ref_12","unstructured":"Robertson, M.A., Borman, S., and Stevenson, R.L. (1999, January 24\u201328). Dynamic Range Improvement Through Multiple Exposures. Proceedings of the 1999 International Conference on Image Processing, Kobe, Japan."},{"key":"ref_13","first-page":"1","article-title":"Image Deblurring Using Inertial Measurement Sensors","volume":"29","author":"Joshi","year":"2010","journal-title":"ACM Trans. Gr."},{"key":"ref_14","first-page":"321","article-title":"Automated blur detection and removal in airborne imaging systems using IMU data","volume":"XXXIX-B1","author":"Shah","year":"2012","journal-title":"ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_15","first-page":"361","article-title":"Automatic isolation of blurred images from uav image sequences","volume":"XL-1\/W2","author":"Sieberth","year":"2013","journal-title":"ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_16","first-page":"33","article-title":"Uav image blur\u2014Its influence and ways to correct it","volume":"XL-1\/W4","author":"Sieberth","year":"2015","journal-title":"ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Klein, G., and Drummond, T. (2002, January 2\u20135). Tightly Integrated Sensor Fusion for Robust Visual Tracking. Proceedings of the British Machine Vision Conference (BMVC\u201902), Cardiff, UK.","DOI":"10.5244\/C.16.77"},{"key":"ref_18","first-page":"267","article-title":"Combining MEMS-based IMU data and vision-based trajectory estimation","volume":"XL-4","author":"Tsai","year":"2014","journal-title":"ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Hol, J.D., Sch\u00f6n, T.B., Gustafsson, F., and Slycke, P.J. (2006). Sensor Fusion for Augmented Reality. 2006 9th International Conference on Information Fusion, IEEE Service Center.","DOI":"10.1109\/ICIF.2006.301604"},{"key":"ref_20","unstructured":"You, S., and Neumann, U. (2001, January 13\u201317). Fusion of vision and gyro tracking for robust augmented reality registration. Proceedings of the IEEE Virtual Reality 2001 Conference, Yokohama, Japan."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Aarts, E., Collier, R.W., van Loenen, E., and de Ruyter, B. (2003, January 3\u20134). Sensor Fusion for Augmented Reality. Proceedings of the Ambient Intelligence: First European Symposium, EUSAI 2003, Veldhoven, The Netherlands.","DOI":"10.1007\/b94080"},{"key":"ref_22","unstructured":"Wang, Y., Hou, Z., Leman, K., and Chang, R. (2011, January 13\u201315). Real-Time Video Stabilization for Unmanned Aerial Vehicles. Proceedings of the MVA2011 IAPR Conference on Machine Vision Applications, Nara, Japan."},{"key":"ref_23","unstructured":"Martin, O., Meynard, C., Pierrot Deseilligny, M., Souchon, J.P., and Thom, C. (2014, January 24\u201326). R\u00e9alisation d\u2019une cam\u00e9ra photogramm\u00e9trique ultral\u00e9g\u00e8re et de haute r\u00e9solution. Proceedings of the colloque drones et moyens l\u00e9gers a\u00e9roport\u00e9s d\u2019observation, Montpellier, France."},{"key":"ref_24","first-page":"269","article-title":"APERO, an open source bundle adjusment software for automatic calibration and orientation of set of images","volume":"XXXVIII-5\/W16","author":"Clery","year":"2011","journal-title":"ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1309","DOI":"10.1109\/TPAMI.2007.1147","article-title":"Parameter-free Radial Distortion Correction with Centre of Distortion Estimation","volume":"29","author":"Hartley","year":"2007","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_26","unstructured":"Pena, M. (2012). A Comparative Study of Three Image Matching Algorithms: Sift, Surf, and Fast, BiblioBazaar."},{"key":"ref_27","unstructured":"Le, X., and Gonzalez, R. (2009, January 16\u201318). Pattern-based corner detection algorithm. Proceedings of the 2009 6th International Symposium on Image and Signal Processing and Analysis, ISPA 2009, Salzburg, Austria."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1016\/j.eij.2013.06.003","article-title":"A comparative study of image low level feature extraction algorithms","volume":"14","author":"Soliman","year":"2013","journal-title":"Egypt. Inform. J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1508","DOI":"10.1109\/ICCV.2005.104","article-title":"Fusing Points and Lines for High Performance Tracking","volume":"Volume 2","author":"Rosten","year":"2005","journal-title":"Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV \u201905)"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Van der Wal, G., Zhang, D., Kandaswamy, I., Marakowitz, J., Kaighn, K., Zhang, J., and Chai, S. (2015, January 7\u201312). FPGA Acceleration for Feature Based Processing Applications. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Boston, MA, USA.","DOI":"10.1109\/CVPRW.2015.7301365"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Schauwecker, K., Klette, R., and Zell, A. (2012, January 7\u201312). A new feature detector and stereo matching method for accurate high-performance sparse stereo matching. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6385492"},{"key":"ref_32","first-page":"12","article-title":"Analysis of Feature Point Distributions for Fast Image Mosaicking Algorithms","volume":"50","author":"Behrens","year":"2010","journal-title":"Acta Polytech. J. Adv. Eng."},{"key":"ref_33","unstructured":"Kraft, M., Schmidt, A., and Kasinski, A.J. (2008, January 22\u201325). High-Speed Image Feature Detection Using FPGA Implementation of Fast Algorithm. Proceedings of the Third International Conference on Computer Vision Theory and Applications (VISAPP 2008), Funchal, Portugal."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"12","DOI":"10.15623\/ijret.2014.0317003","article-title":"Application of normalized cross correlation to image registration","volume":"3","author":"Rao","year":"2014","journal-title":"Int. J. Res. Eng. Tech."},{"key":"ref_35","first-page":"120","article-title":"Fast normalized cross-correlation","volume":"10","author":"Lewis","year":"1995","journal-title":"Vis. Interface"},{"key":"ref_36","unstructured":"Tabib, W., Charnmanee, G., and Koenig, D. (2013). An Hardware Efficient Normalized Cross Correlation FPGA Coprocessor, An Astroteam Production."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1109\/42.585765","article-title":"A rapid and automatic image registration algorithm with subpixel accuracy","volume":"16","author":"Althof","year":"1997","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1427","DOI":"10.1088\/0957-0233\/8\/12\/007","article-title":"On errors of digital particle image velocimetry","volume":"8","author":"Huang","year":"1997","journal-title":"Meas. Sci. Technol."},{"key":"ref_39","unstructured":"Shimizu, M., and Okutomi, M. (2002, January 1\u20133). An analysis of sub-pixel estimation error on area-based image matching. Proceedings of the 2002 14th International Conference on Digital Signal Processing, DSP 2002, Santorini, Greece."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1007\/s00348-005-0942-3","article-title":"Two-dimensional Gaussian regression for sub-pixel displacement estimation in particle image velocimetry or particle position estimation in particle tracking velocimetry","volume":"38","author":"Nobach","year":"2005","journal-title":"Exp. Fluids"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/7\/1646\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:43:01Z","timestamp":1760208181000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/7\/1646"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,7,18]]},"references-count":40,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2017,7]]}},"alternative-id":["s17071646"],"URL":"https:\/\/doi.org\/10.3390\/s17071646","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,7,18]]}}}