{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T20:57:56Z","timestamp":1773435476328,"version":"3.50.1"},"reference-count":23,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2021,10,21]],"date-time":"2021-10-21T00:00:00Z","timestamp":1634774400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Cameras play a prominent role in the context of 3D data, as they can be designed to be very cheap and small and can therefore be used in many 3D reconstruction systems. Typical cameras capture video at 20 to 60 frames per second, resulting in a high number of frames to select from for 3D reconstruction. Many frames are unsuited for reconstruction as they suffer from motion blur or show too little variation compared to other frames. The camera used within this work has built-in inertial sensors. What if one could use the built-in inertial sensors to select a set of key frames well-suited for 3D reconstruction, free from motion blur and redundancy, in real time? A random forest classifier (RF) is trained by inertial data to determine frames without motion blur and to reduce redundancy. Frames are analyzed by the fast Fourier transformation and Lucas\u2013Kanade method to detect motion blur and moving features in frames to label those correctly to train the RF. We achieve a classifier that omits successfully redundant frames and preserves frames with the required quality but exhibits an unsatisfied performance with respect to ideal frames. A 3D reconstruction by Meshroom shows a better result with selected key frames by the classifier. By extracting frames from video, one can comfortably scan objects and scenes without taking single pictures. Our proposed method automatically extracts the best frames in real time without using complex image-processing algorithms.<\/jats:p>","DOI":"10.3390\/a14110303","type":"journal-article","created":{"date-parts":[[2021,10,21]],"date-time":"2021-10-21T14:37:48Z","timestamp":1634827068000},"page":"303","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Selection of Key Frames for 3D Reconstruction in Real Time"],"prefix":"10.3390","volume":"14","author":[{"given":"Alan","family":"Koschel","sequence":"first","affiliation":[{"name":"Fraunhofer Institute for Physical Measurement Techniques IPM, 79110 Freiburg, Germany"}]},{"given":"Christoph","family":"M\u00fcller","sequence":"additional","affiliation":[{"name":"Fraunhofer Institute for Physical Measurement Techniques IPM, 79110 Freiburg, Germany"},{"name":"Department of Digital Media, Furtwangen University, 78120 Furtwangen, Germany"}]},{"given":"Alexander","family":"Reiterer","sequence":"additional","affiliation":[{"name":"Fraunhofer Institute for Physical Measurement Techniques IPM, 79110 Freiburg, Germany"},{"name":"Department of Suistainable Systems Engnineering INATECH, Albert Ludwigs University Freiburg, 79110 Freiburg, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Reiterer, A., W\u00e4schle, K., St\u00f6rk, D., Leydecker, A., and Gitzen, N. (2020). Fully Automated Segmentation of 2D and 3D Mobile Mapping Data for Reliable Modeling of Surface Structures Using Deep Learning. Remote Sens., 12.","DOI":"10.3390\/rs12162530"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Paar, G., Huber, N.B., Bauer, A., Avian, M., and Reiterer, A. (2012). Vision-Based Terrestrial Surface Monitoring. Terrigenous Mass Mov., 283\u2013348.","DOI":"10.1007\/978-3-642-25495-6_10"},{"key":"ref_3","first-page":"461","article-title":"Image-based 3D surface approximation of the bladder using structure-from-motion for enhanced cystoscopy based on phantom data","volume":"63","author":"Hein","year":"2017","journal-title":"Biomed. Technol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.cviu.2017.03.005","article-title":"Model-based motion blur estimation for the improvement of motion tracking","volume":"160","author":"Seibold","year":"2017","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_5","unstructured":"Ahmed, M.T., Dailey, M., Landabaso, J., and Herrero, N. (2010, January 17\u201321). Robust key frame extraction for 3D reconstruction from video strams. Proceedings of the Fifth International Conference on Computer Vision Theory and Applications, Angers, France."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhang, C., Wang, H., Li, H., and Liu, J. (2017, January 28\u201330). A fast key frame extraction algorithm and an accurate feature matching method for 3D reconstruction from aerial video. Proceedings of the 29th Chinese Control and Decision Conference (CCDC), Chongqing, China.","DOI":"10.1109\/CCDC.2017.7978392"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"46014","DOI":"10.1117\/1.JBO.20.4.046014","article-title":"Automated frame selection process for high-resolution microendoscopy","volume":"20","author":"Ishijima","year":"2015","journal-title":"J. Biomed. Opt."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ren, J., Shen, X., Lin, Z., and Mech, R. (2020, January 1\u20135). Best frame selection in a short video. Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093615"},{"key":"ref_9","unstructured":"Lucas, B.D., and Kanade, T. (1981, January 24\u201328). An iterative image registration technique with an application to stereo vision. Proceedings DARPA Image Understanding Workshop, Columnia, UK."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/S0165-1684(98)00128-5","article-title":"A survey of hybrid MC\/DPCM\/DCT video coding distortions","volume":"70","author":"Yuen","year":"1998","journal-title":"Signal Process."},{"key":"ref_11","unstructured":"Gonzales, R.C., and Woods, R.E. (2008). Digital Image Processing, Pearson\/Prentice Hall. Available online: http:\/\/sdeuoc.ac.in\/sites\/default\/files\/sde_videos\/Digital%20Image%20Processing%203rd%20ed.%20-%20R.%20Gonzalez%2C%20R.%20Woods-ilovepdf-compressed.pdf."},{"key":"ref_12","unstructured":"J\u00e4hne, B. (2005). Digitale Bildverarbeitung, Springer."},{"key":"ref_13","unstructured":"Rekleitis, J. (2021, February 23). Visual Motion Estimation Based on Motion Blur Interpretation. School of Computer Science, McGill University. Available online: https:\/\/www.researchgate.net\/publication\/2687203_Visual_Motion_Estimation_based_on_Motion_Blur_Interpretation."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kalalembang, E., Usman, K., and Gunawan, I.P. (2009, January 23\u201325). DCT-based local motion blur detection. Proceedings of the International Conference on Instrumentation, Communication, Information Technology, and Biomedical Engineering, Bandung, Indonesia.","DOI":"10.1109\/ICICI-BME.2009.5417252"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1007\/BF01420984","article-title":"Performance of optical flow techniques","volume":"12","author":"Barron","year":"1994","journal-title":"Int. J. Comput. Vis."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive Image Features from Scale-Invariant Keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_17","first-page":"10","article-title":"A Combined Corner and Edge Detector","volume":"15","author":"Harris","year":"1988","journal-title":"Alvey Vis. Conf."},{"key":"ref_18","unstructured":"OpenCV (2015). Open Source Computer Vision Library, Intel."},{"key":"ref_19","unstructured":"Shi, J., and Tomasi, C. (1994, January 21\u201323). Good Features to Track. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, DC, USA."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kasebzadeh, P., Hendeby, G., Fritsche, C., Gunnarsson, F., and Gustafsson, F. (2017, January 18\u201321). IMU dataset for motion and device mode classification. Proceedings of the 2017 International Conference on Indoor Positioning and Indoor Navigation (IPIN), Sapporo, Japan.","DOI":"10.1109\/IPIN.2017.8115956"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1539","DOI":"10.3390\/s130201539","article-title":"Motion Mode Recognition and Step Detection Algorithms for Mobile Phone Users","volume":"13","author":"Susi","year":"2013","journal-title":"Sensors"},{"key":"ref_22","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Macm. Learn. Res."},{"key":"ref_23","unstructured":"Alice, V. (2021, February 23). Meshroom: A 3D Reconstruction Software. Available online: https:\/\/github.com\/alicevision\/meshroom."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/11\/303\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:20:43Z","timestamp":1760167243000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/11\/303"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,21]]},"references-count":23,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["a14110303"],"URL":"https:\/\/doi.org\/10.3390\/a14110303","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,21]]}}}