{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T23:13:53Z","timestamp":1774307633323,"version":"3.50.1"},"reference-count":60,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2019,6,29]],"date-time":"2019-06-29T00:00:00Z","timestamp":1561766400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Small-scaled unmanned aerial vehicles (UAVs) emerge as ideal image acquisition platforms due to their high maneuverability even in complex and tightly built environments. The acquired images can be utilized to generate high-quality 3D models using current multi-view stereo approaches. However, the quality of the resulting 3D model highly depends on the preceding flight plan which still requires human expert knowledge, especially in complex urban and hazardous environments. In terms of safe flight plans, practical considerations often define prohibited and restricted airspaces to be accessed with the vehicle. We propose a 3D UAV path planning framework designed for detailed and complete small-scaled 3D reconstructions considering the semantic properties of the environment allowing for user-specified restrictions on the airspace. The generated trajectories account for the desired model resolution and the demands on a successful photogrammetric reconstruction. We exploit semantics from an initial flight to extract the target object and to define restricted and prohibited airspaces which have to be avoided during the path planning process to ensure a safe and short UAV path, while still aiming to maximize the object reconstruction quality. The path planning problem is formulated as an orienteering problem and solved via discrete optimization exploiting submodularity and photogrammetrical relevant heuristics. An evaluation of our method on a customized synthetic scene and on outdoor experiments suggests the real-world capability of our methodology by providing feasible, short and safe flight plans for the generation of detailed 3D reconstruction models.<\/jats:p>","DOI":"10.3390\/rs11131550","type":"journal-article","created":{"date-parts":[[2019,7,1]],"date-time":"2019-07-01T03:23:59Z","timestamp":1561951439000},"page":"1550","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":75,"title":["Automatic and Semantically-Aware 3D UAV Flight Planning for Image-Based 3D Reconstruction"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2988-982X","authenticated-orcid":false,"given":"Tobias","family":"Koch","sequence":"first","affiliation":[{"name":"Chair of Remote Sensing Technology, Technical University of Munich, 80333 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9186-4175","authenticated-orcid":false,"given":"Marco","family":"K\u00f6rner","sequence":"additional","affiliation":[{"name":"Chair of Remote Sensing Technology, Technical University of Munich, 80333 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5805-8892","authenticated-orcid":false,"given":"Friedrich","family":"Fraundorfer","sequence":"additional","affiliation":[{"name":"Institute for Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria"},{"name":"Remote Sensing Technology Institute, German Aerospace Center, 82234 Wessling, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2019,6,29]]},"reference":[{"key":"ref_1","unstructured":"(2019, May 28). Pix4D: Pix4Dcapture. Available online: https:\/\/pix4d.com\/product\/pix4dcapture\/."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1145\/1141911.1141964","article-title":"Photo Tourism: Exploring Photo Collections in 3D","volume":"25","author":"Snavely","year":"2006","journal-title":"ACM Trans. Graph."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sch\u00f6nberger, J.L., and Frahm, J.M. (2016, January 27\u201330). Structure-from-Motion Revisited. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.445"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Vacanas, Y., Themistocleous, K., Agapiou, A., and Hadjimitsis, D. (2015, January 16\u201319). Building Information Modelling (BIM) and Unmanned Aerial Vehicle (UAV) Technologies in Infrastructure Construction Project Management and Delay and Disruption Analysis. Proceedings of the International Conference on Remote Sensing and Geoinformation of the Environment, Paphos, Cyprus.","DOI":"10.1117\/12.2192723"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Hallermann, N., and Morgenthal, G. (2014, January 7\u201311). Visual Inspection Strategies for Large Bridges using Unmanned Aerial Vehicles (UAV). Proceedings of the 7th International Conference on Bridge Maintenance, Safety and Management (IABMAS), Shanghai, China.","DOI":"10.1201\/b17063-96"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Mostegel, C., Prettenthaler, R., Fraundorfer, F., and Bischof, H. (2017, January 21\u201326). Scalable Surface Reconstruction from Point Clouds with Extreme Scale and Density Diversity. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.268"},{"key":"ref_7","unstructured":"(2019, May 28). Precisionhawk: Precision Flight. Available online: https:\/\/www.precisionhawk.com\/precisionflight\/."},{"key":"ref_8","unstructured":"(2019, May 28). DJI: Flight Planner. Available online: https:\/\/www.djiflightplanner.com\/."},{"key":"ref_9","unstructured":"(2019, May 28). Ardupilot: Mission Planner. Available online: http:\/\/ardupilot.org\/planner\/."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Roberts, M., Dey, D., Truong, A., Sinha, S., Shah, S., Kapoor, A., Hanrahan, P., and Joshi, N. (2017, January 22\u201329). Submodular Trajectory Optimization for Aerial 3D Scanning. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.569"},{"key":"ref_11","first-page":"4","article-title":"Plan3D: Viewpoint and Trajectory Optimization for Aerial Multi-View Stereo Reconstruction","volume":"38","author":"Hepp","year":"2018","journal-title":"ACM Trans. Graph."},{"key":"ref_12","unstructured":"Cheng, P., Keller, J., and Kumar, V. (2008, January 22\u201326). Time-optimal UAV Trajectory Planning for 3D Urban Structure Coverage. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Nice, France."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chakrabarty, A., and Langelaan, J. (2009, January 10\u201313). Energy Maps for Long-range Path Planning for Small-and Micro-UAVs. Proceedings of the AIAA Guidance, Navigation, and Control Conference (GNC), Chicago, IL, USA.","DOI":"10.2514\/6.2009-6113"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1007\/s10846-016-0348-x","article-title":"Coverage Path Planning for UAVs Photogrammetry with Energy and Resolution Constraints","volume":"83","author":"Buttazzo","year":"2016","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/MGRS.2017.2762307","article-title":"Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources","volume":"5","author":"Zhu","year":"2017","journal-title":"IEEE Geosci. Remote Sens. Mag."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Goesele, M., Snavely, N., Curless, B., Hoppe, H., and Seitz, S.M. (2007, January 14\u201321). Multi-view Stereo for Community Photo Collections. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Rio de Janeiro, Brazil.","DOI":"10.1109\/ICCV.2007.4408933"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Furukawa, Y., Curless, B., Seitz, S.M., and Szeliski, R. (2010, January 13\u201318). Towards Internet-scale Multi-view Stereo. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539802"},{"key":"ref_18","unstructured":"Rumpler, M., Irschara, A., and Bischof, H. (2011, January 26\u201327). Multi-view Stereo: Redundancy Benefits for 3D Reconstruction. Proceedings of the 35th Workshop of the Austrian Association for Pattern Recognition (AAPR), Graz, Austria."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/0600000052","article-title":"Multi-view Stereo: A Tutorial","volume":"9","author":"Furukawa","year":"2015","journal-title":"Found. Trends Comput. Graph. Vis."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s12518-013-0120-x","article-title":"UAV for 3D Mapping Applications: A Review","volume":"6","author":"Nex","year":"2014","journal-title":"Appl. Geomat."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1007\/s11554-013-0386-6","article-title":"Efficient Next-best-scan Planning for Autonomous 3D Surface Reconstruction of Unknown Objects","volume":"10","author":"Kriegel","year":"2015","journal-title":"J. Real-Time Image Process."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Heng, L., Lee, G.H., Fraundorfer, F., and Pollefeys, M. (2011, January 25\u201330). Real-time Photo-realistic 3D Mapping for Micro Aerial Vehicles. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, USA.","DOI":"10.1109\/IROS.2011.6048818"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Sturm, J., Bylow, E., Kerl, C., Kahl, F., and Cremers, D. (2013, January 4\u20136). Dense Tracking and Mapping with a Quadrocopter. Proceedings of the ISPRS\u2014International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XL-1\/W2, Rostock, Germany.","DOI":"10.5194\/isprsarchives-XL-1-W2-395-2013"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Loianno, G., Thomas, J., and Kumar, V. (2015, January 26\u201330). Cooperative Localization and Mapping of MAVs using RGB-D Sensors. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139761"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1002\/rob.21436","article-title":"Collaborative Mapping of an Earthquake Damaged Building via Ground and Aerial Robots","volume":"29","author":"Michael","year":"2012","journal-title":"J. Field Robot."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1145\/2980179.2980225","article-title":"Automated View and Path Planning for Scalable Multi-object 3D Scanning","volume":"35","author":"Fan","year":"2016","journal-title":"ACM Trans. Graph."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hepp, B., Dey, D., Sinha, S.N., Kapoor, A., Joshi, N., and Hilliges, O. (2018, January 8\u201314). Learn-to-Score: Efficient 3D Scene Exploration by Predicting View Utility. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01267-0_27"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1680","DOI":"10.1109\/LRA.2017.2655144","article-title":"A Two-Stage Optimized Next-View Planning Framework for 3-D Unknown Environment Exploration, and Structural Reconstruction","volume":"2","author":"Meng","year":"2017","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Dunn, E., and Frahm, J.M. (2009, January 7\u201310). Next Best View Planning for Active Model Improvement. Proceedings of the British Machine Vision Conference (BMVC), London, UK.","DOI":"10.5244\/C.23.53"},{"key":"ref_30","unstructured":"Von Stumberg, L., Usenko, V., Engel, J., St\u00fcckler, J., and Cremers, D. (2016). Autonomous Exploration with a Low-Cost Quadrocopter Using Semi-Dense Monocular SLAM. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Mendez, O., Hadfield, S., Pugeault, N., and Bowden, R. (2017, January 21\u201326). Taking the Scenic Route to 3D: Optimising Reconstruction from Moving Cameras. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/ICCV.2017.501"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Palazzolo, E., and Stachniss, C. (2018). Effective Exploration for MAVs Based on the Expected Information Gain. Drones, 2.","DOI":"10.3390\/drones2010009"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kumar Ramakrishnan, S., and Grauman, K. (2018, January 8\u201314). Sidekick Policy Learning for Active Visual Exploration. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01258-8_26"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Border, R., Gammell, J.D., and Newman, P. (2018, January 21\u201325). Surface Edge Explorer (SEE): Planning Next Best Views Directly from 3D Observations. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8461098"},{"key":"ref_35","unstructured":"Hoppe, C., Wendel, A., Zollmann, S., Pirker, K., Irschara, A., Bischof, H., and Kluckner, S. (2012, January 4\u20136). Photogrammetric Camera Network Design for Micro Aerial Vehicles. Proceedings of the Computer Vision Winter Workshop (CVWW), Hernstein, Austria."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Jing, W., Polden, J., Tao, P.Y., Lin, W., and Shimada, K. (2016, January 13\u201315). View Planning for 3D Shape Reconstruction of Buildings with Unmanned Aerial Vehicles. Proceedings of the IEEE International Conference on Control, Automation, Robotics and Vision (ICARCV), Phuket, Thailand.","DOI":"10.1109\/ICARCV.2016.7838774"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Smith, N., Moehrle, N., Goesele, M., and Heidrich, W. (2018, January 4\u20137). Aerial Path Planning for Urban Scene Reconstruction: A Continuous Optimization Method and Benchmark. Proceedings of the ACM SIGGRAPH Asia, Tokyo, Japan.","DOI":"10.1145\/3272127.3275010"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Peng, C., and Isler, V. (2018). Adaptive View Planning for Aerial 3D Reconstruction of Complex Scenes. arXiv.","DOI":"10.1109\/ICRA.2019.8793532"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Huang, R., Zou, D., Vaughan, R., and Tan, P. (2018, January 21\u201325). Active Image-based Modeling with a Toy Drone. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460673"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1016\/j.culher.2012.11.007","article-title":"Automated Camera Network Design for 3D Modeling of Cultural Heritage Objects","volume":"14","author":"Alsadik","year":"2013","journal-title":"J. Cult. Herit."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1007\/s10514-015-9517-1","article-title":"Three-dimensional Coverage Path Planning via Viewpoint Resampling and Tour Optimization for Aerial Robots","volume":"40","author":"Bircher","year":"2016","journal-title":"Auton. Robot."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Snavely, N., Seitz, S.M., and Szeliski, R. (2008, January 23\u201328). Skeletal Graphs for Efficient Structure from Motion. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, AK, USA.","DOI":"10.1109\/CVPR.2008.4587678"},{"key":"ref_43","unstructured":"Mostegel, C., Rumpler, M., Fraundorfer, F., and Bischof, H. (July, January 26). UAV-based Autonomous Image Acquisition with Multi-view Stereo Quality Assurance by Confidence Prediction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPR-WS), Las Vegas, NV, USA."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Devrim Kaba, M., Gokhan Uzunbas, M., and Nam Lim, S. (2017, January 21\u201326). A Reinforcement Learning Approach to the View Planning Problem. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.541"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"473","DOI":"10.5194\/isprs-annals-III-3-473-2016","article-title":"Semantic Segmentation of Aerial Images with an Ensemble of CNNs","volume":"3","author":"Marmanis","year":"2016","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"6054","DOI":"10.1109\/TGRS.2017.2719738","article-title":"Learning Aerial Image Segmentation from Online Maps","volume":"55","author":"Kaiser","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1109\/LGRS.2017.2778181","article-title":"Semantic segmentation of aerial images with shuffling convolutional neural networks","volume":"15","author":"Chen","year":"2018","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Wendel, A., Maurer, M., Graber, G., Pock, T., and Bischof, H. (2012, January 16\u201321). Dense Reconstruction on-the-fly. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6247833"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 16\u201321). Fully Convolutional Networks for Semantic Segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_50","unstructured":"(2019, May 28). Semantic Drone Dataset. Available online: http:\/\/dronedataset.icg.tugraz.at."},{"key":"ref_51","unstructured":"(2019, May 28). ISPRS 2D Semantic Labelling Contest\u2014Potsdam. Available online: http:\/\/www2.isprs.org\/commissions\/comm3\/wg4\/2d-sem-label-potsdam.html."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1145\/1276377.1276407","article-title":"Direct Visibility of Point Sets","volume":"26","author":"Katz","year":"2007","journal-title":"ACM Trans. Graph."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Hartley, R., and Zisserman, A. (2003). Multiple View Geometry in Computer Vision, Cambridge University Press.","DOI":"10.1017\/CBO9780511811685"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Luhmann, T., Robson, S., Kyle, S., and Boehm, J. (2013). Close-Range Photogrammetry and 3D Imaging, Walter de Gruyter.","DOI":"10.1515\/9783110302783"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"F\u00f6rstner, W., and Wrobel, B.P. (2016). Photogrammetric Computer Vision, Springer.","DOI":"10.1007\/978-3-319-11550-4"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"251","DOI":"10.5194\/isprsarchives-XL-5-W1-251-2013","article-title":"Image acquisition and model selection for multi-view stereo","volume":"40","author":"Wenzel","year":"2013","journal-title":"ISPRS Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_57","unstructured":"Kraus, K. (2011). Photogrammetry: Geometry from Images and Laser Scans, Walter de Gruyter."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Krause, A., and Golovin, D. (2014). Submodular Function Maximization, Cambridge University Press.","DOI":"10.1017\/CBO9781139177801.004"},{"key":"ref_59","unstructured":"Blender Online Community (2018). Blender\u2014A 3D Modelling and Rendering Package, Blender Foundation, Blender Institute."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1145\/3072959.3073599","article-title":"Tanks and Temples: Benchmarking Large-scale Scene Reconstruction","volume":"36","author":"Knapitsch","year":"2017","journal-title":"ACM Trans. Graph."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/13\/1550\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:02:26Z","timestamp":1760187746000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/11\/13\/1550"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,29]]},"references-count":60,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2019,7]]}},"alternative-id":["rs11131550"],"URL":"https:\/\/doi.org\/10.3390\/rs11131550","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,6,29]]}}}