{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T16:39:54Z","timestamp":1778776794608,"version":"3.51.4"},"reference-count":53,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2022,4,28]],"date-time":"2022-04-28T00:00:00Z","timestamp":1651104000000},"content-version":"vor","delay-in-days":27,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"JSPS","doi-asserted-by":"publisher","award":["JP19K12681"],"award-info":[{"award-number":["JP19K12681"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,4,28]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The integration of augmented reality and drones allows past and future landscapes to be visualized from an aerial perspective. However, these visualizations still suffer from the occlusion problem, where the three-dimensional (3D) virtual model displayed in the real world is in front of a real-world object. Currently, city digital twins are essential for the sustainable development of cities and the development of detailed 3D models of cities. By visualizing the city digital twin, augmented reality can facilitate the participation of nonexpert citizens in the decision-making process of urban design, but research examples are limited. Here, using detailed city 3D models, we develop a digital-twin approach to outdoor augmented reality with occlusion handling for both first-person and bird\u2019s-eye views. In a verification experiment, the occlusion handling accuracy of the prototype system was measured to be about 0.8 using intersection over union. The frame rate of the entire prototype system was about 30 fps, and the delay between the controller and the augmented reality device was about 3 s. The internet-based system architecture was developed to integrate augmented reality and drone systems. Our system allows multiple stakeholders involved in building construction projects to observe aerial perspectives of those projects, both on-site and off-site via an internet browser, using augmented reality with occlusion handling.<\/jats:p>","DOI":"10.1093\/jcde\/qwac032","type":"journal-article","created":{"date-parts":[[2022,4,14]],"date-time":"2022-04-14T11:11:27Z","timestamp":1649934687000},"page":"837-856","source":"Crossref","is-referenced-by-count":51,"title":["Future landscape visualization using a city digital twin: integration of augmented reality and drones with implementation of 3D model-based occlusion handling"],"prefix":"10.1093","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9403-5671","authenticated-orcid":false,"given":"Naoki","family":"Kikuchi","sequence":"first","affiliation":[{"name":"Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, Osaka University, Suita, Osaka 565-0824, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4271-4445","authenticated-orcid":false,"given":"Tomohiro","family":"Fukuda","sequence":"additional","affiliation":[{"name":"Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, Osaka University, Suita, Osaka 565-0824, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2944-4540","authenticated-orcid":false,"given":"Nobuyoshi","family":"Yabuki","sequence":"additional","affiliation":[{"name":"Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, Osaka University, Suita, Osaka 565-0824, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2022,4,28]]},"reference":[{"key":"2022050612543758900_bib1","article-title":"Amazon prime air","author":"Amazon","year":"2013"},{"issue":"4","key":"2022050612543758900_bib2","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1162\/pres.1997.6.4.355","article-title":"A survey of augmented reality","volume":"6","author":"Azuma","year":"1997","journal-title":"Presence: Teleoperators and Virtual Environments"},{"issue":"5","key":"2022050612543758900_bib3","first-page":"817","article-title":"Digital twins","volume":"45","author":"Batty","year":"2018","journal-title":"Environment and Planning B: Urban Analytics and City Science"},{"key":"2022050612543758900_bib4","doi-asserted-by":"crossref","first-page":"1063","DOI":"10.1109\/TSMCA.2007.904779","article-title":"Review of low frame rate effects on human performance","volume":"37","author":"Chen","year":"2007","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics \u2013 Part A: Systems and Humans"},{"issue":"6","key":"2022050612543758900_bib5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/su12062307","article-title":"Urban digital twins for smart cities and citizens: The case study of Herrenberg, Germany","volume":"12","author":"Dembski","year":"2020","journal-title":"Sustainability"},{"issue":"2","key":"2022050612543758900_bib6","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.jmse.2021.03.003","article-title":"A systematic review of a digital twin city: A new pattern of urban governance toward smart cities","volume":"6","author":"Deng","year":"2021","journal-title":"Journal of Management Science and Engineering"},{"key":"2022050612543758900_bib7","first-page":"555","article-title":"Research on construction of spatio-temporal data visualization platform for GIS and BIM fusion","volume-title":"Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences-ISPRS Archives","author":"Dou","year":"2019"},{"key":"2022050612543758900_bib8","first-page":"829","article-title":"DepthLab: Real-time 3D interaction with depth maps for mobile augmented reality","volume-title":"Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (UIST '20), Association for Computing Machinery","author":"Du","year":"2020"},{"key":"2022050612543758900_bib9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/WCNC.2018.8377077","article-title":"Monitoring road traffic with a UAV-based system","volume-title":"2018 IEEE Wireless Communications and Networking Conference (WCNC)","author":"Elloumi","year":"2018"},{"issue":"5","key":"2022050612543758900_bib10","doi-asserted-by":"crossref","first-page":"2417","DOI":"10.3390\/app11052417","article-title":"Mixed reality: A reconsideration based on mixed objects and geospatial modalities","volume":"11","author":"Evangelidis","year":"2021","journal-title":"Applied Sciences"},{"key":"2022050612543758900_bib11","author":"Fast.com"},{"issue":"2","key":"2022050612543758900_bib12","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1016\/j.jcde.2018.05.007","article-title":"An indoor thermal environment design system for renovation using augmented reality","volume":"6","author":"Fukuda","year":"2019","journal-title":"Journal of Computational Design and Engineering"},{"issue":"4","key":"2022050612543758900_bib13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.24102\/ijslup.v3i4.738","article-title":"Drone applications for environmental management in urban spaces: A review","volume":"3","author":"Gallacher","year":"2017","journal-title":"International Journal of Sustainable Land Use and Urban Planning"},{"key":"2022050612543758900_bib14","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/ISMAR-Adjunct.2018.00024","article-title":"Addressing the occlusion problem in augmented reality environments with phantom hollow objects","volume-title":"IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)","author":"Gimeno","year":"2018"},{"key":"2022050612543758900_bib15","first-page":"244","article-title":"Testing the effectiveness of augmented reality in the public participation process: A case study in the city of Bernburg","volume":"2","author":"Goudarznia","year":"2017","journal-title":"Journal of Digital Landscape Architecture"},{"key":"2022050612543758900_bib16","first-page":"1","volume-title":"Digital twin: Manufacturing excellence through virtual factory replication, White Paper","author":"Grieves","year":"2014"},{"issue":"3","key":"2022050612543758900_bib17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1061\/(ASCE)ME.1943-5479.0000748","article-title":"Participatory sensing and digital twin city: Updating virtual city models for enhanced risk-informed decision-making","volume":"36","author":"Ham","year":"2020","journal-title":"Journal of Management in Engineering"},{"key":"2022050612543758900_bib18","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1016\/j.envsoft.2018.05.012","article-title":"Mobile augmented reality for flood visualization","volume":"109","author":"Haynes","year":"2018","journal-title":"Environmental Modelling and Software"},{"issue":"15","key":"2022050612543758900_bib19","doi-asserted-by":"crossref","first-page":"2976","DOI":"10.3390\/rs13152976","article-title":"A comparison of monoscopic and stereoscopic 3D visualizations: Effect on spatial planning in digital twins","volume":"13","author":"Herman","year":"2021","journal-title":"Remote Sensing"},{"issue":"6","key":"2022050612543758900_bib20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3272127.3275083","article-title":"Fast depth densification for occlusion-aware augmented reality","volume":"37","author":"Holynski","year":"2018","journal-title":"ACM Transactions on Graphics"},{"key":"2022050612543758900_bib21","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1016\/j.trc.2018.02.017","article-title":"A range-restricted recharging station coverage model for drone delivery service planning","volume":"90","author":"Hong","year":"2018","journal-title":"Transportation Research Part C: Emerging Technologies"},{"key":"2022050612543758900_bib22","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1007\/978-3-319-70096-0_37","article-title":"Training deep neural networks for detecting drinking glasses using synthetic images","volume-title":"International Conference on Neural Information Processing","author":"Jabbar","year":"2017"},{"key":"2022050612543758900_bib23","first-page":"1","article-title":"Occlusion in outdoor augmented reality using geospatial building data","volume-title":"Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology. Association for Computing Machinery","author":"Kasperi","year":"2017"},{"key":"2022050612543758900_bib24","doi-asserted-by":"crossref","first-page":"101281","DOI":"10.1016\/j.aei.2021.101281","article-title":"Assessing future landscapes using enhanced mixed reality with semantic segmentation by deep learning","volume":"48","author":"Kido","year":"2021","journal-title":"Advanced Engineering Informatics"},{"key":"2022050612543758900_bib25","first-page":"521","article-title":"Landscape visualization by integrating augmented reality and drones with occlusion handling to link real and virtual worlds \u2013 Towards city digital twin realization","volume-title":"Proceedings of the 39th eCAADe Conference","author":"Kikuchi","year":"2021"},{"issue":"3","key":"2022050612543758900_bib26","doi-asserted-by":"crossref","first-page":"910","DOI":"10.1093\/jcde\/qwab022","article-title":"Evaluation framework for BIM-based VR applications in design phase","volume":"8","author":"Kim","year":"2021","journal-title":"Journal of Computational Design and Engineering"},{"key":"2022050612543758900_bib27","first-page":"843","article-title":"Flying augmented reality: Supporting planning and simulation analysis by combining mixed reality methods using multicopter and pattern recognition","volume-title":"Proceedings of the 29th eCAADe Conference","author":"Koch","year":"2011"},{"issue":"1","key":"2022050612543758900_bib28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/ijgi7010032","article-title":"Real-Time location-based rendering of urban underground pipelines","volume":"7","author":"Li","year":"2018","journal-title":"ISPRS International Journal of Geo-Information"},{"key":"2022050612543758900_bib29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1061\/(ASCE)CP.1943-5487.0000958","article-title":"Integrating building information model and augmented reality for drone-based building inspection","volume":"35","author":"Liu","year":"2021","journal-title":"Journal of Computing in Civil Engineering"},{"key":"2022050612543758900_bib30","author":"Microsoft HoloLens","year":"2016"},{"issue":"12","key":"2022050612543758900_bib31","first-page":"1321","article-title":"A taxonomy of mixed reality visual displays","volume":"E77-D","author":"Milgram","year":"1994","journal-title":"IEICE Transactions on Information and Systems"},{"key":"2022050612543758900_bib33","doi-asserted-by":"crossref","first-page":"1995","DOI":"10.24251\/HICSS.2019.241","article-title":"Devising a game theoretic approach to enable smart city digital twin analytics","volume-title":"Hawaii International Conference on System Sciences (HICSS)","author":"Mohammadi","year":"2019"},{"issue":"2","key":"2022050612543758900_bib34","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1109\/MCOM.2017.1600587CM","article-title":"UAV-Based IoT platform: A crowd surveillance use case","volume":"55","author":"Motlagh","year":"2017","journal-title":"IEEE Communications Magazine"},{"key":"2022050612543758900_bib35","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1145\/2682884.2682886","article-title":"Over the rainbow: Information design for low-resolution urban displays","volume-title":"Proceedings of the 2nd Media Architecture Biennale Conference: World Cities (MAB '14). Association for Computing Machinery","author":"Offenhuber","year":"2014"},{"key":"2022050612543758900_bib36","article-title":"OGC city geography markup language (CityGML) 3.0 conceptual model users guide","author":"Open Geospatial Consortium","year":"2021"},{"key":"2022050612543758900_bib32","author":"PLATEAU","year":"2020"},{"key":"2022050612543758900_bib37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3281505.3281546","article-title":"Occlusion handling using semantic segmentation and visibility-based rendering for mixed reality","volume-title":"Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (VRST 2018)","author":"Roxas","year":"2018"},{"key":"2022050612543758900_bib38","first-page":"155","article-title":"Smart city platform enabling digital twin","volume-title":"International Conference on Intelligent Systems (IS)","author":"Ruohom\u00e4ki","year":"2018"},{"key":"2022050612543758900_bib39","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1007\/s41064-020-00092-2","article-title":"The digital twin of the city of Zurich for urban planning","volume":"88","author":"Schrotter","year":"2020","journal-title":"Journal of Photogrammetry, Remote Sensing and Geoinformation Science"},{"key":"2022050612543758900_bib40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/su13063386","article-title":"City digital twin potentials: A review and research agenda","volume":"13","author":"Shahat","year":"2021","journal-title":"Sustainability"},{"key":"2022050612543758900_bib42","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.neucom.2014.12.081","article-title":"Handling occlusions in augmented reality based on 3D reconstruction method","volume":"156","author":"Tian","year":"2015","journal-title":"Neurocomputing"},{"issue":"2","key":"2022050612543758900_bib43","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/mti3020043","article-title":"Interactive landscape design and flood visualisation in augmented reality","volume":"3","author":"Tomkins","year":"2019","journal-title":"Multimodal Technologies and Interaction"},{"key":"2022050612543758900_bib44","first-page":"140","article-title":"Where the wild things will be: Adaptive visualisation with spatial computing","volume":"6","author":"Tomkins","year":"2021","journal-title":"Journal of Digital Landscape Architecture"},{"key":"2022050612543758900_bib45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ISMSIT.2018.8567073","article-title":"Geo-location based augmented reality application for cultural heritage using drones","volume-title":"Proceedings of the 2018 2nd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT)","author":"Unal","year":"2018"},{"key":"2022050612543758900_bib46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.daach.2020.e00140","article-title":"Distant augmented reality: Bringing a new dimension to user experience using drones","volume":"17","author":"Unal","year":"2020","journal-title":"Digital Applications in Archaeology and Cultural Heritage"},{"issue":"6","key":"2022050612543758900_bib47","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3272127.3275041","article-title":"Depth from motion for smartphone AR","volume":"37","author":"Valentin","year":"2018","journal-title":"ACM Transactions on Graphics"},{"key":"2022050612543758900_bib41","author":"Virtual Singapore","year":"2017"},{"key":"2022050612543758900_bib48","first-page":"1570","article-title":"Augmented reality and unmanned aerial vehicle assist in construction management","volume-title":"International Conference on Computing in Civil and Building Engineering","author":"Wen","year":"2014"},{"key":"2022050612543758900_bib49","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1016\/j.geomorph.2012.08.021","article-title":"Structure-from-Motion\u2019 photogrammetry: A low-cost, effective tool for geoscience applications","volume":"179","author":"Westoby","year":"2012","journal-title":"Geomorphology"},{"key":"2022050612543758900_bib50","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cities.2020.103064","article-title":"A digital twin smart city for citizen feedback","volume":"110","author":"White","year":"2021","journal-title":"Cities"},{"key":"2022050612543758900_bib51","doi-asserted-by":"crossref","first-page":"012074","DOI":"10.1088\/1755-1315\/323\/1\/012074","article-title":"Visualisation of KPIs in zero emission neighbourhoods for improved stakeholder participation using virtual reality","volume":"323","author":"Wiberg","year":"2019","journal-title":"IOP Conference Series: Earth and Environmental Science"},{"key":"2022050612543758900_bib52","first-page":"423","article-title":"Integrating UAV development technology with augmented reality toward landscape tele-simulation","volume-title":"Proceedings of the 24th CAADRIA Conference","author":"Yan","year":"2019"},{"issue":"9","key":"2022050612543758900_bib53","doi-asserted-by":"crossref","first-page":"3808","DOI":"10.1109\/TVCG.2020.2986247","article-title":"Visualization techniques in augmented reality: A taxonomy, methods and patterns","volume":"27","author":"Zollmann","year":"2020","journal-title":"IEEE Transactions on Visualization and Computer Graphics"}],"container-title":["Journal of Computational Design and Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jcde\/article-pdf\/9\/2\/837\/43589419\/qwac032.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcde\/article-pdf\/9\/2\/837\/43589419\/qwac032.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,5,6]],"date-time":"2022-05-06T12:59:05Z","timestamp":1651841945000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jcde\/article\/9\/2\/837\/6575496"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4]]},"references-count":53,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,4,28]]}},"URL":"https:\/\/doi.org\/10.1093\/jcde\/qwac032","relation":{},"ISSN":["2288-5048"],"issn-type":[{"value":"2288-5048","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,4]]},"published":{"date-parts":[[2022,4]]}}}