{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,12]],"date-time":"2025-12-12T13:36:45Z","timestamp":1765546605022,"version":"build-2065373602"},"reference-count":37,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2020,4,16]],"date-time":"2020-04-16T00:00:00Z","timestamp":1586995200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Trade, Industry and Energy (MOTIE, Korea)","award":["10063098"],"award-info":[{"award-number":["10063098"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Pan-tilt-zoom (PTZ) and omnidirectional cameras serve as a video-mediated communication interface for telemedicine. Most cases use either PTZ or omnidirectional cameras exclusively; even when used together, images from the two are shown separately on 2D displays. Conventional foveated imaging techniques may offer a solution for exploiting the benefits of both cameras, i.e., the high resolution of the PTZ camera and the wide field-of-view of the omnidirectional camera, but displaying the unified image on a 2D display would reduce the benefit of \u201comni-\u201d directionality. In this paper, we introduce a foveated imaging pipeline designed to support virtual reality head-mounted displays (HMDs). The pipeline consists of two parallel processes: one for estimating parameters for the integration of the two images and another for rendering images in real time. A control mechanism for placing the foveal region (i.e., high-resolution area) in the scene and zooming is also proposed. Our evaluations showed that the proposed pipeline achieved, on average, 17 frames per second when rendering the foveated view on an HMD, and showed angular resolution improvement on the foveal region compared with the omnidirectional camera view. However, the improvement was less significant when the zoom level was 8\u00d7 and more. We discuss possible improvement points and future research directions.<\/jats:p>","DOI":"10.3390\/s20082264","type":"journal-article","created":{"date-parts":[[2020,4,16]],"date-time":"2020-04-16T13:01:39Z","timestamp":1587042099000},"page":"2264","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Foveation Pipeline for 360\u00b0 Video-Based Telemedicine"],"prefix":"10.3390","volume":"20","author":[{"given":"Muhammad Firdaus","family":"Syawaludin","sequence":"first","affiliation":[{"name":"Imaging Media Research Center, Korea Institute of Science and Technology (KIST), Seoul 02792, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0766-6484","authenticated-orcid":false,"given":"Myungho","family":"Lee","sequence":"additional","affiliation":[{"name":"Imaging Media Research Center, Korea Institute of Science and Technology (KIST), Seoul 02792, Korea"}]},{"given":"Jae-In","family":"Hwang","sequence":"additional","affiliation":[{"name":"Imaging Media Research Center, Korea Institute of Science and Technology (KIST), Seoul 02792, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,16]]},"reference":[{"key":"ref_1","unstructured":"Field, M.J. (1996). Telemedicine: A Guide to Assessing Telecommunications for Health Care, National Academies Press."},{"key":"ref_2","unstructured":"Tracy, J. (2004). Telemedicine Technical Assistance Documents A Guide to Getting Started in Telemedicine, University of Missouri-School of Medicine. Office for the Advancement of Telehealth."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"638","DOI":"10.1089\/tmj.2018.0044","article-title":"Development and Evaluation of a User-Centered Mobile Telestroke Platform","volume":"25","author":"Chapman","year":"2019","journal-title":"Telemed. E-Health"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Laetitia, Y., Van Hooff, R.J., De Smedt, A., Alexis, V.E., Van Rita, D., De Rohny, V.C., Andre, C., Ives, H., Door, L., and De Jacques, K. (2014). Feasibility of ambulance-based telemedicine (fact) study: Safety, feasibility and reliability of third generation ambulance telemedicine. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0110043"},{"key":"ref_5","unstructured":"Cadik, M. (2004). Human Perception and Computer Graphics, Czech Technical University."},{"key":"ref_6","first-page":"33043","article-title":"Superimposed Multi-Resolution","volume":"25","author":"Carles","year":"2017","journal-title":"Imaging"},{"key":"ref_7","unstructured":"Sandini, G., Questa, P., Scheffer, D., Diericks, B., and Mannucci, A. (2000, January 16\u201317). A retina-like CMOS sensor and its applications. Proceedings of the IEEE Sensor Array and Multichannel Signal Processing Workshop, Cambridge, UK."},{"key":"ref_8","first-page":"96261A","article-title":"Wide-angle lens miniaturization through foveated imaging","volume":"9626","author":"Dallaire","year":"2015","journal-title":"Opt. Syst. Des. Opt. Des. Eng. VI"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"6081","DOI":"10.1364\/AO.52.006081","article-title":"Demonstration of a multichannel, multiresolution imaging system","volume":"52","author":"Belay","year":"2013","journal-title":"Appl. Opt."},{"key":"ref_10","unstructured":"Ude, A., Gaskett, C., and Cheng, G. (2006, January 15\u201319). Foveated vision systems with two cameras per eye. Proceedings of the IEEE International Conference on Robotics and Automation, Orlando, FL, USA."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1869","DOI":"10.1364\/OL.41.001869","article-title":"Multi-aperture foveated imaging","volume":"41","author":"Carles","year":"2016","journal-title":"Opt. Lett."},{"key":"ref_12","first-page":"24","article-title":"The effect of environmental features, self-avatar, and immersion on object location memory in virtual environments","volume":"3","author":"Steed","year":"2016","journal-title":"Front. ICT"},{"key":"ref_13","unstructured":"Gould, S., Arfvidsson, J., Kaehler, A., Sapp, B., Messner, M., Bradski, G., Baumstarck, P., Chung, S., and Ng, A.Y. (2007, January 6\u201312). Peripheral-foveal vision for real-time object recognition and tracking in video. Proceedings of the International Joint Conference on Artificial Intelligence, Hyderabad, India."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1007\/s10846-010-9504-x","article-title":"A catadioptric and pan-tilt-zoom camera pair object tracking system for UAVs","volume":"61","author":"Tarhan","year":"2011","journal-title":"J. Intell. Robot. Syst. Theory Appl."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2548","DOI":"10.1364\/BOE.5.002548","article-title":"Characterization and in-vivo evaluation of a multi-resolution foveated laparoscope for minimally invasive surgery","volume":"5","author":"Qin","year":"2014","journal-title":"Biomed. Opt. Express"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1456650.1456652","article-title":"A review of overview+detail, zooming, and focus+context interfaces","volume":"41","author":"Cockburn","year":"2008","journal-title":"ACM Comput. Surv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.cviu.2006.06.002","article-title":"Pan-tilt-zoom camera calibration and high-resolution mosaic generation","volume":"103","author":"Sinha","year":"2006","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_18","first-page":"30:1","article-title":"Large-Area, Multilayered, and High-Resolution Visual Monitoring Using a Dual-Camera System","volume":"11","author":"Lin","year":"2014","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl. TOMM"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2168996.2168997","article-title":"Image registration for foveated panoramic sensing","volume":"8","author":"Dornaika","year":"2012","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1256","DOI":"10.1109\/TMM.2011.2165055","article-title":"Multi-resolution design for large-scale and high-resolution monitoring","volume":"13","author":"Chen","year":"2011","journal-title":"IEEE Trans. Multimed."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1364\/AO.47.000317","article-title":"Dual-sensor foveated imaging system","volume":"47","author":"Hua","year":"2008","journal-title":"Appl. Opt."},{"key":"ref_22","first-page":"1","article-title":"Rich360: Optimized Spherical Representation from Structured Panoramic Camera Arrays","volume":"35","author":"Lee","year":"2016","journal-title":"ACM Trans. Graph."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1111\/cgf.13277","article-title":"High-resolution 360 Video Foveated Stitching for Real-time VR","volume":"36","author":"Lee","year":"2017","journal-title":"Comput. Graph. Forum"},{"key":"ref_24","unstructured":"Logitech (2020, January 28). Enable Every Room. Enable Every Person. Available online: https:\/\/www.logitech.com\/assets\/64494\/vc-whitepaper.pdf."},{"key":"ref_25","unstructured":"Kirbiz, D.S. (2020, January 28). TheRandomLab: Logitech c920 and c910 Fields of View for RGBDtoolkit. Available online: https:\/\/therandomlab.blogspot.com\/2013\/03\/logitech-c920-and-c910-fields-of-view.html."},{"key":"ref_26","unstructured":"Pagani, A., Gava, C., Cui, Y., Krolla, B., Hengen, J.M., and Stricker, D. (2011, January 18\u201321). Dense 3D Point Cloud Generation from Multiple High-Resolution Spherical Images. Proceedings of the 12th International Conference on Virtual Reality, Archaeology and Cultural Heritage, Prato, Italy."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_28","unstructured":"Muja, M., and Lowe, D.G. (2009, January 5\u20138). Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration. Proceedings of the Fourth International Conference on Computer Vision Theory and Applications (VISAPP \u201909), Lisboa, Portugal."},{"key":"ref_29","unstructured":"Hartley, R., and Zisserman, A. (2000). Multiple View Geometry in Computer Vision, Cambridge University Press. [2nd ed.]."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Angel, E., and Shreiner, D. (2012). Interactive Computer Graphics: A Top-Down Approach with Shader-Based OpenGL, Addison-Wesley.","DOI":"10.1201\/b12288-3"},{"key":"ref_31","unstructured":"Clark, R.N. (2020, January 28). Clarkvision Photography\u2014Resolution of the Human Eye. Available online: https:\/\/clarkvision.com\/imagedetail\/eye-resolution.html."},{"key":"ref_32","unstructured":"Bian, J., Yang, R., Liu, Y., Zhang, L., Cheng, M.M., Reid, I., and Wu, W. (2018). MatchBench: An Evaluation of Feature Matchers. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/s11263-014-0787-4","article-title":"SPHORB: A Fast and Robust Binary Feature on the Sphere","volume":"113","author":"Zhao","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_34","unstructured":"Fvdbergh (2020, March 20). MTF Mapper. Available online: https:\/\/sourceforge.net\/p\/mtfmapper\/home\/Home\/."},{"key":"ref_35","unstructured":"Fischer, R., Tadic-Galeb, B., and Yoder, P. (2008). Chapter Review of Specific Geometrical Aberrations and How to Get Rid of Them, McGraw-Hill Education."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Martins, H., and Ventura, R. (2009, January 22\u201325). Immersive 3-D teleoperation of a search and rescue robot using a head-mounted display. Proceedings of the ETFA 2009 IEEE Conference on Emerging Technologies and Factory Automation, Palma de Mallorca, Spain.","DOI":"10.1109\/ETFA.2009.5347014"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Piumsomboon, T., Lee, G.A., Irlitti, A., Ens, B., Thomas, B.H., and Billinghurst, M. (2019, January 4\u20139). On the shoulder of the giant: A multi-scale mixed reality collaboration with 360 video sharing and tangible interaction. Proceedings of the Conference on Human Factors in Computing Systems, Scotland, UK.","DOI":"10.1145\/3290605.3300458"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/8\/2264\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:09:14Z","timestamp":1760364554000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/8\/2264"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,16]]},"references-count":37,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["s20082264"],"URL":"https:\/\/doi.org\/10.3390\/s20082264","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,4,16]]}}}