{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T15:17:27Z","timestamp":1772119047197,"version":"3.50.1"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T00:00:00Z","timestamp":1713830400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T00:00:00Z","timestamp":1713830400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["100-241-945"],"award-info":[{"award-number":["100-241-945"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Machine Vision and Applications"],"published-print":{"date-parts":[[2024,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Stereo vision is a key technology for 3D scene reconstruction from image pairs. Most approaches process perspective images from commodity cameras. These images, however, have a very limited field of view and only picture a small portion of the scene. In contrast, omnidirectional images, also known as fisheye images, exhibit a much larger field of view and allow a full 3D scene reconstruction with a small amount of cameras if placed carefully. However, omnidirectional images are strongly distorted which make the 3D reconstruction much more sophisticated. Nowadays, a lot of research is conducted on CNNs for omnidirectional stereo vision. Nevertheless, a significant gap between estimation accuracy and throughput can be observed in the literature. This work aims to bridge this gap by introducing a novel set of transformations, namely\n                    <jats:italic>OmniGlasses<\/jats:italic>\n                    . These are incorporated into the architecture of a fast network, i.e.,\n                    <jats:italic>AnyNet<\/jats:italic>\n                    , originally designed for scene reconstruction on perspective images. Our network,\n                    <jats:italic>Omni-AnyNet<\/jats:italic>\n                    , produces accurate omnidirectional distance maps with a mean absolute error of around 13 cm at 48.4 fps and is therefore real-time capable.\n                  <\/jats:p>","DOI":"10.1007\/s00138-024-01534-2","type":"journal-article","created":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T10:01:27Z","timestamp":1713866487000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["OmniGlasses: an optical aid for stereo vision CNNs to enable omnidirectional image processing"],"prefix":"10.1007","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0636-3385","authenticated-orcid":false,"given":"Julian B.","family":"Seuffert","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1171-903X","authenticated-orcid":false,"given":"Ana C.","family":"Perez\u00a0Grassi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6038-3328","authenticated-orcid":false,"given":"Hamza","family":"Ahmed","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3144-1488","authenticated-orcid":false,"given":"Roman","family":"Seidel","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4393-5354","authenticated-orcid":false,"given":"Gangolf","family":"Hirtz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,4,23]]},"reference":[{"key":"1534_CR1","doi-asserted-by":"publisher","unstructured":"Findeisen, M., Hirtz, G.: Trinocular spherical stereo vision for indoor surveillance. In: 2014 Canadian Conference on Computer and Robot Vision, pp. 364\u2013370. IEEE, New York (2014). https:\/\/doi.org\/10.1109\/CRV.2014.56","DOI":"10.1109\/CRV.2014.56"},{"key":"1534_CR2","doi-asserted-by":"publisher","unstructured":"Findeisen, M., Meinel, L., Hirtz, G.: A trinocular omnidirectional stereo vision system for high-precision RGB-D acquisition. In: Proceedings ELMAR-2014, pp. 1\u20134. IEEE, New York (2014). https:\/\/doi.org\/10.1109\/ELMAR.2014.6923350","DOI":"10.1109\/ELMAR.2014.6923350"},{"key":"1534_CR3","doi-asserted-by":"publisher","unstructured":"Findeisen, M., Meinel, L., Richter, J., Hirtz, G.: An omnidirectional stereo sensor for human behavior analysis in complex indoor environments. In: 2015 IEEE international conference on consumer electronics (ICCE), pp. 17\u201319. IEEE, New York (2015). https:\/\/doi.org\/10.1109\/ICCE.2015.7066302","DOI":"10.1109\/ICCE.2015.7066302"},{"key":"1534_CR4","doi-asserted-by":"publisher","unstructured":"Wang, Y., Lai, Z., Huang, G., Wang, B.H., van\u00a0der Maaten, L., Campbell, M., Weinberger, K.Q.: Anytime stereo image depth estimation on mobile devices. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 5893\u20135900. IEEE, New York (2019). https:\/\/doi.org\/10.1109\/ICRA.2019.8794003","DOI":"10.1109\/ICRA.2019.8794003"},{"key":"1534_CR5","doi-asserted-by":"publisher","unstructured":"Won, C., Ryu, J., Lim, J.: SweepNet: wide-baseline omnidirectional depth estimation. In: 2019 International Conference on Robotics and Automation (ICRA), pp. 6073\u20136079. IEEE, New York (2019). https:\/\/doi.org\/10.1109\/ICRA.2019.8793823","DOI":"10.1109\/ICRA.2019.8793823"},{"key":"1534_CR6","doi-asserted-by":"publisher","unstructured":"Won, C., Ryu, J., Lim, J.: OmniMVS: end-to-end learning for omnidirectional stereo matching. In: 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), pp. 8986\u20138995. IEEE, New York (2019). https:\/\/doi.org\/10.1109\/ICCV.2019.00908","DOI":"10.1109\/ICCV.2019.00908"},{"issue":"11","key":"1534_CR7","doi-asserted-by":"publisher","first-page":"3850","DOI":"10.1109\/TPAMI.2020.2992497","volume":"43","author":"C Won","year":"2021","unstructured":"Won, C., Ryu, J., Lim, J.: End-to-end learning for omnidirectional stereo matching with uncertainty prior. IEEE Trans. Pattern Anal. Mach. Intell. 43(11), 3850\u20133862 (2021). https:\/\/doi.org\/10.1109\/TPAMI.2020.2992497","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"1534_CR8","doi-asserted-by":"publisher","unstructured":"Seuffert, J.B., Perez\u00a0Grassi, A.C., Scheck, T., Hirtz, G.: A Study on the Influence of Omnidirectional Distortion on CNN-based Stereo Vision. In: Proceedings of the 16th international joint conference on computer vision, imaging and computer graphics theory and applications, VISIGRAPP 2021, Volume 5: VISAPP, pp. 809\u2013816. SciTePress, Set\u00fabal (2021). https:\/\/doi.org\/10.5220\/0010324808090816","DOI":"10.5220\/0010324808090816"},{"key":"1534_CR9","doi-asserted-by":"publisher","unstructured":"Hirschm\u00fcller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), vol. 2, pp. 807\u2013814. IEEE, New York (2005). https:\/\/doi.org\/10.1109\/CVPR.2005.56","DOI":"10.1109\/CVPR.2005.56"},{"key":"1534_CR10","doi-asserted-by":"publisher","unstructured":"Wang, N.-H., Solarte, B., Tsai, Y.-H., Chiu, W.-C., Sun, M.: 360SD-Net: 360$${}^\\circ $$ Stereo depth estimation with learnable cost volume. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 582\u2013588. IEEE, New York (2020). https:\/\/doi.org\/10.1109\/ICRA40945.2020.9196975","DOI":"10.1109\/ICRA40945.2020.9196975"},{"key":"1534_CR11","doi-asserted-by":"publisher","unstructured":"Komatsu, R., Fujii, H., Tamura, Y., Yamashita, A., Asama, H.: 360$${}^\\circ $$ Depth estimation from multiple fisheye images with origami crown representation of icosahedron. In: 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10092\u201310099. IEEE, New York (2020). https:\/\/doi.org\/10.1109\/IROS45743.2020.9340981","DOI":"10.1109\/IROS45743.2020.9340981"},{"issue":"24","key":"1534_CR12","doi-asserted-by":"publisher","first-page":"85","DOI":"10.3390\/app10248851","volume":"10","author":"D-M C\u00f3rdova-Esparza","year":"2020","unstructured":"C\u00f3rdova-Esparza, D.-M., Terven, J., Romero-Gonz\u00e1lez, J.-A., Ram\u00edrez-Pedraza, A.: Three-dimensional reconstruction of indoor and outdoor environments using a stereo catadioptric system. Appl. Sci 10(24), 85 (2020). https:\/\/doi.org\/10.3390\/app10248851","journal-title":"Appl. Sci"},{"issue":"3","key":"1534_CR13","doi-asserted-by":"publisher","first-page":"300","DOI":"10.1007\/s11263-016-0908-3","volume":"120","author":"J Revaud","year":"2016","unstructured":"Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: DeepMatching: hierarchical deformable dense matching. Int. J. Comput. Vision 120(3), 300\u2013323 (2016). https:\/\/doi.org\/10.1007\/s11263-016-0908-3","journal-title":"Int. J. Comput. Vision"},{"key":"1534_CR14","doi-asserted-by":"publisher","unstructured":"Lee, J., Park, D., Lee, D., Ji, D.: Semi-supervised 360$${}^\\circ $$ depth estimation from multiple fisheye cameras with pixel-level selective loss. In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2290\u20132294. IEEE, New York (2022). https:\/\/doi.org\/10.1109\/ICASSP43922.2022.9746232","DOI":"10.1109\/ICASSP43922.2022.9746232"},{"key":"1534_CR15","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1016\/j.imavis.2021.104264","volume":"114","author":"M Li","year":"2021","unstructured":"Li, M., Hu, X., Dai, J., Li, Y., Du, S.: Omnidirectional stereo depth estimation based on spherical deep network. Image Vis. Comput, 114, 85 (2021). https:\/\/doi.org\/10.1016\/j.imavis.2021.104264","journal-title":"Image Vis. Comput,"},{"key":"1534_CR16","doi-asserted-by":"publisher","unstructured":"Meuleman, A., Jang, H., Jeon, D.S., Kim, M.H.: Real-time sphere sweeping stereo from multiview fisheye images. In: 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11418\u201311427. IEEE, New York (2021). https:\/\/doi.org\/10.1109\/CVPR46437.2021.01126","DOI":"10.1109\/CVPR46437.2021.01126"},{"key":"1534_CR17","doi-asserted-by":"publisher","unstructured":"Konolige, K.: Small vision systems: hardware and implementation. In: Robotics Research - The Eighth International Symposium, pp. 203\u2013212. Springer, London (1998). https:\/\/doi.org\/10.1007\/978-1-4471-1580-9_19","DOI":"10.1007\/978-1-4471-1580-9_19"},{"key":"1534_CR18","doi-asserted-by":"publisher","unstructured":"Tonioni, A., Tosi, F., Poggi, M., Mattoccia, S., Stefano, L.D.: Real-time self-adaptive deep stereo. In: 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 195\u2013204. IEEE, New York (2019). https:\/\/doi.org\/10.1109\/CVPR.2019.00028","DOI":"10.1109\/CVPR.2019.00028"},{"key":"1534_CR19","doi-asserted-by":"publisher","unstructured":"Kendall, A., Martirosyan, H., Dasgupta, S., Henry, P., Kennedy, R., Bachrach, A., Bry, A.: End-to-end learning of geometry and context for deep stereo regression. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 66\u201375. IEEE, New York (2017). https:\/\/doi.org\/10.1109\/ICCV.2017.17","DOI":"10.1109\/ICCV.2017.17"},{"key":"1534_CR20","doi-asserted-by":"publisher","unstructured":"Lipson, L., Teed, Z., Deng, J.: RAFT-stereo: multilevel recurrent field transforms for stereo matching. In: 2021 International Conference on 3D Vision (3DV), pp. 218\u2013227. IEEE, New York (2021). https:\/\/doi.org\/10.1109\/3DV53792.2021.00032","DOI":"10.1109\/3DV53792.2021.00032"},{"key":"1534_CR21","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1007\/978-3-319-24574-4_28","volume-title":"Medical image computing and computer-assisted intervention - MICCAI 2015","author":"O Ronneberger","year":"2015","unstructured":"Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical image computing and computer-assisted intervention - MICCAI 2015. Lecture Notes in Computer Science, vol. 9351, pp. 234\u2013241. Springer, Cham (2015). https:\/\/doi.org\/10.1007\/978-3-319-24574-4_28"},{"key":"1534_CR22","doi-asserted-by":"publisher","unstructured":"Li, S.: Real-time spherical stereo. In: 18th International Conference on Pattern Recognition (ICPR\u201906), vol. 3, pp. 1046\u20131049. IEEE, New York (2006). https:\/\/doi.org\/10.1109\/ICPR.2006.968","DOI":"10.1109\/ICPR.2006.968"},{"key":"1534_CR23","doi-asserted-by":"publisher","unstructured":"Li, S.: Trinocular spherical stereo. In: 2006 IEEE\/RSJ International Conference on Intelligent Robots and Systems, pp. 4786\u20134791. IEEE, New York (2006). https:\/\/doi.org\/10.1109\/IROS.2006.282350","DOI":"10.1109\/IROS.2006.282350"},{"issue":"8","key":"1534_CR24","doi-asserted-by":"publisher","first-page":"1335","DOI":"10.1109\/TPAMI.2006.153","volume":"28","author":"J Kannala","year":"2006","unstructured":"Kannala, J., Brandt, S.S.: A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Trans. Pattern Anal. Mach. Intell. 28(8), 1335\u20131340 (2006). https:\/\/doi.org\/10.1109\/TPAMI.2006.153","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"1534_CR25","unstructured":"Liu, S., Mello, S.D., Gu, J., Zhong, G., Yang, M.-H., Kautz, J.: Learning affinity via spatial propagation networks. In: Guyon, I., Luxburg, U.v., Bengio, S., Wallach, H.M., Fergus, R., Vishwanathan, S.V.N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, vol. 30, pp. 1520\u20131530. Curran Associates, Inc., Red Hook (2017)"},{"key":"1534_CR26","doi-asserted-by":"publisher","unstructured":"Scheck, T., Seidel, R., Hirtz, G.: Learning from THEODORE: a synthetic omnidirectional top-view indoor dataset for deep transfer learning. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), Snowmass Village, Colorado, USA, pp. 932\u2013941 (2020). https:\/\/doi.org\/10.1109\/WACV45572.2020.9093563","DOI":"10.1109\/WACV45572.2020.9093563"},{"key":"1534_CR27","doi-asserted-by":"crossref","unstructured":"Bourke, P.: iDome: Immersive gaming with the Unity game engine. In: CGAT 09: computer games. Multimedia and Allied Technology 09: Proceedings, pp. 265\u2013272. Research Publishing Services, Singapore (2009)","DOI":"10.1037\/e602482011-012"},{"key":"1534_CR28","doi-asserted-by":"crossref","unstructured":"David Bourke, P., Quintanilha Felinto, D.: Blender and immersive gaming in a hemispherical dome. GSTF Int J Comput 1(1), 280\u2013284 (2010)","DOI":"10.5176\/2010-2283_1.1.05"},{"key":"1534_CR29","doi-asserted-by":"publisher","unstructured":"Scharstein, D., Szeliski, R., Zabih, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. In: Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001), pp. 131\u2013140. IEEE, New York (2001). https:\/\/doi.org\/10.1109\/SMBV.2001.988771","DOI":"10.1109\/SMBV.2001.988771"},{"key":"1534_CR30","doi-asserted-by":"publisher","first-page":"427","DOI":"10.5194\/isprsannals-II-3-W5-427-2015","volume":"2","author":"M Menze","year":"2015","unstructured":"Menze, M., Heipke, C., Geiger, A.: Joint 3D estimation of vehicles and scene flow. ISPRS Ann. Photogr. Remote Sens. Spat. Inf. Sci. 2, 427\u2013434 (2015). https:\/\/doi.org\/10.5194\/isprsannals-II-3-W5-427-2015","journal-title":"ISPRS Ann. Photogr. Remote Sens. Spat. Inf. Sci."},{"key":"1534_CR31","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/j.isprsjprs.2017.09.013","volume":"140","author":"M Menze","year":"2018","unstructured":"Menze, M., Heipke, C., Geiger, A.: Object scene flow. ISPRS J. Photogramm. Remote. Sens. 140, 60\u201376 (2018). https:\/\/doi.org\/10.1016\/j.isprsjprs.2017.09.013","journal-title":"ISPRS J. Photogramm. Remote. Sens."},{"key":"1534_CR32","doi-asserted-by":"publisher","unstructured":"Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, Conference Track Proceedings, San Diego, CA, USA (2015). https:\/\/doi.org\/10.48550\/arXiv.1412.6980","DOI":"10.48550\/arXiv.1412.6980"},{"key":"1534_CR33","doi-asserted-by":"publisher","unstructured":"Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with warm restarts. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings (2017). https:\/\/doi.org\/10.48550\/arXiv.1608.03983","DOI":"10.48550\/arXiv.1608.03983"}],"container-title":["Machine Vision and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00138-024-01534-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00138-024-01534-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00138-024-01534-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,14]],"date-time":"2024-05-14T00:23:33Z","timestamp":1715646213000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00138-024-01534-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,23]]},"references-count":33,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,5]]}},"alternative-id":["1534"],"URL":"https:\/\/doi.org\/10.1007\/s00138-024-01534-2","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-2776786\/v1","asserted-by":"object"}]},"ISSN":["0932-8092","1432-1769"],"issn-type":[{"value":"0932-8092","type":"print"},{"value":"1432-1769","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,23]]},"assertion":[{"value":"4 April 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 January 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 March 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 April 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose. The authors have no Conflict of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"58"}}