{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:51:45Z","timestamp":1760241105830,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2019,11,29]],"date-time":"2019-11-29T00:00:00Z","timestamp":1574985600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005073","name":"Agency for Defense Development","doi-asserted-by":"publisher","award":["UD190018ID"],"award-info":[{"award-number":["UD190018ID"]}],"id":[{"id":"10.13039\/501100005073","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Compound eyes, also known as insect eyes, have a unique structure. They have a hemispheric surface, and a lot of single eyes are deployed regularly on the surface. Thanks to this unique form, using the compound images has several advantages, such as a large field of view (FOV) with low aberrations. We can exploit these benefits in high-level vision applications, such as object recognition, or semantic segmentation for a moving robot, by emulating the compound images that describe the captured scenes from compound eye cameras. In this paper, to the best of our knowledge, we propose the first convolutional neural network (CNN)-based ego-motion classification algorithm designed for the compound eye structure. To achieve this, we introduce a voting-based approach that fully utilizes one of the unique features of compound images, specifically, the compound images consist of a lot of single eye images. The proposed method classifies a number of local motions by CNN, and these local classifications which represent the motions of each single eye image, are aggregated to the final classification by a voting procedure. For the experiments, we collected a new dataset for compound eye camera ego-motion classification which contains scenes of the inside and outside of a certain building. The samples of the proposed dataset consist of two consequent emulated compound images and the corresponding ego-motion class. The experimental results show that the proposed method has achieved the classification accuracy of 85.0%, which is superior compared to the baselines on the proposed dataset. Also, the proposed model is light-weight compared to the conventional CNN-based image recognition algorithms such as AlexNet, ResNet50, and MobileNetV2.<\/jats:p>","DOI":"10.3390\/s19235275","type":"journal-article","created":{"date-parts":[[2019,11,29]],"date-time":"2019-11-29T10:58:21Z","timestamp":1575025101000},"page":"5275","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Deep Ego-Motion Classifiers for Compound Eye Cameras"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3331-3172","authenticated-orcid":false,"given":"Hwiyeon","family":"Yoo","sequence":"first","affiliation":[{"name":"Department of Eletrical and Computer Engineering and ASRI, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 08826, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3008-4642","authenticated-orcid":false,"given":"Geonho","family":"Cha","sequence":"additional","affiliation":[{"name":"Department of Eletrical and Computer Engineering and ASRI, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 08826, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Songhwai","family":"Oh","sequence":"additional","affiliation":[{"name":"Department of Eletrical and Computer Engineering and ASRI, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul 08826, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,11,29]]},"reference":[{"key":"ref_1","unstructured":"Warrant, E., and Nilsson, D.E. (2006). Invertebrate Vision, Cambridge University Press."},{"key":"ref_2","unstructured":"Dudley, R. (2002). The Biomechanics of Insect Flight: Form, Function, Evolution, Princeton University Press."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Floreano, D., Zufferey, J.C., Srinivasan, M.V., and Ellington, C. (2010). Flying Insects and Robots, Springer.","DOI":"10.1007\/978-3-540-89393-6"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"R1","DOI":"10.1088\/1748-3182\/1\/1\/R01","article-title":"Micro-optical artificial compound eyes","volume":"1","author":"Wippermann","year":"2006","journal-title":"Bioinspir. Biomimetics"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1038\/nature12083","article-title":"Digital cameras with designs inspired by the arthropod eye","volume":"497","author":"Song","year":"2013","journal-title":"Nature"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1843","DOI":"10.1364\/AO.51.001843","article-title":"Design and fabrication of a freeform microlens array for a compact large-field-of-view compound-eye camera","volume":"51","author":"Li","year":"2012","journal-title":"Appl. Opt."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1038\/s41377-018-0081-2","article-title":"Xenos peckii vision inspires an ultrathin digital camera","volume":"7","author":"Keum","year":"2018","journal-title":"Light. Sci. Appl."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1719","DOI":"10.1364\/AO.43.001719","article-title":"Reconstruction of a high-resolution image on a compound-eye image-capturing system","volume":"43","author":"Kitamura","year":"2004","journal-title":"Appl. Opt."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yoo, H., Lee, D., Cha, G., and Oh, S. (2017, January 16\u201318). Estimating objectness using a compound eye camera. Proceedings of the IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Daegu, Korea.","DOI":"10.1109\/MFI.2017.8170418"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Cha, G., Yoo, H., Lee, D., and Oh, S. (2017, January 16\u201318). Light-weight semantic segmentation for compound images. Proceedings of the IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Daegu, Korea.","DOI":"10.1109\/MFI.2017.8170444"},{"key":"ref_11","unstructured":"Neumann, J., Fermuller, C., Aloimonos, Y., and Brajovic, V. (October, January 28). Compound eye sensor for 3D ego motion estimation. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Sendai, Japan."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Santamaria, M., and Trujillo, M. (2012, January 1\u20135). A comparison of block-matching motion estimation algorithms. Proceedings of the IEEE 7th Colombian Computing Congress (CCC), Medellin, Colombia.","DOI":"10.1109\/ColombianCC.2012.6398002"},{"key":"ref_13","unstructured":"Farneb\u00e4ck, G. (July, January 29). Two-frame motion estimation based on polynomial expansion. Proceedings of the Scandinavian conference on Image analysis (SCIA), Halmstad, Sweden."},{"key":"ref_14","unstructured":"Hirschmuller, H., Innocent, P.R., and Garibaldi, J.M. (2002, January 2\u20135). Fast, unconstrained camera motion estimation from stereo without tracking and robust statistics. Proceedings of the IEEE 7th International Conference on Control, Automation, Robotics and Vision (ICARCV), Singapore."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1109\/83.826785","article-title":"Efficient, robust, and fast global motion estimation for video coding","volume":"9","author":"Dufaux","year":"2000","journal-title":"IEEE Trans. Image Process."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Eichenseer, A., B\u00e4tz, M., and Kaup, A. (2016, January 25\u201328). Motion estimation for fisheye video sequences combining perspective projection with camera calibration information. Proceedings of the IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7533210"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"De Simone, F., Frossard, P., Birkbeck, N., and Adsumilli, B. (2017, January 16\u201318). Deformable block-based motion estimation in omnidirectional image sequences. Proceedings of the IEEE 19th International Workshop on Multimedia Signal Processing (MMSP), Luton, UK.","DOI":"10.1109\/MMSP.2017.8122254"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1007\/s11045-007-0022-3","article-title":"Super-resolution reconstruction in a computational compound-eye imaging system","volume":"18","author":"Chan","year":"2007","journal-title":"Multidimens. Syst. Signal Process."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1007\/s00422-006-0097-1","article-title":"Depth estimation using the compound eye of dipteran flies","volume":"95","author":"Bitsakos","year":"2006","journal-title":"Biol. Cybern."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/LRA.2015.2505717","article-title":"Exploring representation learning with cnns for frame-to-frame ego-motion estimation","volume":"1","author":"Costante","year":"2015","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., and Brox, T. (2017, January 21\u201326). Demon: Depth and motion network for learning monocular stereo. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.596"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Du, L., Jiang, W., Zhao, Z., and Su, F. (2017, January 19\u201321). Ego-motion classification for driving vehicle. Proceedings of the IEEE 3rd International Conference on Multimedia Big Data (BigMM), Laguna Hills, CA, USA.","DOI":"10.1109\/BigMM.2017.25"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ravanbakhsh, M., Nabi, M., Mousavi, H., Sangineto, E., and Sebe, N. (2018, January 12\u201315). Plug-and-play cnn for crowd motion analysis: An application in abnormal event detection. Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.","DOI":"10.1109\/WACV.2018.00188"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2545","DOI":"10.1109\/TMI.2019.2905917","article-title":"MTBI Identification From Diffusion MR Images Using Bag of Adversarial Visual Features","volume":"38","author":"Minaee","year":"2019","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wang, X., Chan, K.C., Yu, K., Dong, C., and Change Loy, C. (2019, January 16\u201320). Edvr: Video restoration with enhanced deformable convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00247"},{"key":"ref_26","unstructured":"Minaee, S., and Abdolrashidi, A. (2019). Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network. arXiv."},{"key":"ref_27","unstructured":"Bromley, J., Guyon, I., LeCun, Y., S\u00e4ckinger, E., and Shah, R. (December, January 28). Signature verification using a \u201csiamese\u201d time delay neural network. Proceedings of the Advances in Neural Information Processing Systems (NIPS), Denver, CO, USA."},{"key":"ref_28","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2\u20134). Tensorflow: A system for large-scale machine learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI), Savannah, GA, USA."},{"key":"ref_29","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.cviu.2004.03.004","article-title":"The geometric error for homographies","volume":"97","author":"Chum","year":"2005","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18\u201322). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ma, N., Zhang, X., Zheng, H.T., and Sun, J. (2018, January 8\u201314). Shufflenet v2: Practical guidelines for efficient cnn architecture design. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_8"},{"key":"ref_35","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems (NIPS), Lake Tahoe, NV, USA."},{"key":"ref_36","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the International Conference on Learning Representations (ICLR), San Diego, CA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/23\/5275\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:38:50Z","timestamp":1760189930000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/23\/5275"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,29]]},"references-count":36,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["s19235275"],"URL":"https:\/\/doi.org\/10.3390\/s19235275","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,11,29]]}}}