{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T08:09:48Z","timestamp":1776326988000,"version":"3.50.1"},"reference-count":54,"publisher":"American Association for the Advancement of Science (AAAS)","issue":"90","content-domain":{"domain":["www.science.org"],"crossmark-restriction":true},"short-container-title":["Sci. Robot."],"published-print":{"date-parts":[[2024,5,15]]},"abstract":"<jats:p>Arthropods\u2019 eyes are effective biological vision systems for object tracking and wide field of view because of their structural uniqueness; however, unlike mammalian eyes, they can hardly acquire the depth information of a static object because of their monocular cues. Therefore, most arthropods rely on motion parallax to track the object in three-dimensional (3D) space. Uniquely, the praying mantis (Mantodea) uses both compound structured eyes and a form of stereopsis and is capable of achieving object recognition in 3D space. Here, by mimicking the vision system of the praying mantis using stereoscopically coupled artificial compound eyes, we demonstrated spatiotemporal object sensing and tracking in 3D space with a wide field of view. Furthermore, to achieve a fast response with minimal latency, data storage\/transportation, and power consumption, we processed the visual information at the edge of the system using a synaptic device and a federated split learning algorithm. The designed and fabricated stereoscopic artificial compound eye provides energy-efficient and accurate spatiotemporal object sensing and optical flow tracking. It exhibits a root mean square error of 0.3 centimeter, consuming only approximately 4 millijoules for sensing and tracking. These results are more than 400 times lower than conventional complementary metal-oxide semiconductor\u2013based imaging systems. Our biomimetic imager shows the potential of integrating nature\u2019s unique design using hardware and software codesigned technology toward capabilities of edge computing and sensing.<\/jats:p>","DOI":"10.1126\/scirobotics.adl3606","type":"journal-article","created":{"date-parts":[[2024,5,15]],"date-time":"2024-05-15T17:59:00Z","timestamp":1715795940000},"update-policy":"https:\/\/doi.org\/10.34133\/aaas_crossmark","source":"Crossref","is-referenced-by-count":30,"title":["Stereoscopic artificial compound eyes for spatiotemporal perception in three-dimensional space"],"prefix":"10.1126","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4305-3853","authenticated-orcid":true,"given":"Byungjoon","family":"Bae","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22904, USA."}]},{"given":"Doeon","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22904, USA."}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2923-1681","authenticated-orcid":true,"given":"Minseong","family":"Park","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22904, USA."}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-0364-6706","authenticated-orcid":true,"given":"Yujia","family":"Mu","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22904, USA."}]},{"given":"Yongmin","family":"Baek","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22904, USA."}]},{"given":"Inbo","family":"Sim","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22904, USA."}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3148-4453","authenticated-orcid":true,"given":"Cong","family":"Shen","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22904, USA."}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7123-3395","authenticated-orcid":true,"given":"Kyusang","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, VA 22904, USA."},{"name":"Department of Material Science and Engineering, University of Virginia, Charlottesville, VA 22904, USA."}]}],"member":"221","reference":[{"key":"e_1_3_2_2_2","unstructured":"Marshall Cavendish Corporation Insects and Spiders of the World (Marshall Cavendish 2003)."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1163\/18759866-06704001"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6633\/ab6a42"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"M. F. Land D.-E. Nilsson Animal Eyes (Oxford Univ. Press ed. 2 2012).","DOI":"10.1093\/acprof:oso\/9780199581139.001.0001"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2006.12.032"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1002\/adma.201305182"},{"key":"e_1_3_2_8_2","doi-asserted-by":"crossref","unstructured":"D. Floreano M. V. Srinivasan J. C. Zufferey C. Ellington Flying Insects and Robots (Springer 2010).","DOI":"10.1007\/978-3-540-89393-6"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0042-6989(99)00002-4"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1038\/srep18718"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2018.01.039"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-019-10721-z"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-007-0071-y"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"D. H. Warren E. R. Strelow Electronic Spatial Sensing for the Blind: Contributions from Perception Rehabilitation and Computer Vision D. H. Warren E. R. Strelow Eds. vol. 99 of NATO Science Series E: (Springer Netherlands 1985).","DOI":"10.1007\/978-94-017-1400-6"},{"key":"e_1_3_2_15_2","unstructured":"A. Burton J. Radford Thinking in Perspective: Critical Essays in the Study of Thought Processes vol. 66 of Psychology in Progress A. Burton J. Radford Eds. (Methuen 1978)."},{"key":"e_1_3_2_16_2","unstructured":"K. Blachut T. Kryjak M. Gorgon Hardware implementation of multi-scale Lucas-Kanade optical flow computation algorithm \u2013 A demo in Proceedings of the 2018 Conference on Design and Architectures for Signal and Image Processing (DASIP) (IEEE 2018) pp. 60\u201361."},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","unstructured":"C. H. Chang C. N. Chou E. Y. Chang CLKN: Cascaded Lucas-Kanade networks for image alignment in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE 2017) pp. 3777\u20133785.","DOI":"10.1109\/CVPR.2017.402"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.expthermflusci.2017.09.010"},{"key":"e_1_3_2_19_2","first-page":"45","article-title":"Activity-difference training of deep neural networks using memristor crossbars","volume":"6","author":"Yi S.","year":"2023","unstructured":"S. Yi, J. D. Kendall, R. S. Williams, S. Kumar, Activity-difference training of deep neural networks using memristor crossbars. Nat. Electron. 6, 45\u201351 (2023).","journal-title":"Nat. Electron."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2019.2946538"},{"key":"e_1_3_2_21_2","doi-asserted-by":"crossref","unstructured":"D. Li X. Chen M. Becchi Z. Zong Evaluating the energy efficiency of deep convolutional neural networks on CPUs and GPUs in 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud) Social Computing and Networking (SocialCom) Sustainable Computing and Communications (SustainCom) (BdCloud-SocialCom-SustainCom) (IEEE 2016) pp. 477\u2013484.","DOI":"10.1109\/BDCloud-SocialCom-SustainCom.2016.76"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2880694"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2014.07.006"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41565-023-01379-2"},{"key":"e_1_3_2_25_2","first-page":"202201905","article-title":"Hetero-integrated ingaas photodiode and oxide memristor-based artificial optical nerve for in-sensor nir image processing","volume":"11","author":"Bae B.","year":"2022","unstructured":"B. Bae, M. Park, D. Lee, I. Sim, K. Lee, Hetero-integrated ingaas photodiode and oxide memristor-based artificial optical nerve for in-sensor nir image processing. Adv. Opt. Mater. 11, 202201905 (2022).","journal-title":"Adv. Opt. Mater."},{"key":"e_1_3_2_26_2","article-title":"In-sensor image memorization and encoding via optical neurons for bio-stimulus domain reduction toward visual cognitive processing","volume":"13","author":"Lee D.","year":"2022","unstructured":"D. Lee, M. Park, Y. Baek, B. Bae, J. Heo, K. Lee, In-sensor image memorization and encoding via optical neurons for bio-stimulus domain reduction toward visual cognitive processing. Nat. Commun. 13, (2022).","journal-title":"Nat. Commun."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202100159"},{"key":"e_1_3_2_28_2","first-page":"8485","article-title":"SplitFed: When federated learning meets split learning","volume":"36","author":"Thapa C.","year":"2020","unstructured":"C. Thapa, M. A. P. Chamikara, S. Camtepe, SplitFed: When federated learning meets split learning. Proc. AAAI Conf. Artif. Intell. 36, 8485\u20138493 (2020).","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2022.3229443"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCCN.2022.3140788"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2022.3145823"},{"key":"e_1_3_2_32_2","first-page":"2366","article-title":"Depth map prediction from a single image using a multi-scale deep network","volume":"3","author":"Eigen D.","year":"2014","unstructured":"D. Eigen, C. Puhrsch, R. Fergus, Depth map prediction from a single image using a multi-scale deep network. Adv. Neural Inf. Process. Syst. 3, 2366\u20132374 (2014).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2012.10.002"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1021\/acsphotonics.6b00042"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms2553"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.1182383"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.2307\/1311112"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/0042-6989(75)90029-2"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1219068110"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-27575-z"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1002\/adfm.201400453"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.253.5021.776"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-19660-6"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1116\/1.5116021"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1364\/OE.17.006283"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.3390\/mi12050501"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.materresbull.2014.12.023"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.3390\/s21092984"},{"key":"e_1_3_2_49_2","unstructured":"B. Matas C. DeSubercasau Memory 1997: Complete Coverage of DRAM SRAM EPROM and Flash Memory ICs (Integrated Circuit Engineering Corporation 1997)."},{"key":"e_1_3_2_50_2","doi-asserted-by":"crossref","unstructured":"K. Kagawa S. Shishido M. Nunoshita J. Ohta A 3.6pW\/frame-pixel 1.35V PWM CMOS imager with dynamic pixel readout and no static bias current in 2008 IEEE International Solid-State Circuits Conference-Digest of Technical Papers (IEEE 2008) pp. 54\u201356.","DOI":"10.1109\/ISSCC.2008.4523053"},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","unstructured":"M. Y. Ng 0.18um low voltage 12-bit successive-approximation-register analog-to-digital converter (SAR ADC) in 2011 3rd Asia Symposium on Quality Electronic Design (ASQED) (IEEE 2011) pp. 277\u2013281.","DOI":"10.1109\/ASQED.2011.6111760"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1049\/el.2015.1924"},{"key":"e_1_3_2_53_2","unstructured":"Micron Technology Inc. Technical note: DDR3 point-to-point design support (2013)."},{"key":"e_1_3_2_54_2","doi-asserted-by":"crossref","unstructured":"A. Petreto A. Hennequin T. Koehler T. Romera Y. Fargeix B. Gaillard M. Bouyer Q. L. Meunier L. Lacassagne Energy and execution time comparison of optical flow algorithms on SIMD and GPU architectures in 2018 Conference on Design and Architectures for Signal and Image Processing (DASIP) (IEEE 2018) pp. 25\u201330.","DOI":"10.1109\/DASIP.2018.8597004"},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","unstructured":"G. Huang Z. Liu L. Van Der Maaten K. Q. Weinberger Densely connected convolutional networks in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE 2017) pp. 4700\u20134708.","DOI":"10.1109\/CVPR.2017.243"}],"container-title":["Science Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.science.org\/doi\/pdf\/10.1126\/scirobotics.adl3606","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T03:21:58Z","timestamp":1731986518000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.adl3606"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,15]]},"references-count":54,"journal-issue":{"issue":"90","published-print":{"date-parts":[[2024,5,15]]}},"alternative-id":["10.1126\/scirobotics.adl3606"],"URL":"https:\/\/doi.org\/10.1126\/scirobotics.adl3606","relation":{},"ISSN":["2470-9476"],"issn-type":[{"value":"2470-9476","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,15]]},"assertion":[{"value":"2023-10-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-17","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"eadl3606"}}