{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T14:51:09Z","timestamp":1777128669249,"version":"3.51.4"},"reference-count":48,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2025,7,23]],"date-time":"2025-07-23T00:00:00Z","timestamp":1753228800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,7,23]],"date-time":"2025-07-23T00:00:00Z","timestamp":1753228800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"National Science Foundation","award":["CCF2319617"],"award-info":[{"award-number":["CCF2319617"]}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2025,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The ability to predict motion in real time is fundamental to many maneuvering activities in animals, particularly those critical for survival, such as attack and escape responses. Given its significance, it is no surprise that motion prediction (MP) in animals begins in the retina. Similarly, autonomous systems utilizing computer vision could greatly benefit from the capability to predict motion in real time. Therefore, for computer vision applications, MP should be integrated directly at the camera pixel level. Towards that end, we present a retina-inspired neuromorphic framework capable of performing real-time, energy-efficient MP directly within camera pixels. Our hardware-algorithm framework, implemented using GlobalFoundries\u2019 22\u2009nm FDSOI technology, integrates key retinal MP compute blocks, including a biphasic filter, spike adder, non-linear circuit, and a 2D array for multi-directional MP. Additionally, integrating the sensor and MP compute die using a 3D Cu\u2013Cu hybrid bonding approach improves design compactness by minimizing area usage and simplifying routing complexity. Validated on real-world object stimuli, the model delivers efficient, low-latency MP for decision-making scenarios reliant on predictive visual computation, while consuming only 18.56\u2009pJ\/MP in our mixed-signal hardware implementation.<\/jats:p>","DOI":"10.1088\/2634-4386\/adef76","type":"journal-article","created":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T22:51:06Z","timestamp":1752533466000},"page":"034005","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["A retina-inspired pathway to real-time motion prediction inside image sensors for extreme-edge intelligence"],"prefix":"10.1088","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-3325-9099","authenticated-orcid":true,"given":"Subhradip","family":"Chakraborty","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3369-3478","authenticated-orcid":true,"given":"Shay","family":"Snyder","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Md","family":"Abdullah-Al Kaiser","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maryam","family":"Parsa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gregory","family":"Schwartz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Akhilesh R","family":"Jaiswal","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2025,7,23]]},"reference":[{"key":"nceadef76bib1","doi-asserted-by":"publisher","first-page":"R319","DOI":"10.1016\/j.cub.2005.04.041","article-title":"The optical structures of animal eyes","volume":"15","author":"Land","year":"2005","journal-title":"Curr. Biol."},{"key":"nceadef76bib2","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1017\/S0952523810000209","article-title":"Multiple pathways of inhibition shape bipolar cell responses in the retina","volume":"28","author":"Eggers","year":"2011","journal-title":"Vis. Neurosci."},{"key":"nceadef76bib3","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1109\/MCD.2005.el2005cmos","article-title":"CMOS image sensors","volume":"21","author":"Gamal","year":"2005","journal-title":"IEEE Circuits Devices Mag."},{"key":"nceadef76bib4","author":"Schwartz","year":"2021"},{"key":"nceadef76bib5","doi-asserted-by":"publisher","first-page":"320","DOI":"10.1016\/j.neuron.2016.03.012","article-title":"Nonlinear spatiotemporal integration by electrical and chemical synapses in the retina","volume":"90","author":"Kuo","year":"2016","journal-title":"Neuron"},{"key":"nceadef76bib6","doi-asserted-by":"publisher","first-page":"334","DOI":"10.1038\/18678","article-title":"Anticipation of moving stimuli by the retina","volume":"398","author":"Berry","year":"1999","journal-title":"Nature"},{"key":"nceadef76bib7","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1038\/nn.3308","article-title":"Lag normalization in an electrically coupled neural network","volume":"16","author":"Trenholm","year":"2013","journal-title":"Nat. Neurosci."},{"key":"nceadef76bib8","doi-asserted-by":"publisher","first-page":"2187","DOI":"10.1109\/JSSC.2007.905295","article-title":"CMOS camera with in-pixel temporal change detection and ADC","volume":"42","author":"Chi","year":"2007","journal-title":"IEEE J. Solid-State Circuits"},{"key":"nceadef76bib9","first-page":"pp 1373","article-title":"Silicon retina for motion computation","volume":"vol 3","author":"Andreou","year":"1991"},{"key":"nceadef76bib10","first-page":"pp 1","article-title":"A CMOS image sensor with on-chip motion detection and object localization","author":"Zhao","year":"2011"},{"key":"nceadef76bib11","first-page":"pp 2416","article-title":"A low power multi-mode CMOS image sensor with integrated on-chip motion detection","author":"Liu","year":"2013"},{"key":"nceadef76bib12","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1016\/0893-6080(88)90024-X","article-title":"A silicon model of early visual processing","volume":"1","author":"Mead","year":"1988","journal-title":"Neural Netw."},{"key":"nceadef76bib13","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1016\/j.conb.2010.03.007","article-title":"Neuromorphic sensory systems","volume":"20","author":"Liu","year":"2010","journal-title":"Curr. Opin. Neurobiol."},{"key":"nceadef76bib14","doi-asserted-by":"publisher","DOI":"10.1063\/5.0016485","article-title":"Technologies toward next generation human machine interfaces: From machine learning enhanced tactile sensing to neuromorphic sensory systems","volume":"7","author":"Zhu","year":"2020","journal-title":"Appl. Phys. Rev."},{"key":"nceadef76bib15","first-page":"pp 89","article-title":"A neuromorphic circuit that computes differential motion","author":"Tseng","year":"2012"},{"key":"nceadef76bib16","doi-asserted-by":"publisher","first-page":"6261","DOI":"10.1038\/s41467-024-50488-6","article-title":"Adaptative machine vision with microsecond-level accurate perception beyond human retina","volume":"15","author":"Li","year":"2024","journal-title":"Nat. Commun."},{"key":"nceadef76bib17","doi-asserted-by":"publisher","first-page":"8650","DOI":"10.1038\/s41467-024-52982-3","article-title":"Multifunctional human visual pathway-replicated hardware based on 2d materials","volume":"15","author":"Peng","year":"2024","journal-title":"Nat. Commun."},{"key":"nceadef76bib18","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1177\/00368504241275372","article-title":"Design and simulation of artificial retinal stimulation ic with switched capacitor using si nanowire optical properties","volume":"107","author":"Han","year":"2024","journal-title":"Sci. Prog."},{"key":"nceadef76bib19","doi-asserted-by":"publisher","first-page":"2384","DOI":"10.1162\/neco_a_01112","article-title":"Spiking elementary motion detector in neuromorphic systems","volume":"30","author":"Milde","year":"2018","journal-title":"Neural Comput."},{"key":"nceadef76bib20","doi-asserted-by":"publisher","first-page":"1168","DOI":"10.1038\/81435","article-title":"Models of motion detection","volume":"3","author":"Borst","year":"2000","journal-title":"Nat. Neurosci."},{"key":"nceadef76bib21","doi-asserted-by":"publisher","first-page":"2020","DOI":"10.3389\/fnins.2020.00451","article-title":"Event-based eccentric motion detection exploiting time difference encoding","volume":"14","author":"D\u2019Angelo","year":"2020","journal-title":"Front. Neurosci."},{"key":"nceadef76bib22","article-title":"Retina-inspired object motion segmentation","author":"Clerico","year":"2024"},{"key":"nceadef76bib23","first-page":"pp 256","article-title":"Hardware-algorithm re-engineering of retinal circuit for intelligent object motion segmentation","author":"Sinaga","year":"2024"},{"key":"nceadef76bib24","article-title":"Object motion sensitivity: A bio-inspired solution to the ego-motion problem for event-based cameras","author":"Snyder","year":"2023"},{"key":"nceadef76bib25","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2023.1241691","article-title":"Iris: Integrated retinal functionality in image sensors","volume":"17","author":"Yin","year":"2023","journal-title":"Front. Neurosci."},{"key":"nceadef76bib26","first-page":"pp 2636","article-title":"Bdd100k: A diverse driving dataset for heterogeneous multitask learning","author":"Yu","year":"2020"},{"key":"nceadef76bib27","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2024.106502","article-title":"A computationally efficient and robust looming perception model based on dynamic neural field","volume":"179","author":"Qin","year":"2024","journal-title":"Neural Netw."},{"key":"nceadef76bib28","doi-asserted-by":"publisher","first-page":"5177","DOI":"10.1523\/JNEUROSCI.0471-20.2020","article-title":"Mouse retinal cell atlas: Molecular identification of over sixty amacrine cell types","volume":"40","author":"Yan","year":"2020","journal-title":"J. Neurosci."},{"key":"nceadef76bib29","doi-asserted-by":"publisher","DOI":"10.1016\/j.celrep.2022.111040","article-title":"Unified classification of mouse retinal ganglion cells using function, morphology and gene expression","volume":"40","author":"Goetz","year":"2022","journal-title":"Cell Rep."},{"key":"nceadef76bib30","first-page":"pp 246","article-title":"Chapter 13 - motion anticipation","author":"Schwartz","year":"2021"},{"key":"nceadef76bib31","doi-asserted-by":"publisher","first-page":"3311","DOI":"10.1152\/jn.01168.2005","article-title":"Electrical stimulation of mammalian retinal ganglion cells with multielectrode arrays","volume":"95","author":"Sekirnjak","year":"2006","journal-title":"J. Neurophysiol."},{"key":"nceadef76bib32","article-title":"Pytorch: An imperative style, high-performance deep learning library","author":"Paszke","year":"2019"},{"key":"nceadef76bib33","doi-asserted-by":"publisher","first-page":"566","DOI":"10.1109\/JSSC.2007.914337","article-title":"A 128\u00d7 128 120 db 15 \u00b5s latency asynchronous temporal contrast vision sensor","volume":"43","author":"Lichtsteiner","year":"2008","journal-title":"IEEE J. Solid-State Circuits"},{"key":"nceadef76bib34","doi-asserted-by":"publisher","first-page":"827","DOI":"10.1109\/JSSC.2012.2230553","article-title":"A 128 \u00d7 128 1.5% contrast sensitivity 0.9% fpn 3 \u00b5s latency 4 mw asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers","volume":"48","author":"Serrano-Gotarredona","year":"2013","journal-title":"IEEE J. Solid-State Circuits"},{"key":"nceadef76bib35","first-page":"pp 66","article-title":"4.1 a 640\u00d7 480 dynamic vision sensor with a 9\u00b5m pixel and 300meps address-event representation","author":"Son","year":"2017"},{"key":"nceadef76bib36","first-page":"pp 323","article-title":"22nm fd-soi technology with back-biasing capability offers excellent performance for enabling efficient, ultra-low power analog and rf\/millimeter-wave designs","author":"Ong","year":"2019"},{"key":"nceadef76bib37","doi-asserted-by":"publisher","first-page":"416","DOI":"10.1109\/82.842110","article-title":"Point-to-point connectivity between neuromorphic chips using address events","volume":"47","author":"Boahen","year":"2000","journal-title":"IEEE Trans. Circuits Syst. II"},{"key":"nceadef76bib38","doi-asserted-by":"publisher","first-page":"1269","DOI":"10.1109\/TCSI.2004.830703","article-title":"A burst-mode word-serial address-event link-i: transmitter design","volume":"51","author":"Boahen","year":"2004","journal-title":"IEEE Trans. Circuits Syst. I: Regular Pap."},{"key":"nceadef76bib39","first-page":"pp 8","article-title":"Novel stacked CMOS image sensor with advanced cu2cu hybrid bonding","author":"Kagawa","year":"2016"},{"key":"nceadef76bib40","first-page":"pp 1","article-title":"A 2.6 e-rms low-random-noise, 116.2 mw low-power 2-mp global shutter CMOS image sensor with pixel-level adc and in-pixel memory","author":"Seo","year":"2021"},{"key":"nceadef76bib41","first-page":"pp 148","article-title":"Impacts of misalignment on 1 \u00b5m pitch Cu-Cu hybrid bonding","author":"Kagawa","year":"2020"},{"key":"nceadef76bib42","first-page":"pp 613","article-title":"Technology-circuit-algorithm tri-design for processing-in-pixel-in-memory (p2m)","author":"Kaiser","year":"2023"},{"key":"nceadef76bib43","article-title":"Questa onesim: one simulator for your complete verification flow","author":"Siemens","year":"2024"},{"key":"nceadef76bib44","article-title":"Synopsys design compiler","author":"Compiler","year":"2016"},{"key":"nceadef76bib45","first-page":"pp 201","article-title":"Design of 1.8 v lvds transmitter in gf 22nm for associative memory","author":"Teja","year":"2021"},{"key":"nceadef76bib46","doi-asserted-by":"publisher","first-page":"5460","DOI":"10.1109\/TED.2018.2876688","article-title":"Benchmarking digital die-to-die channels in 2.5-D and 3-D heterogeneous integration platforms","volume":"65","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Electron Devices"},{"key":"nceadef76bib47","doi-asserted-by":"publisher","first-page":"5072","DOI":"10.1109\/TCSI.2021.3111202","article-title":"An efficient digital realization of retinal light adaptation in cone photoreceptors","volume":"68","author":"Ghanbarpour","year":"2021","journal-title":"IEEE Trans. Circuits Syst. I: Regular Pap."},{"key":"nceadef76bib48","article-title":"Neuromorphic retina: an fpga-based emulator","author":"Phillip","year":"2025"}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/adef76","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/adef76\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/adef76","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/adef76\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/adef76\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/adef76\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/adef76\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/adef76\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,23]],"date-time":"2025-07-23T09:06:10Z","timestamp":1753261570000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/adef76"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,23]]},"references-count":48,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,7,23]]},"published-print":{"date-parts":[[2025,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/adef76","relation":{},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,23]]},"assertion":[{"value":"A retina-inspired pathway to real-time motion prediction inside image sensors for extreme-edge intelligence","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2025 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2025-04-02","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-07-14","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-07-23","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}