{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T20:32:02Z","timestamp":1772137922225,"version":"3.50.1"},"reference-count":45,"publisher":"IOP Publishing","issue":"2","license":[{"start":{"date-parts":[[2023,6,12]],"date-time":"2023-06-12T00:00:00Z","timestamp":1686528000000},"content-version":"vor","delay-in-days":11,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,12]],"date-time":"2023-06-12T00:00:00Z","timestamp":1686528000000},"content-version":"tdm","delay-in-days":11,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"Ubbo Emmius Fund"},{"DOI":"10.13039\/501100020535","name":"Cluster of Excellence Cognitive Interaction Technology","doi-asserted-by":"crossref","award":["EXC 277"],"award-info":[{"award-number":["EXC 277"]}],"id":[{"id":"10.13039\/501100020535","id-type":"DOI","asserted-by":"crossref"}]},{"name":"MINDROB","award":["PID2019-105556GB-C33\/AEI\/10.13039\/501100011033"],"award-info":[{"award-number":["PID2019-105556GB-C33\/AEI\/10.13039\/501100011033"]}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2023,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Sound source localization (SSL) is used in various applications such as industrial noise-control, speech detection in mobile phones, speech enhancement in hearing aids and many more. Newest video conferencing setups use SSL. The position of a speaker is detected from the difference in the audio waves received by a microphone array. After detection the camera focuses onto the location of the speaker. The human brain is also able to detect the location of a speaker from auditory signals. It uses, among other cues, the difference in amplitude and arrival time of the sound wave at the two ears, called interaural level and time difference. However, the substrate and computational primitives of our brain are different from classical digital computing. Due to its low power consumption of around 20 W and its performance in real time the human brain has become a great source of inspiration for emerging technologies. One of these technologies is neuromorphic hardware which implements the fundamental principles of brain computing identified until today using complementary metal-oxide-semiconductor technologies and new devices. In this work we propose the first neuromorphic closed-loop robotic system that uses the interaural time difference for SSL in real time. Our system can successfully locate sound sources such as human speech. In a closed-loop experiment, the robotic platform turned immediately into the direction of the sound source with a turning velocity linearly proportional to the angle difference between sound source and binaural microphones. After this initial turn, the robotic platform remains at the direction of the sound source. Even though the system only uses very few resources of the available hardware, consumes around 1 W, and was only tuned by hand, meaning it does not contain any learning at all, it already reaches performances comparable to other neuromorphic approaches. The SSL system presented in this article brings us one step closer towards neuromorphic event-based systems for robotics and embodied computing.<\/jats:p>","DOI":"10.1088\/2634-4386\/acdaba","type":"journal-article","created":{"date-parts":[[2023,6,1]],"date-time":"2023-06-01T18:27:20Z","timestamp":1685644040000},"page":"024009","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Closed-loop sound source localization in neuromorphic systems"],"prefix":"10.1088","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7048-7358","authenticated-orcid":true,"given":"Thorben","family":"Schoepe","sequence":"first","affiliation":[]},{"given":"Daniel","family":"Gutierrez-Galan","sequence":"additional","affiliation":[]},{"given":"Juan P","family":"Dominguez-Morales","sequence":"additional","affiliation":[]},{"given":"Hugh","family":"Greatorex","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3061-5922","authenticated-orcid":true,"given":"Angel","family":"Jimenez-Fernandez","sequence":"additional","affiliation":[]},{"given":"Alejandro","family":"Linares-Barranco","sequence":"additional","affiliation":[]},{"given":"Elisabetta","family":"Chicca","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2023,6,12]]},"reference":[{"key":"nceacdababib1","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1121\/10.0011809","article-title":"A survey of sound source localization with deep learning methods","volume":"152","author":"Grumiaux","year":"2022","journal-title":"J. Acoust. Soc. Am."},{"key":"nceacdababib2","doi-asserted-by":"publisher","first-page":"1620","DOI":"10.1109\/TASLP.2020.2990485","article-title":"The locata challenge: acoustic source localization and tracking","volume":"28","author":"Evers","year":"2020","journal-title":"IEEE\/ACM Trans. Audio, Speech Lang. Process."},{"key":"nceacdababib3","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1109\/TNNLS.2018.2830119","article-title":"Enhanced robot speech recognition using biomimetic binaural sound source localization","volume":"30","author":"D\u00e1vila-Chac\u00f3n","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"nceacdababib4","doi-asserted-by":"publisher","first-page":"21","DOI":"10.3389\/fnins.2012.00021","article-title":"Neuromorphic audio-visual sensor fusion on a sound-localising robot","volume":"6","author":"Chan","year":"2012","journal-title":"Front. Neurosci."},{"key":"nceacdababib5","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1109\/TPAMI.2020.3008413","article-title":"Event-based vision: a survey","volume":"44","author":"Gallego","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"nceacdababib6","doi-asserted-by":"publisher","first-page":"804","DOI":"10.1109\/TNNLS.2016.2583223","article-title":"A binaural neuromorphic auditory sensor for FPGA: a spike signal processing approach","volume":"28","author":"Jim\u00e9nez-Fern\u00e1ndez","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"nceacdababib7","doi-asserted-by":"publisher","first-page":"891","DOI":"10.3389\/fnins.2018.00891","article-title":"Large-scale neuromorphic spiking array processors: a quest to mimic the brain","volume":"12","author":"Thakur","year":"2018","journal-title":"Front. Neurosci."},{"key":"nceacdababib8","doi-asserted-by":"publisher","first-page":"2384","DOI":"10.1162\/neco_a_01112","article-title":"Spiking elementary motion detector in neuromorphic systems","volume":"30","author":"Milde","year":"2018","journal-title":"Neural Comput."},{"key":"nceacdababib9","first-page":"pp 1","article-title":"Neuromorphic sensory integration for combining sound source localization and collision avoidance","author":"Schoepe","year":"2019"},{"key":"nceacdababib10","doi-asserted-by":"publisher","first-page":"451","DOI":"10.3389\/fnins.2020.00451","article-title":"Event-based eccentric motion detection exploiting time difference encoding","volume":"14","author":"D\u2019Angelo","year":"2020","journal-title":"Front. Neurosci."},{"key":"nceacdababib11","doi-asserted-by":"publisher","first-page":"420","DOI":"10.3389\/fnins.2020.00420","article-title":"Event-based computation for touch localization based on precise spike timing","volume":"14","author":"Haessig","year":"2020","journal-title":"Front. Neurosci."},{"key":"nceacdababib12","first-page":"pp 1","article-title":"A hardware-friendly neuromorphic spiking neural network for frequency detection and fine texture decoding","author":"Mastella","year":"2021"},{"key":"nceacdababib13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TNNLS.2021.3108047","article-title":"An event-based digital time difference encoder model implementation for neuromorphic systems","volume":"33","author":"Gutierrez-Galan","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"nceacdababib14","doi-asserted-by":"publisher","first-page":"184","DOI":"10.1016\/j.robot.2017.07.011","article-title":"Localization of sound sources in robotics: a review","volume":"96","author":"Rascon","year":"2017","journal-title":"Robot. Auton. Syst."},{"key":"nceacdababib15","first-page":"pp 1260","article-title":"Spiking neural network for sound localization using microphone array","author":"Faraji","year":"2015"},{"key":"nceacdababib16","article-title":"Binaural sound localization in spiking neural networks","author":"Kriener","year":"2014"},{"key":"nceacdababib17","doi-asserted-by":"publisher","first-page":"18","DOI":"10.3389\/fncom.2010.00018","article-title":"A spiking neural network model of the medial superior olive using spike timing dependent plasticity for sound localization","volume":"4","author":"Glackin","year":"2010","journal-title":"Front. Comput. Neurosci."},{"key":"nceacdababib18","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1109\/TNNLS.2011.2178317","article-title":"Spiking neural network model of sound localization using the interaural intensity difference","volume":"23","author":"Wall","year":"2012","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"nceacdababib19","first-page":"pp 784","article-title":"Learning to localise sounds with spiking neural networks","author":"Goodman","year":"2010"},{"key":"nceacdababib20","doi-asserted-by":"publisher","first-page":"2656","DOI":"10.1109\/TASLP.2021.3100684","article-title":"Multi-tone phase coding of interaural time difference for sound source localization with spiking neural networks","volume":"29","author":"Pan","year":"2021","journal-title":"IEEE\/ACM Trans. Audio, Speech Lang. Process."},{"key":"nceacdababib21","first-page":"pp 103","article-title":"A bio-inspired model of sound source localization on neuromorphic hardware","author":"Oess","year":"2020"},{"key":"nceacdababib22","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1016\/j.neucom.2017.12.041","article-title":"Real-time neuro-inspired sound source localization and tracking architecture applied to a robotic platform","volume":"283","author":"Escudero","year":"2018","journal-title":"Neurocomputing"},{"key":"nceacdababib23","first-page":"pp 1282","article-title":"A computational model of filtering, detection and compression in the cochlea","volume":"vol 7","author":"Lyon","year":"1982"},{"key":"nceacdababib24","article-title":"The address-event representation communication protocol"},{"key":"nceacdababib25","first-page":"pp 1","article-title":"Building blocks for spikes signals processing","author":"Jimenez-Fernandez","year":"2010"},{"key":"nceacdababib26","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1016\/j.neucom.2020.12.062","article-title":"Opennas: open source neuromorphic auditory sensor HDL code generator for FPGA implementations","volume":"436","author":"Gutierrez-Galan","year":"2021","journal-title":"Neurocomputing"},{"key":"nceacdababib27","first-page":"pp 1556","article-title":"An AER handshake-less modular infrastructure PCB with x8 2.5Gbps LVDS serial links","author":"Iakymchuk","year":"2014"},{"key":"nceacdababib28","volume":"vol 29","author":"Rosen","year":"2011"},{"key":"nceacdababib29","doi-asserted-by":"publisher","first-page":"1943","DOI":"10.1109\/JSSC.2013.2259038","article-title":"SpiNNaker: a 1-W 18-core system-on-chip for massively-parallel neural network simulation","volume":"48","author":"Painkras","year":"2013","journal-title":"IEEE J. Solid-State Circuits"},{"key":"nceacdababib30","doi-asserted-by":"publisher","first-page":"11","DOI":"10.3389\/neuro.11.011.2008","article-title":"PyNN: a common interface for neuronal network simulators","volume":"2","author":"Davison","year":"2009","journal-title":"Front. Neuroinform."},{"key":"nceacdababib31","doi-asserted-by":"publisher","first-page":"816","DOI":"10.3389\/fnins.2018.00816","article-title":"sPyNNaker: a software package for running PyNN simulations on SpiNNaker","volume":"12","author":"Rhodes","year":"2018","journal-title":"Front. Neurosci."},{"key":"nceacdababib32","doi-asserted-by":"publisher","first-page":"84918","DOI":"10.1109\/ACCESS.2020.2991038","article-title":"spiNNlink: FPGA-based interconnect for the million-core SpiNNaker system","volume":"8","author":"Plana","year":"2020","journal-title":"IEEE Access"},{"key":"nceacdababib33","doi-asserted-by":"publisher","first-page":"771","DOI":"10.1109\/TNN.2006.872253","article-title":"On algorithmic rate-coded AER generation","volume":"17","author":"Linares-Barranco","year":"2006","journal-title":"IEEE Trans. Neural Netw."},{"key":"nceacdababib34","doi-asserted-by":"publisher","first-page":"2112","DOI":"10.1523\/JNEUROSCI.16-06-02112.1996","article-title":"Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory","volume":"16","author":"Zhang","year":"1996","journal-title":"J. Neurosci."},{"key":"nceacdababib35","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.23496","article-title":"Angular velocity integration in a fly heading circuit","volume":"6","author":"Turner-Evans","year":"2017","journal-title":"eLife"},{"key":"nceacdababib36","article-title":"Speech commands: a dataset for limited-vocabulary speech recognition","author":"Warden","year":"2018"},{"key":"nceacdababib37","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.1038\/s41467-022-28487-2","article-title":"Embodied neuromorphic intelligence","volume":"13","author":"Bartolozzi","year":"2022","journal-title":"Nat. Commun."},{"key":"nceacdababib38","doi-asserted-by":"publisher","first-page":"eabl8419","DOI":"10.1126\/scirobotics.abl8419","article-title":"Neuromorphic computing hardware and neural architectures for robotics","volume":"7","author":"Sandamirskaya","year":"2022","journal-title":"Sci. Robot."},{"key":"nceacdababib39","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1016\/j.anorl.2018.04.009","article-title":"Sound source localization","volume":"135","author":"Risoud","year":"2018","journal-title":"Eur. Ann. Otorhinolaryngol. Head Neck Dis."},{"key":"nceacdababib40","first-page":"pp 11","article-title":"Towards the neuromorphic implementation of the auditory perception in the iCub robotic platform","author":"Gutierrez-Galan","year":"2022"},{"key":"nceacdababib41","doi-asserted-by":"publisher","first-page":"3831","DOI":"10.3390\/s120403831","article-title":"A neuro-inspired spike-based PID motor controller for multi-motor robots with low cost FPGAs","volume":"12","author":"Jim\u00e9nez-Fernandez","year":"2012","journal-title":"Sensors"},{"key":"nceacdababib42","first-page":"pp 1","article-title":"High performance computing on spinnaker neuromorphic platform: a case study for energy efficient image processing","author":"Sugiarto","year":"2016"},{"key":"nceacdababib43","first-page":"pp 1052","article-title":"Object contact shape classification using neuromorphic spiking neural network with STDP learning","author":"Dabbous","year":"2022"},{"key":"nceacdababib44","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1109\/TNN.2005.860850","article-title":"A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity","volume":"17","author":"Indiveri","year":"2006","journal-title":"IEEE Trans. Neural Netw."},{"key":"nceacdababib45","author":"Schoepe","year":"2023"}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acdaba","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acdaba\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acdaba","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acdaba\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acdaba\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acdaba\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acdaba\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acdaba\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,12]],"date-time":"2023-06-12T07:12:19Z","timestamp":1686553939000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acdaba"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,1]]},"references-count":45,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,6,12]]},"published-print":{"date-parts":[[2023,6,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/acdaba","relation":{"has-review":[{"id-type":"doi","id":"10.1088\/2634-4386\/ACDABA\/v2\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/ACDABA\/v2\/response1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/ACDABA\/v3\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/ACDABA\/v2\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/ACDABA\/v1\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/ACDABA\/v3\/response1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/ACDABA\/v1\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/ACDABA\/v1\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2634-4386\/ACDABA\/v2\/review1","asserted-by":"object"}]},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,1]]},"assertion":[{"value":"Closed-loop sound source localization in neuromorphic systems","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2023 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2023-02-23","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2023-06-01","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2023-06-12","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}