{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T13:58:05Z","timestamp":1781791085872,"version":"3.54.5"},"reference-count":62,"publisher":"Wiley","issue":"41","license":[{"start":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T00:00:00Z","timestamp":1725840000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["advanced.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Advanced Science"],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Focusing on a specific conversation amidst multiple interfering talkers is challenging, especially for those with hearing loss. Brain\u2010controlled assistive hearing devices aim to alleviate this problem by enhancing the attended speech based on the listener's neural signals using auditory attention decoding (AAD). Departing from conventional AAD studies that relied on oversimplified scenarios with stationary talkers, a realistic AAD task that involves multiple talkers taking turns as they continuously move in space in background noise is presented. Invasive electroencephalography (iEEG) data are collected from three neurosurgical patients as they focused on one of the two moving conversations. An enhanced brain\u2010controlled assistive hearing system that combines AAD and a binaural speaker\u2010independent speech separation model is presented. The separation model unmixes talkers while preserving their spatial location and provides talker trajectories to the neural decoder to improve AAD accuracy. Subjective and objective evaluations show that the proposed system enhances speech intelligibility and facilitates conversation tracking while maintaining spatial cues and voice quality in challenging acoustic environments. This research demonstrates the potential of this approach in real\u2010world scenarios and marks a significant step toward developing assistive hearing technologies that adapt to the intricate dynamics of everyday auditory experiences.<\/jats:p>","DOI":"10.1002\/advs.202401379","type":"journal-article","created":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T09:48:12Z","timestamp":1725875292000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Brain\u2010Controlled Augmented Hearing for Spatially Moving Conversations in Multi\u2010Talker Environments"],"prefix":"10.1002","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-5486-5913","authenticated-orcid":false,"given":"Vishal","family":"Choudhari","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering Columbia University  New York NY 10027 USA"},{"name":"Mortimer B. Zuckerman Mind Brain Behavior Institute  New York NY 10027 USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2121-000X","authenticated-orcid":false,"given":"Cong","family":"Han","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering Columbia University  New York NY 10027 USA"},{"name":"Mortimer B. Zuckerman Mind Brain Behavior Institute  New York NY 10027 USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5265-2447","authenticated-orcid":false,"given":"Stephan","family":"Bickel","sequence":"additional","affiliation":[{"name":"Hofstra Northwell School of Medicine  Uniondale NY 11549 USA"},{"name":"The Feinstein Institutes for Medical Research  Manhasset NY 11030 USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7293-1101","authenticated-orcid":false,"given":"Ashesh D.","family":"Mehta","sequence":"additional","affiliation":[{"name":"Hofstra Northwell School of Medicine  Uniondale NY 11549 USA"},{"name":"The Feinstein Institutes for Medical Research  Manhasset NY 11030 USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4485-7933","authenticated-orcid":false,"given":"Catherine","family":"Schevon","sequence":"additional","affiliation":[{"name":"Department of Neurology Columbia University  New York NY 10027 USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9695-3564","authenticated-orcid":false,"given":"Guy M.","family":"McKhann","sequence":"additional","affiliation":[{"name":"Department of Neurological Surgery, Vagelos College of Physicians and Surgeons Columbia University, New York  New York NY 10027 USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2987-759X","authenticated-orcid":false,"given":"Nima","family":"Mesgarani","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering Columbia University  New York NY 10027 USA"},{"name":"Mortimer B. Zuckerman Mind Brain Behavior Institute  New York NY 10027 USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2024,9,9]]},"reference":[{"key":"e_1_2_10_1_1","doi-asserted-by":"publisher","DOI":"10.1001\/archotol.1970.00770040379010"},{"key":"e_1_2_10_2_1","doi-asserted-by":"publisher","DOI":"10.1155\/ASP.2005.2915"},{"key":"e_1_2_10_3_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.4948445"},{"key":"e_1_2_10_4_1","doi-asserted-by":"publisher","DOI":"10.1097\/00003446-199402000-00002"},{"key":"e_1_2_10_5_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature11020"},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/aa7ab4"},{"key":"e_1_2_10_7_1","doi-asserted-by":"crossref","unstructured":"J. R.Hershey Z.Chen J.Le Roux S.Watanabe in2016 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP) IEEE Shanghai2016 31.","DOI":"10.1109\/ICASSP.2016.7471631"},{"key":"e_1_2_10_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2019.2915167"},{"key":"e_1_2_10_9_1","doi-asserted-by":"crossref","unstructured":"Z.\u2010Q.Wang S.Cornell S.Choi Y.Lee B.\u2010Y.Kim S.Watanabe TF\u2010GridNet: Integrating Full\u2010 and Sub\u2010Band Modeling for Speech Separation2022.","DOI":"10.1109\/TASLP.2023.3304482"},{"key":"e_1_2_10_10_1","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.aav6134"},{"key":"e_1_2_10_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2020.117282"},{"key":"e_1_2_10_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2021.02.020"},{"key":"e_1_2_10_13_1","doi-asserted-by":"publisher","DOI":"10.1093\/cercor\/bht355"},{"key":"e_1_2_10_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2021.3075932"},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.56481"},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNSRE.2016.2571900"},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-47795-0"},{"key":"e_1_2_10_18_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.2109187"},{"key":"e_1_2_10_19_1","first-page":"3505","volume-title":"Binaural Speech Separation of Moving Speakers With Preserved Spatial Cues. in Interspeech","author":"Han C.","year":"2021"},{"key":"e_1_2_10_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2022.07.047"},{"key":"e_1_2_10_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.celrep.2018.07.076"},{"key":"e_1_2_10_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2019.116283"},{"key":"e_1_2_10_23_1","volume":"105313","author":"Makov S.","year":"2022","journal-title":"Cognition"},{"key":"e_1_2_10_24_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-39166-6"},{"key":"e_1_2_10_25_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0043688"},{"key":"e_1_2_10_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASRU.2015.7404837"},{"key":"e_1_2_10_27_1","doi-asserted-by":"publisher","DOI":"10.1097\/AUD.0b013e31822c2549"},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2018.01.033"},{"key":"e_1_2_10_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/WASPAA.2019.8937253"},{"key":"e_1_2_10_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2019.8682884"},{"key":"e_1_2_10_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2017.2726762"},{"key":"e_1_2_10_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2021.3083405"},{"key":"e_1_2_10_33_1","unstructured":"S.Doclo T.Klasen T.Van den Bogaert J.Wouters M.Moonen inInternational Workshop on Acoustic Signal Enhancement2006 1."},{"key":"e_1_2_10_34_1","first-page":"308","volume-title":"Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers. xv","author":"Swets J. A.","year":"1996"},{"key":"e_1_2_10_35_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1245994"},{"key":"e_1_2_10_36_1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.2383-16.2017"},{"key":"e_1_2_10_37_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-34326-1"},{"key":"e_1_2_10_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2001.941023"},{"key":"e_1_2_10_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2016.2585878"},{"key":"e_1_2_10_40_1","doi-asserted-by":"publisher","DOI":"10.1093\/cercor\/bhs359"},{"key":"e_1_2_10_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2019.116360"},{"key":"e_1_2_10_42_1","doi-asserted-by":"publisher","DOI":"10.1093\/cercor\/bhh133"},{"key":"e_1_2_10_43_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1112852109"},{"key":"e_1_2_10_44_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-neuro-072116-031215"},{"key":"e_1_2_10_45_1","doi-asserted-by":"publisher","DOI":"10.1111\/ejn.15616"},{"key":"e_1_2_10_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2012.12.037"},{"key":"e_1_2_10_47_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1205381109"},{"key":"e_1_2_10_48_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06377-x"},{"key":"e_1_2_10_49_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa2027540"},{"key":"e_1_2_10_50_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-33611-3"},{"key":"e_1_2_10_51_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06443-4"},{"key":"e_1_2_10_52_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-37359-z"},{"key":"e_1_2_10_53_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1388-2457(00)00545-9"},{"key":"e_1_2_10_54_1","doi-asserted-by":"publisher","DOI":"10.1152\/jn.90954.2008"},{"key":"e_1_2_10_55_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2017-1386"},{"key":"e_1_2_10_56_1","unstructured":"M.Gorzel A.Allen I. J.Kelly A.Gungormusler Efficient Encoding and Decoding of Binaural Sound with Resonance Audio 2019."},{"key":"e_1_2_10_57_1","first-page":"1","volume-title":"2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","author":"Wang Z.\u2010Q.","year":"2018"},{"key":"e_1_2_10_58_1","unstructured":"R.Gu J.Wu S.\u2010X.Zhang L.Chen Y.Xu M.Yu D.Su Y.Zou D.Yu End\u2010to\u2010End Multi\u2010Channel Speech Separation2019."},{"key":"e_1_2_10_59_1","first-page":"5206","volume-title":"An ASR corpus based on public domain audio books. in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","author":"Panayotov V.","year":"2015"},{"key":"e_1_2_10_60_1","first-page":"035081","volume-title":"Proc. Mtgs. Acoust.","author":"Thiemann J.","year":"2013"},{"key":"e_1_2_10_61_1","unstructured":"D. P.Kingma J. A.Ba A Method for Stochastic Optimization 2014."},{"key":"e_1_2_10_62_1","doi-asserted-by":"publisher","DOI":"10.21105\/joss.05351"}],"container-title":["Advanced Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/advs.202401379","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,6]],"date-time":"2025-10-06T21:33:14Z","timestamp":1759786394000},"score":1,"resource":{"primary":{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/10.1002\/advs.202401379"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,9]]},"references-count":62,"journal-issue":{"issue":"41","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["10.1002\/advs.202401379"],"URL":"https:\/\/doi.org\/10.1002\/advs.202401379","archive":["Portico"],"relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2024.02.05.579018","asserted-by":"object"}]},"ISSN":["2198-3844","2198-3844"],"issn-type":[{"value":"2198-3844","type":"print"},{"value":"2198-3844","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,9]]},"assertion":[{"value":"2024-02-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"2401379"}}