{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T07:25:03Z","timestamp":1769153103156,"version":"3.49.0"},"reference-count":45,"publisher":"IOP Publishing","issue":"2","license":[{"start":{"date-parts":[[2023,5,2]],"date-time":"2023-05-02T00:00:00Z","timestamp":1682985600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,5,2]],"date-time":"2023-05-02T00:00:00Z","timestamp":1682985600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"European Union\u2019s Horizon 2020 Framework Programme for Research and Innovation","award":["785907 (Human Brain Project SGA2)"],"award-info":[{"award-number":["785907 (Human Brain Project SGA2)"]}]},{"name":"German Federal Ministry of Education and Research","award":["16ME0398K (NEUROTEC)"],"award-info":[{"award-number":["16ME0398K (NEUROTEC)"]}]},{"name":"Helmholtz Association Initiative and Networking Fund","award":["SO-092"],"award-info":[{"award-number":["SO-092"]}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"crossref","award":["SFB 917 \"Nanoswitches\""],"award-info":[{"award-number":["SFB 917 \"Nanoswitches\""]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2023,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Machine learning models for sequence learning and processing often suffer from high energy consumption and require large amounts of training data. The brain presents more efficient solutions to how these types of tasks can be solved. While this has inspired the conception of novel brain-inspired algorithms, their realizations remain constrained to conventional von-Neumann machines. Therefore, the potential power efficiency of the algorithm cannot be exploited due to the inherent memory bottleneck of the computing architecture. Therefore, we present in this paper a dedicated hardware implementation of a biologically plausible version of the Temporal Memory component of the Hierarchical Temporal Memory concept. Our implementation is built on a memristive crossbar array and is the result of a hardware-algorithm co-design process. Rather than using the memristive devices solely for data storage, our approach leverages their specific switching dynamics to propose a formulation of the peripheral circuitry, resulting in a more efficient design. By combining a brain-like algorithm with emerging non-volatile memristive device technology we strive for maximum energy efficiency. We present simulation results on the training of complex high-order sequences and discuss how the system is able to predict in a context-dependent manner. Finally, we investigate the energy consumption during the training and conclude with a discussion of scaling prospects.<\/jats:p>","DOI":"10.1088\/2634-4386\/acca45","type":"journal-article","created":{"date-parts":[[2023,4,4]],"date-time":"2023-04-04T22:30:46Z","timestamp":1680647446000},"page":"024002","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["System model of neuromorphic sequence learning on a memristive crossbar array"],"prefix":"10.1088","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9922-4861","authenticated-orcid":true,"given":"Sebastian","family":"Siegel","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4367-8236","authenticated-orcid":true,"given":"Younes","family":"Bouhadjar","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5794-5425","authenticated-orcid":true,"given":"Tom","family":"Tetzlaff","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5426-9967","authenticated-orcid":false,"given":"Rainer","family":"Waser","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1886-1864","authenticated-orcid":false,"given":"Regina","family":"Dittmann","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6766-8553","authenticated-orcid":false,"given":"Dirk J","family":"Wouters","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2023,5,2]]},"reference":[{"key":"nceacca45bib1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10916-019-1345-y","article-title":"Automated depression detection using deep representation and sequence learning with EEG signals","volume":"43","author":"Ay","year":"2019","journal-title":"J. Med. Syst."},{"key":"nceacca45bib2","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v32i1.11891","article-title":"DeepHeart: semi-supervised sequence learning for cardiovascular risk prediction","author":"Ballinger","year":"2018"},{"key":"nceacca45bib3","first-page":"pp 16","article-title":"AI at the edge for sign language learning support","author":"Battistoni","year":"2019"},{"key":"nceacca45bib4","doi-asserted-by":"publisher","first-page":"4618","DOI":"10.1109\/TCSI.2020.3018502","article-title":"Variability-aware modeling of filamentary oxide-based bipolar resistive switching cells using SPICE level compact models","volume":"67","author":"Bengel","year":"2020","journal-title":"IEEE Trans. Circuits Syst. I"},{"key":"nceacca45bib5","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-73254-2","article-title":"Picosecond multilevel resistive switching in tantalum oxide thin films","volume":"10","author":"B\u00f6ttger","year":"2020","journal-title":"Sci. Rep."},{"key":"nceacca45bib6","article-title":"Sequence learning in a spiking neuronal network with memristive synapses","author":"Bouhadjar","year":"2022"},{"key":"nceacca45bib7","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1010233","article-title":"Sequence learning, prediction and replay in networks of spiking neurons","volume":"18","author":"Bouhadjar","year":"2022","journal-title":"PLoS Comput. Biol."},{"key":"nceacca45bib8","doi-asserted-by":"publisher","first-page":"290","DOI":"10.1038\/s41928-019-0270-x","article-title":"A fully integrated reprogrammable memristor\u2013CMOS system for efficient multiply\u2013accumulate operations","volume":"2","author":"Cai","year":"2019","journal-title":"Nat. Electron."},{"key":"nceacca45bib9","doi-asserted-by":"publisher","first-page":"15512","DOI":"10.1073\/pnas.1313114110","article-title":"Reverse engineering the cognitive brain","volume":"110","author":"Cauwenberghs","year":"2013","journal-title":"Proc. Natl Acad. Sci."},{"key":"nceacca45bib10","doi-asserted-by":"publisher","DOI":"10.1088\/2634-4386\/ac4a83","article-title":"2022 roadmap on neuromorphic computing and engineering","volume":"2","author":"Christensen","year":"2022","journal-title":"Neuromorph. Comput. Eng."},{"key":"nceacca45bib11","doi-asserted-by":"publisher","first-page":"2474","DOI":"10.1162\/NECO_a_00893","article-title":"Continuous online sequence learning with an unsupervised neural network model","volume":"28","author":"Cui","year":"2016","journal-title":"Neural Comput."},{"key":"nceacca45bib12","doi-asserted-by":"publisher","DOI":"10.1063\/1.5108654","article-title":"Exploiting the switching dynamics of HfO2-based ReRAM devices for reliable analog memristive behavior","volume":"7","author":"C\u00fcppers","year":"2019","journal-title":"APL Mater."},{"key":"nceacca45bib13","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1080\/00018732.2022.2084006","article-title":"Nanoionic memristive phenomena in metal oxides: the valence change mechanism","volume":"70","author":"Dittmann","year":"2021","journal-title":"Adv. Phys."},{"key":"nceacca45bib14","first-page":"pp 20","article-title":"Temporal sequence learning with a history-sensitive probabilistic learning rule intrinsic to oxygen vacancy-based RRAM","author":"Doevenspeck","year":"2018"},{"key":"nceacca45bib15","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1109\/TED.2019.2957067","article-title":"Modeling and demonstration of oxygen vacancy-based RRAM as probabilistic device for sequence learning","volume":"67","author":"Doevenspeck","year":"2019","journal-title":"IEEE Trans. Electron Devices"},{"key":"nceacca45bib16","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1038\/s41591-018-0316-z","article-title":"A guide to deep learning in healthcare","volume":"25","author":"Esteva","year":"2019","journal-title":"Nat. Med."},{"key":"nceacca45bib17","doi-asserted-by":"publisher","first-page":"1907","DOI":"10.1109\/TNNLS.2015.2462731","article-title":"Hierarchical temporal memory based on spin-neurons and resistive memory for energy-efficient brain-inspired computing","volume":"27","author":"Fan","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"nceacca45bib18","doi-asserted-by":"publisher","first-page":"1430","DOI":"10.4249\/scholarpedia.1430","article-title":"NEST (NEural Simulation Tool)","volume":"2","author":"Gewaltig","year":"2007","journal-title":"Scholarpedia J."},{"key":"nceacca45bib19","first-page":"pp 24","article-title":"Characterizing flash memory: anomalies, observations and applications","author":"Grupp","year":"2009"},{"key":"nceacca45bib20","doi-asserted-by":"publisher","first-page":"812","DOI":"10.3389\/fnins.2019.00812","article-title":"Unsupervised learning on resistive memory array based spiking neural networks","volume":"13","author":"Guo","year":"2019","journal-title":"Front. Neurosci."},{"key":"nceacca45bib21","doi-asserted-by":"publisher","first-page":"3229","DOI":"10.1109\/TED.2018.2849872","article-title":"Improved switching stability and the effect of an internal series resistor in HfO2-TiO x bilayer ReRAM cells","volume":"65","author":"Hardtdegen","year":"2018","journal-title":"IEEE Trans. Electron Devices"},{"key":"nceacca45bib22","author":"Hawkins","year":"2021"},{"key":"nceacca45bib23","author":"Hawkins","year":"2006"},{"key":"nceacca45bib24","doi-asserted-by":"publisher","DOI":"10.1002\/adma.201705914","article-title":"Memristor-based analog computation and neural network classification with a dot product engine","volume":"30","author":"Hu","year":"2018","journal-title":"Adv. Mater."},{"key":"nceacca45bib25","author":"Ielmini","year":"2015"},{"key":"nceacca45bib26","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1007\/s10618-019-00619-1","article-title":"Deep learning for time series classification: a review","volume":"33","author":"Ismail Fawaz","year":"2019","journal-title":"Data Min. Knowl. Disc."},{"key":"nceacca45bib27","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/j.ins.2019.03.066","article-title":"Recurrent neural network-based semantic variational autoencoder for sequence-to-sequence learning","volume":"490","author":"Jang","year":"2019","journal-title":"Inf. Sci."},{"key":"nceacca45bib28","doi-asserted-by":"publisher","DOI":"10.1002\/aelm.201600090","article-title":"Memristors for energy-efficient new computing paradigms","volume":"2","author":"Jeong","year":"2016","journal-title":"Adv. Electron. Mater."},{"key":"nceacca45bib29","first-page":"pp 449","article-title":"A comparative study on transformer vs RNN in speech applications","author":"Karita","year":"2019"},{"key":"nceacca45bib30","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1109\/TETCI.2018.2838124","article-title":"Hierarchical temporal memory using memristor networks: a survey","volume":"2","author":"Krestinskaya","year":"2018","journal-title":"IEEE Trans. Emerg. Top. Comput. Intell."},{"key":"nceacca45bib31","doi-asserted-by":"publisher","first-page":"1143","DOI":"10.1109\/TCAD.2017.2748024","article-title":"Hierarchical temporal memory features with memristor logic circuits for pattern recognition","volume":"37","author":"Krestinskaya","year":"2017","journal-title":"IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst."},{"key":"nceacca45bib32","article-title":"Spontaneous activity in a self-organizing recurrent network reflects prior learning","author":"Lazar","year":"2010"},{"key":"nceacca45bib33","first-page":"pp 4360","article-title":"Learning multi-task communication with message passing for sequence learning","volume":"vol 33","author":"Liu","year":"2019"},{"key":"nceacca45bib34","doi-asserted-by":"publisher","first-page":"197","DOI":"10.1039\/C8FD00117K","article-title":"The ultimate switching speed limit of redox-based resistive switching devices","volume":"213","author":"Menzel","year":"2019","journal-title":"Faraday Discuss."},{"key":"nceacca45bib35","doi-asserted-by":"publisher","first-page":"1031","DOI":"10.1109\/TCAD.2013.2249557","article-title":"Modeling power consumption of NAND flash memories using flashpower","volume":"32","author":"Mohan","year":"2013","journal-title":"IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst."},{"key":"nceacca45bib36","doi-asserted-by":"publisher","first-page":"740","DOI":"10.1109\/LED.2017.2696023","article-title":"Arbitrary spike time dependent plasticity (STDP) in memristor by analog waveform engineering","volume":"38","author":"Panwar","year":"2017","journal-title":"IEEE Electron Device Lett."},{"key":"nceacca45bib37","doi-asserted-by":"publisher","first-page":"5793","DOI":"10.1038\/s41467-022-33476-6","article-title":"Self-organization of an inhomogeneous memristive hardware for sequence learning","volume":"13","author":"Payvand","year":"2022","journal-title":"Nat. Commun."},{"key":"nceacca45bib38","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1109\/MSP.2019.2933719","article-title":"Low-power neuromorphic hardware for signal processing applications: a review of architectural and system-level design approaches","volume":"36","author":"Rajendran","year":"2019","journal-title":"IEEE Signal Process. Mag."},{"key":"nceacca45bib39","doi-asserted-by":"publisher","first-page":"2","DOI":"10.3389\/fnins.2013.00002","article-title":"STDP and STDP variations with memristors for spiking neuromorphic learning systems","volume":"7","author":"Serrano-Gotarredona","year":"2013","journal-title":"Front. Neurosci."},{"key":"nceacca45bib40","first-page":"pp 1","article-title":"Automatic chromosome classification using deep attention based sequence learning of chromosome bands","author":"Sharma","year":"2018"},{"key":"nceacca45bib41","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1145\/3584954.3585000","article-title":"Demonstration of neuromorphic sequence learning on a memristive array","author":"Siegel","year":"2023"},{"key":"nceacca45bib42","first-page":"pp 760","article-title":"A comparative study of state-of-the-art machine learning algorithms for predictive maintenance","author":"Silvestrin","year":"2019"},{"key":"nceacca45bib43","doi-asserted-by":"publisher","first-page":"667","DOI":"10.1109\/JEDS.2021.3095389","article-title":"Determining the electrical charging speed limit of ReRAM devices","volume":"9","author":"von Witzleben","year":"2021","journal-title":"IEEE J. Electron Devices Soc."},{"key":"nceacca45bib44","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1145\/216585.216588","article-title":"Hitting the memory wall: Implications of the obvious","volume":"23","author":"Wulf","year":"1995","journal-title":"ACM SIGARCH Comput. Archit. News"},{"key":"nceacca45bib45","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1038\/s41563-019-0291-x","article-title":"Memristive crossbar arrays for brain-inspired computing","volume":"18","author":"Xia","year":"2019","journal-title":"Nat. Mater."}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acca45","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acca45\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acca45","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acca45\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acca45\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acca45\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acca45\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acca45\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,2]],"date-time":"2023-05-02T09:13:39Z","timestamp":1683018819000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/acca45"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,2]]},"references-count":45,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,5,2]]},"published-print":{"date-parts":[[2023,6,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/acca45","relation":{},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,2]]},"assertion":[{"value":"System model of neuromorphic sequence learning on a memristive crossbar array","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2023 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2022-12-28","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2023-04-04","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2023-05-02","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}