{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T20:39:42Z","timestamp":1772743182455,"version":"3.50.1"},"reference-count":55,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2022,8,5]],"date-time":"2022-08-05T00:00:00Z","timestamp":1659657600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2022,8,5]],"date-time":"2022-08-05T00:00:00Z","timestamp":1659657600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"crossref","award":["EP\/M006727\/1"],"award-info":[{"award-number":["EP\/M006727\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100018919","name":"Peng Cheng Laboratory","doi-asserted-by":"crossref","award":["Major Key Project of PCL2021A12"],"award-info":[{"award-number":["Major Key Project of PCL2021A12"]}],"id":[{"id":"10.13039\/100018919","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62104188"],"award-info":[{"award-number":["62104188"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Neuromorph. Comput. Eng."],"published-print":{"date-parts":[[2022,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Overfitting is a common and critical challenge for neural networks trained with limited dataset. The conventional solution is software-based regularization algorithms such as Gaussian noise injection. Semiconductor noise, such as 1\/<jats:italic>f<\/jats:italic> noise, in artificial neuron\/synapse devices, which is often regarded as undesirable disturbance to the hardware neural networks (HNNs), could also play a useful role in suppressing overfitting, but that is as yet unexplored. In this work, we proposed the idea of using 1\/<jats:italic>f<\/jats:italic> noise injection to suppress overfitting in different neural networks, and demonstrated that: (i) 1\/<jats:italic>f<\/jats:italic> noise could suppress the overfitting in multilayer perceptron (MLP) and long short-term memory (LSTM); (ii) 1\/<jats:italic>f<\/jats:italic> noise and Gaussian noise performs similarly for the MLP but differently for the LSTM; (iii) the superior performance of 1\/<jats:italic>f<\/jats:italic> noise on LSTM can be attributed to its intrinsic long range dependence. This work reveals that 1\/<jats:italic>f<\/jats:italic> noise, which is common in semiconductor devices, can be a useful solution to suppress the overfitting in HNNs, and more importantly, further evidents that the imperfectness of semiconductor devices is a rich mine of solutions to boost the development of brain-inspired hardware technologies in the artificial intelligence era.<\/jats:p>","DOI":"10.1088\/2634-4386\/ac6d05","type":"journal-article","created":{"date-parts":[[2022,5,5]],"date-time":"2022-05-05T22:17:17Z","timestamp":1651789037000},"page":"034006","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Synaptic 1\/f noise injection for overfitting suppression in hardware neural networks"],"prefix":"10.1088","volume":"2","author":[{"given":"Yan","family":"Du","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Shao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3446-7138","authenticated-orcid":true,"given":"Zheng","family":"Chai","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hanzhang","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qihui","family":"Diao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yawei","family":"Gao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xihui","family":"Yuan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qiaoqiao","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3337-6202","authenticated-orcid":true,"given":"Tao","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weidong","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jian Fu","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tai","family":"Min","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2022,8,5]]},"reference":[{"key":"nceac6d05bib1","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"nceac6d05bib2","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1109\/tbdata.2016.2599923","article-title":"Clustering big spatiotemporal-interval data","volume":"2","author":"Shao","year":"2016","journal-title":"IEEE Trans. Big Data"},{"key":"nceac6d05bib3","author":"Anthony","year":"2009"},{"key":"nceac6d05bib4","first-page":"124","article-title":"Parking availability prediction with long short term memory model","volume":"vol 11204","author":"Shao","year":"2019"},{"key":"nceac6d05bib5","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.aaz4261","article-title":"Three-dimensional vectorial holography based on machine learning inverse design","volume":"6","author":"Ren","year":"2020","journal-title":"Sci. Adv."},{"key":"nceac6d05bib6","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1145\/3007787.3001163","article-title":"EIE: efficient inference engine on compressed deep neural network","volume":"44","author":"Han","year":"2016","journal-title":"ACM SIGARCH Comput. Architect. News"},{"key":"nceac6d05bib7","doi-asserted-by":"publisher","first-page":"5227","DOI":"10.1109\/jiot.2018.2868243","article-title":"BLEDoorGuard: a device-free person identification framework using bluetooth signals for door access","volume":"5","author":"Shao","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"nceac6d05bib8","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"nceac6d05bib9","doi-asserted-by":"publisher","first-page":"703","DOI":"10.1038\/nmeth.3968","article-title":"Model selection and overfitting","volume":"13","author":"Lever","year":"2016","journal-title":"Nat. Methods"},{"key":"nceac6d05bib10","first-page":"268","article-title":"Privacy risk in machine learning: analyzing the connection to overfitting","author":"Samuel","year":"2018"},{"key":"nceac6d05bib11","doi-asserted-by":"publisher","first-page":"30063","DOI":"10.1073\/pnas.1907378117","article-title":"Benign overfitting in linear regression","volume":"117","author":"Bartlett","year":"2020","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"nceac6d05bib12","article-title":"Handling overfitting in deep learning models","author":"Carremans","year":"2018"},{"key":"nceac6d05bib13","article-title":"Underfitting","author":"","year":"2021"},{"key":"nceac6d05bib14","doi-asserted-by":"crossref","DOI":"10.1109\/IJCNN48605.2020.9207041","article-title":"Approximating optimisation solutions for the travelling officer problem with neural networks","author":"Shao","year":"2020"},{"key":"nceac6d05bib15","article-title":"Variational dropout sparsifies deep neural networks","author":"Molchanov","year":"2017"},{"key":"nceac6d05bib16","article-title":"Regularizing deep neural networks by noise: its interpretation and optimization","author":"Noh","year":"2017"},{"key":"nceac6d05bib17","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1109\/21.155944","article-title":"Noise injection into inputs in back-propagation learning","volume":"22","author":"Matsuoka","year":"1992","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"nceac6d05bib18","article-title":"How to improve deep learning model robustness by adding noise","author":"Brownlee","year":"2018"},{"key":"nceac6d05bib19","first-page":"384","article-title":"Von Neumann computers","author":"Eigenmann","year":"1999"},{"key":"nceac6d05bib20","author":"Drubach","year":"2000"},{"key":"nceac6d05bib21","article-title":"Supercomputer Fugaku","author":"","year":"2021"},{"key":"nceac6d05bib22","doi-asserted-by":"publisher","first-page":"1652","DOI":"10.1109\/led.2018.2869072","article-title":"Impact of RTN on pattern recognition accuracy of RRAM-based synaptic neural network","volume":"39","author":"Chai","year":"2018","journal-title":"IEEE Electron Device Lett."},{"key":"nceac6d05bib23","doi-asserted-by":"publisher","first-page":"3335","DOI":"10.1109\/ted.2020.3002736","article-title":"Exploring the impact of random telegraph noise-induced accuracy loss on resistive RAM-based deep neural network","volume":"67","author":"Du","year":"2020","journal-title":"IEEE Trans. Electron Devices"},{"key":"nceac6d05bib24","doi-asserted-by":"crossref","DOI":"10.1109\/IEDM.2017.8268340","article-title":"Time-dependent variability in RRAM-based analog neuromorphic system for pattern recognition","author":"Kang","year":"2017"},{"key":"nceac6d05bib25","doi-asserted-by":"publisher","first-page":"2001842","DOI":"10.1002\/advs.202001842","article-title":"Implementation of dropout neuronal units based on stochastic memristive devices in neural networks with high classification accuracy","volume":"7","author":"Huang","year":"2020","journal-title":"Adv. Sci."},{"key":"nceac6d05bib26","doi-asserted-by":"publisher","first-page":"1688","DOI":"10.1109\/led.2020.3021593","article-title":"Quantitatively evaluating the effect of read noise in memristive Hopfield network on solving traveling salesman problem","volume":"41","author":"Lu","year":"2020","journal-title":"IEEE Electron Device Lett."},{"key":"nceac6d05bib27","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1038\/s41928-020-0436-6","article-title":"Power-efficient combinatorial optimization using intrinsic noise in memristor Hopfield neural networks","volume":"3","author":"Cai","year":"2020","journal-title":"Nat. Electron."},{"key":"nceac6d05bib28","author":"Yacoub","year":"2019"},{"key":"nceac6d05bib29","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1109\/mim.2021.9400958","article-title":"Thermal noise measurement and characterization for modern semiconductor devices","volume":"24","author":"Chen","year":"2021","journal-title":"IEEE Instrum. Meas. Mag."},{"key":"nceac6d05bib30","first-page":"1","article-title":"Random telegraph signal noise in advanced high performance and memory devices","author":"Claeys","year":"2016"},{"key":"nceac6d05bib31","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1109\/led.2016.2530849","article-title":"Impact of the effective work function gate metal on the low-frequency noise of gate-all-around silicon-on-insulator NWFETs","volume":"37","author":"Fang","year":"2016","journal-title":"IEEE Electron Device Lett."},{"key":"nceac6d05bib32","author":"Bonani","year":"2001"},{"key":"nceac6d05bib33","author":"Konczakowska","year":"2011"},{"key":"nceac6d05bib34","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1103\/physrevlett.59.381","article-title":"Self-organized criticality: an explanation of the 1\/f noise","volume":"59","author":"Bak","year":"1987","journal-title":"Phys. Rev. Lett."},{"key":"nceac6d05bib35","doi-asserted-by":"publisher","first-page":"1926","DOI":"10.1109\/16.333808","article-title":"1\/f noise sources","volume":"41","author":"Hooge","year":"1994","journal-title":"IEEE Trans. Electron Devices"},{"key":"nceac6d05bib36","doi-asserted-by":"publisher","first-page":"1664","DOI":"10.1063\/1.105130","article-title":"1\/f noise reduction of metal\u2010oxide\u2010semiconductor transistors by cycling from inversion to accumulation","volume":"58","author":"Bloom","year":"1991","journal-title":"Appl. Phys. Lett."},{"key":"nceac6d05bib37","doi-asserted-by":"publisher","first-page":"1624","DOI":"10.1109\/ted.2009.2022700","article-title":"The 1\/f noise and random telegraph noise characteristics in floating-gate nand flash memories","volume":"56","author":"Bae","year":"2009","journal-title":"IEEE Trans. Electron Devices"},{"key":"nceac6d05bib38","first-page":"1","article-title":"Breakdown mechanisms in MgO based magnetic tunnel junctions and correlation with low frequency noise","author":"Amara-Dababi","year":"2014"},{"key":"nceac6d05bib39","doi-asserted-by":"publisher","first-page":"3805","DOI":"10.1109\/ted.2015.2475598","article-title":"Noise-induced resistance broadening in resistive switching memory\u2014part: I. Intrinsic cell behavior","volume":"62","author":"Ambrogio","year":"2015","journal-title":"IEEE Trans. Electron Devices"},{"key":"nceac6d05bib40","doi-asserted-by":"publisher","first-page":"1559","DOI":"10.1109\/led.2012.2214472","article-title":"1\/f noise in 45\u00a0nm RESET-state phase-change memory devices: characterization, impact on memory readout operation, and scaling perspectives","volume":"33","author":"Betti Beneventi","year":"2012","journal-title":"IEEE Electron Device Lett."},{"key":"nceac6d05bib41","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1116\/1.573194","article-title":"1\/f noise in p\u2013n junction diodes","volume":"3","author":"Kleinpenning","year":"1985","journal-title":"J. Vac. Sci. Technol. A"},{"key":"nceac6d05bib42","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1016\/j.microrel.2015.12.029","article-title":"Analysis of origin of measured 1\/f noise in high-power semiconductor laser diodes far below threshold current","volume":"59","author":"Guan","year":"2016","journal-title":"Microelectron. Reliab."},{"key":"nceac6d05bib43","article-title":"Stochastic resonance in thalamic neurons and resonant neuron models","author":"Reinker","year":"2004"},{"key":"nceac6d05bib44","author":"Buzs\u00e1ki","year":"2006"},{"key":"nceac6d05bib45","doi-asserted-by":"publisher","first-page":"1370","DOI":"10.1523\/jneurosci.21-04-01370.2001","article-title":"Long-range temporal correlations and scaling behavior in human brain oscillations","volume":"21","author":"Linkenkaer-Hansen","year":"2001","journal-title":"J. Neurosci."},{"key":"nceac6d05bib46","doi-asserted-by":"publisher","DOI":"10.1103\/physreve.80.061914","article-title":"Spontaneous brain activity as a source of ideal","volume":"80","author":"Allegrini","year":"2009","journal-title":"Phys. Rev. E"},{"key":"nceac6d05bib47","doi-asserted-by":"publisher","first-page":"579","DOI":"10.3758\/bf03196615","article-title":"Estimation and interpretation of 1\/f\u03b1 noise in human cognition","volume":"11","author":"Wagenmakers","year":"2004","journal-title":"Psychon. Bull. Rev."},{"key":"nceac6d05bib48","doi-asserted-by":"publisher","first-page":"2135","DOI":"10.1116\/1.588886","article-title":"Autocorrelation function of 1\/f current fluctuations in vacuum microelectronics devices","volume":"14","author":"Amirkhanov","year":"1996","journal-title":"J. Vac. Sci. Technol. B"},{"key":"nceac6d05bib49","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fphys.2012.00127","article-title":"Measuring fractality","volume":"3","author":"Stadnitski","year":"2012","journal-title":"Front. Physiol."},{"key":"nceac6d05bib50","article-title":"Pinknoise","author":"","year":"2022"},{"key":"nceac6d05bib51","article-title":"Randn","author":"","year":"2022"},{"key":"nceac6d05bib52","article-title":"An overview on multilayer perceptron (MLP)","author":"","year":"2022"},{"key":"nceac6d05bib53","article-title":"The MNIST database of handwritten digits","author":"LeCun","year":"1998"},{"key":"nceac6d05bib54","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1023\/a:1019903710291","article-title":"Associative neural network","volume":"16","author":"Tetko","year":"2002","journal-title":"Neural Process. Lett."},{"key":"nceac6d05bib55","article-title":"Xcorr","author":"","year":"2022"}],"container-title":["Neuromorphic Computing and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac6d05","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac6d05\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac6d05","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac6d05\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac6d05\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac6d05\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac6d05\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac6d05\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,11]],"date-time":"2022-08-11T07:28:57Z","timestamp":1660202937000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2634-4386\/ac6d05"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,5]]},"references-count":55,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,8,5]]},"published-print":{"date-parts":[[2022,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2634-4386\/ac6d05","relation":{},"ISSN":["2634-4386"],"issn-type":[{"value":"2634-4386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,5]]},"assertion":[{"value":"Synaptic 1\/f noise injection for overfitting suppression in hardware neural networks","name":"article_title","label":"Article Title"},{"value":"Neuromorphic Computing and Engineering","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2022 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2021-10-30","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-05-05","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2022-08-05","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}