{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T20:36:46Z","timestamp":1775075806039,"version":"3.50.1"},"reference-count":38,"publisher":"IOP Publishing","issue":"1","license":[{"start":{"date-parts":[[2020,2,4]],"date-time":"2020-02-04T00:00:00Z","timestamp":1580774400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/3.0\/"},{"start":{"date-parts":[[2020,2,4]],"date-time":"2020-02-04T00:00:00Z","timestamp":1580774400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"crossref","award":["EP\/N02379X\/1"],"award-info":[{"award-number":["EP\/N02379X\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2020,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Scanning probe microscopists generally do not rely on complete images to assess the quality of data acquired during a scan. Instead, assessments of the state of the tip apex, which not only determines the resolution in any scanning probe technique, but can also generate a wide array of frustrating artefacts, are carried out in real time on the basis of a few lines of an image (and, typically, their associated line profiles.) The very small number of machine learning approaches to probe microscopy published to date, however, involve classifications based on full images. Given that data acquisition is the most time-consuming task during routine tip conditioning, automated methods are thus currently extremely slow in comparison to the tried-and-trusted strategies and heuristics used routinely by probe microscopists. Here, we explore various strategies by which different STM image classes (arising from changes in the tip state) can be correctly identified from partial scans. By employing a secondary temporal network and a rolling window of a small group of individual scanlines, we find that tip assessment is possible with a small fraction of a complete image. We achieve this with little-to-no performance penalty\u2014or, indeed, markedly improved performance in some cases\u2014and introduce a protocol to detect the state of the tip apex in real time.<\/jats:p>","DOI":"10.1088\/2632-2153\/ab42ec","type":"journal-article","created":{"date-parts":[[2020,2,4]],"date-time":"2020-02-04T17:52:07Z","timestamp":1580838727000},"page":"015001","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":22,"title":["Embedding human heuristics in machine-learning-enabled probe microscopy"],"prefix":"10.1088","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8733-7500","authenticated-orcid":false,"given":"Oliver M","family":"Gordon","sequence":"first","affiliation":[]},{"given":"Filipe L Q","family":"Junqueira","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9926-9004","authenticated-orcid":false,"given":"Philip J","family":"Moriarty","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2020,2,4]]},"reference":[{"key":"mlstab42ecbib1","doi-asserted-by":"publisher","DOI":"10.1063\/1.5003851","article-title":"On the effect of local barrier height in scanning tunneling microscopy: measurement methods and control implications","volume":"89","author":"Tajaddodianfar","year":"2018","journal-title":"Rev. Sci. Instrum."},{"key":"mlstab42ecbib2","doi-asserted-by":"publisher","first-page":"2389","DOI":"10.3762\/bjnano.8.238","article-title":"Robust procedure for creating and characterizing the atomic structure of scanning tunneling microscope tips","volume":"8","author":"Tewari","year":"2017","journal-title":"Beil. J. Nanotechnol."},{"key":"mlstab42ecbib3","doi-asserted-by":"publisher","DOI":"10.1063\/1.5052264","article-title":"The qPlus sensor, a powerful core for the atomic force microscope","volume":"90","author":"Giessibl","year":"2019","journal-title":"Rev. Sci. Instrum."},{"key":"mlstab42ecbib4","doi-asserted-by":"publisher","first-page":"1110","DOI":"10.1126\/science.1176210","article-title":"The chemical structure of a molecule resolved by atomic force microscopy","volume":"325","author":"Gross","year":"2009","journal-title":"Science"},{"key":"mlstab42ecbib5","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.106.046104","article-title":"Quantitative atomic force microscopy with carbon monoxide terminated tips","volume":"106","author":"Sun","year":"2011","journal-title":"Phys. Rev. Lett."},{"key":"mlstab42ecbib6","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.108.268302","article-title":"Precise orientation of a single C-60 molecule on the tip of a scanning probe microscope","volume":"108","author":"Chiutu","year":"2012","journal-title":"Phys. Rev. Lett."},{"key":"mlstab42ecbib7","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1016\/S0927-0256(00)00205-6","article-title":"Atom manipulation with the STM: nanostructuring, tip functionalization, and femtochemistry,","volume":"20","author":"Meyer","year":"2001","journal-title":"Comput. Mater. Sci."},{"key":"mlstab42ecbib8","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.107.086101","article-title":"High-resolution molecular orbital imaging using a p-Wave STM tip","volume":"107","author":"Gross","year":"2011","journal-title":"Phys. Rev. Lett."},{"key":"mlstab42ecbib9","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.85.235305","article-title":"Role of orbital overlap in atomic manipulation","volume":"85","author":"Jarvis","year":"2012","journal-title":"Phys. Rev. B"},{"key":"mlstab42ecbib10","doi-asserted-by":"publisher","first-page":"5185","DOI":"10.1021\/acsnano.8b02208","article-title":"Autonomous scanning probe microscopy in situ tip conditioning through machine learning","volume":"12","author":"Rashidi","year":"2018","journal-title":"ACS Nano"},{"key":"mlstab42ecbib11","article-title":"Autonomous atomic scale manufacturing through machine learning","author":"Rashidi","year":"2019"},{"key":"mlstab42ecbib12","doi-asserted-by":"publisher","DOI":"10.1063\/1.5099590","article-title":"Scanning probe state recognition with multi-class neural network ensembles","volume":"90","author":"Gordon","year":"2019","journal-title":"Rev. Sci. Int."},{"key":"mlstab42ecbib13","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/s41586-019-1319-8","article-title":"Machine learning in electronic-quantum-matter imaging experiments","volume":"570","author":"Zhang","year":"2019","journal-title":"Nature"},{"key":"mlstab42ecbib14","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevMaterials.3.033805","article-title":"Classifying surface probe images in strongly correlated electronic systems via machine learning","volume":"3","author":"Burzawa","year":"2019","journal-title":"Phys. Rev. Mater."},{"key":"mlstab42ecbib15","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1038\/nnano.2007.25","article-title":"Coerced mechanical coarsening of nanoparticle assemblies","volume":"2","author":"Blunt","year":"2007","journal-title":"Nat. Nanotechnol."},{"key":"mlstab42ecbib16","doi-asserted-by":"publisher","first-page":"1985","DOI":"10.1021\/nl070773m","article-title":"A genetic algorithm approach to probing the evolution of self-organized nanostructured systems","volume":"7","author":"Siepmann","year":"2007","journal-title":"Nano Lett."},{"key":"mlstab42ecbib17","doi-asserted-by":"publisher","first-page":"663","DOI":"10.1002\/crat.201300240","article-title":"Double-tip effects on scanning tunneling microscopy imaging of 2D periodic objects: unambiguous detection and limits of their removal by crystallographic averaging in the spatial frequency domain","volume":"49","author":"Straton","year":"2014","journal-title":"Cryst. Res. Technol."},{"key":"mlstab42ecbib18","doi-asserted-by":"publisher","DOI":"10.1063\/1.3600662","article-title":"Automated probe microscopy via evolutionary optimization at the atomic scale","volume":"98","author":"Woolley","year":"2011","journal-title":"Appl. Phys. Lett."},{"key":"mlstab42ecbib19","doi-asserted-by":"publisher","DOI":"10.1063\/1.4827076","article-title":"Scanning probe image wizard: a toolbox for automated scanning probe microscopy data analysis","volume":"84","author":"Stirling","year":"2013","journal-title":"Rev. Sci. Instrum."},{"key":"mlstab42ecbib20","doi-asserted-by":"publisher","first-page":"2774","DOI":"10.1109\/TIP.2016.2532239","article-title":"Double-tip artifact removal from atomic force microscopy images","volume":"25","author":"Wang","year":"2016","journal-title":"IEEE Trans. Image Process."},{"key":"mlstab42ecbib21","doi-asserted-by":"publisher","first-page":"2636","DOI":"10.1103\/PhysRevLett.68.2636","article-title":"Direct observation of an increase in buckled dimers on Si(001) at low temperature","volume":"68","author":"Wolkow","year":"1992","journal-title":"Phys. Rev. Lett."},{"key":"mlstab42ecbib22","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.94.115440","article-title":"Measuring the reactivity of a silicon-terminated probe","volume":"94","author":"Sweetman","year":"2016","journal-title":"Phys. Rev. B"},{"key":"mlstab42ecbib23","doi-asserted-by":"publisher","first-page":"25","DOI":"10.3762\/bjnano.3.3","article-title":"Effect of the tip state during qPlus noncontact atomic force microscopy of Si(100) at 5 K: Probing the probe","volume":"3","author":"Sweetman","year":"2012","journal-title":"Beil. J. Nanotechnol."},{"key":"mlstab42ecbib24","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6528\/28\/7\/075302","article-title":"Automated extraction of single H atoms with STM: tip state dependency","volume":"28","author":"M\u00f8ller","year":"2017","journal-title":"Nanotechnology"},{"key":"mlstab42ecbib25","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1146\/annurev.physchem.040808.090314","article-title":"Atomic-scale templates patterned by ultrahigh vacuum scanning tunneling microscopy on silicon","volume":"60","author":"Walsh","year":"2009","journal-title":"Annu. Rev. Phys. Chem."},{"key":"mlstab42ecbib26","doi-asserted-by":"publisher","first-page":"1590","DOI":"10.1126\/science.268.5217.1590","article-title":"Atomic-scale desorption through electronic and vibrational-excitation mechanisms","volume":"268","author":"Shen","year":"1995","journal-title":"Science"},{"key":"mlstab42ecbib27","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1038\/35017519","article-title":"Self-directed growth of molecular nanostructures on silicon","volume":"406","author":"Lopinski","year":"2000","journal-title":"Nature"},{"key":"mlstab42ecbib28","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1038\/nnano.2012.21","article-title":"A single-atom transistor","volume":"7","author":"Fuechsle","year":"2012","journal-title":"Nat. Nanotechnol."},{"key":"mlstab42ecbib29","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1126\/science.1214319","article-title":"Ohm\u2019s law survives to the atomic scale","volume":"335","author":"Weber","year":"2012","journal-title":"Science"},{"key":"mlstab42ecbib30","doi-asserted-by":"publisher","first-page":"2778","DOI":"10.1038\/s41467-018-05171-y","article-title":"Lithography for robust and editable atomic-scale silicon devices and memories","volume":"9","author":"Achal","year":"2018","journal-title":"Nat. Commun."},{"key":"mlstab42ecbib31","doi-asserted-by":"publisher","first-page":"636","DOI":"10.1038\/s41928-018-0180-3","article-title":"Binary atomic silicon logic","volume":"1","author":"Huff","year":"2018","journal-title":"Nat. Electron."},{"key":"mlstab42ecbib32","author":"Goodfellow","year":"2016"},{"key":"mlstab42ecbib33","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An introduction to ROC analysis","volume":"27","author":"Fawcett","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"mlstab42ecbib34","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"mlstab42ecbib35","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1109\/MCI.2018.2840738","article-title":"Recent trends in deep learning based natural language processing","volume":"13","author":"Young","year":"2018","journal-title":"IEEE Comput. Intell. Mag."},{"key":"mlstab42ecbib36","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2014"},{"key":"mlstab42ecbib37","first-page":"2625","article-title":"Long-term recurrent convolutional networks for visual recognition and description","author":"Donahue","year":"2015"},{"key":"mlstab42ecbib38","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ab42ec\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ab42ec","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ab42ec","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ab42ec\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ab42ec\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ab42ec\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ab42ec","content-type":"text\/html","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ab42ec\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,11,22]],"date-time":"2021-11-22T23:31:20Z","timestamp":1637623880000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ab42ec"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,4]]},"references-count":38,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,2,4]]},"published-print":{"date-parts":[[2020,3,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ab42ec","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,4]]},"assertion":[{"value":"Embedding human heuristics in machine-learning-enabled probe microscopy","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2020 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2019-07-31","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2019-09-10","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2020-02-04","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}