{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T15:43:45Z","timestamp":1792079025219,"version":"4.4.0"},"reference-count":42,"publisher":"American Chemical Society (ACS)","issue":"7","license":[{"start":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T00:00:00Z","timestamp":1679875200000},"content-version":"stm-asf","delay-in-days":0,"URL":"https:\/\/doi.org\/10.15223\/policy-029"},{"start":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T00:00:00Z","timestamp":1679875200000},"content-version":"stm-asf","delay-in-days":0,"URL":"https:\/\/doi.org\/10.15223\/policy-037"},{"start":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T00:00:00Z","timestamp":1679875200000},"content-version":"stm-asf","delay-in-days":0,"URL":"https:\/\/doi.org\/10.15223\/policy-045"}],"funder":[{"DOI":"10.13039\/100000185","name":"Defense Advanced Research Projects Agency","doi-asserted-by":"publisher","award":["HR00111920025"],"award-info":[{"award-number":["HR00111920025"]}],"id":[{"id":"10.13039\/100000185","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Machine Learning for Pharmaceutical Discovery and Synthesis Consortium"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,4,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Molecular structure recognition is the task of translating a molecular image into its graph structure. Significant variation in drawing styles and conventions exhibited in chemical literature poses a significant challenge for automating this task. In this paper, we propose MolScribe, a novel image-to-graph generation model that explicitly predicts atoms and bonds, along with their geometric layouts, to construct the molecular structure. Our model flexibly incorporates symbolic chemistry constraints to recognize chirality and expand abbreviated structures. We further develop data augmentation strategies to enhance the model robustness against domain shifts. In experiments on both synthetic and realistic molecular images, MolScribe significantly outperforms previous models, achieving 76\u201393% accuracy on public benchmarks. Chemists can also easily verify MolScribe\u2019s prediction, informed by its confidence estimation and atom-level alignment with the input image. MolScribe is publicly available through Python and web interfaces: https:\/\/github.com\/thomas0809\/MolScribe.<\/jats:p>","DOI":"10.1021\/acs.jcim.2c01480","type":"journal-article","created":{"date-parts":[[2023,3,27]],"date-time":"2023-03-27T07:57:01Z","timestamp":1679903821000},"page":"1925-1934","source":"Crossref","is-referenced-by-count":78,"title":["MolScribe: Robust\nMolecular Structure Recognition\nwith Image-to-Graph Generation"],"prefix":"10.1021","volume":"63","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3747-4552","authenticated-orcid":true,"given":"Yujie","family":"Qian","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology , , , ,","place":["Cambridge, Massachusetts, United States, 02139"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9816-805X","authenticated-orcid":true,"given":"Jiang","family":"Guo","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology , , , ,","place":["Cambridge, Massachusetts, United States, 02139"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhengkai","family":"Tu","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology , , , ,","place":["Cambridge, Massachusetts, United States, 02139"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhening","family":"Li","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology , , , ,","place":["Cambridge, Massachusetts, United States, 02139"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8271-8723","authenticated-orcid":true,"given":"Connor W.","family":"Coley","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology , , , ,","place":["Cambridge, Massachusetts, United States, 02139"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Regina","family":"Barzilay","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology , , , ,","place":["Cambridge, Massachusetts, United States, 02139"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"316","published-online":{"date-parts":[[2023,3,27]]},"reference":[{"key":"2026081804441020000_cit1","doi-asserted-by":"publisher","first-page":"740","DOI":"10.1021\/ci800067r","article-title":"Optical structure recognition software to recover chemical\ninformation: OSRA, an open source solution","volume":"49","author":"Filippov","year":"2009","journal-title":"J. Chem. Inf. Model."},{"key":"2026081804441020000_cit2","article-title":"MOLVEC: Open source library\nfor chemical structure recognition","volume-title":"Abstracts\nof papers of the American Chemical Society","author":"Peryea","year":"2019"},{"key":"2026081804441020000_cit3","doi-asserted-by":"publisher","first-page":"1017","DOI":"10.1021\/acs.jcim.8b00669","article-title":"Molecular Structure\nExtraction from Documents Using Deep Learning","volume":"59","author":"Staker","year":"2019","journal-title":"J. Chem. Inf. Model."},{"key":"2026081804441020000_cit4","doi-asserted-by":"publisher","first-page":"14174","DOI":"10.1039\/D1SC01839F","article-title":"Img2Mol \u2013\naccurate SMILES\nrecognition from molecular graphical depictions","volume":"12","author":"Clevert","year":"2021","journal-title":"Chem. Sci."},{"key":"2026081804441020000_cit5","doi-asserted-by":"crossref","unstructured":"Yoo, S.; Kwon, O.; Lee, H.\n          Image-to-Graph Transformers for Chemical\nStructure Recognition. IEEE International\nConference on Acoustics, Speech and Signal Processing, ICASSP 2022,\nVirtual and Singapore, 23\u201327 May 2022; 2022; pp 3393\u20133397,10.1109\/ICASSP43922.2022.9746088.","DOI":"10.1109\/ICASSP43922.2022.9746088"},{"key":"2026081804441020000_cit6","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1186\/s13321-020-00465-0","article-title":"A review of\noptical chemical structure recognition tools","volume":"12","author":"Rajan","year":"2020","journal-title":"J. Cheminf."},{"key":"2026081804441020000_cit7","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1021\/ci00008a018","article-title":"Kekule: OCR-optical\nchemical (structure) recognition","volume":"32","author":"McDaniel","year":"1992","journal-title":"J. Chem. Inf.\nComput. Sci."},{"key":"2026081804441020000_cit8","doi-asserted-by":"crossref","unstructured":"Casey, R. G.; Boyer, S.; Healey, P.; Miller, A.; Oudot, B.; Zilles, K.\n          Optical recognition of chemical\ngraphics. 2nd International Conference Document\nAnalysis and Recognition, ICDAR \u201993, October 20\u201322,\n1993, Tsukuba City, Japan; 1993; pp 627\u2013631,10.1109\/ICDAR.1993.395658.","DOI":"10.1109\/ICDAR.1993.395658"},{"key":"2026081804441020000_cit9","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1021\/ci00013a010","article-title":"Chemical\nliterature\ndata extraction: the CLiDE Project","volume":"33","author":"Ibison","year":"1993","journal-title":"J. Chem.\nInf. Comput. Sci."},{"key":"2026081804441020000_cit10","doi-asserted-by":"publisher","first-page":"780","DOI":"10.1021\/ci800449t","article-title":"CLiDE Pro: the latest generation of CLiDE, a tool for\noptical chemical structure recognition","volume":"49","author":"Valko","year":"2009","journal-title":"J. Chem.\nInf. Model."},{"key":"2026081804441020000_cit11","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1186\/1752-153X-3-4","article-title":"Automated extraction\nof chemical structure information\nfrom digital raster images","volume":"3","author":"Park","year":"2009","journal-title":"Chem. Cent. J."},{"key":"2026081804441020000_cit12","doi-asserted-by":"crossref","unstructured":"Sadawi, N. M.; Sexton, A. P.; Sorge, V.\n          Chemical structure\nrecognition: a rule-based approach. Document Recognition and Retrieval\nXIX, part of the IS&T-SPIE Electronic\nImaging Symposium, Burlingame, California, USA, January 25\u201326,\n2012, Proceedings; 2012; p 82970E,10.1117\/12.912185.","DOI":"10.1117\/12.912185"},{"key":"2026081804441020000_cit13","doi-asserted-by":"crossref","unstructured":"Algorri, M.-E.; Zimmermann, M.; Friedrich, C. M.; Akle, S.; Hofmann-Apitius, M.\n          Reconstruction\nof Chemical Molecules from Images. 2007 29th\nAnnual International Conference of the IEEE Engineering in Medicine\nand Biology Society; 2007; pp 4609\u20134612,10.1109\/IEMBS.2007.4353366.","DOI":"10.1109\/IEMBS.2007.4353366"},{"key":"2026081804441020000_cit14","doi-asserted-by":"publisher","first-page":"2380","DOI":"10.1021\/ci5002197","article-title":"Markov logic networks\nfor optical chemical structure recognition","volume":"54","author":"Frasconi","year":"2014","journal-title":"J. Chem. Inf. Model."},{"key":"2026081804441020000_cit15","doi-asserted-by":"crossref","unstructured":"Smolov, V.; Zentsev, F.; Rybalkin, M.\n          Imago: Open-Source\nToolkit for 2D Chemical Structure Image Recognition. TREC; 2011.","DOI":"10.6028\/NIST.SP.500-296.chemical-GGA"},{"key":"2026081804441020000_cit16","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1186\/s13321-020-00469-w","article-title":"DECIMER: towards deep learning for\nchemical image recognition","volume":"12","author":"Rajan","year":"2020","journal-title":"J. Cheminf."},{"key":"2026081804441020000_cit17","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1186\/s13321-021-00538-8","article-title":"DECIMER 1.0:\ndeep learning for chemical\nimage recognition using transformers","volume":"13","author":"Rajan","year":"2021","journal-title":"J. Cheminf."},{"key":"2026081804441020000_cit18","doi-asserted-by":"publisher","DOI":"10.1002\/cmtd.202100069","article-title":"Image2SMILES: Transformer-Based\nMolecular Optical Recognition Engine","volume":"2","author":"Khokhlov","year":"2022","journal-title":"Chem.:\nMethods"},{"key":"2026081804441020000_cit19","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1186\/s13321-022-00624-5","article-title":"SwinOCSR: end-to-end\noptical chemical structure recognition using a Swin Transformer","volume":"14","author":"Xu","year":"2022","journal-title":"J. Cheminf."},{"key":"2026081804441020000_cit20","doi-asserted-by":"publisher","first-page":"10622","DOI":"10.1039\/D1SC02957F","article-title":"ChemPix:\nautomated recognition of hand-drawn hydrocarbon\nstructures using deep learning","volume":"12","author":"Weir","year":"2021","journal-title":"Chem. Sci."},{"key":"2026081804441020000_cit21","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1186\/s13321-022-00620-9","article-title":"DECIMER\ue5f8hand-drawn\nmolecule images dataset","volume":"14","author":"Brinkhaus","year":"2022","journal-title":"J. Cheminf."},{"key":"2026081804441020000_cit22","doi-asserted-by":"publisher","first-page":"4506","DOI":"10.1021\/acs.jcim.0c00459","article-title":"ChemGrapher: optical\ngraph recognition of chemical compounds by deep learning","volume":"60","author":"Oldenhof","year":"2020","journal-title":"J. Chem. Inf. Model."},{"key":"2026081804441020000_cit23","doi-asserted-by":"publisher","first-page":"5321","DOI":"10.1021\/acs.jcim.2c00733","article-title":"MolMiner: You Only Look Once for Chemical Structure\nRecognition","volume":"62","author":"Xu","year":"2022","journal-title":"J. Chem. Inf. Model."},{"key":"2026081804441020000_cit24","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1021\/ci00057a005","article-title":"SMILES, a\nchemical language and information system. 1. Introduction to methodology\nand encoding rules","volume":"28","author":"Weininger","year":"1988","journal-title":"J. Chem. Inf. Comput. Sci."},{"key":"2026081804441020000_cit25","doi-asserted-by":"publisher","first-page":"244","DOI":"10.1021\/ci00007a012","article-title":"Description\nof several chemical structure\nfile formats used by computer programs developed at Molecular Design\nLimited","volume":"32","author":"Dalby","year":"1992","journal-title":"J. Chem. Inf. Comput. Sci."},{"key":"2026081804441020000_cit26","unstructured":"Chen, T.; Saxena, S.; Li, L.; Fleet, D.\nJ.; Hinton, G.\n          Pix2seq: A Language Modeling\nFramework for Object Detection. International\nConference on Learning Representations; 2021."},{"key":"2026081804441020000_cit27","unstructured":"SmilesPE\natomwise_tokenizer. https:\/\/xinhaoli74.github.io\/SmilesPE\/pretokenizer\/#atomwise_tokenizer (accessed 2022-10-01)."},{"key":"2026081804441020000_cit28","unstructured":"Landrum, G.\n          \n          RDKit: Open-Source\nCheminformatics Software; 2016."},{"key":"2026081804441020000_cit29","unstructured":"Battaglia, P. W.; Hamrick, J. B.; Bapst, V.; Sanchez-Gonzalez, A.; Zambaldi, V.; Malinowski, M.; Tacchetti, A.; Raposo, D.; Santoro, A.; Faulkner, R.; \n               , et al.\n          Relational inductive biases, deep learning, and graph networks. 2018, arXiv:1806.01261. arXiv preprint. https:\/\/arxiv.org\/abs\/1806.01261 (accessed 2023-20-23)."},{"key":"2026081804441020000_cit30","doi-asserted-by":"publisher","first-page":"D1202","DOI":"10.1093\/nar\/gkv951","article-title":"PubChem\nSubstance and Compound databases","volume":"44","author":"Kim","year":"2016","journal-title":"Nucleic Acids\nRes."},{"key":"2026081804441020000_cit31","doi-asserted-by":"publisher","first-page":"P4","DOI":"10.1186\/1758-2946-3-S1-P4","article-title":"Indigo: universal cheminformatics\nAPI","volume":"3","author":"Pavlov","year":"2011","journal-title":"J. Cheminf."},{"key":"2026081804441020000_cit32","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1186\/s13321-022-00609-4","article-title":"RanDepict:\nRandom chemical structure depiction generator","volume":"14","author":"Brinkhaus","year":"2022","journal-title":"J. Cheminf."},{"key":"2026081804441020000_cit33","doi-asserted-by":"publisher","first-page":"598","DOI":"10.1002\/qsar.200290002","article-title":"Identification\nof Diverse Database Subsets using Property-Based and Fragment-Based\nMolecular Descriptions","volume":"21","author":"Ashton","year":"2002","journal-title":"Quant. Struct.-Act.\nRelat."},{"key":"2026081804441020000_cit34","unstructured":"United States Patent and Trademark\nOffice\n          \n          Grant Red Book. https:\/\/bulkdata.uspto.gov (accessed 2021-12-31)."},{"key":"2026081804441020000_cit35","doi-asserted-by":"publisher","first-page":"930","DOI":"10.1021\/ci950173l","article-title":"The sheffield generic\nstructures project a retrospective\nreview","volume":"36","author":"Lynch","year":"1996","journal-title":"J. Chem. Inf. Comput. Sci."},{"key":"2026081804441020000_cit36","doi-asserted-by":"crossref","unstructured":"Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; Guo, B.\n          Swin Transformer:\nHierarchical Vision Transformer Using Shifted Windows. Proceedings of the IEEE\/CVF International Conference on\nComputer Vision (ICCV); 2021; pp 10012\u201310022,10.1109\/ICCV48922.2021.00986.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"2026081804441020000_cit37","unstructured":"Swin\nTransformer. https:\/\/github.com\/microsoft\/Swin-Transformer (accessed 2022-10-01)."},{"key":"2026081804441020000_cit38","unstructured":"Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.\nN.; Kaiser, \u0141.; Polosukhin, I.\n          Attention is all you need. Advances in neural information processing systems; 2017; Vol. 30."},{"key":"2026081804441020000_cit39","unstructured":"DECIMER\nImage Transformer V2. https:\/\/github.com\/Kohulan\/DECIMER-Image_Transformer (accessed 2023-1-20)."},{"key":"2026081804441020000_cit40","doi-asserted-by":"publisher","first-page":"2059","DOI":"10.1021\/acs.jcim.0c00042","article-title":"ChemSchematicResolver:\nA toolkit to decode 2D chemical\ndiagrams with labels and R-groups into annotated chemical named entities","volume":"60","author":"Beard","year":"2020","journal-title":"J. Chem. Inf. Model."},{"key":"2026081804441020000_cit41","doi-asserted-by":"publisher","first-page":"4962","DOI":"10.1021\/acs.jcim.1c01017","article-title":"ReactionDataExtractor: A Tool for\nAutomated Extraction\nof Information from Chemical Reaction Schemes","volume":"61","author":"Wilary","year":"2021","journal-title":"J. Chem. Inf. Model."},{"key":"2026081804441020000_cit42","doi-asserted-by":"publisher","first-page":"2035","DOI":"10.1021\/acs.jcim.1c00284","article-title":"Automated Chemical\nReaction Extraction\nfrom Scientific Literature","volume":"62","author":"Guo","year":"2022","journal-title":"J. Chem. Inf. Model"}],"container-title":["Journal of Chemical Information\nand Modeling"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.acs.org\/doi\/pdf\/10.1021\/acs.jcim.2c01480","content-type":"application\/pdf","content-version":"vor","intended-application":"unspecified"},{"URL":"https:\/\/pubs.acs.org\/jcisd8\/article-pdf\/63\/7\/1925\/8910293\/ci2c01480.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/pubs.acs.org\/jcisd8\/article-pdf\/63\/7\/1925\/8910293\/ci2c01480.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T09:55:39Z","timestamp":1787046939000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.acs.org\/jcisd8\/article\/63\/7\/1925\/850390\/MolScribe-Robust-Molecular-Structure-Recognition"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,27]]},"references-count":42,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,3,27]]},"published-print":{"date-parts":[[2023,4,10]]}},"URL":"https:\/\/doi.org\/10.1021\/acs.jcim.2c01480","relation":{},"ISSN":["1549-9596","1549-960X"],"issn-type":[{"value":"1549-9596","type":"print"},{"value":"1549-960X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,27]]}}}