{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T17:12:10Z","timestamp":1790961130404,"version":"4.1.0"},"reference-count":39,"publisher":"American Chemical Society (ACS)","issue":"10","license":[{"start":{"date-parts":[[2020,9,14]],"date-time":"2020-09-14T00:00:00Z","timestamp":1600041600000},"content-version":"stm-asf","delay-in-days":0,"URL":"https:\/\/doi.org\/10.15223\/policy-029"},{"start":{"date-parts":[[2020,9,14]],"date-time":"2020-09-14T00:00:00Z","timestamp":1600041600000},"content-version":"stm-asf","delay-in-days":0,"URL":"https:\/\/doi.org\/10.15223\/policy-037"},{"start":{"date-parts":[[2020,9,14]],"date-time":"2020-09-14T00:00:00Z","timestamp":1600041600000},"content-version":"stm-asf","delay-in-days":0,"URL":"https:\/\/doi.org\/10.15223\/policy-045"}],"funder":[{"DOI":"10.13039\/501100010767","name":"Innovative Medicines Initiative","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100010767","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011878","name":"Vlaamse regering","doi-asserted-by":"publisher","award":["VR 2019 2203"],"award-info":[{"award-number":["VR 2019 2203"]}],"id":[{"id":"10.13039\/501100011878","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004497","name":"Onderzoeksraad, KU Leuven","doi-asserted-by":"publisher","award":["C14\/18\/092 Sym-BioSys3"],"award-info":[{"award-number":["C14\/18\/092 Sym-BioSys3"]}],"id":[{"id":"10.13039\/501100004497","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003132","name":"Agentschap voor Innovatie door Wetenschap en Technologie","doi-asserted-by":"publisher","award":["FWO 06260"],"award-info":[{"award-number":["FWO 06260"]}],"id":[{"id":"10.13039\/501100003132","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Central Europe Leuven Strategic Alliance"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,10,26]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>In drug discovery, knowledge of the graph structure of chemical compounds is essential. Many thousands of scientific articles and patents in chemistry and pharmaceutical sciences have investigated chemical compounds, but in many cases, the details of the structure of these chemical compounds are published only as an image. A tool to analyze these images automatically and convert them into a chemical graph structure would be useful for many applications, such as drug discovery. A few such tools are available and they are mostly derived from optical character recognition. However, our evaluation of the performance of these tools reveals that they often make mistakes in recognizing the correct bond multiplicity and stereochemical information. In addition, errors sometimes even lead to missing atoms in the resulting graph. In our work, we address these issues by developing a compound recognition method based on machine learning. More specifically, we develop a deep neural network model for optical compound recognition. The deep learning solution presented here consists of a segmentation model, followed by three classification models that predict atom locations, bonds, and charges. Furthermore, this model not only predicts the graph structure of the molecule but also provides all information necessary to relate each component of the resulting graph to the source image. This solution is scalable and can rapidly process thousands of images. Finally, we empirically compare the proposed method with the well-established tool OSRA1 and observe significant error reduction.<\/jats:p>","DOI":"10.1021\/acs.jcim.0c00459","type":"journal-article","created":{"date-parts":[[2020,9,14]],"date-time":"2020-09-14T13:38:51Z","timestamp":1600090731000},"page":"4506-4517","source":"Crossref","is-referenced-by-count":59,"title":["ChemGrapher: Optical Graph Recognition of Chemical\nCompounds by Deep Learning"],"prefix":"10.1021","volume":"60","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4916-3014","authenticated-orcid":true,"given":"Martijn","family":"Oldenhof","sequence":"first","affiliation":[{"name":"KU Leuven , , Kasteelpark Arenberg 10 , ,","place":["Leuven, Belgium, 3001"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adam","family":"Arany","sequence":"additional","affiliation":[{"name":"KU Leuven , , Kasteelpark Arenberg 10 , ,","place":["Leuven, Belgium, 3001"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yves","family":"Moreau","sequence":"additional","affiliation":[{"name":"KU Leuven , , Kasteelpark Arenberg 10 , ,","place":["Leuven, Belgium, 3001"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jaak","family":"Simm","sequence":"additional","affiliation":[{"name":"KU Leuven , , Kasteelpark Arenberg 10 , ,","place":["Leuven, Belgium, 3001"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"316","published-online":{"date-parts":[[2020,9,14]]},"reference":[{"key":"2026081804381190100_cit1","doi-asserted-by":"publisher","first-page":"740","DOI":"10.1021\/ci800067r","article-title":"Optical Structure\nRecognition Software To Recover Chemical\nInformation: OSRA, An Open Source Solution","volume":"49","author":"Filippov","year":"2009","journal-title":"J. Chem. Inf. Model."},{"key":"2026081804381190100_cit2","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1021\/ci00057a005","article-title":"SMILES, a\nChemical Language and Information System. 1. Introduction to Methodology\nand Encoding Rules","volume":"28","author":"Weininger","year":"1988","journal-title":"J. Chem. Inf. Comput. Sci."},{"key":"2026081804381190100_cit3","doi-asserted-by":"publisher","first-page":"244","DOI":"10.1021\/ci00007a012","article-title":"Description\nof Several Chemical Structure\nFile Formats Used by Computer Programs Developed at Molecular Design\nLimited","volume":"32","author":"Dalby","year":"1992","journal-title":"J. Chem. Inf. Comput. Sci."},{"key":"2026081804381190100_cit4","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1186\/1752-153x-3-4","article-title":"Automated Extraction of Chemical Structure Information\nfrom Digital Raster Images","volume":"3","author":"Park","year":"2009","journal-title":"Chem. Cent. J."},{"key":"2026081804381190100_cit5","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1021\/ci00008a018","article-title":"Kekule: OCR-Optical Chemical (Structure) Recognition","volume":"32","author":"McDaniel","year":"1992","journal-title":"J. Chem. Inf. Comput. Sci."},{"key":"2026081804381190100_cit6","doi-asserted-by":"publisher","first-page":"780","DOI":"10.1021\/ci800449t","article-title":"CLiDE Pro: The Latest Generation of CLiDE, a Tool for\nOptical Chemical Structure Recognition","volume":"49","author":"Valko","year":"2009","journal-title":"J. Chem.\nInf. Model."},{"key":"2026081804381190100_cit7","doi-asserted-by":"crossref","unstructured":"Sadawi, M. N.; Sexton, A. P.; Sorge, V.\n          Chemical Structure\nRecognition: A Rule Based Approach. Proceedings of\nSPIE. Document Recognition and Retrieval XIX, 2012; Vol. 8297.","DOI":"10.1117\/12.912185"},{"key":"2026081804381190100_cit8","doi-asserted-by":"publisher","first-page":"1017","DOI":"10.1021\/acs.jcim.8b00669","article-title":"Molecular Structure\nExtraction from Documents Using Deep Learning","volume":"59","author":"Staker","year":"2019","journal-title":"J. Chem. Inf. Model."},{"key":"2026081804381190100_cit9","unstructured":"Soderberg, T.\n          \n          Chirality and\nstereoisomers. https:\/\/chem.libretexts.org\/Bookshelves\/Organic_Chemistry\/Book%3A_Organic_Chemistry_with_a_Biological_Emphasis_(Soderberg)\/Chapter_03%3A_Conformations_and_Stereochemistry\/03.3%3A_Stereoisomerism_%E2%80%93_chirality%2C_stereocenters%2C_enantiomers (accessed on 03.06.2019)."},{"key":"2026081804381190100_cit10","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"2026081804381190100_cit11","unstructured":"LeCun, Y.; Bengio, Y.\n          Convolutional\nNetworks for Images, Speech, and Time Series. The Handbook of Brain Theory and Neural Networks, 1995; Vol. 3361, p 1995."},{"key":"2026081804381190100_cit12","doi-asserted-by":"crossref","unstructured":"Ronneberger, O.; Fischer, O.; Brox, T.\n          U-Net: Convolutional\nNetworks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI); MICCAI, 2015; pp 234\u2013241.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"2026081804381190100_cit13","doi-asserted-by":"crossref","DOI":"10.1109\/ICCV.2017.322","article-title":"Mask R-CNN","author":"He","year":"2017"},{"key":"2026081804381190100_cit14","unstructured":"Jansson, A.; Humphrey, E. J.; Montecchio, N.; Bittner, R. M.; Kumar, A.; Weyde, T.\n          Singing Voice Separation with\nDeep U-Net Convolutional Networks. ISMIR, 2017."},{"key":"2026081804381190100_cit15","article-title":"3D\nU-Net: Learning Dense Volumetric Segmentation from Sparse Annotation","author":"C\u0327ic\u0327ek","year":"2016"},{"key":"2026081804381190100_cit16","article-title":"Multi-Scale Context\nAggregation by Dilated Convolutions","author":"Yu","year":"2016"},{"key":"2026081804381190100_cit17","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1109\/72.279181","article-title":"Learning Long-term Dependencies with\nGradient Descent is Difficult","volume":"5","author":"Bengio","year":"1994","journal-title":"IEEE Trans. Neural\nNetwork."},{"key":"2026081804381190100_cit18","doi-asserted-by":"crossref","unstructured":"He, K.; Zhang, X.; Ren, S.; Sun, J.\n          Deep Residual Learning for\nImage Recognition. 2016 IEEE Conference on\nComputer Vision and Pattern Recognition\n(CVPR), 2016; pp 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"2026081804381190100_cit19","article-title":"Road Extraction by Deep Residual U-Net","author":"Zhang","year":"2017"},{"key":"2026081804381190100_cit20","article-title":"Batch Normalization:\nAccelerating Deep Network Training by Reducing Internal Covariate\nShift","author":"Ioffe","year":"2015"},{"key":"2026081804381190100_cit21","article-title":"Fast and Accurate\nDeep Network Learning by Exponential Linear Units (ELUs)","author":"Clevert","year":"2015"},{"key":"2026081804381190100_cit22","doi-asserted-by":"publisher","first-page":"1241","DOI":"10.1016\/j.drudis.2018.01.039","article-title":"The Rise of Deep Learning in Drug\nDiscovery","volume":"23","author":"Chen","year":"2018","journal-title":"Drug Discovery Today"},{"key":"2026081804381190100_cit23","doi-asserted-by":"crossref","unstructured":"Xu, Z.; Wang, S.; Zhu, F.; Huang, J.\n          Seq2Seq Fingerprint: An Unsupervised\nDeep Molecular\nEmbedding for Drug Discovery. Proceedings\nof the 8th ACM International Conference on Bioinformatics, Computational\nBiology, and Health Informatics, New York,\nNY, USA, 2017; pp 285\u2013294.","DOI":"10.1145\/3107411.3107424"},{"key":"2026081804381190100_cit24","doi-asserted-by":"publisher","first-page":"1692","DOI":"10.1039\/c8sc04175j","article-title":"Learning Continuous\nand Data-driven Molecular Descriptors by Translating Equivalent Chemical\nRepresentations","volume":"10","author":"Winter","year":"2019","journal-title":"Chem. Sci."},{"key":"2026081804381190100_cit25","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1021\/acscentsci.7b00572","article-title":"Automatic\nChemical Design Using a Data-Driven Continuous Representation of Molecules","volume":"4","author":"Go\u0301mez-Bombarelli","year":"2018","journal-title":"ACS Cent. Sci."},{"key":"2026081804381190100_cit26","unstructured":"Duvenaud, D. K.; Maclaurin, D.; Iparraguirre, J.; Bombarell, R.; Hirzel, T.; Aspuru-Guzik, A.; Adams, R.\nP.\n          Convolutional Networks\non Graphs for Learning Molecular Fingerprints. Advances in Neural Information Processing Systems, 2015; pp 2224\u20132232."},{"key":"2026081804381190100_cit27","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1007\/s10822-016-9938-8","article-title":"Molecular\nGraph Convolutions: Moving Beyond Fingerprints","volume":"30","author":"Kearnes","year":"2016","journal-title":"J. Comput.-Aided Mol. Des."},{"key":"2026081804381190100_cit28","doi-asserted-by":"publisher","first-page":"1757","DOI":"10.1021\/acs.jcim.6b00601","article-title":"Convolutional\nEmbedding of Attributed Molecular Graphs for Physical Property Prediction","volume":"57","author":"Coley","year":"2017","journal-title":"J. Chem. Inf. Model."},{"key":"2026081804381190100_cit29","article-title":"Graph Informer Networks for\nMolecules","author":"Simm","year":"2019"},{"key":"2026081804381190100_cit30","doi-asserted-by":"publisher","first-page":"4066","DOI":"10.1021\/acs.jmedchem.5b00104","article-title":"pkCSM: Predicting Small-Molecule\nPharmacokinetic and Toxicity Properties Using Graph-Based Signatures","volume":"58","author":"Pires","year":"2015","journal-title":"J. Med. Chem."},{"key":"2026081804381190100_cit31","article-title":"Chemception: A Deep Neural\nNetwork with Minimal Chemistry Knowledge Matches the Performance of\nExpert-developed QSAR\/QSPR Models","author":"Goh","year":"2017"},{"key":"2026081804381190100_cit32","doi-asserted-by":"publisher","first-page":"640","DOI":"10.1109\/tpami.2016.2572683","article-title":"Fully Convolutional\nNetworks for\nSemantic Segmentation","volume":"39","author":"Shelhamer","year":"2017","journal-title":"IEEE Trans. Pattern Anal.\nMach. Intell."},{"key":"2026081804381190100_cit33","doi-asserted-by":"crossref","unstructured":"Chollet, F.\n          \n          Xception: Deep\nLearning With Depthwise Separable Convolutions. The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.","DOI":"10.1109\/CVPR.2017.195"},{"key":"2026081804381190100_cit34","doi-asserted-by":"publisher","first-page":"D945","DOI":"10.1093\/nar\/gkw1074","article-title":"The ChEMBL\ndatabase\nin 2017","volume":"45","author":"Gaulton","year":"2017","journal-title":"Nucleic Acids Res."},{"key":"2026081804381190100_cit35","unstructured":"Landrum, G.\n          \n          RDKit: Open-source\ncheminformatics. http:\/\/www.rdkit.org (accessed on 03.06.2019)."},{"key":"2026081804381190100_cit36","unstructured":"Fork of the official sources for the RDKit library. https:\/\/github.com\/biolearning-stadius\/rdkit (accessed on 17.07.2020)."},{"key":"2026081804381190100_cit37","unstructured":"Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; Lerer, A.\n          Automatic Differentiation in PyTorch, NIPS-W, 2017."},{"key":"2026081804381190100_cit38","unstructured":"F1 score. 2020. https:\/\/en.wikipedia.org\/w\/index.php?title=F1_score&oldid=935689499, Page Version ID: 935689499 (accessed on 20.01.2020)."},{"key":"2026081804381190100_cit39","unstructured":"Maybridge dataset. https:\/\/www.cs.bham.ac.uk\/research\/groupings\/reasoning\/sdag\/mol-dataset.php (accessed on 10.06.2020)."}],"container-title":["Journal of Chemical Information and Modeling"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.acs.org\/doi\/pdf\/10.1021\/acs.jcim.0c00459","content-type":"application\/pdf","content-version":"vor","intended-application":"unspecified"},{"URL":"https:\/\/pubs.acs.org\/jcisd8\/article-pdf\/60\/10\/4506\/8896912\/ci0c00459.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/pubs.acs.org\/jcisd8\/article-pdf\/60\/10\/4506\/8896912\/ci0c00459.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T09:42:33Z","timestamp":1787046153000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.acs.org\/jcisd8\/article\/60\/10\/4506\/849621\/ChemGrapher-Optical-Graph-Recognition-of-Chemical"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,14]]},"references-count":39,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2020,9,14]]},"published-print":{"date-parts":[[2020,10,26]]}},"URL":"https:\/\/doi.org\/10.1021\/acs.jcim.0c00459","relation":{},"ISSN":["1549-9596","1549-960X"],"issn-type":[{"value":"1549-9596","type":"print"},{"value":"1549-960X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,14]]}}}