{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T02:13:32Z","timestamp":1790820812671,"version":"4.1.0"},"reference-count":44,"publisher":"American Chemical Society (ACS)","issue":"7","license":[{"start":{"date-parts":[[2023,3,20]],"date-time":"2023-03-20T00:00:00Z","timestamp":1679270400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000287","name":"Royal Academy of Engineering","doi-asserted-by":"publisher","award":["RCSRC\\1819\\7\\10"],"award-info":[{"award-number":["RCSRC\\1819\\7\\10"]}],"id":[{"id":"10.13039\/501100000287","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000590","name":"Christ's College, University of Cambridge","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000590","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004543","name":"China Scholarship Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004543","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003343","name":"Cambridge Trust","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003343","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,4,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Text mining in the optical-materials domain is becoming increasingly important as the number of scientific publications in this area grows rapidly. Language models such as Bidirectional Encoder Representations from Transformers (BERT) have opened up a new era and brought a significant boost to state-of-the-art natural-language-processing (NLP) tasks. In this paper, we present two \u201cmaterials-aware\u201d text-based language models for optical research, OpticalBERT and OpticalPureBERT, which are trained on a large corpus of scientific literature in the optical-materials domain. These two models outperform BERT and previous state-of-the-art models in a variety of text-mining tasks about optical materials. We also release the first \u201cmaterials-aware\u201d table-based language model, OpticalTable-SQA. This is a querying facility that solicits answers to questions about optical materials using tabular information that pertains to this scientific domain. The OpticalTable-SQA model was realized by fine-tuning the Tapas-SQA model using a manually annotated OpticalTableQA data set which was curated specifically for this work. While preserving its sequential question-answering performance on general tables, the OpticalTable-SQA model significantly outperforms Tapas-SQA on optical-materials-related tables. All models and data sets are available to the optical-materials-science community.<\/jats:p>","DOI":"10.1021\/acs.jcim.2c01259","type":"journal-article","created":{"date-parts":[[2023,3,20]],"date-time":"2023-03-20T15:48:46Z","timestamp":1679327326000},"page":"1961-1981","source":"Crossref","is-referenced-by-count":52,"title":["OpticalBERT and\nOpticalTable-SQA: Text- and Table-Based\nLanguage Models for the Optical-Materials Domain"],"prefix":"10.1021","volume":"63","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2561-352X","authenticated-orcid":true,"given":"Jiuyang","family":"Zhao","sequence":"first","affiliation":[{"name":"University of Cambridge , , J. J. Thomson Avenue , ,","place":["Cambridge, U.K., CB3 0HE"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1901-8361","authenticated-orcid":true,"given":"Shu","family":"Huang","sequence":"additional","affiliation":[{"name":"University of Cambridge , , J. J. Thomson Avenue , ,","place":["Cambridge, U.K., CB3 0HE"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1552-8743","authenticated-orcid":true,"given":"Jacqueline M.","family":"Cole","sequence":"additional","affiliation":[{"name":"University of Cambridge , , J. J. Thomson Avenue , ,","place":["Cambridge, U.K., CB3 0HE"]},{"name":"Rutherford Appleton Laboratory, Harwell Science and Innovation Campus , , , ,","place":["Didcot, Oxfordshire, U.K., OX11 0QX"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"316","published-online":{"date-parts":[[2023,3,20]]},"reference":[{"key":"2026081804443111100_cit1","doi-asserted-by":"crossref","unstructured":"Garcia de Abajo, F. J.\n          \n          Engineering Materials with Extreme Optical Properties. In Proceedings of the Photonic Metamaterials: From Random\nto Periodic, TuA2; Optical Society of America, 2006.","DOI":"10.1364\/META.2006.TuA2"},{"issue":"7","key":"2026081804443111100_cit2","doi-asserted-by":"publisher","first-page":"1915","DOI":"10.1021\/ma502569r","article-title":"Recent Progress\nin High Refractive Index Polymers","volume":"48","author":"Higashihara","year":"2015","journal-title":"Macromolecules"},{"issue":"7","key":"2026081804443111100_cit3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1117\/1.JBO.24.7.071606","article-title":"Machine learning approach\nfor rapid and accurate estimation of optical\nproperties using spatial frequency domain imaging","volume":"24","author":"Panigrahi","year":"2019","journal-title":"J. Biomed. Opt."},{"issue":"10","key":"2026081804443111100_cit4","doi-asserted-by":"publisher","first-page":"1894","DOI":"10.1021\/acs.jcim.6b00207","article-title":"ChemDataExtractor:\nA Toolkit for Automated Extraction\nof Chemical Information from the Scientific Literature","volume":"56","author":"Swain","year":"2016","journal-title":"J. Chem. Inf. Model."},{"issue":"9","key":"2026081804443111100_cit5","doi-asserted-by":"publisher","first-page":"4280","DOI":"10.1021\/acs.jcim.1c00446","article-title":"ChemDataExtractor\n2.0: Autopopulated Ontologies for Materials Science","volume":"61","author":"Mavrac\u030cic\u0301","year":"2021","journal-title":"J. Chem. Inf. Model."},{"issue":"1","key":"2026081804443111100_cit6","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1038\/s41597-020-00602-2","article-title":"A database of battery\nmaterials auto-generated using\nChemDataExtractor","volume":"7","author":"Huang","year":"2020","journal-title":"Sci. Data"},{"key":"2026081804443111100_cit7","doi-asserted-by":"publisher","first-page":"180111","DOI":"10.1038\/sdata.2018.111","article-title":"Auto-generated materials\ndatabase of Curie and Ne\u0301el\ntemperatures via semi-supervised relationship extraction","volume":"5","author":"Court","year":"2018","journal-title":"Sci. Data"},{"issue":"1","key":"2026081804443111100_cit8","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1038\/s41597-022-01295-5","article-title":"A database of refractive\nindices and dielectric constants\nauto-generated using ChemDataExtractor","volume":"9","author":"Zhao","year":"2022","journal-title":"Sci.\nData"},{"issue":"1","key":"2026081804443111100_cit9","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1038\/s41597-019-0306-0","article-title":"Comparative dataset of experimental and computational\nattributes of UV\/vis absorption spectra","volume":"6","author":"Beard","year":"2019","journal-title":"Sci.\nData"},{"key":"2026081804443111100_cit10","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1038\/s41597-022-01355-w","article-title":"Perovskite- and\nDye-Sensitized Solar-Cell Device Databases\nAuto-generated Using ChemDataExtractor","volume":"9","author":"Beard","year":"2022","journal-title":"Sci.\nData"},{"issue":"1","key":"2026081804443111100_cit11","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1038\/s41597-022-01294-6","article-title":"Auto-generated database\nof semiconductor band gaps\nusing ChemDataExtractor","volume":"9","author":"Dong","year":"2022","journal-title":"Scientific Data"},{"issue":"1","key":"2026081804443111100_cit12","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1038\/s41597-022-01301-w","article-title":"Auto-generating\ndatabases of Yield\nStrength and Grain Size using ChemDataExtractor","volume":"9","author":"Kumar","year":"2022","journal-title":"Sci. Data"},{"key":"2026081804443111100_cit13","doi-asserted-by":"crossref","unstructured":"Peters, M. E.; Neumann, M.; Iyyer, M.; Gardner, M.; Clark, C.; Lee, K.; Zettlemoyer, L.\n          Deep Contextualized Word Representations. In Proceedings of the 2018 Conference of the North American\nChapter of the Association for Computational Linguistics: Human Language\nTechnologies, Vol. 1 (Long Papers), 2227\u20132237; Association for Computational Linguistics, New Orleans, LA, 2018;10.18653\/v1\/N18-1202.","DOI":"10.18653\/v1\/N18-1202"},{"key":"2026081804443111100_cit14","unstructured":"Brown, T. B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T.; Child, R.; Ramesh, A.; Ziegler, D. M.; Wu, J.; Winter, C.; Hesse, C.; Chen, M.; Sigler, E.; Litwin, M.; Gray, S.; Chess, B.; Clark, J.; Berner, C.; McCandlish, S.; Radford, A.; Sutskever, I.; Amodei, D.\n          Language Models\nare Few-Shot Learners. 2020, 2005.14165. ArXiv2005.14165. https:\/\/arxiv.org\/abs\/2005.14165 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit15","unstructured":"Devlin, J.; Chang, M.; Lee, K.; Toutanova, K.\n          BERT: Pre-training of Deep\nBidirectional\nTransformers for Language Understanding. 2018, 1810.04805. ArXiv1810.04805. https:\/\/arxiv.org\/abs\/1810.04805 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit16","unstructured":"Lee, J.; Yoon, W.; Kim, S.; Kim, D.; Kim, S.; So, C. H.; Kang, J.\n          BioBERT:\na pre-trained biomedical language representation model for biomedical\ntext mining. 2019, 1901.08746. ArXiv1901.08746. https:\/\/arxiv.org\/abs\/1901.08746 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit17","unstructured":"Gupta, T.; Zaki, M.; Krishnan, N. M. A.; Mausam. MatSciBERT: A Materials Domain Language\nModel for Text Mining and\nInformation Extraction. 2021, 2109.15290. ArXiv2109.15290. https:\/\/arxiv.org\/abs\/2109.15290 (accessed 2023-03-06)."},{"issue":"4","key":"2026081804443111100_cit18","doi-asserted-by":"publisher","first-page":"100488","DOI":"10.1016\/j.patter.2022.100488","article-title":"Quantifying the advantage of domain-specific\npre-training on named\nentity recognition tasks in materials science","volume":"3","author":"Trewartha","year":"2022","journal-title":"Patterns"},{"key":"2026081804443111100_cit19","doi-asserted-by":"publisher","first-page":"6365","DOI":"10.1021\/acs.jcim.2c00035","article-title":"BatteryBERT: A Pretrained Language Model for Battery\nDatabase Enhancement","volume":"62","author":"Huang","year":"2022","journal-title":"J. Chem. Inf. Model"},{"key":"2026081804443111100_cit20","unstructured":"National Science\nand Technology\nCouncil. Materials Genome Initiative for Global Competitiveness; Executive Office of the President, National Science\nand Technology Council, 2011."},{"key":"2026081804443111100_cit21","unstructured":"Vakulenko, S.; Savenkov, V.\n          TableQA:\nQuestion\nAnswering on Tabular Data.  2017, 1705.06504. ArXiv1705.06504. https:\/\/arxiv.org\/abs\/1705.06504 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit22","doi-asserted-by":"crossref","unstructured":"Herzig, J.; Nowak, P.\nK.; Mu\u0308ller, T.; Piccinno, F.; Eisenschlos, J. M.\n          Tapas: Weakly\nSupervised Table Parsing via Pre-training. In Proceedings of the 58th Annual Meeting of the Association for Computational\nLinguistics (Vol. 1: Long Papers); Seattle,\nWashington, United States, 2020;10.18653\/v1\/2020.acl-main.398.","DOI":"10.18653\/v1\/2020.acl-main.398"},{"key":"2026081804443111100_cit23","unstructured":"Chemmengath, S. A.; Kumar, V.; Bharadwaj, S.; Sen, J.; Canim, M.; Chakrabarti, S.; Gliozzo, A.; Sankaranarayanan, K.\n          Topic Transferable Table Question\nAnswering. 2021, 2109.07377. ArXiv2109.07377. https:\/\/arxiv.org\/abs\/2109.07377 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit24","unstructured":"Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.\nN.; Kaiser, L.; Polosukhin, I.\n          Attention Is All You Need. 2017, 1706.03762. ArXiv1706.03762.  https:\/\/arxiv.org\/abs\/1706.03762 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit25","unstructured":"Beltagy, I.; Cohan, A.; Lo, K.\n          SciBERT: Pretrained Contextualized\nEmbeddings for Scientific Text. 2019,\n1903.10676. ArXiv1903.10676. https:\/\/arxiv.org\/abs\/1903.10676 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit26","unstructured":"Wolf, T.; Debut, L.; Sanh, V.; Chaumond, J.; Delangue, C.; Moi, A.; Cistac, P.; Rault, T.; Louf, R.; Funtowicz, M.; Brew, J.\n          HuggingFace\u2019s\nTransformers: State-of-the-art Natural Language Processing. 2019, 1910.03771. ArXiv1910.03771. https:\/\/arxiv.org\/abs\/1910.03771 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit27","unstructured":"Wu, Y.; Schuster, M.; Chen, Z.; Le, Q. V.; Norouzi, M.; Macherey, W.; Krikun, M.; Cao, Y.; Gao, Q.; Macherey, K.; Klingner, J.; Shah, A.; Johnson, M.; Liu, X.; Kaiser, L.; Gouws, S.; Kato, Y.; Kudo, T.; Kazawa, H.; Stevens, K.; Kurian, G.; Patil, N.; Wang, W.; Young, C.; Smith, J.; Riesa, J.; Rudnick, A.; Vinyals, O.; Corrado, G.; Hughes, M.; Dean, J.\n          Google\u2019s Neural Machine Translation\nSystem: Bridging the Gap between Human and Machine Translation. 2016, 1609.08144. ArXiv1609.08144. https:\/\/arxiv.org\/abs\/1609.08144 (accessed 2023-03-06)."},{"issue":"59","key":"2026081804443111100_cit28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/14786448008626877","article-title":"On the diagrammatic and mechanical representation of propositions\nand reasonings","volume":"10","author":"Venn\nM.A","year":"1880","journal-title":"London, Edinburgh, and Dublin\nPhilosophical Magazine and Journal of Science"},{"key":"2026081804443111100_cit29","unstructured":"A\u0301cs, J.\n          \n          Exploring\nBERT\u2019s Vocabulary. http:\/\/juditacs.github.io\/2019\/02\/19\/bert-tokenization-stats.html (accessed 2022-05-30)."},{"key":"2026081804443111100_cit30","unstructured":"Rust, P.; Pfeiffer, J.; Vulic, I.; Ruder, S.; Gurevych, I.\n          How Good is Your Tokenizer? On the\nMonolingual Performance of Multilingual Language Models. 2020, 2012.15613. ArXiv2012.15613. https:\/\/arxiv.org\/abs\/2012.15613 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit31","unstructured":"Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; Lerer, A.\n          Automatic Differentiation in PyTorch. In NIPS 2017 Workshop on Autodiff ; 2017."},{"key":"2026081804443111100_cit32","unstructured":"Liu, P.; Yuan, W.; Fu, J.; Jiang, Z.; Hayashi, H.; Neubig, G.\n          Pre-train, Prompt, and Predict:\nA Systematic Survey\nof Prompting Methods in Natural Language Processing. 2021, 2107.13586. ArXiv2107.13586. https:\/\/arxiv.org\/abs\/2107.13586 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit33","unstructured":"Rajpurkar, P.; Zhang, J.; Lopyrev, K.; Liang, P.\n          SQuAD: 100, 000+ Questions for Machine\nComprehension of Text. 2016, \/1606.05250. ArXiv1606.05250. https:\/\/arxiv.org\/abs\/1606.05250 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit34","unstructured":"Wiese, G.; Weissenborn, D.; Neves, M.\nL.\n          Neural Domain Adaptation\nfor Biomedical Question Answering. 2017, 1706.03610. ArXiv1706.03610. https:\/\/arxiv.org\/abs\/1706.03610 (accessed 2023-03-06)."},{"issue":"23","key":"2026081804443111100_cit35","doi-asserted-by":"publisher","first-page":"4087","DOI":"10.1093\/bioinformatics\/bty449","article-title":"Transfer\nlearning for biomedical named entity recognition\nwith neural networks","volume":"34","author":"Giorgi","year":"2018","journal-title":"Bioinformatics"},{"key":"2026081804443111100_cit36","unstructured":"Wang, X.; Zhang, Y.; Ren, X.; Zhang, Y.; Zitnik, M.; Shang, J.; Langlotz, C. P.; Han, J.\n          Cross-type Biomedical Named Entity Recognition with Deep Multi-Task\nLearning. 2018, 1801.09851. ArXiv1801.09851. https:\/\/arxiv.org\/abs\/1801.09851 (accessed 2023-03-06)."},{"issue":"5","key":"2026081804443111100_cit37","doi-asserted-by":"publisher","first-page":"1207","DOI":"10.1021\/acs.jcim.1c01199","article-title":"Single Model for\nOrganic and Inorganic Chemical Named\nEntity Recognition in ChemDataExtractor","volume":"62","author":"Isazawa","year":"2022","journal-title":"J.\nChem. Inf. Model"},{"key":"2026081804443111100_cit38","unstructured":"Sang, E. F. T. K.; Buchholz, S.\n          Introduction to\nthe CoNLL-2000 Shared Task: Chunking. 2000, cs.CL\/0009008. arXiv:cs\/0009008. https:\/\/arxiv.org\/abs\/cs\/0009008 (accessed 2023-03-06)."},{"issue":"1","key":"2026081804443111100_cit39","doi-asserted-by":"publisher","first-page":"S2","DOI":"10.1186\/1758-2946-7-S1-S2","article-title":"The CHEMDNER corpus of chemicals\nand drugs and its annotation principles","volume":"7","author":"Krallinger","year":"2015","journal-title":"J.\nCheminformatics"},{"issue":"9","key":"2026081804443111100_cit40","doi-asserted-by":"publisher","first-page":"3692","DOI":"10.1021\/acs.jcim.9b00470","article-title":"Named Entity\nRecognition\nand Normalization Applied to Large-Scale Information Extraction from\nthe Materials Science Literature","volume":"59","author":"Weston","year":"2019","journal-title":"J. Chem. Inf.\nModel"},{"key":"2026081804443111100_cit41","doi-asserted-by":"crossref","unstructured":"Iyyer, M.; Yih, W.-t.; Chang, M.-W.\n          Search-based\nNeural Structured\nLearning for Sequential Question Answering. In Proceedings of the 55th Annual Meeting of the Association for Computational\nLinguistics (Vol. 1: Long Papers); Association\nfor Computational Linguistics, Vancouver,\nCanada, 2017; pp 1821\u20131831,10.18653\/v1\/P17-1167.","DOI":"10.18653\/v1\/P17-1167"},{"issue":"60","key":"2026081804443111100_cit42","doi-asserted-by":"publisher","first-page":"3021","DOI":"10.21105\/joss.03021","article-title":"seaborn:\nstatistical data visualization","volume":"6","author":"Waskom","year":"2021","journal-title":"J. Open Source\nSoftw."},{"key":"2026081804443111100_cit43","unstructured":"Liu, P.; Yuan, W.; Fu, J.; Jiang, Z.; Hayashi, H.; Neubig, G.\n          Pre-train, Prompt, and Predict: A Systematic Survey\nof Prompting Methods in Natural Language Processing. 2021, abs\/2107.13586. ArXiv2107.13586. https:\/\/arxiv.org\/abs\/2107.13586 (accessed 2023-03-06)."},{"key":"2026081804443111100_cit44","unstructured":"Jain, S.; Wallace, B. C.\n          Attention is\nnot Explanation. 2019, 1902.10186. ArXiv1902.10186.  https:\/\/arxiv.org\/abs\/1902.10186 (accessed 2023-03-06)."}],"container-title":["Journal of Chemical Information\nand Modeling"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubs.acs.org\/doi\/pdf\/10.1021\/acs.jcim.2c01259","content-type":"application\/pdf","content-version":"vor","intended-application":"unspecified"},{"URL":"https:\/\/pubs.acs.org\/jcisd8\/article-pdf\/63\/7\/1961\/8911200\/ci2c01259.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/pubs.acs.org\/jcisd8\/article-pdf\/63\/7\/1961\/8911200\/ci2c01259.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T09:01:19Z","timestamp":1787043679000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubs.acs.org\/jcisd8\/article\/63\/7\/1961\/850442\/OpticalBERT-and-OpticalTable-SQA-Text-and-Table"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,20]]},"references-count":44,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,3,20]]},"published-print":{"date-parts":[[2023,4,10]]}},"URL":"https:\/\/doi.org\/10.1021\/acs.jcim.2c01259","relation":{},"ISSN":["1549-9596","1549-960X"],"issn-type":[{"value":"1549-9596","type":"print"},{"value":"1549-960X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,20]]}}}