{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T09:55:36Z","timestamp":1773654936632,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T00:00:00Z","timestamp":1773360000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012190","name":"Ministry of Science and Higher Education of the Russian Federation","doi-asserted-by":"crossref","award":["075-15-2025-461"],"award-info":[{"award-number":["075-15-2025-461"]}],"id":[{"id":"10.13039\/501100012190","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Large language models (LLMs) hold promise for automated extraction of structured biological information from scientific literature, yet their reliability in some domain-specific tasks, such as DNA probe parsing remains underexplored. We developed a verification-focused, schema-guided extraction pipeline that transforms unstructured texts from scientific articles into a normalized database of oligonucleotide probes, primers, and associated metadata. The system combined multi-turn JSON generation, strict schema validation, sequence-specific rule checks, and a post-processing recovery module that rescues systematically corrupted nucleotide outputs. Benchmarking across nine contemporary LLMs revealed distinct accuracy\u2013hallucination trade-offs, with the context-optimized Qwen3 model achieving the highest overall extraction efficiency while maintaining low hallucination rates. Iterative prompting substantially improved fidelity but introduced notable latency and variance. Across all models, stable error profiles and the success of the recovery module indicated that most extraction failures stem from systematic and correctable formatting issues rather than semantic misunderstandings. These findings highlight both the potential and the current limitations of LLMs for structured scientific data extraction. The research provides a reproducible benchmark and extensible framework for future large-scale curation of molecular biology datasets.<\/jats:p>","DOI":"10.3390\/a19030214","type":"journal-article","created":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T08:26:19Z","timestamp":1773390379000},"page":"214","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Efficient and Verified Research Data Extraction with LLM"],"prefix":"10.3390","volume":"19","author":[{"given":"Aleksandr","family":"Serdiukov","sequence":"first","affiliation":[{"name":"Faculty of Information Technology and Programming, ITMO University, 197101 St. Petersburg, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vitaliy","family":"Dravgelis","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Programming, ITMO University, 197101 St. Petersburg, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1460-4108","authenticated-orcid":false,"given":"Daniil","family":"Smutin","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Programming, ITMO University, 197101 St. Petersburg, Russia"},{"name":"Institute of Biomedical Chemistry, 119121 Moscow, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Amir","family":"Taldaev","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Programming, ITMO University, 197101 St. Petersburg, Russia"},{"name":"Institute of Biomedical Chemistry, 119121 Moscow, Russia"},{"name":"Research Center for Molecular Mechanisms of Aging and Age-Related Diseases, Moscow Center for Advanced Studies, 123592 Moscow, Russia"},{"name":"Shemyakin-Ovchinnikov Institute of Bioorganic Chemistry RAS, 117997 Moscow, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7997-0637","authenticated-orcid":false,"given":"Artem","family":"Ivanov","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Programming, ITMO University, 197101 St. Petersburg, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1563-4615","authenticated-orcid":false,"given":"Leonid","family":"Adonin","sequence":"additional","affiliation":[{"name":"Faculty of Radioelectronic Systems and Robotics, The Bonch-Bruevich Saint-Petersburg State University of Telecommunications, 193232 St. Petersburg, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sergey","family":"Muravyov","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology and Programming, ITMO University, 197101 St. Petersburg, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,3,13]]},"reference":[{"key":"ref_1","unstructured":"Garcia, G.L., Manesco, J.R.R., Paiola, P.H., Miranda, L., de Salvo, M.P., and Papa, J.P. (2024). A Review on Scientific Knowledge Extraction Using Large Language Models in Biomedical Sciences. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1763","DOI":"10.7326\/ANNALS-25-00739","article-title":"AI-Assisted Data Extraction with a Large Language Model: A Study Within Reviews","volume":"178","author":"Gartlehner","year":"2025","journal-title":"Ann. Intern. Med."},{"key":"ref_3","unstructured":"Schmidt, L., Hair, K., Graziosi, S., Campbell, F., Kapp, C., Khanteymoori, A., Craig, D., Engelbert, M., and Thomas, J. (2024). Exploring the Use of a Large Language Model for Data Extraction in Systematic Reviews: A Rapid Feasibility Study. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"526","DOI":"10.1515\/cdbme-2024-2129","article-title":"Using Large Language Models for Extracting Structured Information from Scientific Texts","volume":"10","author":"Rettenberger","year":"2024","journal-title":"Curr. Dir. Biomed. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"104128","DOI":"10.1016\/j.ipm.2025.104128","article-title":"Traceable LLM-Based Validation of Statements in Knowledge Graphs","volume":"62","author":"Adam","year":"2025","journal-title":"Inf. Process. Manag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1038\/s44185-024-00043-9","article-title":"Testing the Reliability of an AI-Based Large Language Model to Extract Ecological Information from the Scientific Literature","volume":"3","author":"Gougherty","year":"2024","journal-title":"npj Biodivers."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Keck, F., Broadbent, H., and Altermatt, F. (2025). Extracting Massive Ecological Data on State and Interactions of Species Using Large Language Models. bioRxiv.","DOI":"10.1101\/2025.01.24.634685"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Jung, S.J., Kim, H., and Jang, K.S. (2024). LLM Based Biological Named Entity Recognition from Scientific Literature. Proceedings of the 2024 IEEE International Conference on Big Data and Smart Computing (BigComp), Bangkok, Thailand, 18 February 2024, IEEE.","DOI":"10.1109\/BigComp60711.2024.00095"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1002\/jrsm.1732","article-title":"Performance of Two Large Language Models for Data Extraction in Evidence Synthesis","volume":"15","author":"Konet","year":"2024","journal-title":"Res. Synth. Methods"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"576","DOI":"10.1002\/jrsm.1710","article-title":"Data Extraction for Evidence Synthesis Using a Large Language Model: A Proof-of-concept Study","volume":"15","author":"Gartlehner","year":"2024","journal-title":"Res. Synth. Methods"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"giaf070","DOI":"10.1093\/gigascience\/giaf070","article-title":"Extraction of Biological Terms Using Large Language Models Enhances the Usability of Metadata in the BioSample Database","volume":"14","author":"Ikeda","year":"2025","journal-title":"GigaScience"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3280","DOI":"10.1038\/s41467-025-56989-2","article-title":"Benchmarking Large Language Models for Biomedical Natural Language Processing Applications and Recommendations","volume":"16","author":"Chen","year":"2025","journal-title":"Nat. Commun."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ivanisenko, T.V., Demenkov, P.S., and Ivanisenko, V.A. (2024). An Accurate and Efficient Approach to Knowledge Extraction from Scientific Publications Using Structured Ontology Models, Graph Neural Networks, and Large Language Models. Int. J. Mol. Sci., 25.","DOI":"10.3390\/ijms252111811"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1145\/3571730","article-title":"Survey of Hallucination in Natural Language Generation","volume":"55","author":"Ji","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Li, J., Yuan, R., Tian, Y., and Li, J. (2024). Towards Instruction-Tuned Verification for Improving Biomedical Information Extraction with Large Language Models. Proceedings of the 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Lisbon, Portugal, 3 December 2024, IEEE.","DOI":"10.1109\/BIBM62325.2024.10822624"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"638","DOI":"10.1093\/jamia\/ocae325","article-title":"Collaborative Large Language Models for Automated Data Extraction in Living Systematic Reviews","volume":"32","author":"Khan","year":"2025","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_17","unstructured":"R Core Team (2024). R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1686","DOI":"10.21105\/joss.01686","article-title":"Welcome to the Tidyverse","volume":"4","author":"Wickham","year":"2019","journal-title":"J. Open Source Softw."},{"key":"ref_19","unstructured":"Ooms, J. (2014). The Jsonlite Package: A Practical and Consistent Mapping Between JSON Data and R Objects. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"111","DOI":"10.32614\/RJ-2014-011","article-title":"The Stringdist Package for Approximate String Matching","volume":"6","year":"2014","journal-title":"R J."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wickham, H. (2016). Ggplot2: Elegant Graphics for Data Analysis, Springer.","DOI":"10.1007\/978-3-319-24277-4_9"},{"key":"ref_22","unstructured":"Kassambara, A. (2026, March 01). ggpubr: \u2018ggplot2\u2019 Based Publication Ready Plots. Available online: https:\/\/CRAN.R-project.org\/package=ggpubr."},{"key":"ref_23","unstructured":"Yu, G. (2026, March 01). Aplot: Decorate a \u2018Ggplot\u2019 with Associated Information. Available online: https:\/\/CRAN.R-project.org\/package=aplot."},{"key":"ref_24","unstructured":"Smutin, D. (2026, March 01). Ggviolinbox: Half-Violin and Half-Boxplot Geoms for Ggplot2. Available online: https:\/\/github.com\/dsmutin\/ggviolinbox\/."},{"key":"ref_25","unstructured":"Wilke, C.O. (2026, March 01). Ggridges: Ridgeline Plots in \u2018Ggplot2\u2019. Available online: https:\/\/cran.r-project.org\/web\/packages\/ggridges\/index.html."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v067.i01","article-title":"Fitting Linear Mixed-Effects Models Using Lme4","volume":"67","author":"Bates","year":"2015","journal-title":"J. Stat. Softw."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v082.i13","article-title":"lmerTest Package: Tests in Linear Mixed Effects Models","volume":"82","author":"Kuznetsova","year":"2017","journal-title":"J. Stat. Softw."},{"key":"ref_28","unstructured":"(2026, March 01). Datalab-to\/Marker. Available online: https:\/\/github.com\/datalab-to\/marker\/."},{"key":"ref_29","unstructured":"Willard, B.T., and Louf, R. (2023). Efficient Guided Generation for Large Language Models. arXiv."},{"key":"ref_30","unstructured":"Wright, A., Andrews, H., Hutton, B., and Dennis, G. (2020). JSON Schema: A Media Type for Describing JSON Documents, Internet Engineering Task Force. Available online: https:\/\/json-schema.org\/draft\/2020-12\/json-schema-core."},{"key":"ref_31","unstructured":"Hipp, R.D. (2026, March 01). SQLite. Available online: https:\/\/sqlite.org\/."},{"key":"ref_32","unstructured":"Yang, A., Yu, B., Li, C., Liu, D., Huang, F., Huang, H., Jiang, J., Tu, J., Zhang, J., and Zhou, J. (2025). Qwen2.5-1M Technical Report. arXiv."},{"key":"ref_33","unstructured":"Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., and Lv, C. (2025). Qwen3 Technical Report. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C.H., Gonzalez, J., Zhang, H., and Stoica, I. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention. Proceedings of the 29th Symposium on Operating Systems Principles, Koblenz, Germany, 23 October 2023, ACM.","DOI":"10.1145\/3600006.3613165"},{"key":"ref_35","unstructured":"Gemma Team, Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., Ram\u00e9, A., and Rivi\u00e8re, M. (2025). Gemma 3 Technical Report. arXiv."},{"key":"ref_36","unstructured":"Abdin, M., Aneja, J., Behl, H., Bubeck, S., Eldan, R., Gunasekar, S., Harrison, M., Hewett, R.J., Javaheripi, M., and Kauffmann, P. (2024). Phi-4 Technical Report. arXiv."},{"key":"ref_37","unstructured":"(2026, March 01). Ollama\u2014Open Language Model Management. Available online: https:\/\/ollama.com."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"258","DOI":"10.54254\/2755-2721\/76\/20240608","article-title":"A Review of Methods for Alleviating Hallucination Issues in Large Language Models","volume":"76","author":"Yin","year":"2024","journal-title":"Appl. Comput. Eng."},{"key":"ref_39","unstructured":"Abolhasani, M., and Pan, R. (2024). Leveraging LLM for Automated Ontology Extraction and Knowledge Graph Generation. arXiv."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/19\/3\/214\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T08:50:46Z","timestamp":1773651046000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/19\/3\/214"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,13]]},"references-count":39,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["a19030214"],"URL":"https:\/\/doi.org\/10.3390\/a19030214","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,13]]}}}