{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,3]],"date-time":"2025-12-03T03:34:32Z","timestamp":1764732872404,"version":"3.37.3"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2020,5,22]],"date-time":"2020-05-22T00:00:00Z","timestamp":1590105600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,5,22]],"date-time":"2020-05-22T00:00:00Z","timestamp":1590105600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Office of research integrity","award":["ORIIR180038"],"award-info":[{"award-number":["ORIIR180038"]}]},{"name":"Post-Truth Initiative, a Sydney University Research Excellence Initiative","award":["SREI 2020"],"award-info":[{"award-number":["SREI 2020"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Scientometrics"],"published-print":{"date-parts":[[2020,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In an idealised vision of science the scientific literature is error-free. Errors reported during peer review are supposed to be corrected prior to publication, as further research establishes new knowledge based on the body of literature. It happens, however, that errors pass through peer review, and a minority of cases errata and retractions follow. Automated screening software can be applied to detect errors in manuscripts and publications. The contribution of this paper is twofold. First, we designed the erroneous reagent checking () benchmark to assess the accuracy of fact-checkers screening biomedical publications for dubious mentions of nucleotide sequence reagents. It comes with a test collection comprised of 1679 nucleotide sequence reagents that were curated by biomedical experts. Second, we benchmarked our own screening software called Seek&amp;Blastn with three input formats to assess the extent of performance loss when operating on various publication formats. Our findings stress the superiority of markup formats (a 79% detection rate on XML and HTML) over the prominent PDF format (a 69% detection rate at most) regarding an error flagging task. This is the first published baseline on error detection involving reagents reported in biomedical scientific publications. The  benchmark is designed to facilitate the development and validation of software bricks to enhance the reliability of the peer review process.<\/jats:p>","DOI":"10.1007\/s11192-020-03463-z","type":"journal-article","created":{"date-parts":[[2020,5,22]],"date-time":"2020-05-22T06:02:45Z","timestamp":1590127365000},"page":"1139-1156","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Flagging incorrect nucleotide sequence reagents in biomedical papers: To what extent does the leading publication format impede automatic error detection?"],"prefix":"10.1007","volume":"124","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4855-7038","authenticated-orcid":false,"given":"Cyril","family":"Labb\u00e9","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3060-6241","authenticated-orcid":false,"given":"Guillaume","family":"Cabanac","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3162-4425","authenticated-orcid":false,"given":"Rachael A.","family":"West","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2292-4680","authenticated-orcid":false,"given":"Thierry","family":"Gautier","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8356-0219","authenticated-orcid":false,"given":"Bertrand","family":"Favier","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8923-0587","authenticated-orcid":false,"given":"Jennifer A.","family":"Byrne","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2020,5,22]]},"reference":[{"doi-asserted-by":"publisher","unstructured":"Acuna, D. E., Brookes, P. S., & Kording, K. P. (2018). Bioscience-scale automated detection of figure element reuse. bioRXiv. https:\/\/doi.org\/10.1101\/269415.","key":"3463_CR1","DOI":"10.1101\/269415"},{"issue":"17","key":"3463_CR2","doi-asserted-by":"publisher","first-page":"3389","DOI":"10.1093\/nar\/25.17.3389","volume":"25","author":"SF Altschul","year":"1997","unstructured":"Altschul, S. F., Madden, T. L., Sch\u00e4ffer, A. A., Zhang, J., Zhang, Z., Miller, W., et al. (1997). Gapped BLAST and PSI-BLAST: A new generation of protein database search programs. Nucleic Acids Research, 25(17), 3389\u20133402. https:\/\/doi.org\/10.1093\/nar\/25.17.3389.","journal-title":"Nucleic Acids Research"},{"key":"3463_CR3","doi-asserted-by":"publisher","DOI":"10.1038\/nature.2016.19802","author":"M Baker","year":"2016","unstructured":"Baker, M. (2016). Problematic images found in 4% of biomedical papers [News]. Nature,. https:\/\/doi.org\/10.1038\/nature.2016.19802.","journal-title":"Nature"},{"doi-asserted-by":"publisher","unstructured":"Bast, H., & Korzen, C. (2013). The Icecite research paper management system. In WISE\u201913: Proceedings of the international conference on web information systems engineering (Vol. 8181, pp. 396\u2013409). https:\/\/doi.org\/10.1007\/978-3-642-41154-0_30.","key":"3463_CR4","DOI":"10.1007\/978-3-642-41154-0_30"},{"doi-asserted-by":"publisher","unstructured":"Bast, H., & Korzen, C. (2017). A benchmark and evaluation for text extraction from PDF. In JCDL\u201917: Proceedings of the 17th ACM\/IEEE joint conference on digital libraries (pp. 99\u2013108). IEEE. https:\/\/doi.org\/10.1109\/jcdl.2017.7991564.","key":"3463_CR5","DOI":"10.1109\/jcdl.2017.7991564"},{"doi-asserted-by":"publisher","unstructured":"Beck, J. (2011). NISO Z39.96 The Journal Article Tag Suite (JATS): What Happened to the NLM DTDs? The Journal of Electronic Publishing. https:\/\/doi.org\/10.3998\/3336451.0014.106.","key":"3463_CR6","DOI":"10.3998\/3336451.0014.106"},{"key":"3463_CR7","doi-asserted-by":"publisher","DOI":"10.1126\/science.aav8384","author":"J Brainard","year":"2018","unstructured":"Brainard, J., & You, J. (2018). What a massive database of retracted papers reveals about science publishing\u2019s \u201cdeath penalty\u201d. Science,. https:\/\/doi.org\/10.1126\/science.aav8384.","journal-title":"Science"},{"issue":"7","key":"3463_CR8","first-page":"24","volume":"23","author":"F Brischoux","year":"2009","unstructured":"Brischoux, F., & Legagneux, P. (2009). Don\u2019t format manuscripts. The Scientist, 23(7), 24.","journal-title":"The Scientist"},{"key":"3463_CR9","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-68690-3","volume-title":"Literature-based discovery","author":"P Bruza","year":"2008","unstructured":"Bruza, P., & Weeber, M. (2008). Literature-based discovery (Vol. 15). Berlin: Springer. https:\/\/doi.org\/10.1007\/978-3-540-68690-3."},{"issue":"3","key":"3463_CR10","doi-asserted-by":"publisher","first-page":"1471","DOI":"10.1007\/s11192-016-2209-6","volume":"110","author":"JA Byrne","year":"2017","unstructured":"Byrne, J. A., & Labb\u00e9, C. (2017). Striking similarities between publications from china describing single gene knockdown experiments in human cancer cell lines. Scientometrics, 110(3), 1471\u20131493. https:\/\/doi.org\/10.1007\/s11192-016-2209-6.","journal-title":"Scientometrics"},{"issue":"18","key":"3463_CR11","doi-asserted-by":"publisher","first-page":"3027","DOI":"10.1002\/1873-3468.13201","volume":"592","author":"J Christopher","year":"2018","unstructured":"Christopher, J. (2018). Systematic fabrication of scientific images revealed. FEBS Letters, 592(18), 3027\u20133029. https:\/\/doi.org\/10.1002\/1873-3468.13201.","journal-title":"FEBS Letters"},{"issue":"1","key":"3463_CR12","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1073\/pnas.1415135111","volume":"112","author":"DT Citron","year":"2015","unstructured":"Citron, D. T., & Ginsparg, P. (2015). Patterns of text reuse in a scientific corpus. Proceedings of the National Academy of Sciences, 112(1), 25\u201330.","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"3463_CR13","volume-title":"Bibliometrics and citation analysis: From the Science Citation Index to cybermetrics","author":"N De Bellis","year":"2009","unstructured":"De Bellis, N. (2009). Bibliometrics and citation analysis: From the Science Citation Index to cybermetrics. Lanham, MD: Scarecrow Press."},{"issue":"14","key":"3463_CR14","doi-asserted-by":"publisher","first-page":"2474","DOI":"10.1093\/bioinformatics\/bty152","volume":"34","author":"D Galea","year":"2018","unstructured":"Galea, D., Laponogov, I., & Veselkov, K. (2018). Exploiting and assessing multi-source data for supervised biomedical named entity recognition. Bioinformatics, 34(14), 2474\u20132482. https:\/\/doi.org\/10.1093\/bioinformatics\/bty152.","journal-title":"Bioinformatics"},{"issue":"1","key":"3463_CR15","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1186\/1471-2105-11-85","volume":"11","author":"M Gerner","year":"2010","unstructured":"Gerner, M., Nenadic, G., & Bergman, C. M. (2010). LINNAEUS: A species name identification system for biomedical literature. BMC Bioinformatics, 11(1), 85. https:\/\/doi.org\/10.1186\/1471-2105-11-85.","journal-title":"BMC Bioinformatics"},{"issue":"1","key":"3463_CR16","doi-asserted-by":"publisher","first-page":"280","DOI":"10.1093\/bioinformatics\/btz504","volume":"36","author":"JM Giorgi","year":"2020","unstructured":"Giorgi, J. M., & Bader, G. D. (2020). Towards reliable named entity recognition in the biomedical domain. Bioinformatics, 36(1), 280\u2013286. https:\/\/doi.org\/10.1093\/bioinformatics\/btz504.","journal-title":"Bioinformatics"},{"issue":"2","key":"3463_CR17","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1002\/leap.1092","volume":"30","author":"J Hartley","year":"2017","unstructured":"Hartley, J., & Cabanac, G. (2017). The delights, discomforts, and downright furies of the manuscript submission process [Opinion Piece]. Learned Publishing, 30(2), 167\u2013172. https:\/\/doi.org\/10.1002\/leap.1092.","journal-title":"Learned Publishing"},{"issue":"5","key":"3463_CR18","doi-asserted-by":"publisher","first-page":"429","DOI":"10.3233\/IDA-2002-6504","volume":"6","author":"N Japkowicz","year":"2002","unstructured":"Japkowicz, N., & Stephen, S. (2002). The class imbalance problem: A systematic study. Intelligent Data Analysis, 6(5), 429\u2013449.","journal-title":"Intelligent Data Analysis"},{"issue":"3","key":"3463_CR19","doi-asserted-by":"publisher","first-page":"e0213266","DOI":"10.1371\/journal.pone.0213266","volume":"14","author":"C Labb\u00e9","year":"2019","unstructured":"Labb\u00e9, C., Grima, N., Gautier, T., Favier, B., & Byrne, J. A. (2019). Semi-automated factchecking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool. PLoS ONE, 14(3), e0213266. https:\/\/doi.org\/10.1371\/journal.pone.0213266.","journal-title":"PLoS ONE"},{"issue":"1","key":"3463_CR20","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1007\/s11192-012-0781-y","volume":"94","author":"C Labb\u00e9","year":"2013","unstructured":"Labb\u00e9, C., & Labb\u00e9, D. (2013). Duplicate and fake publications in the scientific literature: how many scigen papers in computer science? Scientometrics, 94(1), 379\u2013396.","journal-title":"Scientometrics"},{"issue":"7685","key":"3463_CR21","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1038\/d41586-017-07763-y","volume":"552","author":"H Ledford","year":"2017","unstructured":"Ledford, H., Castelvecchi, D., Dolgin, E., Reardon, S., Gibney, E., Phillips, N., et al. (2017). Nature\u2019s 10: Ten people who mattered this year. Nature, 552(7685), 315\u2013324. https:\/\/doi.org\/10.1038\/d41586-017-07763-y.","journal-title":"Nature"},{"issue":"3","key":"3463_CR22","doi-asserted-by":"publisher","first-page":"305","DOI":"10.3138\/jsp.49.3.02","volume":"49","author":"X Liu","year":"2018","unstructured":"Liu, X., & Chen, X. (2018). Journal retractions: Some unique features of research misconduct in China. Journal of Scholarly Publishing, 49(3), 305\u2013319. https:\/\/doi.org\/10.3138\/jsp.49.3.02.","journal-title":"Journal of Scholarly Publishing"},{"doi-asserted-by":"publisher","unstructured":"Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for scholarship publications. In ECDL\u201909: Proceedings of the international conference on theory and practice of digital libraries (Vol. 5714, pp. 473\u2013474). https:\/\/doi.org\/10.1007\/978-3-642-04346-8_62.","key":"3463_CR23","DOI":"10.1007\/978-3-642-04346-8_62"},{"key":"3463_CR24","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1017\/cbo9781139107860.00","volume-title":"The handbook of journal publishing","author":"S Morris","year":"2013","unstructured":"Morris, S., Barnas, E., LaFrenier, D., & Reich, M. (2013). The production process. In S. Morris, E. Barnas, D. LaFrenier, & M. Reich (Eds.), The handbook of journal publishing (pp. 104\u2013132). Cambridge: Cambridge University Press. https:\/\/doi.org\/10.1017\/cbo9781139107860.00."},{"issue":"4","key":"3463_CR25","doi-asserted-by":"publisher","first-page":"1205","DOI":"10.3758\/s13428-015-0664-2","volume":"48","author":"MB Nuijten","year":"2016","unstructured":"Nuijten, M. B., Hartgerink, C. H., van Assen, M. A., Epskamp, S., & Wicherts, J. M. (2016). The prevalence of statistical reporting errors in psychology (1985\u20132013). Behavior Research Methods, 48(4), 1205\u20131226. https:\/\/doi.org\/10.3758\/s13428-015-0664-2.","journal-title":"Behavior Research Methods"},{"key":"3463_CR26","doi-asserted-by":"publisher","first-page":"316","DOI":"10.1038\/d41586-019-02142-1","volume":"571","author":"P Pulla","year":"2019","unstructured":"Pulla, P. (2019). The plan to mine the world\u2019s research papers. Nature, 571, 316\u2013318. https:\/\/doi.org\/10.1038\/d41586-019-02142-1.","journal-title":"Nature"},{"issue":"1","key":"3463_CR27","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1002\/leap.1218","volume":"32","author":"P Smart","year":"2019","unstructured":"Smart, P., & Gaston, T. (2019). How prevalent are plagiarized submissions? Global survey of editors. Learned Publishing, 32(1), 47\u201356. https:\/\/doi.org\/10.1002\/leap.1218.","journal-title":"Learned Publishing"},{"unstructured":"Soete, L., Schneegans, S., Er\u00f6cal, D., Angathevar, B., & Rasiah, R. (2015). A world in search of an effective growth strategy. In S. Schneegans (Ed.), UNESCO science report: Towards 2030 (pp. 20\u201355). Paris. Retrieved from http:\/\/unesdoc.unesco.org\/images\/0023\/002354\/235406e.pdf.","key":"3463_CR28"},{"key":"3463_CR29","doi-asserted-by":"publisher","DOI":"10.1038\/nature.2014.14763","author":"R Van Noorden","year":"2014","unstructured":"Van Noorden, R. (2014). Publishers withdraw more than 120 gibberish papers. Nature. https:\/\/doi.org\/10.1038\/nature.2014.14763.","journal-title":"Nature"},{"key":"3463_CR30","doi-asserted-by":"publisher","DOI":"10.1038\/nature.2015.17749","author":"R Van Noorden","year":"2015","unstructured":"Van Noorden, R. (2015). The image detective who roots out manuscript flaws. Nature. https:\/\/doi.org\/10.1038\/nature.2015.17749.","journal-title":"Nature"},{"issue":"4","key":"3463_CR31","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1108\/AP-03-2012-0025","volume":"65","author":"R Volentine","year":"2013","unstructured":"Volentine, R., & Tenopir, C. (2013). Value of academic reading and value of the library in academics\u2019 own words. Aslib Proceedings, 65(4), 425\u2013440. https:\/\/doi.org\/10.1108\/AP-03-2012-0025.","journal-title":"Aslib Proceedings"},{"issue":"11","key":"3463_CR32","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1145\/1297797.1297822","volume":"50","author":"EM Voorhees","year":"2007","unstructured":"Voorhees, E. M. (2007). TREC: Continuing information retrieval\u2019s tradition of experimentation. Communications of the ACM, 50(11), 51\u201354. https:\/\/doi.org\/10.1145\/1297797.1297822.","journal-title":"Communications of the ACM"},{"issue":"9","key":"3463_CR33","doi-asserted-by":"publisher","first-page":"6047","DOI":"10.1073\/pnas.092143499","volume":"99","author":"J-Y Yu","year":"2002","unstructured":"Yu, J.-Y., DeRuiter, S. L., & Turner, D. L. (2002). RNA interference by expression of shortinterfering RNAs and hairpin RNAs in mammalian cells. Proceedings of the National Academy of Sciences, 99(9), 6047\u20136052. https:\/\/doi.org\/10.1073\/pnas.092143499.","journal-title":"Proceedings of the National Academy of Sciences"}],"container-title":["Scientometrics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-020-03463-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11192-020-03463-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11192-020-03463-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,21]],"date-time":"2021-05-21T23:07:57Z","timestamp":1621638477000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11192-020-03463-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,22]]},"references-count":33,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,8]]}},"alternative-id":["3463"],"URL":"https:\/\/doi.org\/10.1007\/s11192-020-03463-z","relation":{},"ISSN":["0138-9130","1588-2861"],"issn-type":[{"type":"print","value":"0138-9130"},{"type":"electronic","value":"1588-2861"}],"subject":[],"published":{"date-parts":[[2020,5,22]]},"assertion":[{"value":"1 October 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 May 2020","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}