{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T14:16:16Z","timestamp":1784038576058,"version":"3.55.0"},"reference-count":130,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,12,1]],"date-time":"2019-12-01T00:00:00Z","timestamp":1575158400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,12,5]],"date-time":"2019-12-05T00:00:00Z","timestamp":1575504000000},"content-version":"vor","delay-in-days":4,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Machine learning can assist with multiple tasks during systematic reviews to facilitate the rapid retrieval of relevant references during screening and to identify and extract information relevant to the study characteristics, which include the PICO elements of patient\/population, intervention, comparator, and outcomes. The latter requires techniques for identifying and categorising fragments of text, known as named entity recognition.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>A publicly available corpus of PICO annotations on biomedical abstracts is used to train a named entity recognition model, which is implemented as a recurrent neural network. This model is then applied to a separate collection of abstracts for references from systematic reviews within biomedical and health domains. The occurrences of words tagged in the context of specific PICO contexts are used as additional features for a relevancy classification model. Simulations of the machine learning-assisted screening are used to evaluate the work saved by the relevancy model with and without the PICO features. Chi-squared and statistical significance of positive predicted values are used to identify words that are more indicative of relevancy within PICO contexts.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Inclusion of PICO features improves the performance metric on 15 of the 20 collections, with substantial gains on certain systematic reviews. Examples of words whose PICO context are more precise can explain this increase.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>Words within PICO tagged segments in abstracts are predictive features for determining inclusion. Combining PICO annotation model into the relevancy classification pipeline is a promising approach. The annotations may be useful on their own to aid users in pinpointing necessary information for data extraction, or to facilitate semantic search.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s12911-019-0992-8","type":"journal-article","created":{"date-parts":[[2019,12,5]],"date-time":"2019-12-05T17:17:04Z","timestamp":1575566224000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":42,"title":["Improving reference prioritisation with PICO recognition"],"prefix":"10.1186","volume":"19","author":[{"given":"Austin J.","family":"Brockmeier","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meizhi","family":"Ju","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Piotr","family":"Przyby\u0142a","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4097-9191","authenticated-orcid":false,"given":"Sophia","family":"Ananiadou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2019,12,5]]},"reference":[{"key":"992_CR1","volume-title":"Cochrane Handbook for Systematic Reviews of Interventions, Chap. 7. Version 5.1.0","author":"JP Higgins","year":"2011","unstructured":"Higgins JP, Deeks JJ. Selecting studies and collecting data In: Higgins JP, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions, Chap. 7. Version 5.1.0. Chichester: The Cochrane Collaboration. John Wiley & Sons: 2011. updated March 2011."},{"key":"992_CR2","unstructured":"Huang X, Lin J, Demner-Fushman D. Evaluation of PICO as a knowledge representation for clinical questions, vol. 2006. In: AMIA Annual Symposium Proceedings. American Medical Informatics Association: 2006. p. 359."},{"issue":"17","key":"992_CR3","doi-asserted-by":"crossref","first-page":"2093","DOI":"10.1001\/jama.1993.03510170083036","volume":"270","author":"AD Oxman","year":"1993","unstructured":"Oxman AD, Sackett DL, Guyatt GH, Browman G, Cook D, Gerstein H, Haynes B, Hayward R, Levine M, Nishikawa J, et al.Users\u2019 guides to the medical literature: I. how to get started. JAMA. 1993; 270(17):2093\u20135.","journal-title":"JAMA"},{"issue":"3","key":"992_CR4","doi-asserted-by":"crossref","first-page":"12","DOI":"10.7326\/ACPJC-1995-123-3-A12","volume":"123","author":"WS Richardson","year":"1995","unstructured":"Richardson WS, Wilson MC, Nishikawa J, Hayward RSA. The well-built clinical question: a key to evidence-based decisions. ACP J Club. 1995; 123(3):12.","journal-title":"ACP J Club"},{"issue":"1","key":"992_CR5","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1186\/1472-6947-7-16","volume":"7","author":"C Schardt","year":"2007","unstructured":"Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the PICO framework to improve searching PubMed for clinical questions. BMC Med Inf Dec Making. 2007; 7(1):16.","journal-title":"BMC Med Inf Dec Making"},{"key":"992_CR6","doi-asserted-by":"publisher","unstructured":"Wagner G, Nussbaumer-Streit B, Greimel J, Ciapponi A, Gartlehner G. Trading certainty for speed - how much uncertainty are decisionmakers and guideline developers willing to accept when using rapid reviews: an international survey. BMC Med Res Methodol. 2017; 17(1):121. https:\/\/doi.org\/10.1186\/s12874-017-0406-5.","DOI":"10.1186\/s12874-017-0406-5"},{"issue":"1","key":"992_CR7","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1002\/jrsm.1093","volume":"5","author":"Ian Shemilt","year":"2013","unstructured":"Shemilt I, Simon A, Hollands GJ, Marteau TM, Ogilvie D, O\u2019Mara-Eves A, Kelly MP, Thomas J. Pinpointing needles in giant haystacks: use of text mining to reduce impractical screening workload in extremely large scoping reviews. Res Synth Methods. 2014; 5(1):31\u201349. https:\/\/doi.org\/10.1002\/jrsm.1093.","journal-title":"Research Synthesis Methods"},{"issue":"7","key":"992_CR8","doi-asserted-by":"crossref","first-page":"634","DOI":"10.1001\/jama.282.7.634","volume":"282","author":"IE Allen","year":"1999","unstructured":"Allen IE, Olkin I. Estimating time to conduct a meta-analysis from number of citations retrieved. Jama. 1999; 282(7):634\u20135.","journal-title":"Jama"},{"key":"992_CR9","doi-asserted-by":"publisher","unstructured":"Lefebvre C, Glanville J, Wieland LS, Coles B, Weightman AL. Methodological developments in searching for studies for systematic reviews: past, present and future?Syst Rev. 2013; 2(78). https:\/\/doi.org\/10.1186\/2046-4053-2-78.","DOI":"10.1186\/2046-4053-2-78"},{"key":"992_CR10","doi-asserted-by":"publisher","unstructured":"O\u2019Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015; 4(5). https:\/\/doi.org\/10.1186\/2046-4053-4-5.","DOI":"10.1186\/2046-4053-4-5"},{"key":"992_CR11","doi-asserted-by":"publisher","unstructured":"Shemilt I, Khan N, Park S, Thomas J. Use of cost-effectiveness analysis to compare the efficiency of study identification methods in systematic reviews. Syst Rev. 2016; 5(140). https:\/\/doi.org\/10.1186\/s13643-016-0315-4.","DOI":"10.1186\/s13643-016-0315-4"},{"key":"992_CR12","unstructured":"Kanoulas E, Li D, Azzopardi L, Spijker R. CLEF 2017 technologically assisted reviews in empirical medicine overview. In: CEUR Workshop Proceedings, vol. 1866: 2017. p. 1\u201329. http:\/\/ceur-ws.org\/Vol-1866\/invited_paper_12.pdf. Accessed 27 Sept 2018."},{"key":"992_CR13","unstructured":"Kanoulas E, Li D, Azzopardi L, Spijker R. Clef 2018 technologically assisted reviews in empirical medicine overview. In: CEUR Workshop Proceedings, vol. 2125: 2018. p. 1\u201334. http:\/\/ceur-ws.org\/Vol-2125\/invited_paper_6.pdf. Accessed 27 Sept 2018."},{"issue":"2","key":"992_CR14","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1197\/jamia.M1929","volume":"13","author":"A. M. Cohen","year":"2006","unstructured":"Cohen AM, Hersh WR, Peterson K, Yen PY. Reducing workload in systematic review preparation using automated citation classification. J Am Med Inform Assoc. 2006; 13(2):206\u201319. https:\/\/doi.org\/10.1197\/jamia.M1929.","journal-title":"Journal of the American Medical Informatics Association"},{"issue":"1","key":"992_CR15","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1186\/1471-2105-11-55","volume":"11","author":"BC Wallace","year":"2010","unstructured":"Wallace BC, Trikalinos TA, Lau J, Brodley C, Schmid CH. Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinf. 2010; 11(1):55.","journal-title":"BMC Bioinf"},{"key":"992_CR16","doi-asserted-by":"publisher","unstructured":"Rathbone J, Hoffmann T, Glasziou P. Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers. Syst Rev. 2015; 4(80). https:\/\/doi.org\/10.1186\/s13643-015-0067-6.","DOI":"10.1186\/s13643-015-0067-6"},{"key":"992_CR17","doi-asserted-by":"publisher","unstructured":"Przyby\u0142a P, Brockmeier AJ, Kontonatsios G, Le Pogam M-A, McNaught J, von Elm E, Nolan K, Ananiadou S. Prioritising references for systematic reviews with robotanalyst: A user study. Res Synth Meth; 9(3):470\u201388. https:\/\/doi.org\/10.1002\/jrsm.1311.","DOI":"10.1002\/jrsm.1311"},{"issue":"1","key":"992_CR18","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1186\/s13643-018-0724-7","volume":"7","author":"G Tsafnat","year":"2018","unstructured":"Tsafnat G, Glasziou P, Karystianis G, Coiera E. Automated screening of research studies for systematic reviews using study characteristics. Syst Rev. 2018; 7(1):64.","journal-title":"Syst Rev"},{"key":"992_CR19","doi-asserted-by":"crossref","unstructured":"Ju M, Miwa M, Ananiadou S. A neural layered model for nested named entity recognition. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). New Orleans: Association for Computational Linguistics: 2018. p. 1446\u20131459. http:\/\/aclweb.org\/anthology\/N18-1131.","DOI":"10.18653\/v1\/N18-1131"},{"key":"992_CR20","doi-asserted-by":"publisher","unstructured":"Nye B, Li JJ, Patel R, Yang Y, Marshall IJ, Nenkova A, Wallace BC. A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature. 2018:197\u2013207. https:\/\/doi.org\/10.18653\/v1\/p18-1019.","DOI":"10.18653\/v1\/p18-1019"},{"key":"992_CR21","doi-asserted-by":"publisher","unstructured":"Chiu B, Crichton G, Korhonen A, Pyysalo S. How to train good word embeddings for biomedical nlp. In: Proceedings of the 15th Workshop on Biomedical Natural Language Processing: 2016. p. 166\u201374. https:\/\/doi.org\/10.18653\/v1\/w16-2922.","DOI":"10.18653\/v1\/w16-2922"},{"key":"992_CR22","doi-asserted-by":"publisher","unstructured":"Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C. Neural architectures for named entity recognition. In: Proceedings of NAACL-HLT: 2016. p. 260\u201370. https:\/\/doi.org\/10.18653\/v1\/n16-1030.","DOI":"10.18653\/v1\/n16-1030"},{"key":"992_CR23","volume-title":"Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics","author":"P Stenetorp","year":"2012","unstructured":"Stenetorp P, Pyysalo S, Topi\u0107 G, Ohta T, Ananiadou S, Tsujii J. Brat: a web-based tool for nlp-assisted text annotation. In: Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics: 2012. p. 102\u20137."},{"key":"992_CR24","unstructured":"Pacific Northwest Evidence-based Practice. OHSU Center for Evidence-Based Policy: Drug Effectiveness Review Project (DERP) Systematic Drug Class Review Gold Standard Data. https:\/\/dmice.ohsu.edu\/cohenaa\/systematic-drug-class-review-data.html. Accessed 16 Jan 2018."},{"key":"992_CR25","unstructured":"Pacific Northwest Evidence-based Practice Center. Drug Effectiveness Review Project (DERP). https:\/\/www.ohsu.edu\/xd\/research\/centers-institutes\/evidence-based-practice-center\/drug-effectiveness-review-project\/current-past-reports.cfm. Accessed 16 Jan 2018."},{"issue":"1","key":"992_CR26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/jrsm.27","volume":"2","author":"J Thomas","year":"2011","unstructured":"Thomas J, McNaught J, Ananiadou S. Applications of text mining within systematic reviews. Res Synth Meth. 2011; 2(1):1\u201314.","journal-title":"Res Synth Meth"},{"issue":"1","key":"992_CR27","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1186\/2046-4053-3-74","volume":"3","author":"G Tsafnat","year":"2014","unstructured":"Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Syst Rev. 2014; 3(1):74.","journal-title":"Syst Rev"},{"issue":"1","key":"992_CR28","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1186\/s13643-018-0740-7","volume":"7","author":"E Beller","year":"2018","unstructured":"Beller E, Clark J, Tsafnat G, Adams C, Diehl H, Lund H, Ouzzani M, Thayer K, Thomas J, Turner T, et al.Making progress with the automation of systematic reviews: principles of the international collaboration for the automation of systematic reviews (ICASR). Syst Rev. 2018; 7(1):77.","journal-title":"Syst Rev"},{"key":"992_CR29","unstructured":"Aphinyanaphongs Y, Aliferis CF. Text categorization models for retrieval of high quality articles in internal medicine. In: AMIA Annual Symposium Proceedings, vol. 2003. Bethesda: American Medical Informatics Association: 2003. p. 31\u20135. https:\/\/www.ncbi.nlm.nih.gov\/pmc\/articles\/PMC1480096\/."},{"issue":"2","key":"992_CR30","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1197\/jamia.M1641","volume":"12","author":"Y Aphinyanaphongs","year":"2005","unstructured":"Aphinyanaphongs Y, Tsamardinos I, Statnikov A, Hardin D, Aliferis CF. Text categorization models for high-quality article retrieval in internal medicine. J Am Med Inform Assoc. 2005; 12(2):207\u2013216.","journal-title":"J Am Med Inform Assoc"},{"key":"992_CR31","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.ins.2012.05.027","volume":"214","author":"S Choi","year":"2012","unstructured":"Choi S, Ryu B, Yoo S, Choi J. Combining relevancy and methodological quality into a single ranking for evidence-based medicine. Inf Sci. 2012; 214:76\u201390.","journal-title":"Inf Sci"},{"key":"992_CR32","doi-asserted-by":"publisher","unstructured":"Del Fiol G, Michelson M, Iorio A, Cotoi C, Haynes RB. A deep learning method to automatically identify reports of scientifically rigorous clinical research from the biomedical literature: Comparative analytic study. J Med Int Res. 2018; 20(6). https:\/\/doi.org\/10.2196\/preprints.10281.","DOI":"10.2196\/preprints.10281"},{"issue":"4","key":"992_CR33","doi-asserted-by":"crossref","first-page":"1406","DOI":"10.1109\/JBHI.2015.2431314","volume":"19","author":"IJ Marshall","year":"2015","unstructured":"Marshall IJ, Kuiper J, Wallace BC. Automating risk of bias assessment for clinical trials. IEEE J Biomed Health Inf. 2015; 19(4):1406\u201312.","journal-title":"IEEE J Biomed Health Inf"},{"issue":"1","key":"992_CR34","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1093\/jamia\/ocv044","volume":"23","author":"IJ Marshall","year":"2015","unstructured":"Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc. 2015; 23(1):193\u2013201.","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"992_CR35","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1093\/ije\/dyv306","volume":"45","author":"LA Millard","year":"2015","unstructured":"Millard LA, Flach PA, Higgins JP. Machine learning to assist risk-of-bias assessments in systematic reviews. Int J Epidemiol. 2015; 45(1):266\u201377.","journal-title":"Int J Epidemiol"},{"key":"992_CR36","doi-asserted-by":"publisher","unstructured":"Zhang Y, Marshall I, Wallace BC. Rationale-augmented convolutional neural networks for text classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, vol. 2016. NIH Public Access: 2016. p. 795. https:\/\/doi.org\/10.18653\/v1\/d16-1076.","DOI":"10.18653\/v1\/d16-1076"},{"issue":"3","key":"992_CR37","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1093\/jamia\/ocu025","volume":"22","author":"AM Cohen","year":"2015","unstructured":"Cohen AM, Smalheiser NR, McDonagh MS, Yu C, Adams CE, Davis JM, Yu PS. Automated confidence ranked classification of randomized controlled trial articles: an aid to evidence-based medicine. J Am Med Inform Assoc. 2015; 22(3):707\u201317.","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"992_CR38","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1002\/jrsm.1287","volume":"9","author":"Iain J. Marshall","year":"2018","unstructured":"Marshall IJ, Noel-Storr A, Kuiper J, Thomas J, Wallace BC. Machine learning for identifying randomized controlled trials: an evaluation and practitioner\u2019s guide. Res Synth Methods. 2018. https:\/\/doi.org\/10.1002\/jrsm.1287.","journal-title":"Research Synthesis Methods"},{"issue":"1","key":"992_CR39","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1186\/s13643-015-0066-7","volume":"4","author":"SR Jonnalagadda","year":"2015","unstructured":"Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: a systematic review. Syst Rev. 2015; 4(1):78.","journal-title":"Syst Rev"},{"key":"992_CR40","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.jbi.2013.11.002","volume":"52","author":"I Sim","year":"2014","unstructured":"Sim I, Tu SW, Carini S, Lehmann HP, Pollock BH, Peleg M, Wittkowski KM. The ontology of clinical research (OCRe): an informatics foundation for the science of clinical research. J Biomed Inf. 2014; 52:78\u201391.","journal-title":"J Biomed Inf"},{"issue":"1","key":"992_CR41","doi-asserted-by":"crossref","first-page":"9","DOI":"10.14236\/jhi.v15i1.640","volume":"15","author":"M Dawes","year":"2007","unstructured":"Dawes M, Pluye P, Shea L, Grad R, Greenberg A, Nie J-Y. The identification of clinically important elements within medical journal abstracts: Patient_population_problem, exposure_intervention, comparison, outcome, duration and results (PECODR). J Innov Health Inf. 2007; 15(1):9\u201316.","journal-title":"J Innov Health Inf"},{"issue":"3","key":"992_CR42","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1007\/s00354-007-0017-5","volume":"25","author":"K Hara","year":"2007","unstructured":"Hara K, Matsumoto Y. Extracting clinical trial design information from medline abstracts. N Gener Comput. 2007; 25(3):263\u201375.","journal-title":"N Gener Comput"},{"key":"992_CR43","volume-title":"The Sixth Midwest Computational Linguistics Colloquium (MCLC 2009)","author":"R Summerscales","year":"2009","unstructured":"Summerscales R, Argamon S, Hupert J, Schwartz A. Identifying treatments, groups, and outcomes in medical abstracts. In: The Sixth Midwest Computational Linguistics Colloquium (MCLC 2009). Bloomington: Indiana University: 2009."},{"key":"992_CR44","doi-asserted-by":"publisher","unstructured":"Summerscales RL, Argamon S, Bai S, Hupert J, Schwartz A. Automatic summarization of results from clinical trials. In: 2011 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE: 2011. p. 372\u2013377. https:\/\/doi.org\/10.1109\/bibm.2011.72.","DOI":"10.1109\/bibm.2011.72"},{"key":"992_CR45","unstructured":"Niu Y, Hirst G. Analysis of semantic classes in medical text for question answering. In: Proceedings of the Conference on Question Answering in Restricted Domains: 2004."},{"issue":"1","key":"992_CR46","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1162\/coli.2007.33.1.63","volume":"33","author":"D Demner-Fushman","year":"2007","unstructured":"Demner-Fushman D, Lin J. Answering clinical questions with knowledge-based and statistical techniques. Comput Linguist. 2007; 33(1):63\u2013103.","journal-title":"Comput Linguist"},{"key":"992_CR47","volume-title":"Proceedings of the AAAI-05 Workshop on Question Answering in Restricted Domains","author":"D Demner-Fushman","year":"2005","unstructured":"Demner-Fushman D, Lin J. Knowledge extraction for clinical question answering: Preliminary results. In: Proceedings of the AAAI-05 Workshop on Question Answering in Restricted Domains. Pittsburgh: AAAI Press (American Association for Artificial Intelligence): 2005. p. 9\u201313."},{"key":"992_CR48","volume-title":"Medinfo 2007: Proceedings of the 12th World Congress on Health (Medical) Informatics; Building Sustainable Health Systems, Stud Health Technol Inform","author":"R Xu","year":"2007","unstructured":"Xu R, Garten Y, Supekar KS, Das AK, Altman RB, Garber AM. Extracting subject demographic information from abstracts of randomized clinical trial reports. In: Medinfo 2007: Proceedings of the 12th World Congress on Health (Medical) Informatics; Building Sustainable Health Systems, Stud Health Technol Inform. Amsterdam: IOS Press: 2007. p. 550\u201354."},{"key":"992_CR49","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1186\/1471-2105-12-S2-S5","volume":"12","author":"SN Kim","year":"2011","unstructured":"Kim SN, Martinez D, Cavedon L, Yencken L. Automatic classification of sentences to support evidence based medicine. BMC Bioinformatics. 2011; 12:5. BioMed Central.","journal-title":"BMC Bioinformatics"},{"issue":"1","key":"992_CR50","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1186\/1472-6947-10-29","volume":"10","author":"F Boudin","year":"2010","unstructured":"Boudin F, Nie J-Y, Bartlett JC, Grad R, Pluye P, Dawes M. Combining classifiers for robust PICO element detection. BMC Med Inf Dec Making. 2010; 10(1):29.","journal-title":"BMC Med Inf Dec Making"},{"key":"992_CR51","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1007\/978-3-642-12275-0_8","volume-title":"Lecture Notes in Computer Science","author":"Florian Boudin","year":"2010","unstructured":"Boudin F, Shi L, Nie J-Y. Improving medical information retrieval with pico element detection. In: European Conference on Information Retrieval. Springer: 2010. p. 50\u201361. https:\/\/doi.org\/10.1007\/978-3-642-12275-0_8."},{"key":"992_CR52","volume-title":"AMIA Annual Symposium Proceedings, vol. 2010","author":"J Zhao","year":"2010","unstructured":"Zhao J, Kan M-Y, Procter PM, Zubaidah S, Yip WK, Li GM. Improving search for evidence-based practice using information extraction. In: AMIA Annual Symposium Proceedings, vol. 2010. Bethesda: American Medical Informatics Association: 2010. p. 937."},{"key":"992_CR53","volume-title":"AMIA Annual Symposium Proceedings, vol. 2012","author":"J Zhao","year":"2012","unstructured":"Zhao J, Bysani P, Kan M-Y. Exploiting classification correlations for the extraction of evidence-based practice information. In: AMIA Annual Symposium Proceedings, vol. 2012. Bethesda: American Medical Informatics Association: 2012. p. 1070."},{"issue":"2","key":"992_CR54","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1515\/jib-2013-222","volume":"10","author":"C Kelly","year":"2013","unstructured":"Kelly C, Yang H. A system for extracting study design parameters from nutritional genomics abstracts. J Integr Bioinform. 2013; 10(2):82\u201393.","journal-title":"J Integr Bioinform"},{"issue":"5","key":"992_CR55","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1016\/j.jbi.2008.12.011","volume":"42","author":"GY-C Chung","year":"2009","unstructured":"Chung GY-C. Towards identifying intervention arms in randomized controlled trials: extracting coordinating constructions. J Biomed Inform. 2009; 42(5):790\u2013800.","journal-title":"J Biomed Inform"},{"issue":"7","key":"992_CR56","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1258\/jtt.2008.007007","volume":"14","author":"MJ Hansen","year":"2008","unstructured":"Hansen MJ, Rasmussen N. \u00d8., Chung G. A method of extracting the number of trial participants from abstracts describing randomized controlled trials. J Telemed Telecare. 2008; 14(7):354\u20138.","journal-title":"J Telemed Telecare"},{"key":"992_CR57","doi-asserted-by":"publisher","unstructured":"Chung GY, Coiera E. A study of structured clinical abstracts and the semantic classification of sentences. In: Proceedings of the Workshop on BioNLP 2007: Biological, Translational, and Clinical Language Processing. Association for Computational Linguistics: 2007. p. 121\u2013128. https:\/\/doi.org\/10.3115\/1572392.1572415.","DOI":"10.3115\/1572392.1572415"},{"issue":"1","key":"992_CR58","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1186\/1472-6947-9-10","volume":"9","author":"GY Chung","year":"2009","unstructured":"Chung GY. Sentence retrieval for abstracts of randomized controlled trials. BMC Med Inf Dec Making. 2009; 9(1):10.","journal-title":"BMC Med Inf Dec Making"},{"key":"992_CR59","doi-asserted-by":"crossref","unstructured":"Dernoncourt F, Lee JY, Szolovits P. Neural networks for joint sentence classification in medical paper abstracts. arXiv preprint arXiv:1612.05251. 2016.","DOI":"10.18653\/v1\/E17-2110"},{"key":"992_CR60","doi-asserted-by":"publisher","unstructured":"Jin D, Szolovits P. Pico element detection in medical text via long short-term memory neural networks. In: Proceedings of the BioNLP 2018 Workshop: 2018. p. 67\u201375. https:\/\/doi.org\/10.18653\/v1\/w18-2308.","DOI":"10.18653\/v1\/w18-2308"},{"key":"992_CR61","volume-title":"AMIA Ann Symp Proc, vol. 2008","author":"B De Bruijn","year":"2008","unstructured":"De Bruijn B, Carini S, Kiritchenko S, Martin J, Sim I. Automated information extraction of key trial design elements from clinical trial publications. In: AMIA Ann Symp Proc, vol. 2008. Bethesda: American Medical Informatics Association: 2008. p. 141."},{"issue":"1","key":"992_CR62","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1186\/1472-6947-10-56","volume":"10","author":"S Kiritchenko","year":"2010","unstructured":"Kiritchenko S, de Bruijn B, Carini S, Martin J, Sim I. ExaCT: automatic extraction of clinical trial characteristics from journal publications. BMC Med Inf Dec Making. 2010; 10(1):56.","journal-title":"BMC Med Inf Dec Making"},{"key":"992_CR63","volume-title":"AMIA Annual Symposium Proceedings, vol. 2012","author":"W Hsu","year":"2012","unstructured":"Hsu W, Speier W, Taira RK. Automated extraction of reported statistical analyses: towards a logical representation of clinical trial literature. In: AMIA Annual Symposium Proceedings, vol. 2012. Bethesda: American Medical Informatics Association: 2012. p. 350."},{"key":"992_CR64","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/j.jbi.2016.10.014","volume":"64","author":"DDA Bui","year":"2016","unstructured":"Bui DDA, Del Fiol G, Hurdle JF, Jonnalagadda S. Extractive text summarization system to aid data extraction from full text in systematic review development. J Biomed Inf. 2016; 64:265\u201372.","journal-title":"J Biomed Inf"},{"key":"992_CR65","first-page":"1","volume":"17","author":"BC Wallace","year":"2016","unstructured":"Wallace BC, Marshall IJ. Extracting PICO sentences from clinical trial reports using supervised distant supervision. J Mach Learn Res. 2016; 17:1\u201325.","journal-title":"J Mach Learn Res"},{"issue":"suppl_1","key":"992_CR66","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1093\/nar\/gkh061","volume":"32","author":"O Bodenreider","year":"2004","unstructured":"Bodenreider O. The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Res. 2004; 32(suppl_1):267\u201370.","journal-title":"Nucleic Acids Res"},{"key":"992_CR67","doi-asserted-by":"publisher","unstructured":"Singh G, Marshall IJ, Thomas J, Shawe-Taylor J, Wallace BC. A neural candidate-selector architecture for automatic structured clinical text annotation. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. ACM: 2017. p. 1519\u201328. https:\/\/doi.org\/10.1145\/3132847.3132989.","DOI":"10.1145\/3132847.3132989"},{"key":"992_CR68","doi-asserted-by":"publisher","unstructured":"Aronson AR, Mork JG, Gay CW, Humphrey SM, Rogers WJ. The NLM indexing initiative\u2019s medical text indexer. 2004; 107:268\u201372. https:\/\/doi.org\/10.1037\/e615572012-009.","DOI":"10.1037\/e615572012-009"},{"issue":"Aug","key":"992_CR69","first-page":"2493","volume":"12","author":"R Collobert","year":"2011","unstructured":"Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P. Natural language processing (almost) from scratch. J Mach Learn Res. 2011; 12(Aug):2493\u2013537.","journal-title":"J Mach Learn Res"},{"issue":"1","key":"992_CR70","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1075\/li.30.1.03nad","volume":"30","author":"D Nadeau","year":"2007","unstructured":"Nadeau D, Sekine S. A survey of named entity recognition and classification. Lingvisticae Investigationes. 2007; 30(1):3\u201326.","journal-title":"Lingvisticae Investigationes"},{"key":"992_CR71","volume-title":"Pacific Symposium on Biocomputing, vol. 13","author":"R Leaman","year":"2008","unstructured":"Leaman R, Gonzalez G. BANNER: an executable survey of advances in biomedical named entity recognition. In: Pacific Symposium on Biocomputing, vol. 13. Hawaii: World Scientific: 2008. p. 652\u2013663."},{"key":"992_CR72","volume-title":"Proceedings of the 2010 Workshop on Biomedical Natural Language Processing","author":"M Chowdhury","year":"2010","unstructured":"Chowdhury M, Faisal M, et al.Disease mention recognition with specific features. In: Proceedings of the 2010 Workshop on Biomedical Natural Language Processing. Stroudsburg: Association for Computational Linguistics: 2010. p. 83\u201390."},{"key":"992_CR73","unstructured":"Huang Z, Xu W, Yu K. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991. 2015."},{"key":"992_CR74","doi-asserted-by":"publisher","unstructured":"Ma X, Hovy E. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1: 2016. p. 1064\u201374. https:\/\/doi.org\/10.18653\/v1\/p16-1101.","DOI":"10.18653\/v1\/p16-1101"},{"key":"992_CR75","doi-asserted-by":"publisher","unstructured":"Restificar A, Ananiadou S. Inferring appropriate eligibility criteria in clinical trial protocols without labeled data. In: Proceedings of the ACM Sixth International Workshop on Data and Text Mining in Biomedical Informatics, DTMBIO \u201912. New York: ACM: 2012. p. 21\u20138. https:\/\/doi.org\/10.1145\/2390068.2390074.","DOI":"10.1145\/2390068.2390074"},{"key":"992_CR76","doi-asserted-by":"publisher","unstructured":"Restificar A, Korkontzelos I, Ananiadou S. A method for discovering and inferring appropriate eligibility criteria in clinical trial protocols without labeled data. In: BMC Medical Informatics and Decision Making, vol. 13: 2013. p. 6. https:\/\/doi.org\/10.1186\/1472-6947-13-s1-s6.","DOI":"10.1186\/1472-6947-13-s1-s6"},{"issue":"1","key":"992_CR77","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1186\/2041-1480-5-22","volume":"5","author":"G Karystianis","year":"2014","unstructured":"Karystianis G, Buchan I, Nenadic G. Mining characteristics of epidemiological studies from medline: a case study in obesity. J Biomed Semant. 2014; 5(1):22.","journal-title":"J Biomed Semant"},{"key":"992_CR78","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.jbi.2017.04.004","volume":"70","author":"G Karystianis","year":"2017","unstructured":"Karystianis G, Thayer K, Wolfe M, Tsafnat G. Evaluation of a rule-based method for epidemiological document classification towards the automation of systematic reviews. J Biomed Inf. 2017; 70:27\u201334.","journal-title":"J Biomed Inf"},{"key":"992_CR79","volume-title":"AMIA Annual Symposium Proceedings, vol. 2008","author":"AM Cohen","year":"2008","unstructured":"Cohen AM. Optimizing feature representation for automated systematic review work prioritization. In: AMIA Annual Symposium Proceedings, vol. 2008. Bethesda: American Medical Informatics Association: 2008. p. 121\u2013125."},{"key":"992_CR80","volume-title":"AMIA Annual Symposium Proceedings, vol. 2010","author":"AM Cohen","year":"2010","unstructured":"Cohen AM, Ambert K, McDonagh M. A prospective evaluation of an automated classification system to support evidence-based medicine and systematic review. In: AMIA Annual Symposium Proceedings, vol. 2010. Bethesda: American Medical Informatics Association: 2010. p. 121."},{"key":"992_CR81","volume-title":"World Congress on Medical Informatics (MEDINFO), Stud Health Technol Inform, vol. 160","author":"T Bekhuis","year":"2010","unstructured":"Bekhuis T, Demner-Fushman D. Towards automating the initial screening phase of a systematic review In: Safran C, Reti S, Marin H, editors. World Congress on Medical Informatics (MEDINFO), Stud Health Technol Inform, vol. 160. Amsterdam: IOS Press: 2010. p. 146\u201350."},{"issue":"3","key":"992_CR82","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.artmed.2012.05.002","volume":"55","author":"T Bekhuis","year":"2012","unstructured":"Bekhuis T, Demner-Fushman D. Screening nonrandomized studies for medical systematic reviews: a comparative study of classifiers. Artif Intell Med. 2012; 55(3):197\u2013207.","journal-title":"Artif Intell Med"},{"issue":"1","key":"992_CR83","doi-asserted-by":"crossref","first-page":"86277","DOI":"10.1371\/journal.pone.0086277","volume":"9","author":"T Bekhuis","year":"2014","unstructured":"Bekhuis T, Tseytlin E, Mitchell KJ, Demner-Fushman D. Feature engineering and a proposed decision-support system for systematic reviewers of medical evidence. PLoS ONE. 2014; 9(1):86277.","journal-title":"PLoS ONE"},{"issue":"4","key":"992_CR84","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1136\/jamia.2010.004325","volume":"17","author":"S Matwin","year":"2010","unstructured":"Matwin S, Kouznetsov A, Inkpen D, Frunza O, O\u2019Blenis P. A new algorithm for reducing the workload of experts in performing systematic reviews. J Am Med Inform Assoc. 2010; 17(4):446\u201353.","journal-title":"J Am Med Inform Assoc"},{"key":"992_CR85","volume-title":"International Conference on Computational Linguistics (COLING)","author":"O Frunza","year":"2010","unstructured":"Frunza O, Inkpen D, Matwin S. Building systematic reviews using automatic text classification techniques. In: International Conference on Computational Linguistics (COLING). Stroudsburg: Association for Computational Linguistics: 2010. p. 303\u201311."},{"issue":"1","key":"992_CR86","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.artmed.2010.10.005","volume":"51","author":"O Frunza","year":"2011","unstructured":"Frunza O, Inkpen D, Matwin S, Klement W, O\u2019Blenis P. Exploiting the systematic review protocol for classification of medical abstracts. Artif Intell Med. 2011; 51(1):17\u201325.","journal-title":"Artif Intell Med"},{"key":"992_CR87","volume-title":"International Conference on Machine Learning (ICML-11)","author":"K Small","year":"2011","unstructured":"Small K, Wallace B, Trikalinos T, Brodley CE. The constrained weight space SVM: learning with ranked features. In: International Conference on Machine Learning (ICML-11). Norristown: Omnipress: 2011. p. 865\u201372."},{"key":"992_CR88","doi-asserted-by":"publisher","unstructured":"Wallace BC, Small K, Brodley CE, Lau J, Trikalinos TA. Deploying an interactive machine learning system in an evidence-based practice center: abstrackr. ACM SIGHIT Symp Int Health Inf. 2012; 819. https:\/\/doi.org\/10.1145\/2110363.2110464.","DOI":"10.1145\/2110363.2110464"},{"issue":"1-2","key":"992_CR89","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1504\/IJCBDD.2013.052198","volume":"6","author":"S Jonnalagadda","year":"2013","unstructured":"Jonnalagadda S, Petitti D. A new iterative method to reduce workload in systematic review process. Int J Comput Biol Drug Des. 2013; 6(1-2):5\u201317.","journal-title":"Int J Comput Biol Drug Des"},{"issue":"3","key":"992_CR90","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1177\/0272989X12457243","volume":"33","author":"SR Dalal","year":"2013","unstructured":"Dalal SR, Shekelle PG, Hempel S, Newberry SJ, Motala A, Shetty KD. A pilot study using machine learning and domain knowledge to facilitate comparative effectiveness review updating. Med Decis Making. 2013; 33(3):343\u201355.","journal-title":"Med Decis Making"},{"key":"992_CR91","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1016\/j.jbi.2014.06.005","volume":"51","author":"Makoto Miwa","year":"2014","unstructured":"Miwa M, Thomas J, O\u2019Mara-Eves A, Ananiadou S. Reducing systematic review workload through certainty-based screening. J Biomed Inform. 2014; 51:242\u201353. https:\/\/doi.org\/10.1016\/j.jbi.2014.06.005.","journal-title":"Journal of Biomedical Informatics"},{"issue":"2","key":"992_CR92","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1007\/s10796-015-9589-7","volume":"18","author":"P Timsina","year":"2016","unstructured":"Timsina P, Liu J, El-Gayar O. Advanced analytics for the automation of medical systematic reviews. Inf Syst Front. 2016; 18(2):237\u201352.","journal-title":"Inf Syst Front"},{"issue":"3","key":"992_CR93","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1007\/s10994-015-5535-7","volume":"102","author":"M Khabsa","year":"2016","unstructured":"Khabsa M, Elmagarmid A, Ilyas I, Hammady H, Ouzzani M. Learning to identify relevant studies for systematic reviews using random forest and external information. Mach Learn. 2016; 102(3):465\u201382.","journal-title":"Mach Learn"},{"key":"992_CR94","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.jbi.2016.06.001","volume":"62","author":"K Hashimoto","year":"2016","unstructured":"Hashimoto K, Kontonatsios G, Miwa M, Ananiadou S. Topic detection using paragraph vectors to support active learning in systematic reviews. J Biomed Inform. 2016; 62:59\u201365.","journal-title":"J Biomed Inform"},{"key":"992_CR95","doi-asserted-by":"publisher","unstructured":"Howard BE, Phillips J, Miller K, Tandon A, Mav D, Shah MR, Holmgren S, Pelch KE, Walker V, Rooney AA, Macleod M, Shah RR, Thayer K. SWIFT-Review: a text-mining workbench for systematic review. Syst Rev. 2016; 5(87). https:\/\/doi.org\/10.1186\/s13643-016-0263-z.","DOI":"10.1186\/s13643-016-0263-z"},{"key":"992_CR96","volume-title":"Proceedings of the Second Biocreative Challenge Workshop, vol. 209","author":"R S\u00e6tre","year":"2007","unstructured":"S\u00e6tre R, Yoshida K, Yakushiji A, Miyao Y, Matsubayashi Y, Ohta T. AKANE system: protein-protein interaction pairs in BioCreAtIvE2 challenge, PPI-IPS subtask. In: Proceedings of the Second Biocreative Challenge Workshop, vol. 209. Madrid: CNIO Centro Nacional de Investigaciones Oncol\u00f3gicas: 2007. p. 212."},{"key":"992_CR97","doi-asserted-by":"publisher","unstructured":"Ohta T, Tateisi Y, Kim J-D. The genia corpus: An annotated research abstract corpus in molecular biology domain. In: Proceedings of the Second International Conference on Human Language Technology Research. Morgan Kaufmann Publishers Inc.: 2002. p. 82\u201386. https:\/\/doi.org\/10.3115\/1289189.1289260.","DOI":"10.3115\/1289189.1289260"},{"issue":"suppl_1","key":"992_CR98","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1093\/bioinformatics\/btg1023","volume":"19","author":"J-D Kim","year":"2003","unstructured":"Kim J-D, Ohta T, Tateisi Y, Tsujii J. Genia corpus\u2014a semantically annotated corpus for bio-textmining. Bioinformatics. 2003; 19(suppl_1):180\u20132.","journal-title":"Bioinformatics"},{"key":"992_CR99","doi-asserted-by":"publisher","unstructured":"Tsuruoka Y, Tsujii J. Bidirectional inference with the easiest-first strategy for tagging sequence data. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing. Association for Computational Linguistics: 2005. p. 467\u201374. https:\/\/doi.org\/10.3115\/1220575.1220634.","DOI":"10.3115\/1220575.1220634"},{"key":"992_CR100","doi-asserted-by":"publisher","unstructured":"Sang EF, Veenstra J. Representing text chunks. In: Conference of the European Chapter of the Association for Computational Linguistics (EACL). Association for Computational Linguistics: 1999. p. 173\u20139. https:\/\/doi.org\/10.3115\/977035.977059.","DOI":"10.3115\/977035.977059"},{"issue":"Feb","key":"992_CR101","first-page":"1137","volume":"3","author":"Y Bengio","year":"2003","unstructured":"Bengio Y, Ducharme R, Vincent P, Jauvin C. A neural probabilistic language model. J Mach Learn Res. 2003; 3(Feb):1137\u201355.","journal-title":"J Mach Learn Res"},{"key":"992_CR102","unstructured":"Santos CD, Zadrozny B. Learning character-level representations for part-of-speech tagging. In: Proceedings of the 31st International Conference on Machine Learning (ICML-14). PMLR: 2014. p. 1818\u20131826. http:\/\/proceedings.mlr.press\/v32\/."},{"issue":"5-6","key":"992_CR103","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1016\/j.neunet.2005.06.042","volume":"18","author":"A Graves","year":"2005","unstructured":"Graves A, Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 2005; 18(5-6):602\u201310.","journal-title":"Neural Netw"},{"issue":"8","key":"992_CR104","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997; 9(8):1735\u201380.","journal-title":"Neural Comput"},{"key":"992_CR105","volume-title":"Proceedings of the Eighteenth International Conference on Machine Learning","author":"JD Lafferty","year":"2001","unstructured":"Lafferty JD, McCallum A, Pereira FCN. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning. San Francisco: Morgan Kaufmann Publishers Inc.: 2001. p. 282\u20139."},{"issue":"2","key":"992_CR106","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1109\/TIT.1967.1054010","volume":"13","author":"A Viterbi","year":"1967","unstructured":"Viterbi A. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Trans Inf Theory. 1967; 13(2):260\u20139.","journal-title":"IEEE Trans Inf Theory"},{"key":"992_CR107","unstructured":"Kinga D, Adam JB. A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR). San Diego: 2015."},{"key":"992_CR108","first-page":"1929","volume":"15","author":"N Srivastava","year":"2014","unstructured":"Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014; 15:1929\u201358.","journal-title":"J Mach Learn Res"},{"key":"992_CR109","volume-title":"Advances in Neural Information Processing Systems","author":"J Snoek","year":"2012","unstructured":"Snoek J, Larochelle H, Adams RP. Practical bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems. Red Hook: Curran Associates, Inc.: 2012. p. 2951\u20139."},{"key":"992_CR110","first-page":"993","volume":"3","author":"DM Blei","year":"2003","unstructured":"Blei DM, Ng AY, Jordan MI. Latent Dirichlet allocation. J Mach Learn Res. 2003; 3:993\u20131022.","journal-title":"J Mach Learn Res"},{"key":"992_CR111","unstructured":"McCallum AK. MALLET: A machine learning for language toolkit. 2002. http:\/\/mallet.cs.umass.edu. Accessed 16 Jan 2018."},{"issue":"5","key":"992_CR112","doi-asserted-by":"publisher","first-page":"978","DOI":"10.1038\/oby.2011.351","volume":"20","author":"Heather W. Brown","year":"2011","unstructured":"Fan R-E, Chang K-W, Hsieh C-J, Wang X-R, Lin C-J. LIBLINEAR: A library for large linear classification. J Mach Learn Res. 2008; 9:1871\u20134. https:\/\/doi.org\/10.1038\/oby.2011.351.","journal-title":"Obesity"},{"issue":"2","key":"992_CR113","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1111\/j.0006-341X.2000.00345.x","volume":"56","author":"W Leisenring","year":"2000","unstructured":"Leisenring W, Alono T, Pepe MS. Comparisons of predictive values of binary medical diagnostic tests for paired designs. Biometrics. 2000; 56(2):345\u201351.","journal-title":"Biometrics"},{"issue":"6","key":"992_CR114","doi-asserted-by":"crossref","first-page":"964","DOI":"10.1002\/sim.5587","volume":"32","author":"AS Kosinski","year":"2013","unstructured":"Kosinski AS. A weighted generalized score statistic for comparison of predictive values of diagnostic tests. Stat Med. 2013; 32(6):964\u201377.","journal-title":"Stat Med"},{"key":"992_CR115","unstructured":"Nye B, Li JJ, Patel R, Yang Y, Marshall IJ, Nenkova A, Wallace BC. EBM-NLP. https:\/\/ebm-nlp.herokuapp.com. Accessed 9 June 2018."},{"key":"992_CR116","volume-title":"AMIA Annual Symposium Proceedings, vol. 2006","author":"AM Cohen","year":"2006","unstructured":"Cohen AM. An effective general purpose approach for automated biomedical document classification. In: AMIA Annual Symposium Proceedings, vol. 2006. Bethesda: American Medical Informatics Association: 2006. p. 161."},{"issue":"4","key":"992_CR117","first-page":"288","volume":"41","author":"G Salton","year":"1990","unstructured":"Salton G, Buckley C. Improving retrieval performance by relevance feedback. J Assoc Inf Sci Technol. 1990; 41(4):288\u201397.","journal-title":"J Assoc Inf Sci Technol"},{"key":"992_CR118","unstructured":"Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies. CAMARADES. www.camarades.info. http:\/\/www.dcn.ed.ac.uk\/camarades\/default.htm."},{"issue":"1","key":"992_CR119","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1136\/jamia.2010.008177","volume":"18","author":"AM Cohen","year":"2011","unstructured":"Cohen AM. Performance of support-vector-machine-based classification on 15 systematic review topics evaluated with the WSS@ 95 measure. J Am Med Inf Assoc: JAMIA. 2011; 18(1):104.","journal-title":"J Am Med Inf Assoc: JAMIA"},{"issue":"3","key":"992_CR120","doi-asserted-by":"publisher","first-page":"e28","DOI":"10.2196\/medinform.3982","volume":"3","author":"Xiaonan Ji","year":"2015","unstructured":"Ji X, Yen P-Y. Using medline elemental similarity to assist in the article screening process for systematic reviews. JMIR Med Inf. 2015; 3(3). https:\/\/doi.org\/10.2196\/medinform.3982.","journal-title":"JMIR Medical Informatics"},{"key":"992_CR121","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/j.jbi.2017.03.007","volume":"69","author":"X Ji","year":"2017","unstructured":"Ji X, Ritter A, Yen P-Y. Using ontology-based semantic similarity to facilitate the article screening process for systematic reviews. J Biomed Inf. 2017; 69:33\u201342.","journal-title":"J Biomed Inf"},{"key":"992_CR122","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.jbi.2017.06.018","volume":"72","author":"G Kontonatsios","year":"2017","unstructured":"Kontonatsios G, Brockmeier AJ, Przyby\u0142a P, McNaught J, Mu T, Goulermas JY, Ananiadou S. A semi-supervised approach using label propagation to support citation screening. J Biomed Inform. 2017; 72:67\u201376.","journal-title":"J Biomed Inform"},{"key":"992_CR123","volume-title":"Adv Neural Inf Process Syst","author":"T Mikolov","year":"2013","unstructured":"Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed representations of words and phrases and their compositionality. In: Adv Neural Inf Process Syst. Red Hook: Curran Associates, Inc.: 2013. p. 3111\u20139."},{"key":"992_CR124","doi-asserted-by":"publisher","unstructured":"Pennington J, Socher R, Manning CD. Glove: Global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP): 2014. p. 1532\u201343. http:\/\/www.aclweb.org\/anthology\/D14-1162. https:\/\/doi.org\/10.3115\/v1\/d14-1162.","DOI":"10.3115\/v1\/d14-1162"},{"key":"992_CR125","doi-asserted-by":"publisher","unstructured":"Peters M, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L. Deep contextualized word representations. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). New Orleans: Association for Computational Linguistics: 2018. p. 2227\u201337. https:\/\/doi.org\/10.18653\/v1\/N18-1202.","DOI":"10.18653\/v1\/N18-1202"},{"key":"992_CR126","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis: Association for Computational Linguistics: 2018. p. 4171\u201386. http:\/\/arxiv.org\/abs\/1810.04805."},{"key":"992_CR127","doi-asserted-by":"crossref","unstructured":"Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, Kang J. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. arXiv:1901.08746. 2019.","DOI":"10.1093\/bioinformatics\/btz682"},{"issue":"5","key":"992_CR128","doi-asserted-by":"crossref","first-page":"690","DOI":"10.1197\/jamia.M3162","volume":"16","author":"AM Cohen","year":"2009","unstructured":"Cohen AM, Ambert K, McDonagh M. Cross-topic learning for work prioritization in systematic review creation and update. J Am Med Inform Assoc. 2009; 16(5):690\u2013704.","journal-title":"J Am Med Inform Assoc"},{"issue":"10","key":"992_CR129","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1145\/3233231","volume":"61","author":"Zachary C. Lipton","year":"2018","unstructured":"Lipton ZC. The Mythos of Model Interpretability. In: Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016). New York: 2016. https:\/\/doi.org\/10.1145\/3233231.","journal-title":"Communications of the ACM"},{"issue":"10","key":"992_CR130","doi-asserted-by":"crossref","first-page":"1799","DOI":"10.1093\/bioinformatics\/bty871","volume":"35","author":"AJ Soto","year":"2018","unstructured":"Soto AJ, Przyby\u0142a P, Ananiadou S. Thalia: semantic search engine for biomedical abstracts. Bioinformatics. 2018; 35(10):1799\u2013801.","journal-title":"Bioinformatics"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-019-0992-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12911-019-0992-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-019-0992-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,7]],"date-time":"2022-10-07T20:13:43Z","timestamp":1665173623000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-019-0992-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12]]},"references-count":130,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["992"],"URL":"https:\/\/doi.org\/10.1186\/s12911-019-0992-8","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12]]},"assertion":[{"value":"5 April 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 November 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 December 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The author declares that he\/she has no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"256"}}