{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:20:05Z","timestamp":1750220405716,"version":"3.41.0"},"reference-count":76,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,12,20]],"date-time":"2021-12-20T00:00:00Z","timestamp":1639958400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100004490","name":"University of Waterloo","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004490","id-type":"DOI","asserted-by":"crossref"}]},{"name":"NSERC, the Natural Sciences and Engineering Research Council of Canada"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Healthcare"],"published-print":{"date-parts":[[2022,4,30]]},"abstract":"<jats:p>\n            The standard approach to expert-in-the-loop machine learning is active learning, where, repeatedly, an expert is asked to annotate one or more records and the machine finds a classifier that respects all annotations made until that point. We propose an alternative approach,\n            <jats:italic>IQRef<\/jats:italic>\n            , in which the expert iteratively designs a classifier and the machine helps him or her to determine how well it is performing and, importantly, when to stop, by reporting statistics on a fixed, hold-out sample of annotated records. We justify our approach based on prior work giving a theoretical model of how to re-use hold-out data. We compare the two approaches in the context of identifying a cohort of EHRs and examine their strengths and weaknesses through a case study arising from an optometric research problem. We conclude that both approaches are complementary, and we recommend that they both be employed in conjunction to address the problem of cohort identification in health research.\n          <\/jats:p>","DOI":"10.1145\/3483411","type":"journal-article","created":{"date-parts":[[2021,12,20]],"date-time":"2021-12-20T16:29:43Z","timestamp":1640017783000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Computer-Assisted Cohort Identification in Practice"],"prefix":"10.1145","volume":"3","author":[{"given":"Besat","family":"Kassaie","sequence":"first","affiliation":[{"name":"University of Waterloo, Waterloo, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elizabeth L.","family":"Irving","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Frank Wm.","family":"Tompa","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,12,20]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/21.87078"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/170036.170072"},{"key":"e_1_3_2_4_2","doi-asserted-by":"crossref","first-page":"66","DOI":"10.3389\/fmed.2019.00066","article-title":"The revival of the notes field: Leveraging the unstructured content in eHealth records","volume":"6","author":"Assale Michela","year":"2019","unstructured":"Michela Assale, Linda Greta Dui, Andrea Cina, Andrea Seveso, and Federico Cabitza. 2019. The revival of the notes field: Leveraging the unstructured content in eHealth records. Frontiers in Medicine 6 (2019), 66. DOI:https:\/\/doi.org\/10.3389\/fmed.2019.00066","journal-title":"Frontiers in Medicine"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2019.05.009"},{"key":"e_1_3_2_6_2","volume-title":"Proceedings of the 21st Text REtrieval Conference (TREC\u201912) (NIST Special Publication)","volume":"500","author":"Bedrick Steven","year":"2012","unstructured":"Steven Bedrick, Tracy Edinger, Aaron M. Cohen, and William R. Hersh. 2012. Identifying patients for clinical studies from electronic health records: TREC 2012 Medical Records Track at OHSU. In Proceedings of the 21st Text REtrieval Conference (TREC\u201912) (NIST Special Publication), Ellen M. Voorhees and Lori P. Buckland (Eds.), Vol. 500-298. National Institute of Standards and Technology (NIST), 18 pages. http:\/\/trec.nist.gov\/pubs\/trec21\/papers\/OHSU.medical.final.pdf."},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","DOI":"10.1089\/heat.2019.0006","article-title":"Deep learning in orthopedics: How do we build trust in the machine?","author":"Borjali Alireza","year":"2020","unstructured":"Alireza Borjali, Antonia F. Chen, Orhun K. Muratoglu, Mohammad A. Morid, and Kartik M. Varadarajan. 2020. Deep learning in orthopedics: How do we build trust in the machine?Healthcare Transformation (2020) 6 pages. http:\/\/doi.org\/10.1089\/heat.2019.0006","journal-title":"Healthcare Transformation"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/956750.956761"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooaa026"},{"issue":"1","key":"e_1_3_2_10_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1214\/aoms\/1177728844","article-title":"The estimation of biological populations","volume":"25","author":"Chapman Douglas G.","year":"1954","unstructured":"Douglas G. Chapman. 1954. The estimation of biological populations. Annals of Mathematical Statistics 25, 1 (1954), 1\u201315. DOI:https:\/\/doi.org\/10.1214\/aoms\/1177728844","journal-title":"Annals of Mathematical Statistics"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1006\/jbin.2001.1029"},{"issue":"2","key":"e_1_3_2_12_2","first-page":"e253\u2013e259","article-title":"Applying active learning to high-throughput phenotyping algorithms for electronic health records data","volume":"20","author":"Chen Yukun","year":"2013","unstructured":"Yukun Chen, Robert J. Carroll, Eugenia R. McPeek Hinz, Anushi Shah, Anne E. Eyler, Joshua C. Denny, and Hua Xu. 2013. Applying active learning to high-throughput phenotyping algorithms for electronic health records data. Journal of the American Medical Informatics Association 20, e2 (2013), e253\u2013e259.","journal-title":"Journal of the American Medical Informatics Association"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911510"},{"key":"e_1_3_2_14_2","volume-title":"American Medical Informatics Association Annual Symposium (AIMA\u201912)","author":"Cui Licong","year":"2012","unstructured":"Licong Cui, Samden D. Lhatoo, Guo-Qiang Zhang, Satya Sanket Sahoo, and Alireza Bozorgi. 2012. EpiDEA: Extracting structured epilepsy and seizure information from patient discharge summaries for cohort identification. In American Medical Informatics Association Annual Symposium (AIMA\u201912). AMIA. http:\/\/knowledge.amia.org\/amia-55142-a2012a-1.636547\/t-003-1.640625\/f-001-1.640626\/a-134-1.640826\/a-135-1.640823."},{"issue":"8","key":"e_1_3_2_15_2","doi-asserted-by":"crossref","first-page":"1257","DOI":"10.1097\/JTO.0b013e31825bd9f5","article-title":"Automated identification of patients with pulmonary nodules in an integrated health system using administrative health plan data, radiology reports, and natural language processing","volume":"7","author":"Danforth Kim N.","year":"2012","unstructured":"Kim N. Danforth, Megan I. Early, Sharon Ngan, Anne E. Kosco, Chengyi Zheng, and Michael K. Gould. 2012. Automated identification of patients with pulmonary nodules in an integrated health system using administrative health plan data, radiology reports, and natural language processing. Journal of Thoracic Oncology 7, 8 (Aug. 2012), 1257\u20131262.","journal-title":"Journal of Thoracic Oncology"},{"key":"e_1_3_2_16_2","doi-asserted-by":"crossref","unstructured":"Jessica K. De Freitas Kipp W. Johnson Eddye Golden Girish N. Nadkarni Joel T. Dudley Erwin P. Bottinger Benjamin S. Glicksberg and Riccardo Miotto. 2021. Phe2vec: Automated disease phenotyping based on unsupervised embeddings from electronic health records. Patterns 2 100337 (September 10 2021) https:\/\/doi.org\/10.1016\/j.patter.2021.100337","DOI":"10.1016\/j.patter.2021.100337"},{"key":"e_1_3_2_17_2","volume-title":"Proceedings of the 21st Text REtrieval Conference (TREC\u201912) (NIST Special Publication)","volume":"500","author":"Demner-Fushman Dina","year":"2012","unstructured":"Dina Demner-Fushman, Swapna Abhyankar, Antonio Jimeno-Yepes, Russell F. Loane, Fran\u00e7ois-Michel Lang, James G. Mork, Nicolas Ide, and Alan R. Aronson. 2012. NLM at TREC 2012 Medical Records Track. In Proceedings of the 21st Text REtrieval Conference (TREC\u201912) (NIST Special Publication), Ellen M. Voorhees and Lori P. Buckland (Eds.), Vol. 500\u2013298. National Institute of Standards and Technology (NIST), 5 pages. http:\/\/trec.nist.gov\/pubs\/trec21\/papers\/NLM.medical.final.pdf."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/288627.288651"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.5555\/2969442.2969502"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3051088"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2746539.2746580"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/1081870.1081878"},{"key":"e_1_3_2_23_2","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1007\/978-3-319-23344-4_38","volume-title":"International Conference on Brain Informatics and Health","author":"Girardi Dominic","year":"2015","unstructured":"Dominic Girardi, Josef Kueng, and Andreas Holzinger. 2015. A domain-expert centered process model for knowledge discovery in medical research: Putting the expert-in-the-loop. In International Conference on Brain Informatics and Health. Springer, 389\u2013398."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/1378773.1378804"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF02163027"},{"key":"e_1_3_2_26_2","volume-title":"Proceedings of the 25th Text REtrieval Conference (TREC\u201916) (NIST Special Publication)","volume":"500","author":"Grossman Maura R.","year":"2016","unstructured":"Maura R. Grossman, Gordon V. Cormack, and Adam Roegiest. 2016. TREC 2016 total recall track overview. In Proceedings of the 25th Text REtrieval Conference (TREC\u201916) (NIST Special Publication), Ellen M. Voorhees and Angela Ellis (Eds.), Vol. 500-321. National Institute of Standards and Technology (NIST), 17 pages. http:\/\/trec.nist.gov\/pubs\/trec25\/papers\/Overview-TR.pdf."},{"key":"e_1_3_2_27_2","first-page":"477","volume-title":"Pacific-Asia Conference on Knowledge Discovery and Data Mining","author":"Guo Xuan","year":"2016","unstructured":"Xuan Guo, Qi Yu, Rui Li, Cecilia Ovesdotter Alm, Cara Calvelli, Pengcheng Shi, and Anne Haake. 2016. An expert-in-the-loop paradigm for learning medical image grouping. In Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 477\u2013488."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/2.781635"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40708-016-0042-6"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.01.008"},{"issue":"7","key":"e_1_3_2_31_2","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1097\/OPX.0000000000000863","article-title":"Value of routine eye examinations in asymptomatic patients","volume":"93","author":"Irving Elizabeth L.","year":"2016","unstructured":"Elizabeth L. Irving, Joel D. Harris, Carolyn M. Machan, Barbara E. Robinson, Patricia K. Hrynchak, Susan J. Leat, and Linda Lillakas. 2016. Value of routine eye examinations in asymptomatic patients. Optometry and Vision Science: Official Publication of the American Academy of Optometry 93, 7 (July 2016), 660\u2013666.","journal-title":"Optometry and Vision Science: Official Publication of the American Academy of Optometry"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.5555\/645326.649721"},{"key":"e_1_3_2_33_2","volume-title":"Abductive Inference: Computation, Philosophy, Technology","author":"Josephson John R.","year":"1996","unstructured":"John R. Josephson and Susan G. Josephson. 1996. Abductive Inference: Computation, Philosophy, Technology. Cambridge University Press."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.5555\/3015812.3015961"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/293347.293351"},{"issue":"12","key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"A1359","DOI":"10.1016\/S0735-1097(14)61359-0","article-title":"Natural language processing improves phenotypic accuracy in an electronic medical record cohort of type 2 diabetes and cardiovascular disease","volume":"63","author":"Kumar Vishesh","year":"2014","unstructured":"Vishesh Kumar, Katherine Liao, Su-Chun Cheng, Sheng Yu, Uri Kartoun, Ari Brettman, Vivian Gainer, Andrew Cagan, Shawn Murphy, Guergana Savova, Pei Chen, Peter Szolovits, Zongqi Xia, Elizabeth Karlson, Robert Plenge, Ashwin Ananthakrishnan, Susanne Churchill, Tianxi Cai, Isaac Kohane, and Stanley Shaw. 2014. Natural language processing improves phenotypic accuracy in an electronic medical record cohort of type 2 diabetes and cardiovascular disease. Journal of the American College of Cardiology 63, 12 Supplement (2014), A1359.","journal-title":"Journal of the American College of Cardiology"},{"issue":"4","key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1038\/scientificamerican1081-54","article-title":"The patient record in epidemiology","volume":"245","author":"Kurland Leonard T","year":"1981","unstructured":"Leonard T Kurland and Craig A Molgaard. 1981. The patient record in epidemiology. Scientific American 245, 4 (1981), 54\u201363.","journal-title":"Scientific American"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1012491419635"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.5555\/188490.188495"},{"key":"e_1_3_2_40_2","article-title":"Development of phenotype algorithms using electronic medical records and incorporating natural language processing","author":"Liao Katherine P.","year":"2015","unstructured":"Katherine P. Liao, Tianxi Cai, Guergana K. Savova, Shawn N. Murphy, Elizabeth W. Karlson, Ashwin N. Ananthakrishnan, Vivian S. Gainer, Stanley Y. Shaw, Zongqi Xia, Peter Szolovits, Susanne Churchill, and Isaac Kohane. 2015. Development of phenotype algorithms using electronic medical records and incorporating natural language processing. BMJ 350:h1885 (2015), 6 pages. https:\/\/doi.org\/10.1136\/bmj.h1885","journal-title":"BMJ"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2348283.2348471"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/2505515.2505706"},{"key":"e_1_3_2_43_2","volume-title":"Pretrained Transformers for Text Ranking: BERT and Beyond","author":"Lin Jimmy","year":"2021","unstructured":"Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. 2021. Pretrained Transformers for Text Ranking: BERT and Beyond. version 3, Synthesis Lectures on Human Language Technologies, Morgan & Claypool Publishers, forthcoming. 204 pages. https:\/\/arxiv.org\/abs\/2010.06467"},{"key":"e_1_3_2_44_2","unstructured":"Sijia Liu Yanshan Wang Andrew Wen Liwei Wang Na Hong Feichen Shen Steven Bedrick William Hersh and Hongfang Liu. 2019. Create: Cohort retrieval enhanced by analysis of text from electronic health records using OMOP common data model. (2019). arXiv:1901.07601"},{"issue":"5","key":"e_1_3_2_45_2","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1097\/OPX.0b013e3182120514","article-title":"Waterloo Eye Study: Data abstraction and population representation","volume":"88","author":"Machan Carolyn M.","year":"2011","unstructured":"Carolyn M. Machan, Patricia K. Hrynchak, and Elizabeth L. Irving. 2011. Waterloo Eye Study: Data abstraction and population representation. Optometry and Vision Science 88, 5 (2011), 613\u2013620.","journal-title":"Optometry and Vision Science"},{"issue":"2","key":"e_1_3_2_46_2","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1097\/OPX.0b013e31823ee062","article-title":"Modeling the prevalence of age-related cataract: Waterloo eye study","volume":"89","author":"Machan Carolyn M.","year":"2012","unstructured":"Carolyn M. Machan, Patricia K. Hrynchak, and Elizabeth L. Irving. 2012. Modeling the prevalence of age-related cataract: Waterloo eye study. Optometry and Vision Science: Official Publication of the American Academy of Optometry 89, 2 (Feb. 2012), 130\u2013136.","journal-title":"Optometry and Vision Science: Official Publication of the American Academy of Optometry"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.5555\/2999792.2999959"},{"key":"e_1_3_2_48_2","unstructured":"Nabil H. Mustafa and Kasturi R. Varadarajan. 2017. Epsilon-approximations and epsilon-nets."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2018.06.002"},{"issue":"6","key":"e_1_3_2_50_2","first-page":"281","article-title":"Electronic medical records for clinical research: Application to the identification of heart failure","volume":"13","author":"Pakhomov Serguei","year":"2007","unstructured":"Serguei Pakhomov, Susan A. Weston, Steven J. Jacobsen, Christopher G. Chute, Ryan Meverden, and V\u00e9ronique L. Roger. 2007. Electronic medical records for clinical research: Application to the identification of heart failure. American Journal of Managed Care 13, 6 Part 1 (2007), 281\u2013288.","journal-title":"American Journal of Managed Care"},{"key":"e_1_3_2_51_2","first-page":"165","volume-title":"Computer Safety, Reliability, and Security - 38th International Conference (SAFECOMP\u201919), Proceedings (Lecture Notes in Computer Science)","volume":"11698","author":"Picardi Chiara","year":"2019","unstructured":"Chiara Picardi, Richard Hawkins, Colin Paterson, and Ibrahim Habli. 2019. A pattern for arguing the assurance of machine learning in medical diagnosis systems. In Computer Safety, Reliability, and Security - 38th International Conference (SAFECOMP\u201919), Proceedings (Lecture Notes in Computer Science), Alexander B. Romanovsky, Elena Troubitsyna, and Friedemann Bitsch (Eds.), Vol. 11698. Springer, 165\u2013179. DOI:https:\/\/doi.org\/10.1007\/978-3-030-26601-1_12"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/310663.310667"},{"key":"e_1_3_2_53_2","volume-title":"Proceedings of the 24th Text REtrieval Conference (TREC\u201915) (NIST Special Publication)","volume":"500","author":"Roegiest Adam","year":"2015","unstructured":"Adam Roegiest, Gordon V. Cormack, Charles L. A. Clarke, and Maura R. Grossman. 2015. TREC 2015 total recall track overview. In Proceedings of the 24th Text REtrieval Conference (TREC\u201915) (NIST Special Publication), Ellen M. Voorhees and Angela Ellis (Eds.), Vol. 500-319. National Institute of Standards and Technology (NIST), 29 pages. https:\/\/trec.nist.gov\/pubs\/trec24\/papers\/Overview-TR.pdf."},{"key":"e_1_3_2_54_2","first-page":"2830","volume-title":"The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS\u201920), Online (Proceedings of Machine Learning Research)","volume":"108","author":"Rogers Ryan","year":"2020","unstructured":"Ryan Rogers, Aaron Roth, Adam D. Smith, Nathan Srebro, Om Thakkar, and Blake E. Woodworth. 2020. Guaranteed validity for empirical approaches to adaptive data analysis. In The 23rd International Conference on Artificial Intelligence and Statistics (AISTATS\u201920), Online (Proceedings of Machine Learning Research), Silvia Chiappa and Roberto Calandra (Eds.), Vol. 108. PMLR, 2830\u20132840. http:\/\/proceedings.mlr.press\/v108\/rogers20a.html."},{"key":"e_1_3_2_55_2","first-page":"5 pp","volume-title":"The SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR\u201916)","author":"Roy Dwaipayan","year":"2016","unstructured":"Dwaipayan Roy, Debjyoti Paul, Mandar Mitra, and Utpal Garain. 2016. Using word embeddings for automatic query expansion. In The SIGIR 2016 Workshop on Neural Information Retrieval (Neu-IR\u201916). 5 pp. arxiv:1606.07608"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2019.2945779"},{"issue":"2","key":"e_1_3_2_57_2","first-page":"e9\u2013e14","article-title":"Validation of case finding algorithms for hepatocellular cancer from administrative data and electronic health records using natural language processing","volume":"54","author":"Sada Yvonne","year":"2016","unstructured":"Yvonne Sada, Jason Hou, Peter Richardson, Hashem El-Serag, and Jessica Davila. 2016. Validation of case finding algorithms for hepatocellular cancer from administrative data and electronic health records using natural language processing. Medical Care 54, 2 (February 2016), e9\u2013e14. DOI:https:\/\/doi.org\/10.1097\/MLR.0b013e3182a30373","journal-title":"Medical Care"},{"key":"e_1_3_2_58_2","volume-title":"Secondary Analysis of Electronic Health Records","author":"Sarmiento Raymond Francis","year":"2016","unstructured":"Raymond Francis Sarmiento and Franck Dernoncourt. 2016. Improving patient cohort identification using natural language processing. In Secondary Analysis of Electronic Health Records. Springer, Chapter 28. DOI:https:\/\/doi.org\/10.1007\/978-3-319-43742-2_28"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.5555\/3019233"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.5555\/2981562.2981724"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/130385.130417"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.5555\/2621980"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2013-001935"},{"issue":"2","key":"e_1_3_2_64_2","first-page":"47","article-title":"Building trust in artificial intelligence, machine learning, and robotics","volume":"31","author":"Siau Keng","year":"2018","unstructured":"Keng Siau and Weiyu Wang. 2018. Building trust in artificial intelligence, machine learning, and robotics. Cutter Business Technology Journal 31, 2 (2018), 47\u201353.","journal-title":"Cutter Business Technology Journal"},{"key":"e_1_3_2_65_2","unstructured":"Statistics Canada. 2001. Statistics: Power From Data! (2001). Retrieved April 30 2020 from https:\/\/www150.statcan.gc.ca\/n1\/edu\/power-pouvoir\/toc-tdm\/5214718-eng.htm. Catalog no. 12-004-X."},{"key":"e_1_3_2_66_2","volume-title":"Sampling Algorithms","author":"Till\u00e9 Yves","year":"2006","unstructured":"Yves Till\u00e9. 2006. Sampling Algorithms. Springer Science & Business Media."},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1162\/153244302760185243"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1197\/j.aem.2004.08.012"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/2506583.2506624"},{"key":"e_1_3_2_70_2","series-title":"The Information Retrieval Series","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1007\/978-3-030-22948-1_2","volume-title":"Information Retrieval Evaluation in a Changing World - Lessons Learned from 20 Years of CLEF","author":"Voorhees Ellen M.","year":"2019","unstructured":"Ellen M. Voorhees. 2019. The evolution of Cranfield. In Information Retrieval Evaluation in a Changing World - Lessons Learned from 20 Years of CLEF, Nicola Ferro and Carol Peters (Eds.). The Information Retrieval Series, Vol. 41. Springer, 45\u201369. DOI:https:\/\/doi.org\/10.1007\/978-3-030-22948-1_2"},{"key":"e_1_3_2_71_2","volume-title":"Proceedings of the 20th Text REtrieval Conference (TREC\u201911)","volume":"500","author":"Voorhees Ellen M.","year":"2011","unstructured":"Ellen M. Voorhees and Lori P. Buckland (Eds.). 2011. Proceedings of the 20th Text REtrieval Conference (TREC\u201911). Vol. Special Publication 500-296. National Institute of Standards and Technology (NIST). https:\/\/trec.nist.gov\/pubs\/trec20\/t20.proceedings.html."},{"key":"e_1_3_2_72_2","unstructured":"NIST Special Publication Proceedings of the 21st Text REtrieval Conference (TREC\u201912) 500-298 Ellen M. Voorhees Lori P. Buckland 2012"},{"key":"e_1_3_2_73_2","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.ijmedinf.2015.10.005","article-title":"Computer-assisted expert case definition in electronic health records","volume":"86","author":"Walker Alexander M.","year":"2016","unstructured":"Alexander M. Walker, Xiaofeng Zhou, Ashwin N. Ananthakrishnan, Lisa S. Weiss, Rongjun Shen, Rachel E. Sobel, Andrew Bate, and Robert F. Reynolds. 2016. Computer-assisted expert case definition in electronic health records. International Journal of Medical Informatics 86 (2016), 62\u201370. DOI:https:\/\/doi.org\/10.1016\/j.ijmedinf.2015.10.005","journal-title":"International Journal of Medical Informatics"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/2414782.2414784"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.5555\/3294996.3295013"},{"key":"e_1_3_2_76_2","unstructured":"Aston Zhang Zachary C. Lipton Mu Li and Alexander J Smola. 2021. Dive into Deep Learning v2. 839. https:\/\/arxiv.org\/abs\/2106.11342."},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2914685"}],"container-title":["ACM Transactions on Computing for Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3483411","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3483411","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:58Z","timestamp":1750191538000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3483411"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,20]]},"references-count":76,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,4,30]]}},"alternative-id":["10.1145\/3483411"],"URL":"https:\/\/doi.org\/10.1145\/3483411","relation":{},"ISSN":["2691-1957","2637-8051"],"issn-type":[{"type":"print","value":"2691-1957"},{"type":"electronic","value":"2637-8051"}],"subject":[],"published":{"date-parts":[[2021,12,20]]},"assertion":[{"value":"2021-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}