{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,2]],"date-time":"2025-10-02T06:06:21Z","timestamp":1759385181172,"version":"3.41.0"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2017,3,31]],"date-time":"2017-03-31T00:00:00Z","timestamp":1490918400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NoemaLife SpA in the framework of the ConnectToLife project"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Data and Information Quality"],"published-print":{"date-parts":[[2017,3,31]]},"abstract":"<jats:p>In the last five years there has been a flurry of work on information extraction from clinical documents, that is, on algorithms capable of extracting, from the informal and unstructured texts that are generated during everyday clinical practice, mentions of concepts relevant to such practice. Many of these research works are about methods based on supervised learning, that is, methods for training an information extraction system from manually annotated examples. While a lot of work has been devoted to devising learning methods that generate more and more accurate information extractors, no work has been devoted to investigating the effect of the quality of training data on the learning process for the clinical domain. Low quality in training data often derives from the fact that the person who has annotated the data is different from the one against whose judgment the automatically annotated data must be evaluated. In this article, we test the impact of such data quality issues on the accuracy of information extraction systems as applied to the clinical domain. We do this by comparing the accuracy deriving from training data annotated by the authoritative coder (i.e., the one who has also annotated the test data and by whose judgment we must abide) with the accuracy deriving from training data annotated by a different coder, equally expert in the subject matter. The results indicate that, although the disagreement between the two coders (as measured on the training set) is substantial, the difference is (surprisingly enough) not always statistically significant. While the dataset used in the present work originated in a clinical context, the issues we study in this work are of more general interest.<\/jats:p>","DOI":"10.1145\/3106235","type":"journal-article","created":{"date-parts":[[2017,9,11]],"date-time":"2017-09-11T12:12:26Z","timestamp":1505131946000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["On the Effects of Low-Quality Training Data on Information Extraction from Clinical Reports"],"prefix":"10.1145","volume":"9","author":[{"given":"Diego","family":"Marcheggiani","sequence":"first","affiliation":[{"name":"University of Amsterdam, Amsterdam, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4221-6427","authenticated-orcid":false,"given":"Fabrizio","family":"Sebastiani","sequence":"additional","affiliation":[{"name":"Italian National Council of Research, Pisa, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,9,8]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the 20th International Conference on Machine Learning (ICML\u201903)","author":"Altun Yasemin","year":"2003","unstructured":"Yasemin Altun , Ioannis Tsochantaridis , and Thomas Hofmann . 2003 . Hidden markov support vector machines . In Proceedings of the 20th International Conference on Machine Learning (ICML\u201903) . 3--10. Yasemin Altun, Ioannis Tsochantaridis, and Thomas Hofmann. 2003. Hidden markov support vector machines. In Proceedings of the 20th International Conference on Machine Learning (ICML\u201903). 3--10."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli.07-034-R2"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00074"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2669368"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S16-1165"},{"volume-title":"Semi-Supervised Learning","author":"Chapelle Olivier","key":"e_1_2_1_6_1","unstructured":"Olivier Chapelle , Bernard Sch\u00f6lkopf , and Alexander Zien &nbsp;(Eds.). 2006. Semi-Supervised Learning . The MIT Press , Cambridge, MA . Olivier Chapelle, Bernard Sch\u00f6lkopf, and Alexander Zien&nbsp;(Eds.). 2006. Semi-Supervised Learning. The MIT Press, Cambridge, MA."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2005.06.004"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/972487.972488"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v56:6"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1162\/089120104773633402"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S15-2051"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2013.01.006"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04417-5_4"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/1889174.1889192"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2516889"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24027-5_44"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.5555\/1866696.1866723"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/BIBM.2013.6732541"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2014-002669"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 6th Conference on Language Resources and Evaluation (LREC\u201908)","author":"Haddow Barry","year":"2008","unstructured":"Barry Haddow and Beatrice Alex . 2008 . Exploiting multiply annotated corpora in biomedical information extraction tasks . In Proceedings of the 6th Conference on Language Resources and Evaluation (LREC\u201908) . Barry Haddow and Beatrice Alex. 2008. Exploiting multiply annotated corpora in biomedical information extraction tasks. In Proceedings of the 6th Conference on Language Resources and Evaluation (LREC\u201908)."},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the 2009 IEEE World Congress on Engineering and Computer Science (WCECS\u201909)","author":"Jassar Surinder","year":"2009","unstructured":"Surinder Jassar , Zaiyi Liao , and Lian Zhao . 2009 . Impact of data quality on predictive accuracy of ANFIS-based soft sensor models . In Proceedings of the 2009 IEEE World Congress on Engineering and Computer Science (WCECS\u201909) , Vol. II . Surinder Jassar, Zaiyi Liao, and Lian Zhao. 2009. Impact of data quality on predictive accuracy of ANFIS-based soft sensor models. In Proceedings of the 2009 IEEE World Congress on Engineering and Computer Science (WCECS\u201909), Vol. II."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000163"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the 16th International Conference on Machine Learning (ICML\u201999)","author":"Joachims Thorsten","year":"1999","unstructured":"Thorsten Joachims . 1999 . Transductive inference for text classification using support vector machines . In Proceedings of the 16th International Conference on Machine Learning (ICML\u201999) . 200--209. Thorsten Joachims. 1999. Transductive inference for text classification using support vector machines. In Proceedings of the 16th International Conference on Machine Learning (ICML\u201999). 200--209."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2011.10.007"},{"key":"e_1_2_1_25_1","volume-title":"Kors","author":"Kang Ning","year":"2012","unstructured":"Ning Kang , Erik M. van Mulligen , and Jan A . Kors . 2012 . Training text chunkers on a silver standard corpus: Can silver replace gold? BMC Bioinform . 13, 17 (2012). Ning Kang, Erik M. van Mulligen, and Jan A. Kors. 2012. Training text chunkers on a silver standard corpus: Can silver replace gold? BMC Bioinform. 13, 17 (2012)."},{"volume-title":"Proceedings of the 5th International Conference of the CLEF Initiative (CLEF\u201914)","author":"Kelly Liadh","key":"e_1_2_1_26_1","unstructured":"Liadh Kelly , Lorraine Goeuriot , Hanna Suominen , Tobias Schreck , Gondy Leroy , Danielle L. Mowery , Sumithra Velupillai , Wendy W. Chapman , David Mart\u00ednez , Guido Zuccon , and Jo\u00e3o R. M. Palotti . 2014. Overview of the ShARe\/CLEF eHealth evaluation lab 2014 . In Proceedings of the 5th International Conference of the CLEF Initiative (CLEF\u201914) . 172--191. Liadh Kelly, Lorraine Goeuriot, Hanna Suominen, Tobias Schreck, Gondy Leroy, Danielle L. Mowery, Sumithra Velupillai, Wendy W. Chapman, David Mart\u00ednez, Guido Zuccon, and Jo\u00e3o R. M. Palotti. 2014. Overview of the ShARe\/CLEF eHealth evaluation lab 2014. In Proceedings of the 5th International Conference of the CLEF Initiative (CLEF\u201914). 172--191."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.3923\/jas.2005.1394.1398"},{"key":"e_1_2_1_28_1","volume-title":"Content Analysis: An Introduction to its Methodology","author":"Krippendorff Klaus","year":"2004","unstructured":"Klaus Krippendorff . 2004 . Content Analysis: An Introduction to its Methodology . Sage , Thousand Oaks, CA . Klaus Krippendorff. 2004. Content Analysis: An Introduction to its Methodology. Sage, Thousand Oaks, CA."},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of the 18th International Conference on Machine Learning (ICML\u201901)","author":"Lafferty John","year":"2001","unstructured":"John Lafferty , Andrew McCallum , and Fernando Pereira . 2001 . Conditional random fields: Probabilistic models for segmenting and labeling sequence data . In Proceedings of the 18th International Conference on Machine Learning (ICML\u201901) . 282--289. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning (ICML\u201901). 282--289."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/1572306.1572326"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1882992.1883105"},{"key":"e_1_2_1_32_1","volume-title":"Hurdle","author":"Meystre Stephane M.","year":"2008","unstructured":"Stephane M. Meystre , Guerguana K. Savova , Karin C. Kipper-Schuler , and John F . Hurdle . 2008 . Extracting information from textual documents in the electronic health record: A review of recent research. In IMIA Yearbook of Medical Informatics, A. Geissbuhler and C. Kulikowski&nbsp;(Eds.). Schattauer Publishers , Stuttgart, DE, 128--144. Stephane M. Meystre, Guerguana K. Savova, Karin C. Kipper-Schuler, and John F. Hurdle. 2008. Extracting information from textual documents in the electronic health record: A review of recent research. In IMIA Yearbook of Medical Informatics, A. Geissbuhler and C. Kulikowski&nbsp;(Eds.). Schattauer Publishers, Stuttgart, DE, 128--144."},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of the IJCAI 2009 Workshop on Intelligence and Interaction.","author":"Nallapati Ramesh","year":"2009","unstructured":"Ramesh Nallapati , Mihai Surdeanu , and Christopher Manning . 2009 . CorrActive learning: Learning from noisy data through human interaction . In Proceedings of the IJCAI 2009 Workshop on Intelligence and Interaction. Ramesh Nallapati, Mihai Surdeanu, and Christopher Manning. 2009. CorrActive learning: Learning from noisy data through human interaction. In Proceedings of the IJCAI 2009 Workshop on Intelligence and Interaction."},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the Working Notes of CLEF 2016\u2014Conference and Labs of the Evaluation Forum. \u00c9vora, PT, 28--42","author":"N\u00e9v\u00e9ol Aur\u00e9lie","year":"2016","unstructured":"Aur\u00e9lie N\u00e9v\u00e9ol , Kevin Bretonnel Cohen , Cyril Grouin , Thierry Hamon , Thomas Lavergne , Liadh Kelly , Lorraine Goeuriot , Gr\u00e9goire Rey , Aude Robert , Xavier Tannier , and Pierre Zweigenbaum . 2016 . Clinical information extraction at the CLEF eHealth evaluation lab 2016 . In Proceedings of the Working Notes of CLEF 2016\u2014Conference and Labs of the Evaluation Forum. \u00c9vora, PT, 28--42 . Aur\u00e9lie N\u00e9v\u00e9ol, Kevin Bretonnel Cohen, Cyril Grouin, Thierry Hamon, Thomas Lavergne, Liadh Kelly, Lorraine Goeuriot, Gr\u00e9goire Rey, Aude Robert, Xavier Tannier, and Pierre Zweigenbaum. 2016. Clinical information extraction at the CLEF eHealth evaluation lab 2016. In Proceedings of the Working Notes of CLEF 2016\u2014Conference and Labs of the Evaluation Forum. \u00c9vora, PT, 28--42."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-35101-3_38"},{"volume-title":"Mining Text Data, Charu C. Aggarwal and ChengXiang Zhai&nbsp;(Eds.)","author":"Pan Weike","key":"e_1_2_1_36_1","unstructured":"Weike Pan , Erheng Zhong , and Qiang Yang . 2012. Transfer learning for text mining . In Mining Text Data, Charu C. Aggarwal and ChengXiang Zhai&nbsp;(Eds.) . Springer , Heidelberg , 223--258. Weike Pan, Erheng Zhong, and Qiang Yang. 2012. Transfer learning for text mining. In Mining Text Data, Charu C. Aggarwal and ChengXiang Zhai&nbsp;(Eds.). Springer, Heidelberg, 223--258."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2010.003939"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/S14-2007"},{"key":"e_1_2_1_39_1","volume-title":"Lawrence&nbsp;(Eds.)","author":"Qui\u00f1onero-Candela Joaquin","year":"2009","unstructured":"Joaquin Qui\u00f1onero-Candela , Masashi Sugiyama , Anton Schwaighofer , and Neil D . Lawrence&nbsp;(Eds.) . 2009 . Dataset Shift in Machine Learning. The MIT Press , Cambridge, MA. Joaquin Qui\u00f1onero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence&nbsp;(Eds.). 2009. Dataset Shift in Machine Learning. The MIT Press, Cambridge, MA."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0219720010004562"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2008.12.013"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2015.06.010"},{"key":"e_1_2_1_43_1","article-title":"Data errors in neural network and linear regression models: An experimental comparison","volume":"5","author":"Rossin Donald F.","year":"1999","unstructured":"Donald F. Rossin and Barbara D. Klein . 1999 . Data errors in neural network and linear regression models: An experimental comparison . Data Quality J. 5 , 1 (1999). Donald F. Rossin and Barbara D. Klein. 1999. Data errors in neural network and linear regression models: An experimental comparison. Data Quality J. 5, 1 (1999).","journal-title":"Data Quality J."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJKEDM.2015.071284"},{"volume-title":"Encyclopedia of Machine Learning, Claude Sammut and Geoffrey I. Webb&nbsp;(Eds.)","author":"Sammut Claude","key":"e_1_2_1_45_1","unstructured":"Claude Sammut and Michael Harries . 2011. Concept drift . In Encyclopedia of Machine Learning, Claude Sammut and Geoffrey I. Webb&nbsp;(Eds.) . Springer , Heidelberg , 202--205. Claude Sammut and Michael Harries. 2011. Concept drift. In Encyclopedia of Machine Learning, Claude Sammut and Geoffrey I. Webb&nbsp;(Eds.). Springer, Heidelberg, 202--205."},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the Annual Symposium of the American Medical Informatics Association (AMIA\u201906)","author":"Sibanda Tawanda","year":"2006","unstructured":"Tawanda Sibanda , Tian He , Peter Szolovits , and \u00d6zlem Uzuner . 2006 . Syntactically-informed semantic category recognition in discharge summaries . In Proceedings of the Annual Symposium of the American Medical Informatics Association (AMIA\u201906) . 714--718. Tawanda Sibanda, Tian He, Peter Szolovits, and \u00d6zlem Uzuner. 2006. Syntactically-informed semantic category recognition in discharge summaries. In Proceedings of the Annual Symposium of the American Medical Informatics Association (AMIA\u201906). 714--718."},{"volume-title":"Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP\u201908)","author":"Snow Rion","key":"e_1_2_1_47_1","unstructured":"Rion Snow , Brendan O\u2019Connor , Daniel Jurafsky , and Andrew Y. Ng . 2008. Cheap and fast - But is it good? Evaluating non-expert annotations for natural language tasks . In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP\u201908) . 254--263. Rion Snow, Brendan O\u2019Connor, Daniel Jurafsky, and Andrew Y. Ng. 2008. Cheap and fast - But is it good? Evaluating non-expert annotations for natural language tasks. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP\u201908). 254--263."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2013-001628"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40802-1_24"},{"volume-title":"Introduction to Statistical Relational Learning, Lise Getoor and Ben Taskar&nbsp;(Eds.)","author":"Sutton Charles","key":"e_1_2_1_50_1","unstructured":"Charles Sutton and Andrew McCallum . 2007. An introduction to conditional random fields for relational learning . In Introduction to Statistical Relational Learning, Lise Getoor and Ben Taskar&nbsp;(Eds.) . The MIT Press , Cambridge, MA , 93--127. Charles Sutton and Andrew McCallum. 2007. An introduction to conditional random fields for relational learning. In Introduction to Statistical Relational Learning, Lise Getoor and Ben Taskar&nbsp;(Eds.). The MIT Press, Cambridge, MA, 93--127."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1561\/2200000013"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.3115\/1220175.1220203"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2390068.2390073"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000155"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.5555\/1046920.1088722"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000784"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000203"},{"key":"e_1_2_1_58_1","volume-title":"Harman&nbsp;(Eds.)","author":"Voorhees Ellen M.","year":"2005","unstructured":"Ellen M. Voorhees and Donna K . Harman&nbsp;(Eds.) . 2005 . TREC. Experiment and Evaluation in Information Retrieval. The MIT Press , Cambridge, MA. Ellen M. Voorhees and Donna K. Harman&nbsp;(Eds.). 2005. TREC. Experiment and Evaluation in Information Retrieval. The MIT Press, Cambridge, MA."},{"key":"e_1_2_1_59_1","volume-title":"Pooling annotated corpora for clinical concept extraction. J. Biomed. Sem. 4","author":"Wagholikar Kavishwar B.","year":"2013","unstructured":"Kavishwar B. Wagholikar , Manabu Torii , Siddhartha Jonnalagadda , and Hongfang Liu . 2013. Pooling annotated corpora for clinical concept extraction. J. Biomed. Sem. 4 ( 2013 ), Article 3. Kavishwar B. Wagholikar, Manabu Torii, Siddhartha Jonnalagadda, and Hongfang Liu. 2013. Pooling annotated corpora for clinical concept extraction. J. Biomed. Sem. 4 (2013), Article 3."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1561\/2200000001"},{"key":"e_1_2_1_61_1","volume-title":"Proceedings of the RANLP 2009 Workshop on Biomedical Information Extraction. 42--49","author":"Wang Yefeng","year":"2009","unstructured":"Yefeng Wang and Jon Patrick . 2009 . Cascading classifiers for named entity recognition in clinical notes . In Proceedings of the RANLP 2009 Workshop on Biomedical Information Extraction. 42--49 . Yefeng Wang and Jon Patrick. 2009. Cascading classifiers for named entity recognition in clinical notes. In Proceedings of the RANLP 2009 Workshop on Biomedical Information Extraction. 42--49."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484156"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.3115\/992730.992783"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/S14-2142"}],"container-title":["Journal of Data and Information Quality"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3106235","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3106235","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:30:36Z","timestamp":1750217436000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3106235"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,3,31]]},"references-count":64,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2017,3,31]]}},"alternative-id":["10.1145\/3106235"],"URL":"https:\/\/doi.org\/10.1145\/3106235","relation":{},"ISSN":["1936-1955","1936-1963"],"issn-type":[{"type":"print","value":"1936-1955"},{"type":"electronic","value":"1936-1963"}],"subject":[],"published":{"date-parts":[[2017,3,31]]},"assertion":[{"value":"2015-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-09-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}