{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,8,16]],"date-time":"2023-08-16T20:58:32Z","timestamp":1692219512051},"reference-count":49,"publisher":"MIT Press - Journals","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Linguistics"],"published-print":{"date-parts":[[2014,3]]},"abstract":"<jats:p> Philosophy of science has pointed out a problem of theoretical terms in empirical sciences. This problem arises if all known measuring procedures for a quantity of a theory presuppose the validity of this very theory, because then statements containing theoretical terms are circular. We argue that a similar circularity can happen in empirical computational linguistics, especially in cases where data are manually annotated by experts. We define a criterion of T-non-theoretical grounding as guidance to avoid such circularities, and exemplify how this criterion can be met by crowdsourcing, by task-related data annotation, or by data in the wild. We argue that this criterion should be considered as a necessary condition for an empirical science, in addition to measures for reliability of data annotation. <\/jats:p>","DOI":"10.1162\/coli_a_00182","type":"journal-article","created":{"date-parts":[[2013,11,4]],"date-time":"2013-11-04T17:26:46Z","timestamp":1383586006000},"page":"235-245","source":"Crossref","is-referenced-by-count":6,"title":["On the Problem of Theoretical Terms in Empirical Computational Linguistics"],"prefix":"10.1162","volume":"40","author":[{"given":"Stefan","family":"Riezler","sequence":"first","affiliation":[{"name":"Computational Linguistics Heidelberg University, Germany"}]}],"member":"281","reference":[{"key":"R1","doi-asserted-by":"publisher","DOI":"10.1162\/coli.07-034-R2"},{"key":"R2","doi-asserted-by":"publisher","DOI":"10.1515\/9783110879421"},{"key":"R3","doi-asserted-by":"publisher","DOI":"10.1515\/9783110879421"},{"key":"R4","doi-asserted-by":"publisher","DOI":"10.1162\/coli.2009.35.4.35402"},{"key":"R5","doi-asserted-by":"publisher","DOI":"10.1162\/coli.2009.35.1.111"},{"key":"R6","first-page":"440","volume-title":"Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL'07)","author":"Blitzer John","year":"2007"},{"key":"R7","first-page":"1","volume-title":"Proceedings of the NAACL-HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk","author":"Callison-Burch Chris","year":"2010"},{"issue":"2","key":"R8","first-page":"1","volume":"22","author":"Carletta Jean","year":"1996","journal-title":"Computational Linguistics"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390173"},{"key":"R10","first-page":"859","volume-title":"Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI'11)","author":"Chen David L.","year":"2011"},{"key":"R11","first-page":"18","volume-title":"Proceedings of the 14th Conference on Natural Language Learning (CoNLL'10)","author":"Clarke James","year":"2010"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.1007\/s10838-009-9091-3"},{"key":"R13","first-page":"66","volume-title":"Proceedings of the NAACL-HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk","author":"Denkowski Michael","year":"2010"},{"key":"R14","first-page":"162","volume-title":"Proceedings of 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2012)","author":"Dreyer Markus","year":"2012"},{"key":"R15","first-page":"756","volume-title":"Proceedings of the 20th International Joint Conference on Artifical Intelligence (IJCAI'07)","author":"Dundar Murat","year":"2007"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-322-86605-9"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_a_00057"},{"key":"R18","volume-title":"Methoden empirischer Sozialforschung.","author":"Friedrichs J\u00fcrgen","year":"1973","edition":"14"},{"key":"R19","doi-asserted-by":"publisher","DOI":"10.1007\/BF01800832"},{"key":"R20","first-page":"60","volume-title":"Proceedings of the 2nd International Workshop on Evaluating Information Access (EVIA)","author":"Graf Erik","year":"2008"},{"key":"R21","first-page":"1,058","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI'09)","author":"Guo Yunsong","year":"2009"},{"key":"R22","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2009.36"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.3115\/1034678.1034731"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.3115\/1614049.1614064"},{"key":"R25","doi-asserted-by":"publisher","DOI":"10.3115\/977035.977084"},{"key":"R26","first-page":"218","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL'13)","author":"Kim Joohyun","year":"2013"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.1023\/B:MIND.0000045986.90956.7f"},{"key":"R28","volume-title":"Content Analysis. An Introduction to Its Methodology.","author":"Krippendorff Klaus","year":"1980","edition":"2013"},{"key":"R29","first-page":"69","volume-title":"Computerstrategien f\u00fcr die Kommunikationsanalyse.","author":"Krippendorff Klaus","year":"1980"},{"issue":"3","key":"R30","first-page":"411","volume":"30","author":"Krippendorff Klaus","year":"2004","journal-title":"Human Communication Research"},{"key":"R31","first-page":"46","volume-title":"Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT'08)","author":"Miyao Yusuke","year":"2008"},{"key":"R32","first-page":"109","volume-title":"Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics (EACL'12)","author":"Nikoulina Vassilina","year":"2012"},{"key":"R33","doi-asserted-by":"publisher","DOI":"10.3115\/1118693.1118704"},{"key":"R34","doi-asserted-by":"publisher","DOI":"10.1145\/2448116.2448119"},{"key":"R35","doi-asserted-by":"publisher","DOI":"10.1162\/coli.2008.34.3.319"},{"key":"R36","doi-asserted-by":"publisher","DOI":"10.1016\/S0885-2308(02)00024-4"},{"key":"R37","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-010-3066-3"},{"key":"R38","doi-asserted-by":"publisher","DOI":"10.3115\/1613715.1613751"},{"key":"R39","doi-asserted-by":"publisher","DOI":"10.3115\/1075812.1075833"},{"key":"R40","volume-title":"The Structuralist View of Theories. A Possible Analogue of the Bourbaki Programme in Physical Science.","author":"Stegm\u00fcller Wolfgang","year":"1979"},{"key":"R41","volume-title":"Probleme und Resultate der Wissenschaftstheorie und Analytischen Philosophie. Band II: Theorie und Erfahrung.","author":"Stegm\u00fcller Wolfgang","year":"1986"},{"key":"R42","first-page":"719","volume-title":"Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics (ACL'08)","author":"Surdeanu Mihai","year":"2008"},{"key":"R43","first-page":"1","volume-title":"Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP'04)","author":"Taskar Ben","year":"2004"},{"key":"R44","first-page":"1453","volume":"5","author":"Tsochantaridis Ioannis","year":"2005","journal-title":"Journal of Machine Learning Research"},{"key":"R45","doi-asserted-by":"publisher","DOI":"10.1145\/985692.985733"},{"key":"R46","volume-title":"Grundfragen der Erkenntnistheorie.","author":"von Kutschera Franz","year":"1982"},{"key":"R47","first-page":"1","volume-title":"Proceedings of the ICML 2011 Workshop on Unsupervised and Transfer Learning","author":"von Luxburg Ulrike","year":"2012"},{"key":"R48","first-page":"488","volume-title":"Proceedings of the 19th National Conference on Artificial Intelligence (AAAI'04)","author":"Yu Chen","year":"2004"},{"key":"R49","first-page":"53","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL'13)","author":"Yu Haonan","year":"2013"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/COLI_a_00182","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:27:34Z","timestamp":1615584454000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/40\/1\/235-245\/1452"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,3]]},"references-count":49,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2014,3]]}},"alternative-id":["10.1162\/COLI_a_00182"],"URL":"https:\/\/doi.org\/10.1162\/coli_a_00182","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,3]]}}}