{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:11:16Z","timestamp":1750306276096,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":33,"publisher":"ACM","license":[{"start":{"date-parts":[[2016,5,14]],"date-time":"2016-05-14T00:00:00Z","timestamp":1463184000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2016,5,14]]},"DOI":"10.1145\/2897022.2897029","type":"proceedings-article","created":{"date-parts":[[2016,5,22]],"date-time":"2016-05-22T02:39:36Z","timestamp":1463884776000},"page":"46-52","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Cost-efficient quality assurance of natural language processing tools through continuous monitoring with continuous integration"],"prefix":"10.1145","author":[{"given":"Marc","family":"Schreiber","sequence":"first","affiliation":[{"name":"University of Applied Sciences"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bodo","family":"Kraft","sequence":"additional","affiliation":[{"name":"University of Applied Sciences"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Albert","family":"Z\u00fcndorf","sequence":"additional","affiliation":[{"name":"University of Kassel"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2016,5,14]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"BMC Bioinformatics 12.S-3","author":"Barrett N.","year":"2011","unstructured":"N. Barrett and J. H. Weber-Jahnke . \" Building a biomedical tokenizer using the token lattice design pattern and the adapted Viterbi algorithm \". In: BMC Bioinformatics 12.S-3 ( 2011 ). N. Barrett and J. H. Weber-Jahnke. \"Building a biomedical tokenizer using the token lattice design pattern and the adapted Viterbi algorithm\". In: BMC Bioinformatics 12.S-3 (2011)."},{"key":"e_1_3_2_1_2_1","volume-title":"Jenkins Continuous Integration Cookbook","author":"Berg A. M.","year":"2015","unstructured":"A. M. Berg . Jenkins Continuous Integration Cookbook . 2 nd Edition. Packt Publishing , 2015 . ISBN: 978-1-78439-008-2. A. M. Berg. Jenkins Continuous Integration Cookbook. 2nd Edition. Packt Publishing, 2015. ISBN: 978-1-78439-008-2.","edition":"2"},{"key":"e_1_3_2_1_3_1","first-page":"83","volume-title":"Proceedings of the International Conference on Recent Advances in Natural Language Processing. Hissar, Bulgaria: INCOMA Ltd. Shoumen, BULGARIA","author":"Bontcheva K.","year":"2013","unstructured":"K. Bontcheva : An Open-Source Information Extraction Pipeline for Microblog Text \". In: Proceedings of the International Conference on Recent Advances in Natural Language Processing. Hissar, Bulgaria: INCOMA Ltd. Shoumen, BULGARIA , 2013 , pp. 83 -- 90 . K. Bontcheva et al. \"TwitIE: An Open-Source Information Extraction Pipeline for Microblog Text\". In: Proceedings of the International Conference on Recent Advances in Natural Language Processing. Hissar, Bulgaria: INCOMA Ltd. Shoumen, BULGARIA, 2013, pp. 83--90."},{"key":"e_1_3_2_1_4_1","unstructured":"E. Buyko etal \"Automatically adapting an NLP core engine to the biology domain\". In: Proceedings of the Joint BioLINK-Bio-Ontologies Meeting. A Joint Meeting of the ISMB Special Interest Group on Bio-Ontologies and the BioLINK Special Interest Group on Text Data Mining in Association with ISMB. 2006 pp. 65--68.  E. Buyko et al. \"Automatically adapting an NLP core engine to the biology domain\". In: Proceedings of the Joint BioLINK-Bio-Ontologies Meeting. A Joint Meeting of the ISMB Special Interest Group on Bio-Ontologies and the BioLINK Special Interest Group on Text Data Mining in Association with ISMB. 2006 pp. 65--68."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2014.2307227"},{"key":"e_1_3_2_1_6_1","volume-title":"The Data Warehouse ETL Toolkit","author":"Caserta R.","year":"2004","unstructured":"R. Caserta . The Data Warehouse ETL Toolkit . Wiley India Pvt . Limited, 2004 . ISBN: 978-8-12650-554-8. R. Caserta. The Data Warehouse ETL Toolkit. Wiley India Pvt. Limited, 2004. ISBN: 978-8-12650-554-8."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-5201"},{"key":"e_1_3_2_1_8_1","first-page":"056","volume-title":"Pearson Education","author":"Cohn M.","year":"2009","unstructured":"M. Cohn . Succeeding with Agile : Software Development Using Scrum . Pearson Education , 2009 . ISBN: 978-0-32166- 056 - 055 . M. Cohn. Succeeding with Agile: Software Development Using Scrum. Pearson Education, 2009. ISBN: 978-0-32166-056-5."},{"key":"e_1_3_2_1_9_1","volume-title":"Continuous Integration: Improving Software Quality and Reducing Risk","author":"Duvall P.","year":"2007","unstructured":"P. Duvall , S. M. Matyas , and A. Glover . Continuous Integration: Improving Software Quality and Reducing Risk . Addison-Wesley Professional , 2007 . ISBN: 0-32133-638-0. P. Duvall, S. M. Matyas, and A. Glover. Continuous Integration: Improving Software Quality and Reducing Risk. Addison-Wesley Professional, 2007. ISBN: 0-32133-638-0."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2012-001453"},{"key":"e_1_3_2_1_11_1","first-page":"095","volume-title":"Manning","author":"Gaudin O.","year":"2013","unstructured":"O. Gaudin . SonarQube in Action . Manning , 2013 . ISBN: 978-1-61729- 095 - 094 . O. Gaudin. SonarQube in Action. Manning, 2013. ISBN: 978-1-61729-095-4."},{"key":"e_1_3_2_1_12_1","first-page":"27","volume-title":"Proceedings of the 5th Web as Corpus Workshop. WAC5.","author":"Giesbrecht E.","year":"2009","unstructured":"E. Giesbrecht and S. Evert . \" Is Part-of-Speech Tagging a Solved Task? An Evaluation of POS Taggers for the German Web as Corpus \". In: Proceedings of the 5th Web as Corpus Workshop. WAC5. 2009 , pp. 27 -- 35 . E. Giesbrecht and S. Evert. \"Is Part-of-Speech Tagging a Solved Task? An Evaluation of POS Taggers for the German Web as Corpus\". In: Proceedings of the 5th Web as Corpus Workshop. WAC5. 2009, pp. 27--35."},{"key":"e_1_3_2_1_13_1","first-page":"40","article-title":"Continuous Integration and Quality Assurance: A Case Study of Two Open Source Projects","author":"Holck J.","year":"2007","unstructured":"J. Holck and N. J\u00f8rgensen . \" Continuous Integration and Quality Assurance: A Case Study of Two Open Source Projects \". In: Australasian Journal of Information Systems ( 2007 ), pp. 40 -- 53 . J. Holck and N. J\u00f8rgensen. \"Continuous Integration and Quality Assurance: A Case Study of Two Open Source Projects\". In: Australasian Journal of Information Systems (2007), pp. 40--53.","journal-title":"Australasian Journal of Information Systems ("},{"key":"e_1_3_2_1_14_1","volume-title":"Proceeding of Embedded Systems Conference. Silicon Valley","author":"Karlesky M.","year":"2007","unstructured":"M. Karlesky : Test-Driven Development, Continuous Integration, and Design Patterns \". In: Proceeding of Embedded Systems Conference. Silicon Valley , 2007 . M. Karlesky et al. \"Mocking the Embedded World: Test-Driven Development, Continuous Integration, and Design Patterns\". In: Proceeding of Embedded Systems Conference. Silicon Valley, 2007."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btg1023"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btn181"},{"key":"e_1_3_2_1_17_1","volume-title":"DISI","author":"Krapivin M.","year":"2008","unstructured":"M. Krapivin , A. Autayeu , and M. Marchese . Large Dataset for Keyphrases Extraction. Tech. rep. DISI-09-055 . DISI , Trento, Italy , 2008 . M. Krapivin, A. Autayeu, and M. Marchese. Large Dataset for Keyphrases Extraction. Tech. rep. DISI-09-055. DISI, Trento, Italy, 2008."},{"key":"e_1_3_2_1_18_1","volume-title":"Proceedings of the Eighth Workshop on Statistical Machine Translation, ACL 2013","author":"Lewis W.","year":"2013","unstructured":"W. Lewis and S. Eetemadi . \" Dramatically Reducing Training Data Size through Vocabulary Saturation \". In: Proceedings of the Eighth Workshop on Statistical Machine Translation, ACL 2013 . ACL, 2013 . W. Lewis and S. Eetemadi. \"Dramatically Reducing Training Data Size through Vocabulary Saturation\". In: Proceedings of the Eighth Workshop on Statistical Machine Translation, ACL 2013. ACL, 2013."},{"key":"e_1_3_2_1_19_1","first-page":"1527","article-title":"Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements","author":"Lingren T.","year":"2013","unstructured":"T. Lingren \" Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements .\" In: Journal of the American Medical Informatics Association ( 2013 ). ISSN: 1527 - 1974 X. T. Lingren et al. \"Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements.\" In: Journal of the American Medical Informatics Association (2013). ISSN: 1527-974X.","journal-title":"Journal of the American Medical Informatics Association ("},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1186\/2041-1480-3-3"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/972470.972475"},{"key":"e_1_3_2_1_22_1","first-page":"139","volume-title":"Language Processing and Knowledge in the Web. Ed. by I. Gurevych, C. Biemann, and T. Zesch.","author":"Neunerdt M.","year":"2013","unstructured":"M. Neunerdt \" Part-Of-Speech Tagging for Social Media Texts \". In: Language Processing and Knowledge in the Web. Ed. by I. Gurevych, C. Biemann, and T. Zesch. Vol. 8105 . Lecture Notes in Computer Science. Springer Berlin Heidelberg , 2013 , pp. 139 -- 150 . M. Neunerdt et al. \"Part-Of-Speech Tagging for Social Media Texts\". In: Language Processing and Knowledge in the Web. Ed. by I. Gurevych, C. Biemann, and T. Zesch. Vol. 8105. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2013, pp. 139--150."},{"key":"e_1_3_2_1_23_1","volume-title":"Proceedings of the Eight International Conference on Language Resources and Evaluation.","author":"Petrov S.","year":"2012","unstructured":"S. Petrov , D. Das , and R. McDonald . \" A Universal Part-of-Speech Tagset \". In: Proceedings of the Eight International Conference on Language Resources and Evaluation. 2012 . S. Petrov, D. Das, and R. McDonald. \"A Universal Part-of-Speech Tagset\". In: Proceedings of the Eight International Conference on Language Resources and Evaluation. 2012."},{"key":"e_1_3_2_1_24_1","volume-title":"Natural Language Annotation for Machine Learning","author":"Pustejovsky J.","year":"2012","unstructured":"J. Pustejovsky and A. Stubbs . Natural Language Annotation for Machine Learning . O'Reilly Media , 2012 . ISBN: 978-1-4493-0666-3. J. Pustejovsky and A. Stubbs. Natural Language Annotation for Machine Learning. O'Reilly Media, 2012. ISBN: 978-1-4493-0666-3."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12130-999-1026-0"},{"key":"e_1_3_2_1_26_1","first-page":"894","volume-title":"Packt Publishing Ltd","author":"Reese R. M.","year":"2015","unstructured":"R. M. Reese . Natural Language Processing with Java . Packt Publishing Ltd , 2015 . ISBN: 978-1-78439- 894 - 891 . R. M. Reese. Natural Language Processing with Java. Packt Publishing Ltd, 2015. ISBN: 978-1-78439-894-1."},{"key":"e_1_3_2_1_27_1","first-page":"356","volume-title":"Second Joint Conference on Lexical and Computational Semantics (*SEM)","volume":"2013","author":"Rockt\u00e4schel T.","unstructured":"T. Rockt\u00e4schel : The impact of domain-specific features on the performance of identifying and classifying mentions of drugs \". In: Second Joint Conference on Lexical and Computational Semantics (*SEM) , Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013). Atlanta, Georgia, USA: Association for Computational Linguistics, 2013 , pp. 356 -- 363 . T. Rockt\u00e4schel et al. \"WBI-NER: The impact of domain-specific features on the performance of identifying and classifying mentions of drugs\". In: Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013). Atlanta, Georgia, USA: Association for Computational Linguistics, 2013, pp. 356--363."},{"key":"e_1_3_2_1_28_1","first-page":"1","volume-title":"5th International Conference on Information and Communication Systems. Irbid, Jordan: IEEE, 2014","author":"Schreiber M.","year":"1958","unstructured":"M. Schreiber , K. Barkschat , and B. Kraft . \" Using Continuous Integration to Organize and Monitor the Annotation Process of Domain Specific Corpora \". In: 5th International Conference on Information and Communication Systems. Irbid, Jordan: IEEE, 2014 , pp. 1 -- 6 . DOI: 10.1109\/IACS.2014. 684 1958 . 10.1109\/IACS.2014 M. Schreiber, K. Barkschat, and B. Kraft. \"Using Continuous Integration to Organize and Monitor the Annotation Process of Domain Specific Corpora\". In: 5th International Conference on Information and Communication Systems. Irbid, Jordan: IEEE, 2014, pp. 1--6. DOI: 10.1109\/IACS.2014. 6841958."},{"key":"e_1_3_2_1_29_1","first-page":"1","volume-title":"3th Symposium of Applied Graph Technology","author":"Schreiber M.","unstructured":"M. Schreiber , B. Kraft , and A. Z\u00fcndorf . \" Testing Vast Amounts of Natural Language Processing Pipelines in Continuous Integration Environments \". In: 3th Symposium of Applied Graph Technology . Dortmund, Germany , pp. 1 -- 6 . Forthcoming. M. Schreiber, B. Kraft, and A. Z\u00fcndorf. \"Testing Vast Amounts of Natural Language Processing Pipelines in Continuous Integration Environments\". In: 3th Symposium of Applied Graph Technology. Dortmund, Germany, pp. 1--6. Forthcoming."},{"key":"e_1_3_2_1_30_1","first-page":"131","volume-title":"Quick Pad Tagger: An Efficient Graphical User Interface for Building Annotated Corpora with Multiple Annotation Layers","author":"Schreiber M.","year":"2015","unstructured":"M. Schreiber \" Quick Pad Tagger: An Efficient Graphical User Interface for Building Annotated Corpora with Multiple Annotation Layers \". In : Computer Science & Information Technology 4 ( 2015 ). Presented at NLP 2015 in Sydney, Australia , pp. 131 -- 143 . DOI: 10.5121\/csit.2015.50413. 10.5121\/csit.2015.50413 M. Schreiber et al. \"Quick Pad Tagger: An Efficient Graphical User Interface for Building Annotated Corpora with Multiple Annotation Layers\". In: Computer Science & Information Technology 4 (2015). Presented at NLP 2015 in Sydney, Australia, pp. 131--143. DOI: 10.5121\/csit.2015.50413."},{"key":"e_1_3_2_1_31_1","first-page":"619","volume-title":"Proceedings of the International Conference Recent Advances in Natural Language Processing. Hissar, Bulgaria: INCOMA Ltd. Shoumen, BULGARIA","author":"Silva A. P.","year":"2013","unstructured":"A. P. Silva , A. Silva , and I. Rodrigues . \" A New Approach to the POS Tagging Problem Using Evolutionary Computation \". In: Proceedings of the International Conference Recent Advances in Natural Language Processing. Hissar, Bulgaria: INCOMA Ltd. Shoumen, BULGARIA , 2013 , pp. 619 -- 625 . A. P. Silva, A. Silva, and I. Rodrigues. \"A New Approach to the POS Tagging Problem Using Evolutionary Computation\". In: Proceedings of the International Conference Recent Advances in Natural Language Processing. Hissar, Bulgaria: INCOMA Ltd. Shoumen, BULGARIA, 2013, pp. 619--625."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063935"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1177\/0165551509360123"}],"event":{"name":"ICSE '16: 38th International Conference on Software Engineering","sponsor":["ACM Association for Computing Machinery","SIGSOFT ACM Special Interest Group on Software Engineering","IEEE-CS\\DATC IEEE Computer Society","TCSE IEEE Computer Society's Tech. Council on Software Engin."],"location":"Austin Texas","acronym":"ICSE '16"},"container-title":["Proceedings of the 3rd International Workshop on Software Engineering Research and Industrial Practice"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2897022.2897029","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2897022.2897029","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:38:47Z","timestamp":1750221527000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2897022.2897029"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,5,14]]},"references-count":33,"alternative-id":["10.1145\/2897022.2897029","10.1145\/2897022"],"URL":"https:\/\/doi.org\/10.1145\/2897022.2897029","relation":{},"subject":[],"published":{"date-parts":[[2016,5,14]]},"assertion":[{"value":"2016-05-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}