{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T20:45:07Z","timestamp":1776113107292,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":23,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,4,19]],"date-time":"2021-04-19T00:00:00Z","timestamp":1618790400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,4,19]]},"DOI":"10.1145\/3442442.3458602","type":"proceedings-article","created":{"date-parts":[[2021,6,3]],"date-time":"2021-06-03T15:56:42Z","timestamp":1622735802000},"page":"673-677","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["TagRuler: Interactive Tool for Span-Level Data Programming by Demonstration"],"prefix":"10.1145","author":[{"given":"Dongjin","family":"Choi","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sara","family":"Evensen","sequence":"additional","affiliation":[{"name":"Megagon Labs, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"\u00c7a\u011fatay","family":"Demiralp","sequence":"additional","affiliation":[{"name":"Sigma Computing, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Estevam","family":"Hruschka","sequence":"additional","affiliation":[{"name":"Megagon Labs, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,6,3]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"2014. Yelp Dataset. http:\/\/www.yelp.com\/dataset_challenge.  2014. Yelp Dataset. http:\/\/www.yelp.com\/dataset_challenge."},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 157\u2013162","author":"Akbik Alan","year":"2013","unstructured":"Alan Akbik , Oresti Konomi , and Michail Melnikov . 2013 . Propminer: A workflow for interactive information extraction and exploration using dependency trees . In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 157\u2013162 . Alan Akbik, Oresti Konomi, and Michail Melnikov. 2013. Propminer: A workflow for interactive information extraction and exploration using dependency trees. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 157\u2013162."},{"key":"e_1_3_2_1_3_1","unstructured":"Benedikt Boecking Willie Neiswanger Eric Xing and Artur Dubrawski. 2020. Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling. arXiv preprint arXiv:2012.06046(2020).  Benedikt Boecking Willie Neiswanger Eric Xing and Artur Dubrawski. 2020. Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling. arXiv preprint arXiv:2012.06046(2020)."},{"key":"e_1_3_2_1_4_1","volume-title":"Proceedings of the 2013 conference on empirical methods in natural language processing. 827\u2013832","author":"Chiticariu Laura","year":"2013","unstructured":"Laura Chiticariu , Yunyao Li , and Frederick Reiss . 2013 . Rule-based information extraction is dead! long live rule-based information extraction systems! . In Proceedings of the 2013 conference on empirical methods in natural language processing. 827\u2013832 . Laura Chiticariu, Yunyao Li, and Frederick Reiss. 2013. Rule-based information extraction is dead! long live rule-based information extraction systems!. In Proceedings of the 2013 conference on empirical methods in natural language processing. 827\u2013832."},{"key":"e_1_3_2_1_5_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2018 . Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018). Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018)."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.181"},{"key":"e_1_3_2_1_7_1","volume-title":"International Conference on Machine Learning. PMLR, 3280\u20133291","author":"Fu Daniel","year":"2020","unstructured":"Daniel Fu , Mayee Chen , Frederic Sala , Sarah Hooper , Kayvon Fatahalian , and Christopher R\u00e9 . 2020 . Fast and three-rious: Speeding up weak supervision with triplet methods . In International Conference on Machine Learning. PMLR, 3280\u20133291 . Daniel Fu, Mayee Chen, Frederic Sala, Sarah Hooper, Kayvon Fatahalian, and Christopher R\u00e9. 2020. Fast and three-rious: Speeding up weak supervision with triplet methods. In International Conference on Machine Learning. PMLR, 3280\u20133291."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-1611"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1175"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002541"},{"key":"e_1_3_2_1_11_1","unstructured":"Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings convolutional neural networks and incremental parsing. (2017). To appear.  Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings convolutional neural networks and incremental parsing. (2017). To appear."},{"key":"e_1_3_2_1_12_1","unstructured":"David\u00a0R Karger Sewoong Oh and Devavrat Shah. 2011. Iterative learning for reliable crowdsourcing systems. NeurIPS.  David\u00a0R Karger Sewoong Oh and Devavrat Shah. 2011. Iterative learning for reliable crowdsourcing systems. NeurIPS."},{"key":"e_1_3_2_1_13_1","volume-title":"Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 577\u2013589","author":"Jia-Jun Li Toby","year":"2019","unstructured":"Toby Jia-Jun Li , Marissa Radensky , Justin Jia , Kirielle Singarajah , Tom\u00a0 M Mitchell , and Brad\u00a0 A Myers . 2019 . Pumice: A multi-modal agent that learns concepts and conditionals from natural language and demonstrations . In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 577\u2013589 . Toby Jia-Jun Li, Marissa Radensky, Justin Jia, Kirielle Singarajah, Tom\u00a0M Mitchell, and Brad\u00a0A Myers. 2019. Pumice: A multi-modal agent that learns concepts and conditionals from natural language and demonstrations. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology. 577\u2013589."},{"key":"e_1_3_2_1_14_1","unstructured":"Xiaoya Li Xiaofei Sun Yuxian Meng Junjun Liang Fei Wu and Jiwei Li. 2019. Dice loss for data-imbalanced NLP tasks. arXiv preprint arXiv:1911.02855(2019).  Xiaoya Li Xiaofei Sun Yuxian Meng Junjun Liang Fei Wu and Jiwei Li. 2019. Dice loss for data-imbalanced NLP tasks. arXiv preprint arXiv:1911.02855(2019)."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.3115\/1690219.1690287"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Matthew\u00a0E Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365(2018).  Matthew\u00a0E Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365(2018).","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.14778\/3157794.3157797"},{"key":"e_1_3_2_1_18_1","volume-title":"Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084(2019).","author":"Reimers Nils","year":"2019","unstructured":"Nils Reimers and Iryna Gurevych . 2019 . Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084(2019). Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084(2019)."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i04.6009"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"crossref","unstructured":"Sahaana Suri Raghuveer Chanda Neslihan Bulut Pradyumna Narayana Yemao Zeng Peter Bailis Sugato Basu Girija Narlikar Christopher R\u00e9 and Abishek Sethi. 2020. Leveraging organizational resources to adapt models to new data modalities. arXiv preprint arXiv:2008.09983(2020).  Sahaana Suri Raghuveer Chanda Neslihan Bulut Pradyumna Narayana Yemao Zeng Peter Bailis Sugato Basu Girija Narlikar Christopher R\u00e9 and Abishek Sethi. 2020. Leveraging organizational resources to adapt models to new data modalities. arXiv preprint arXiv:2008.09983(2020).","DOI":"10.14778\/3415478.3415559"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.14778\/3291264.3291268"},{"key":"e_1_3_2_1_22_1","volume-title":"Proc. Int. Joint Conf. on Artificial Intelligence (Washington DC","author":"Waldinger RJ","year":"1969","unstructured":"RJ Waldinger and RC T\u00a0PROW LEE. [n.d.]. A step toward automatic program writing . In Proc. Int. Joint Conf. on Artificial Intelligence (Washington DC , May 1969 ). 241\u2013252. RJ Waldinger and RCT\u00a0PROW LEE. [n.d.]. A step toward automatic program writing. In Proc. Int. Joint Conf. on Artificial Intelligence (Washington DC, May 1969). 241\u2013252."},{"key":"e_1_3_2_1_23_1","volume-title":"Proceedings of the fifth BioCreative challenge evaluation workshop, Vol.\u00a014","author":"Wei Chih-Hsuan","year":"2015","unstructured":"Chih-Hsuan Wei , Yifan Peng , Robert Leaman , Allan\u00a0Peter Davis , Carolyn\u00a0 J Mattingly , Jiao Li , Thomas\u00a0 C Wiegers , and Zhiyong Lu . 2015 . Overview of the BioCreative V chemical disease relation (CDR) task . In Proceedings of the fifth BioCreative challenge evaluation workshop, Vol.\u00a014 . Chih-Hsuan Wei, Yifan Peng, Robert Leaman, Allan\u00a0Peter Davis, Carolyn\u00a0J Mattingly, Jiao Li, Thomas\u00a0C Wiegers, and Zhiyong Lu. 2015. Overview of the BioCreative V chemical disease relation (CDR) task. In Proceedings of the fifth BioCreative challenge evaluation workshop, Vol.\u00a014."}],"event":{"name":"WWW '21: The Web Conference 2021","location":"Ljubljana Slovenia","acronym":"WWW '21","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web"]},"container-title":["Companion Proceedings of the Web Conference 2021"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3442442.3458602","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3442442.3458602","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:24:36Z","timestamp":1750195476000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3442442.3458602"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,19]]},"references-count":23,"alternative-id":["10.1145\/3442442.3458602","10.1145\/3442442"],"URL":"https:\/\/doi.org\/10.1145\/3442442.3458602","relation":{},"subject":[],"published":{"date-parts":[[2021,4,19]]},"assertion":[{"value":"2021-06-03","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}