{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T02:46:34Z","timestamp":1780368394229,"version":"3.54.1"},"reference-count":84,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2024,11,21]],"date-time":"2024-11-21T00:00:00Z","timestamp":1732147200000},"content-version":"vor","delay-in-days":325,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,11,18]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Weakly supervised learning aims to reduce the cost of labeling data by using expert-designed labeling rules. However, existing methods require experts to design effective rules in a single shot, which is difficult in the absence of proper guidance and tooling. Therefore, it is still an open question whether experts should spend their limited time writing rules or instead providing instance labels via active learning. In this paper, we investigate how to exploit an expert\u2019s limited time to create effective supervision. First, to develop practical guidelines for rule creation, we conduct an exploratory analysis of diverse collections of existing expert-designed rules and find that rule precision is more important than coverage across datasets. Second, we compare rule creation to individual instance labeling via active learning and demonstrate the importance of both across 6 datasets. Third, we propose an interactive learning framework, INTERVAL, that achieves efficiency by automatically extracting candidate rules based on rich patterns (e.g., by prompting a language model), and effectiveness by soliciting expert feedback on both candidate rules and individual instances. Across 6 datasets, INTERVAL outperforms state-of-the-art weakly supervised approaches by 7% in F1. Furthermore, it requires as few as 10 queries for expert feedback to reach F1 values that existing active learning methods cannot match even with 100 queries.<\/jats:p>","DOI":"10.1162\/tacl_a_00707","type":"journal-article","created":{"date-parts":[[2024,11,21]],"date-time":"2024-11-21T19:15:54Z","timestamp":1732216554000},"page":"1441-1459","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":3,"title":["Interactive Machine Teaching by Labeling Rules and\n                    Instances"],"prefix":"10.1162","volume":"12","author":[{"given":"Giannis","family":"Karamanolakis","sequence":"first","affiliation":[{"name":"Amazon AGI, New York, NY 10001, USA. karamai@amazon.com"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Hsu","sequence":"additional","affiliation":[{"name":"Columbia University, New York, NY 10027, USA. djhsu@cs.columbia.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luis","family":"Gravano","sequence":"additional","affiliation":[{"name":"Columbia University, New York, NY 10027, USA. gravano@cs.columbia.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2024,11,18]]},"reference":[{"key":"2025040814554262400_bib1","first-page":"487","article-title":"Fast algorithms for mining association\n                        rules in large databases","volume-title":"Proceedings of the 20th\n                        International Conference on Very Large Data Bases","author":"Agrawal","year":"1994"},{"key":"2025040814554262400_bib2","doi-asserted-by":"publisher","DOI":"10.1109\/ICMLA.2015.37","article-title":"Tubespam: Comment spam filtering on\n                        youtube","volume-title":"2015 IEEE 14th International Conference\n                        on Machine Learning and Applications (ICMLA)","author":"Alberto","year":"2015"},{"key":"2025040814554262400_bib3","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1145\/2034691.2034742","article-title":"Contributions to the study of SMS spam\n                        filtering: New collection and results","volume-title":"Proceedings of the 11th ACM Symposium on Document\n                        Engineering","author":"Almeida","year":"2011"},{"key":"2025040814554262400_bib4","article-title":"Deep batch active learning by diverse,\n                        uncertain gradient lower bounds","volume-title":"International\n                        Conference on Learning Representations","author":"Ash","year":"2019"},{"key":"2025040814554262400_bib5","doi-asserted-by":"publisher","first-page":"876","DOI":"10.18653\/v1\/D16-1084","article-title":"Stance detection with bidirectional\n                        conditional encoding","volume-title":"Proceedings of the 2016\n                        Conference on Empirical Methods in Natural Language Processing","author":"Augenstein","year":"2016"},{"key":"2025040814554262400_bib6","article-title":"Learning from rules generalizing labeled\n                        exemplars","volume-title":"International Conference on Learning\n                        Representations","author":"Awasthi","year":"2020"},{"key":"2025040814554262400_bib7","doi-asserted-by":"publisher","first-page":"93","DOI":"10.18653\/v1\/2022.acl-demo.9","article-title":"Promptsource: An integrated development environment and\n                        repository for natural language prompts","volume-title":"Proceedings of the 60th Annual Meeting of the Association for\n                        Computational Linguistics: System Demonstrations","author":"Bach","year":"2022"},{"key":"2025040814554262400_bib8","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1145\/3299869.3314036","article-title":"Snorkel drybell: A case study in deploying\n                        weak supervision at industrial scale","volume-title":"Proceedings\n                        of the 2019 International Conference on Management of Data","author":"Bach","year":"2019"},{"key":"2025040814554262400_bib9","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1061","article-title":"Data programming for learning discourse\n                        structure","volume-title":"Association for Computational\n                        Linguistics (ACL)","author":"Badene","year":"2019"},{"key":"2025040814554262400_bib10","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.ijcnlp-main.45","article-title":"A multitask, multilingual, multimodal evaluation of ChatGPT\n                        on reasoning, hallucination, and interactivity","volume-title":"Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the\n                        Association for Computational Linguistics","author":"Bang","year":"2022"},{"key":"2025040814554262400_bib11","article-title":"Mixmatch: A holistic approach to\n                        semi-supervised learning","volume":"32","author":"Berthelot","year":"2019","journal-title":"Advances in Neural\n                        Information Processing Systems"},{"key":"2025040814554262400_bib12","first-page":"199","article-title":"Agnostic active learning without\n                        constraints","volume-title":"Advances in Neural Information\n                        Processing Systems","author":"Beygelzimer","year":"2010"},{"key":"2025040814554262400_bib13","article-title":"Interactive weak supervision: Learning\n                        useful heuristics for data labeling","volume-title":"International Conference on Learning\n                    Representations","author":"Boecking","year":"2020"},{"key":"2025040814554262400_bib14","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.189","article-title":"Active imitation learning with noisy\n                        guidance","volume-title":"Proceedings of the 58th Annual Meeting\n                        of the Association for Computational Linguistics","author":"Brantley","year":"2020"},{"key":"2025040814554262400_bib15","first-page":"177","article-title":"Bottom-up relational learning of pattern\n                        matching rules for information extraction","author":"Califf","year":"2003","journal-title":"Journal\n                        of Machine Learning Research"},{"key":"2025040814554262400_bib16","article-title":"Scaling instruction-finetuned language\n                    models","author":"Chung","year":"2022","journal-title":"arXiv preprint\n                    arXiv:2210.11416"},{"key":"2025040814554262400_bib17","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1217","article-title":"Semi-supervised sequence modeling with cross-view\n                        training","volume-title":"Proceedings of the 2018 Conference on\n                        Empirical Methods in Natural Language Processing","author":"Clark","year":"2018"},{"key":"2025040814554262400_bib18","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1613\/jair.295","article-title":"Active learning with statistical\n                        models","volume":"4","author":"Cohn","year":"1996","journal-title":"Journal of Artificial Intelligence\n                        Research"},{"key":"2025040814554262400_bib19","first-page":"3955","article-title":"Learning from discriminative feature\n                        feedback","volume-title":"Advances in Neural Information\n                        Processing Systems","author":"Dasgupta","year":"2018"},{"key":"2025040814554262400_bib20","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1145\/1390156.1390183","article-title":"Hierarchical sampling for active learning","volume-title":"Proceedings of the 25th International Conference on Machine\n                        Learning","author":"Dasgupta","year":"2008"},{"key":"2025040814554262400_bib21","first-page":"353","article-title":"A general agnostic active learning\n                        algorithm","volume":"20","author":"Dasgupta","year":"2007","journal-title":"Advances in Neural information Processing\n                        Systems"},{"issue":"1","key":"2025040814554262400_bib22","doi-asserted-by":"publisher","first-page":"20","DOI":"10.2307\/2346806","article-title":"Maximum likelihood estimation of observer\n                        error-rates using the em algorithm","volume":"28","author":"Dawid","year":"1979","journal-title":"Journal of the\n                        Royal Statistical Society: Series C (Applied Statistics)"},{"key":"2025040814554262400_bib23","article-title":"Bert: Pre-training of deep bidirectional\n                        transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of\n                        the Association for Computational Linguistics: Human Language\n                        Technologies","author":"Devlin","year":"2019"},{"key":"2025040814554262400_bib24","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.638","article-title":"Active learning for bert: An empirical\n                        study","volume-title":"Proceedings of the 2020 Conference on\n                        Empirical Methods in Natural Language Processing (EMNLP)","author":"Dor","year":"2020"},{"key":"2025040814554262400_bib25","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1145\/1390334.1390436","article-title":"Learning from labeled features using\n                        generalized expectation criteria","volume-title":"Proceedings of\n                        the 31st Annual International ACM SIGIR Conference on Research and\n                        Development in Information Retrieval","author":"Druck","year":"2008"},{"key":"2025040814554262400_bib26","first-page":"3280","article-title":"Fast and three-rious: Speeding up weak\n                        supervision with triplet methods","volume-title":"International\n                        Conference on Machine Learning","author":"Daniel","year":"2020"},{"key":"2025040814554262400_bib27","doi-asserted-by":"publisher","first-page":"3816","DOI":"10.18653\/v1\/2021.acl-long.295","article-title":"Making pre-trained language models better few-shot\n                        learners","volume-title":"Proceedings of the 59th Annual Meeting\n                        of the Association for Computational Linguistics and the 11th International\n                        Joint Conference on Natural Language Processing (Volume 1: Long\n                        Papers)","author":"Gao","year":"2021"},{"key":"2025040814554262400_bib28","doi-asserted-by":"publisher","first-page":"1107","DOI":"10.18653\/v1\/2022.emnlp-main.73","article-title":"Zero-shot text classification with\n                        self-training","volume-title":"Proceedings of the 2022 Conference\n                        on Empirical Methods in Natural Language Processing","author":"Gera","year":"2022"},{"key":"2025040814554262400_bib29","article-title":"DEBERTA: Decoding-enhanced bert with disentangled\n                        attention","volume-title":"International Conference on Learning\n                        Representations","author":"He","year":"2020"},{"key":"2025040814554262400_bib30","article-title":"Bayesian active learning for\n                        classification and preference learning","author":"Houlsby","year":"2011","journal-title":"arXiv\n                        preprint arXiv:1112.5745"},{"key":"2025040814554262400_bib31","doi-asserted-by":"publisher","DOI":"10.14778\/3565838.3565859","article-title":"Nemo: Guiding and contextualizing weak\n                        supervision for interactive data programming","author":"Hsieh","year":"2022","journal-title":"arXiv\n                        preprint arXiv:2203.01382"},{"key":"2025040814554262400_bib32","first-page":"204","article-title":"Incorporating lexical priors into topic\n                        models","volume-title":"Proceedings of the 13th Conference of the\n                        European Chapter of the Association for Computational Linguistics","author":"Jagarlamudi","year":"2012"},{"key":"2025040814554262400_bib33","doi-asserted-by":"publisher","DOI":"10.1145\/1341531.1341560","article-title":"Opinion spam and analysis","volume-title":"Proceedings of the 2008 International Conference on Web Search and\n                        Data Mining","author":"Jindal","year":"2008"},{"key":"2025040814554262400_bib34","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1468","article-title":"Leveraging just a few keywords for\n                        fine-grained aspect detection through weakly supervised\n                        co-training","volume-title":"Proceedings of the 2019 Conference\n                        on Empirical Methods in Natural Language Processing","author":"Karamanolakis","year":"2019"},{"key":"2025040814554262400_bib35","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.66","article-title":"Self-training with weak\n                        supervision","volume-title":"NAACL","author":"Karamanolakis","year":"2021"},{"issue":"1","key":"2025040814554262400_bib36","doi-asserted-by":"publisher","first-page":"211","DOI":"10.3390\/ai3010013","article-title":"Rule-enhanced active learning for\n                        semi-automated weak supervision","volume":"3","author":"Kartchner","year":"2022","journal-title":"AI"},{"key":"2025040814554262400_bib37","article-title":"Batchbald: Efficient and diverse batch acquisition for deep\n                        bayesian active learning","volume":"32","author":"Kirsch","year":"2019","journal-title":"Advances in Neural\n                        information Processing Systems"},{"key":"2025040814554262400_bib38","article-title":"Pseudo-label: The simple and efficient semi-supervised\n                        learning method for deep neural networks","volume-title":"Workshop on Challenges in Representation Learning, ICML","author":"Lee","year":"2013"},{"key":"2025040814554262400_bib39","article-title":"GrASP: A library for extracting and exploring\n                        human-interpretable textual patterns","volume-title":"Proceedings\n                        of the Thirteenth Language Resources and Evaluation Conference","author":"Lertvittayakumjorn","year":"2022"},{"key":"2025040814554262400_bib40","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/978-1-4471-2099-5_1","article-title":"A sequential algorithm for training text\n                        classifiers","volume-title":"SIGIR\u201994","author":"Lewis","year":"1994"},{"key":"2025040814554262400_bib41","doi-asserted-by":"publisher","DOI":"10.3115\/1072228.1072378","article-title":"Learning question classifiers","volume-title":"COLING 2002: The 19th International Conference on Computational\n                        Linguistics","author":"Li","year":"2002"},{"key":"2025040814554262400_bib42","doi-asserted-by":"publisher","DOI":"10.1145\/3560815","article-title":"Pre-train, prompt, and predict: A\n                        systematic survey of prompting methods in natural language\n                        processing","author":"Liu","year":"2023","journal-title":"ACM Computing Surveys"},{"key":"2025040814554262400_bib43","article-title":"Roberta: A robustly optimized BERT\n                        pretraining approach","author":"Liu","year":"2019","journal-title":"arXiv preprint\n                        arXiv:1907.11692"},{"key":"2025040814554262400_bib44","article-title":"Learning word vectors for sentiment\n                        analysis","volume-title":"Proceedings of the 49th Annual Meeting\n                        of the Association for Computational Linguistics: Human Language\n                        Technologies","author":"Maas","year":"2011"},{"key":"2025040814554262400_bib45","doi-asserted-by":"publisher","first-page":"650","DOI":"10.18653\/v1\/2021.emnlp-main.51","article-title":"Active learning by acquiring contrastive\n                        examples","volume-title":"Proceedings of the 2021 Conference on\n                        Empirical Methods in Natural Language Processing","author":"Margatina","year":"2021"},{"key":"2025040814554262400_bib46","doi-asserted-by":"publisher","first-page":"1275","DOI":"10.1145\/1557019.1557156","article-title":"Sentiment analysis of blogs by combining\n                        lexical knowledge with text classification","volume-title":"Proceedings of the 15th ACM SIGKDD International Conference on\n                        Knowledge Discovery and Data Mining","author":"Melville","year":"2009"},{"key":"2025040814554262400_bib47","article-title":"Generative representational instruction\n                        tuning","author":"Muennighoff","year":"2024","journal-title":"arXiv preprint\n                    arXiv:2402.09906"},{"key":"2025040814554262400_bib48","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1145\/354756.354805","article-title":"Analyzing the effectiveness and applicability\n                        of co-training","volume-title":"Proceedings of the 9th\n                        International Conference on Information and Knowledge Management","author":"Nigam","year":"2000"},{"key":"2025040814554262400_bib49","article-title":"Training language models to follow instructions with human\n                        feedback","volume-title":"Advances in Neural Information\n                        Processing Systems","author":"Ouyang","year":"2022"},{"key":"2025040814554262400_bib50","first-page":"11054","article-title":"True few-shot learning with language models","volume":"34","author":"Perez","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2025040814554262400_bib51","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202","article-title":"Deep contextualized word\n                        representations","volume-title":"Proceedings of the 2018\n                        Conference of the North American Chapter of the Association for\n                        Computational Linguistics: Human Language Technologies","author":"Peters","year":"2018"},{"key":"2025040814554262400_bib52","first-page":"1104","article-title":"Learning with feature feedback: From\n                        theory to practice","volume-title":"Artificial Intelligence and\n                        Statistics","author":"Poulis","year":"2017"},{"key":"2025040814554262400_bib53","doi-asserted-by":"publisher","first-page":"269","DOI":"10.14778\/3157794.3157797","article-title":"Snorkel: Rapid training data creation with\n                        weak supervision","volume-title":"Proceedings of the VLDB\n                        Endowment. International Conference on Very Large Data Bases","author":"Ratner","year":"2017"},{"key":"2025040814554262400_bib54","doi-asserted-by":"publisher","first-page":"4763","DOI":"10.1609\/aaai.v33i01.33014763","article-title":"Training complex models with multi-task\n                        weak supervision","volume-title":"Proceedings of the AAAI\n                        Conference on Artificial Intelligence","author":"Ratner","year":"2019"},{"key":"2025040814554262400_bib55","article-title":"Data programming: Creating large training\n                        sets, quickly","volume-title":"Advances in Neural Information\n                        Processing Systems","author":"Ratner","year":"2016"},{"key":"2025040814554262400_bib56","first-page":"441","article-title":"Toward optimal active learning through\n                        sampling estimation of error reduction","volume-title":"Proceedings of the 18th International Conference on Machine\n                        Learning","author":"Roy","year":"2001"},{"key":"2025040814554262400_bib57","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1096","article-title":"Strong baselines for neural semi-supervised\n                        learning under domain shift","volume-title":"Proceedings of the\n                        Annual Meeting of the Association for Computational Linguistics","author":"Ruder","year":"2018"},{"key":"2025040814554262400_bib58","article-title":"Toolformer: Language models can teach\n                        themselves to use tools","author":"Schick","year":"2023","journal-title":"arXiv preprint\n                        arXiv:2302.04761"},{"key":"2025040814554262400_bib59","doi-asserted-by":"publisher","first-page":"255","DOI":"10.18653\/v1\/2021.eacl-main.20","article-title":"Exploiting cloze-questions for few-shot\n                        text classification and natural language inference","volume-title":"Proceedings of the 16th Conference of the European Chapter of the\n                        Association for Computational Linguistics: Main Volume","author":"Schick","year":"2021"},{"key":"2025040814554262400_bib60","doi-asserted-by":"crossref","unstructured":"Matthias\n              Seeger\n            \n          .\n                        2006. A taxonomy for semi-supervised learning\n                        methods. Technical report,\n                        MIT Press. 10.7551\/mitpress\/6173.003.0005","DOI":"10.7551\/mitpress\/9780262033589.003.0002"},{"key":"2025040814554262400_bib61","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/P19-3023","article-title":"Heidl: Learning linguistic expressions\n                        with deep learning and human-in-the-loop","volume-title":"Proceedings of the 57th Annual Meeting of the Association for\n                        Computational Linguistics: System Demonstrations","author":"Sen","year":"2019"},{"key":"2025040814554262400_bib62","unstructured":"Burr\n              Settles\n            \n          .\n                        2009. Active learning literature\n                        survey. Technical report,\n                        University of Wisconsin-Madison Department of Computer\n                        Sciences."},{"key":"2025040814554262400_bib63","first-page":"1467","article-title":"Closing the loop: Fast, interactive\n                        semi-supervised annotation with queries on features and\n                        instances","volume-title":"Proceedings of the 2011 Conference on\n                        Empirical Methods in Natural Language Processing","author":"Settles","year":"2011"},{"key":"2025040814554262400_bib64","doi-asserted-by":"publisher","first-page":"252","DOI":"10.18653\/v1\/W17-2630","article-title":"Deep active learning for named entity\n                        recognition","volume-title":"Proceedings of the 2nd Workshop on\n                        Representation Learning for NLP","author":"Shen","year":"2017"},{"key":"2025040814554262400_bib65","article-title":"Learning syntactic patterns for automatic hypernym\n                        discovery","volume":"17","author":"Snow","year":"2004","journal-title":"Advances in Neural Information Processing\n                        Systems"},{"key":"2025040814554262400_bib66","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0014140","article-title":"Mining sequential patterns:\n                        Generalizations and performance improvements","volume-title":"International Conference on Extending Database Technology","author":"Srikant","year":"1996"},{"key":"2025040814554262400_bib67","article-title":"One embedder, any task:\n                        Instruction-finetuned text embeddings","volume-title":"Findings\n                        of the Association for Computational Linguistics: ACL 2023","author":"Hongjin","year":"2023"},{"key":"2025040814554262400_bib68","doi-asserted-by":"publisher","first-page":"4980","DOI":"10.18653\/v1\/2021.emnlp-main.407","article-title":"Improving and simplifying pattern\n                        exploiting training","volume-title":"Proceedings of the 2021\n                        Conference on Empirical Methods in Natural Language Processing","author":"Tam","year":"2021"},{"key":"2025040814554262400_bib69","article-title":"Llama: Open and efficient foundation\n                        language models","author":"Touvron","year":"2023"},{"key":"2025040814554262400_bib70","article-title":"Llama 2: Open foundation and fine-tuned\n                        chat models","author":"Touvron","year":"2023","journal-title":"arXiv preprint\n                        arXiv:2307.09288"},{"key":"2025040814554262400_bib71","doi-asserted-by":"publisher","first-page":"223","DOI":"10.14778\/3291264.3291268","article-title":"Snuba: Automating weak supervision to\n                        label training data","volume-title":"Proceedings of the VLDB\n                        Endowment. International Conference on Very Large Data Bases","author":"Varma","year":"2018"},{"key":"2025040814554262400_bib72","article-title":"Improving text embeddings with large language\n                        models","author":"Wang","year":"2023","journal-title":"arXiv preprint\n                    arXiv:2401.00368"},{"key":"2025040814554262400_bib73","doi-asserted-by":"publisher","DOI":"10.3115\/974147.974186","article-title":"Unsupervised discovery of scenario-level\n                        patterns for information extraction","volume-title":"Sixth\n                        Applied Natural Language Processing Conference","author":"Yangarber","year":"2000"},{"key":"2025040814554262400_bib74","article-title":"A comprehensive capability analysis of GPT-3\n                        and GPT-3.5 series models","author":"Ye","year":"2023"},{"key":"2025040814554262400_bib75","doi-asserted-by":"publisher","first-page":"3914","DOI":"10.18653\/v1\/D19-1404","article-title":"Benchmarking zero-shot text classification: Datasets,\n                        evaluation and entailment approach","volume-title":"Proceedings\n                        of the 2019 Conference on Empirical Methods in Natural Language Processing\n                        and the 9th International Joint Conference on Natural Language Processing\n                        (EMNLP-IJCNLP)","author":"Yin","year":"2019"},{"key":"2025040814554262400_bib76","doi-asserted-by":"publisher","first-page":"7935","DOI":"10.18653\/v1\/2020.emnlp-main.637","article-title":"Cold-start active learning through\n                        self-supervised language modeling","volume-title":"Proceedings of\n                        the 2020 Conference on Empirical Methods in Natural Language Processing\n                        (EMNLP)","author":"Yuan","year":"2020"},{"key":"2025040814554262400_bib77","first-page":"703","article-title":"Active learning from weak and strong\n                        labelers","volume-title":"Advances in Neural Information\n                        Processing Systems","author":"Zhang","year":"2015"},{"key":"2025040814554262400_bib78","article-title":"A survey on programmatic weak\n                        supervision","author":"Zhang","year":"2022","journal-title":"arXiv preprint\n                        arXiv:2202.05433"},{"key":"2025040814554262400_bib79","article-title":"Wrench: A comprehensive benchmark for weak\n                        supervision","volume-title":"35th Conference on Neural\n                        Information Processing Systems Datasets and Benchmarks Track (Round\n                        2)","author":"Zhang","year":"2021"},{"key":"2025040814554262400_bib80","doi-asserted-by":"publisher","first-page":"745","DOI":"10.18653\/v1\/2022.acl-long.55","article-title":"Prompt- based rule discovery and boosting\n                        for interactive weakly-supervised learning","volume-title":"Proceedings of the 60th Annual Meeting of the Association for\n                        Computational Linguistics (Volume 1: Long Papers)","author":"Zhang","year":"2022"},{"key":"2025040814554262400_bib81","first-page":"649","article-title":"Character-level convolutional networks for\n                        text classification","volume-title":"Advances in Neural\n                        Information Processing Systems","author":"Zhang","year":"2015"},{"key":"2025040814554262400_bib82","article-title":"A survey on multi-task learning","author":"Zhang","year":"2021","journal-title":"IEEE\n                        Transactions on Knowledge and Data Engineering"},{"key":"2025040814554262400_bib83","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.414","article-title":"A survey of active learning for natural\n                        language processing","volume-title":"Proceedings of the 2022\n                        Conference on Empirical Methods in Natural Language Processing","author":"Zhang","year":"2022"},{"key":"2025040814554262400_bib84","first-page":"12697","article-title":"Calibrate before use: Improving few-shot\n                        performance of language models","volume-title":"International\n                        Conference on Machine Learning","author":"Zhao","year":"2021"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00707\/2480376\/tacl_a_00707.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00707\/2480376\/tacl_a_00707.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,8]],"date-time":"2025-04-08T18:57:23Z","timestamp":1744138643000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00707\/125276\/Interactive-Machine-Teaching-by-Labeling-Rules-and"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":84,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00707","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}