{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T05:33:43Z","timestamp":1777613623619,"version":"3.51.4"},"reference-count":47,"publisher":"MIT Press - Journals","license":[{"start":{"date-parts":[[2022,5,17]],"date-time":"2022-05-17T00:00:00Z","timestamp":1652745600000},"content-version":"vor","delay-in-days":136,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,5,16]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The task of ultra-fine entity typing (UFET) seeks to predict diverse and free-form words or phrases that describe the appropriate types of entities mentioned in sentences. A key challenge for this task lies in the large number of types and the scarcity of annotated data per type. Existing systems formulate the task as a multi-way classification problem and train directly or distantly supervised classifiers. This causes two issues: (i) the classifiers do not capture the type semantics because types are often converted into indices; (ii) systems developed in this way are limited to predicting within a pre-defined type set, and often fall short of generalizing to types that are rarely seen or unseen in training.<\/jats:p>\n               <jats:p>This work presents LITE\ud83c\udf7b, a new approach that formulates entity typing as a natural language inference (NLI) problem, making use of (i) the indirect supervision from NLI to infer type information meaningfully represented as textual hypotheses and alleviate the data scarcity issue, as well as (ii) a learning-to-rank objective to avoid the pre-defining of a type set. Experiments show that, with limited training data, LITE obtains state-of-the-art performance on the UFET task. In addition, LITE demonstrates its strong generalizability by not only yielding best results on other fine-grained entity typing benchmarks, more importantly, a pre-trained LITE system works well on new data containing unseen types.1<\/jats:p>","DOI":"10.1162\/tacl_a_00479","type":"journal-article","created":{"date-parts":[[2022,5,17]],"date-time":"2022-05-17T15:04:59Z","timestamp":1652799899000},"page":"607-622","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":13,"title":["Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference"],"prefix":"10.1162","volume":"10","author":[{"given":"Bangzheng","family":"Li","sequence":"first","affiliation":[{"name":"University of Southern California, USA"},{"name":"University of Illinois at Urbana-Champaign, USA. vincentleebang@gmail.com"}]},{"given":"Wenpeng","family":"Yin","sequence":"additional","affiliation":[{"name":"Temple University, USA. wenpeng.yin@temple.edu"}]},{"given":"Muhao","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Southern California, USA. muhaoche@usc.edu"}]}],"member":"281","published-online":{"date-parts":[[2022,5,16]]},"reference":[{"key":"2022051714395773400_bib1","doi-asserted-by":"crossref","first-page":"632","DOI":"10.18653\/v1\/D15-1075","article-title":"A large annotated corpus for learning natural language inference","volume-title":"Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing","author":"Bowman","year":"2015"},{"key":"2022051714395773400_bib2","doi-asserted-by":"crossref","first-page":"4283","DOI":"10.18653\/v1\/2021.findings-acl.375","article-title":"Grounding \u2018grounding\u2019 in NLP","volume-title":"Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021","author":"Chandu","year":"2021"},{"key":"2022051714395773400_bib3","doi-asserted-by":"crossref","first-page":"3841","DOI":"10.18653\/v1\/2021.findings-emnlp.324","article-title":"Can NLI models verify QA systems\u2019 predictions?","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Chen","year":"2021"},{"key":"2022051714395773400_bib4","doi-asserted-by":"crossref","first-page":"8465","DOI":"10.18653\/v1\/2020.acl-main.749","article-title":"Hierarchical entity typing via multi-level learning to rank","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Chen","year":"2020"},{"key":"2022051714395773400_bib5","doi-asserted-by":"crossref","first-page":"8465","DOI":"10.18653\/v1\/2020.acl-main.749","article-title":"Hierarchical entity typing via multi-level learning to rank","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Chen","year":"2020"},{"key":"2022051714395773400_bib6","doi-asserted-by":"crossref","first-page":"87","DOI":"10.18653\/v1\/P18-1009","article-title":"Ultra-fine entity typing","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Choi","year":"2018"},{"key":"2022051714395773400_bib7","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1007\/11736790_9","article-title":"The pascal recognising textual entailment challenge","volume-title":"Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment","author":"Dagan","year":"2006"},{"key":"2022051714395773400_bib8","doi-asserted-by":"crossref","first-page":"6210","DOI":"10.18653\/v1\/D19-1643","article-title":"Improving fine-grained entity typing with entity linking","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Dai","year":"2019"},{"key":"2022051714395773400_bib9","first-page":"1790","article-title":"Ultra-fine entity typing with weak supervision from a masked language model","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Dai","year":"2021"},{"key":"2022051714395773400_bib10","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2022051714395773400_bib11","first-page":"9175","article-title":"Reducing network agnostophobia","volume-title":"Proceedings of the 32nd Annual Conference on Neural Information Processing Systems (NeurIPS)","author":"Dhamija","year":"2018"},{"key":"2022051714395773400_bib12","article-title":"Prompt-learning for fine-grained entity typing","author":"Ding","year":"2021","journal-title":"arXiv preprint arXiv:2108.10604"},{"key":"2022051714395773400_bib13","first-page":"3816","article-title":"Making pre-trained language models better few-shot learners","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Gao","year":"2021"},{"key":"2022051714395773400_bib14","article-title":"Context-dependent fine-grained entity type tagging","author":"Gillick","year":"2014","journal-title":"arXiv preprint arXiv:1412.1820"},{"issue":"8","key":"2022051714395773400_bib15","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"key":"2022051714395773400_bib16","first-page":"2251","article-title":"Diversified semantic attention model for fine-grained entity typing","volume":"9","author":"Yanfeng","year":"2020","journal-title":"IEEE Access"},{"key":"2022051714395773400_bib17","doi-asserted-by":"crossref","first-page":"4969","DOI":"10.18653\/v1\/D19-1502","article-title":"Fine-grained entity typing via hierarchical multi graph convolutional networks","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Jin","year":"2019"},{"key":"2022051714395773400_bib18","doi-asserted-by":"crossref","first-page":"1891","DOI":"10.3115\/v1\/D14-1203","article-title":"Type-aware distantly supervised relation extraction with linked arguments","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Koch","year":"2014"},{"key":"2022051714395773400_bib19","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1162\/tacl_a_00141","article-title":"Design challenges for entity linking","volume":"3","author":"Ling","year":"2015","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"2022051714395773400_bib20","article-title":"Fine-grained entity recognition","volume-title":"Twenty-Sixth AAAI Conference on Artificial Intelligence","author":"Ling","year":"2012"},{"key":"2022051714395773400_bib21","doi-asserted-by":"crossref","first-page":"4611","DOI":"10.18653\/v1\/2021.emnlp-main.378","article-title":"Fine-grained entity typing via label reasoning","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Liu","year":"2021"},{"key":"2022051714395773400_bib22","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","author":"Liu","year":"2019","journal-title":"arXiv preprint arXiv:1907.11692"},{"key":"2022051714395773400_bib23","article-title":"Decoupled weight decay regularization","volume-title":"International Conference on Learning Representations","author":"Loshchilov","year":"2018"},{"key":"2022051714395773400_bib24","first-page":"807","article-title":"Description-based zero-shot fine-grained entity typing","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Obeidat","year":"2019"},{"key":"2022051714395773400_bib25","first-page":"2051","article-title":"Modeling fine-grained entity types with box embeddings","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Onoe","year":"2021"},{"key":"2022051714395773400_bib26","first-page":"2407","article-title":"Learning to denoise distantly-labeled data for entity typing","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Onoe","year":"2019"},{"key":"2022051714395773400_bib27","doi-asserted-by":"crossref","first-page":"612","DOI":"10.18653\/v1\/2020.findings-emnlp.54","article-title":"Interpretable entity representations through large-scale typing","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Onoe","year":"2020"},{"key":"2022051714395773400_bib28","doi-asserted-by":"crossref","first-page":"6459","DOI":"10.18653\/v1\/2020.coling-main.568","article-title":"FASTMATCH: Accelerating the inference of BERT- based text matching","volume-title":"Proceedings of the 28th International Conference on Computational Linguistics","author":"Pang","year":"2020"},{"key":"2022051714395773400_bib29","doi-asserted-by":"crossref","first-page":"2249","DOI":"10.18653\/v1\/D16-1244","article-title":"A decomposable attention model for natural language inference","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Parikh","year":"2016"},{"key":"2022051714395773400_bib30","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3115\/v1\/D14-1162","article-title":"GloVe: Global vectors for word representation","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Pennington","year":"2014"},{"key":"2022051714395773400_bib31","first-page":"2227","article-title":"Deep contextualized word representations","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Peters","year":"2018"},{"key":"2022051714395773400_bib32","doi-asserted-by":"crossref","first-page":"67","DOI":"10.18653\/v1\/D18-1007","article-title":"Collecting diverse natural language inference problems for sentence representation evaluation","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Poliak","year":"2018"},{"issue":"140","key":"2022051714395773400_bib33","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"Journal of Machine Learning Research"},{"key":"2022051714395773400_bib34","doi-asserted-by":"crossref","first-page":"1825","DOI":"10.1145\/2939672.2939822","article-title":"Label noise reduction in entity typing by heterogeneous partial-label embedding","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Ren","year":"2016"},{"key":"2022051714395773400_bib35","first-page":"4239","article-title":"TaxoClass: Hierarchical multi-label text classification using only class names","volume-title":"Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Shen","year":"2021"},{"key":"2022051714395773400_bib36","doi-asserted-by":"crossref","first-page":"263","DOI":"10.18653\/v1\/P18-1025","article-title":"Probabilistic embedding of knowledge graphs with box lattice measures","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Vilnis","year":"2018"},{"key":"2022051714395773400_bib37","doi-asserted-by":"crossref","first-page":"1405","DOI":"10.18653\/v1\/2021.findings-acl.121","article-title":"K-Adapter: Infusing knowledge into pre-trained models with adapters","volume-title":"Findings of the Associationfor Computational Linguistics: ACL-IJCNLP 2021","author":"Wang","year":"2021"},{"key":"2022051714395773400_bib38","first-page":"996","article-title":"Inference is everything: Recasting semantic resources into a unified evaluation framework","volume-title":"Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"White","year":"2017"},{"key":"2022051714395773400_bib39","first-page":"1112","article-title":"A broad-coverage challenge corpus for sentence understanding through inference","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Williams","year":"2018"},{"key":"2022051714395773400_bib40","first-page":"773","article-title":"Imposing label-relational inductive bias for extremely fine-grained entity typing","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Xiong","year":"2019"},{"key":"2022051714395773400_bib41","first-page":"16","article-title":"Neural fine-grained entity type classification with hierarchy-aware loss","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Peng","year":"2018"},{"key":"2022051714395773400_bib42","first-page":"5412","article-title":"Syntax-enhanced pre-trained model","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Zenan","year":"2021"},{"key":"2022051714395773400_bib43","doi-asserted-by":"crossref","first-page":"149","DOI":"10.18653\/v1\/D16-1015","article-title":"Improving semantic parsing via answer type inference","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing","author":"Yavuz","year":"2016"},{"key":"2022051714395773400_bib44","doi-asserted-by":"crossref","first-page":"3914","DOI":"10.18653\/v1\/D19-1404","article-title":"Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Yin","year":"2019"},{"key":"2022051714395773400_bib45","doi-asserted-by":"crossref","first-page":"8229","DOI":"10.18653\/v1\/2020.emnlp-main.660","article-title":"Universal natural language processing with limited annotations: Try few-shot textual entailment as a start","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Yin","year":"2020"},{"key":"2022051714395773400_bib46","first-page":"291","article-title":"Embedding methods for fine grained entity type classification","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)","author":"Yogatama","year":"2015"},{"key":"2022051714395773400_bib47","doi-asserted-by":"crossref","first-page":"1441","DOI":"10.18653\/v1\/P19-1139","article-title":"ERNIE: Enhanced language representation with informative entities","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Zhang","year":"2019"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00479\/2022954\/tacl_a_00479.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00479\/2022954\/tacl_a_00479.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,5,17]],"date-time":"2022-05-17T15:05:51Z","timestamp":1652799951000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00479\/111220\/Ultra-fine-Entity-Typing-with-Indirect-Supervision"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022]]},"references-count":47,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00479","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022]]},"published":{"date-parts":[[2022]]}}}