{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T21:07:10Z","timestamp":1761599230946,"version":"3.41.0"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2022,4,12]],"date-time":"2022-04-12T00:00:00Z","timestamp":1649721600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"US National Science Foundation","doi-asserted-by":"crossref","award":["IIS-1838730"],"award-info":[{"award-number":["IIS-1838730"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Amazon AWS credits"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2022,6,30]]},"abstract":"<jats:p>\n            Short text is ambiguous and often relies predominantly on the domain and context at hand in order to attain semantic relevance. Existing classification models perform poorly on short text due to data sparsity and inadequate context. Auxiliary context, which can often provide sufficient background regarding the domain, is typically available in several application scenarios. While some of the existing works aim to leverage real-world knowledge to enhance short-text representations, they fail to place appropriate emphasis on the auxiliary context. Such models do not harness the full potential of the available context in auxiliary sources. To address this challenge, we reformulate short-text classification as a dual channel self-supervised learning problem (that leverages auxiliary context) with a generation network and a corresponding prediction model. We propose a self-supervised framework,\n            <jats:italic>Pseudo-Auxiliary Context generation network for Short-text Modeling (PACS)<\/jats:italic>\n            , to comprehensively leverage auxiliary context and it is jointly learned with a prediction network in an end-to-end manner. Our PACS model consists of two sub-networks: a Context Generation Network (CGN) that models the auxiliary context\u2019s distribution and a Prediction Network (PN) to map the short-text features and auxiliary context distribution to the final class label. Our experimental results on diverse datasets demonstrate that PACS outperforms formidable state-of-the-art baselines. We also demonstrate the performance of our model on cold-start scenarios (where contextual information is non-existent) during prediction. Furthermore, we perform interpretability and ablation studies to analyze various representational features captured by our model and the individual contribution of its modules to the overall performance of PACS, respectively.\n          <\/jats:p>","DOI":"10.1145\/3511712","type":"journal-article","created":{"date-parts":[[2022,4,12]],"date-time":"2022-04-12T09:31:42Z","timestamp":1649755902000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Self-supervised Short-text Modeling through Auxiliary Context Generation"],"prefix":"10.1145","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4471-8968","authenticated-orcid":false,"given":"Nurendra","family":"Choudhary","sequence":"first","affiliation":[{"name":"Virginia Tech, Arlington, VA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Charu C.","family":"Aggarwal","sequence":"additional","affiliation":[{"name":"IBM T.J. Watson Research Center, Yorktown Heights, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Karthik","family":"Subbian","sequence":"additional","affiliation":[{"name":"University of Minnesota, Minneapolis, MN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chandan K.","family":"Reddy","sequence":"additional","affiliation":[{"name":"Virginia Tech, Arlington, VA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,4,12]]},"reference":[{"key":"e_1_3_3_2_2","first-page":"265","volume-title":"Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201916)","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201916). USENIX Association, 265\u2013283."},{"key":"e_1_3_3_3_2","volume-title":"3rd International Conference on Learning Representations (ICLR\u201915), Conference Track Proceedings","author":"Bahdanau Dzmitry","year":"2015","unstructured":"Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations (ICLR\u201915), Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http:\/\/arxiv.org\/abs\/1409.0473."},{"key":"e_1_3_3_4_2","first-page":"993","article-title":"Latent Dirichlet allocation","volume":"3","author":"Blei David M.","year":"2003","unstructured":"David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research 3, (Jan.2003), 993\u20131022.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_3_5_2","first-page":"161","volume-title":"Proceedings of 3rd Annual Symposium on Document Analysis and Information Retrieval (SDAIR\u201994)","volume":"161175","author":"Cavnar William B.","year":"1994","unstructured":"William B. Cavnar and John M. Trenkle. 1994. N-gram-based text categorization. In Proceedings of 3rd Annual Symposium on Document Analysis and Information Retrieval (SDAIR\u201994), Vol. 161175. Citeseer, 161\u2013175."},{"doi-asserted-by":"publisher","key":"e_1_3_3_6_2","DOI":"10.1609\/aaai.v33i01.33016252"},{"doi-asserted-by":"publisher","key":"e_1_3_3_7_2","DOI":"10.1145\/3488560.3498456"},{"key":"e_1_3_3_8_2","article-title":"Neural network architecture for credibility assessment of textual claims","author":"Choudhary Nurendra","year":"2018","unstructured":"Nurendra Choudhary, Rajat Singh, Ishita Bindlish, and Manish Shrivastava. 2018. Neural network architecture for credibility assessment of textual claims. arXiv preprint arXiv:1803.10547 (2018).","journal-title":"arXiv preprint arXiv:1803.10547"},{"doi-asserted-by":"publisher","key":"e_1_3_3_9_2","DOI":"10.18653\/v1\/n19-1423"},{"doi-asserted-by":"publisher","key":"e_1_3_3_10_2","DOI":"10.1016\/j.scitotenv.2021.149797"},{"unstructured":"Evgeniy Gabrilovich and Shaul Markovitch. 2007. Computing semantic relatedness using Wikipedia-based explicit semantic analysis. In IJcAI Vol. 7. 1606\u20131611.","key":"e_1_3_3_11_2"},{"key":"e_1_3_3_12_2","first-page":"225","article-title":"Linear hinge loss and average margin","volume":"11","author":"Gentile Claudio","year":"1998","unstructured":"Claudio Gentile and Manfred K. K. Warmuth. 1998. Linear hinge loss and average margin. Advances in Neural Information Processing Systems 11 (1998), 225\u2013231.","journal-title":"Advances in Neural Information Processing Systems"},{"doi-asserted-by":"publisher","key":"e_1_3_3_13_2","DOI":"10.1016\/0004-3702(88)90012-4"},{"doi-asserted-by":"publisher","key":"e_1_3_3_14_2","DOI":"10.1145\/1526709.1526773"},{"doi-asserted-by":"publisher","key":"e_1_3_3_15_2","DOI":"10.1145\/2505515.2505665"},{"doi-asserted-by":"publisher","key":"e_1_3_3_16_2","DOI":"10.1007\/11744085_41"},{"key":"e_1_3_3_17_2","volume-title":"3rd International Conference on Learning Representations (ICLR\u201915), Conference Track Proceedings","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations (ICLR\u201915), Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http:\/\/arxiv.org\/abs\/1412.6980."},{"key":"e_1_3_3_18_2","volume-title":"International Conference on Learning Representations (ICLR\u201917)","author":"Kipf Thomas N.","year":"2017","unstructured":"Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR\u201917)."},{"doi-asserted-by":"publisher","key":"e_1_3_3_19_2","DOI":"10.1609\/aaai.v29i1.9513"},{"doi-asserted-by":"publisher","key":"e_1_3_3_20_2","DOI":"10.18653\/v1\/n16-1062"},{"doi-asserted-by":"publisher","key":"e_1_3_3_21_2","DOI":"10.18653\/v1\/2020.acl-main.703"},{"doi-asserted-by":"publisher","key":"e_1_3_3_22_2","DOI":"10.1145\/3091108"},{"key":"e_1_3_3_23_2","volume-title":"5th International Conference on Learning Representations (ICLR\u201917), Conference Track Proceedings","author":"Lin Zhouhan","year":"2017","unstructured":"Zhouhan Lin, Minwei Feng, C\u00edcero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. In 5th International Conference on Learning Representations (ICLR\u201917), Conference Track Proceedings. OpenReview.net. https:\/\/openreview.net\/forum?id=BJC_jUqxe."},{"doi-asserted-by":"publisher","key":"e_1_3_3_24_2","DOI":"10.1016\/B978-1-4832-0771-1.50021-7"},{"doi-asserted-by":"publisher","key":"e_1_3_3_25_2","DOI":"10.1145\/1102351.1102422"},{"doi-asserted-by":"publisher","key":"e_1_3_3_26_2","DOI":"10.1609\/aaai.v33i01.33016826"},{"key":"e_1_3_3_27_2","first-page":"3111","volume-title":"Advances in Neural Information Processing Systems","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111\u20133119."},{"key":"e_1_3_3_28_2","article-title":"Representation learning with contrastive predictive coding","author":"Oord Aaron van den","year":"2018","unstructured":"Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).","journal-title":"arXiv preprint arXiv:1807.03748"},{"doi-asserted-by":"publisher","key":"e_1_3_3_29_2","DOI":"10.1109\/taslp.2016.2520371"},{"doi-asserted-by":"publisher","key":"e_1_3_3_30_2","DOI":"10.3115\/1118693.1118704"},{"doi-asserted-by":"publisher","key":"e_1_3_3_31_2","DOI":"10.1007\/978-3-540-45224-9_72"},{"doi-asserted-by":"publisher","key":"e_1_3_3_32_2","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_3_33_2","first-page":"866","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Post Matt","year":"2013","unstructured":"Matt Post and Shane Bergsma. 2013. Explicit and implicit syntactic features for text classification. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 866\u2013872."},{"doi-asserted-by":"publisher","key":"e_1_3_3_34_2","DOI":"10.1109\/ICDM.2016.0151"},{"key":"e_1_3_3_35_2","first-page":"45","volume-title":"Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks","author":"\u0158eh\u016f\u0159ek Radim","year":"2010","unstructured":"Radim \u0158eh\u016f\u0159ek and Petr Sojka. 2010. Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. ELRA, Valletta, Malta, 45\u201350."},{"doi-asserted-by":"publisher","key":"e_1_3_3_36_2","DOI":"10.1145\/3178876.3186009"},{"doi-asserted-by":"publisher","key":"e_1_3_3_37_2","DOI":"10.5555\/2390948.2391084"},{"doi-asserted-by":"publisher","key":"e_1_3_3_38_2","DOI":"10.21437\/Interspeech.2012-65"},{"doi-asserted-by":"publisher","key":"e_1_3_3_39_2","DOI":"10.18653\/v1\/2020.coling-industry.22"},{"doi-asserted-by":"crossref","unstructured":"Alex Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy and Samuel R. Bowman. 2019. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In The Proceedings of ICLR .","key":"e_1_3_3_40_2","DOI":"10.18653\/v1\/W18-5446"},{"doi-asserted-by":"publisher","key":"e_1_3_3_41_2","DOI":"10.1145\/2661829.2662067"},{"doi-asserted-by":"publisher","key":"e_1_3_3_42_2","DOI":"10.1109\/5.58337"},{"key":"e_1_3_3_43_2","volume-title":"3rd International Conference on Learning Representations (ICLR\u201915), Conference Track Proceedings","author":"Weston Jason","year":"2015","unstructured":"Jason Weston, Sumit Chopra, and Antoine Bordes. 2015. Memory networks. In 3rd International Conference on Learning Representations (ICLR\u201915), Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http:\/\/arxiv.org\/abs\/1410.3916."},{"doi-asserted-by":"publisher","key":"e_1_3_3_44_2","DOI":"10.24963\/ijcai.2020\/549"},{"key":"e_1_3_3_45_2","first-page":"5754","volume-title":"Advances in Neural Information Processing Systems","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R. Salakhutdinov, and Quoc V. Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in Neural Information Processing Systems. 5754\u20135764."},{"doi-asserted-by":"publisher","key":"e_1_3_3_46_2","DOI":"10.18653\/v1\/N16-1174"},{"doi-asserted-by":"publisher","key":"e_1_3_3_47_2","DOI":"10.18653\/v1\/d18-1351"},{"doi-asserted-by":"publisher","key":"e_1_3_3_48_2","DOI":"10.18653\/v1\/2021.emnlp-main.222"},{"key":"e_1_3_3_49_2","first-page":"649","volume-title":"Advances in Neural Information Processing Systems","author":"Zhang Xiang","year":"2015","unstructured":"Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in Neural Information Processing Systems. 649\u2013657."},{"doi-asserted-by":"publisher","key":"e_1_3_3_50_2","DOI":"10.1145\/3292500.3330781"},{"doi-asserted-by":"publisher","key":"e_1_3_3_51_2","DOI":"10.1109\/TKDE.2021.3073195"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511712","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3511712","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:11:47Z","timestamp":1750191107000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511712"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,12]]},"references-count":50,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,6,30]]}},"alternative-id":["10.1145\/3511712"],"URL":"https:\/\/doi.org\/10.1145\/3511712","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"type":"print","value":"2157-6904"},{"type":"electronic","value":"2157-6912"}],"subject":[],"published":{"date-parts":[[2022,4,12]]},"assertion":[{"value":"2021-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-04-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}