{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T08:17:17Z","timestamp":1770279437762,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":55,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,7,25]],"date-time":"2020-07-25T00:00:00Z","timestamp":1595635200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,7,25]]},"DOI":"10.1145\/3397271.3401121","type":"proceedings-article","created":{"date-parts":[[2020,7,25]],"date-time":"2020-07-25T07:50:08Z","timestamp":1595663408000},"page":"1051-1060","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["Learning with Weak Supervision for Email Intent Detection"],"prefix":"10.1145","author":[{"given":"Kai","family":"Shu","sequence":"first","affiliation":[{"name":"Arizona State University, Tempe, AZ, USA"}]},{"given":"Subhabrata","family":"Mukherjee","sequence":"additional","affiliation":[{"name":"Microsoft Research AI, Redmond, WA, USA"}]},{"given":"Guoqing","family":"Zheng","sequence":"additional","affiliation":[{"name":"Microsoft Research AI, Redmond, WA, USA"}]},{"given":"Ahmed Hassan","family":"Awadallah","sequence":"additional","affiliation":[{"name":"Microsoft Research AI, Redmond, WA, USA"}]},{"given":"Milad","family":"Shokouhi","sequence":"additional","affiliation":[{"name":"Microsoft, Bellevue, WA, USA"}]},{"given":"Susan","family":"Dumais","sequence":"additional","affiliation":[{"name":"Microsoft Research AI, Redmond, WA, USA"}]}],"member":"320","published-online":{"date-parts":[[2020,7,25]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"INC.","author":"Email","year":"2015","unstructured":"Email statistics report. The Radicati Group , INC. , 2015 . Email statistics report. The Radicati Group, INC., 2015."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148177"},{"key":"e_1_3_2_1_3_1","volume-title":"WWW","author":"Ai Qingyao","year":"2017","unstructured":"Qingyao Ai , Susan T Dumais , Nick Craswell , and Dan Liebling . Characterizing email search using large-scale behavioral logs and surveys . In WWW , 2017 . Qingyao Ai, Susan T Dumais, Nick Craswell, and Dan Liebling. Characterizing email search using large-scale behavioral logs and surveys. In WWW, 2017."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3290984"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553380"},{"key":"e_1_3_2_1_6_1","volume-title":"SIGIR","author":"Paul","year":"2005","unstructured":"Paul N. Bennett and Jaime Carbonell. Detecting action-items in e-mail . In SIGIR , 2005 . Paul N. Bennett and Jaime Carbonell. Detecting action-items in e-mail. In SIGIR, 2005."},{"key":"e_1_3_2_1_7_1","volume-title":"Detecting action items in email","author":"Bennett Paul N","year":"2005","unstructured":"Paul N Bennett and Jaime G Carbonell . Detecting action items in email . 2005 . Paul N Bennett and Jaime G Carbonell. Detecting action items in email. 2005."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3055399.3055491"},{"key":"e_1_3_2_1_9_1","volume-title":"The social economy: Unlocking value and productivity through social technologies","author":"Chui Michael","year":"2012","unstructured":"Michael Chui , James Manyika , Jacques Bughin , Richard Dobbs , Charles Roxburgh , Hugo Sarrazin , Georey Sands , and Magdalena Westergren . The social economy: Unlocking value and productivity through social technologies . McKinsey Global Institute ., 2012 . Michael Chui, James Manyika, Jacques Bughin, Richard Dobbs, Charles Roxburgh, Hugo Sarrazin, Georey Sands, and Magdalena Westergren. The social economy: Unlocking value and productivity through social technologies. McKinsey Global Institute., 2012."},{"key":"e_1_3_2_1_10_1","volume-title":"A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37--46","author":"Cohen Jacob","year":"1960","unstructured":"Jacob Cohen . A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37--46 , 1960 . Jacob Cohen. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37--46, 1960."},{"key":"e_1_3_2_1_11_1","volume-title":"In Proceedings of Empirical Methods in Natural Language Processing","author":"Cohen William W.","year":"2004","unstructured":"William W. Cohen , Vitor R. Carvalho , and Tom M. Mitchell . Learning to classify email into speech acts . In In Proceedings of Empirical Methods in Natural Language Processing , 2004 . William W. Cohen, Vitor R. Carvalho, and Tom M. Mitchell. Learning to classify email into speech acts. In In Proceedings of Empirical Methods in Natural Language Processing, 2004."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1054972.1055068"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1054972.1055068"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080832"},{"key":"e_1_3_2_1_15_1","volume-title":"Classification in the presence of label noise: a survey","author":"Fr\u00e9nay Beno\u00eet","year":"2013","unstructured":"Beno\u00eet Fr\u00e9nay and Michel Verleysen . Classification in the presence of label noise: a survey . IEEE transactions on neural networks and learning systems, 25(5):845--869, 2013 . Beno\u00eet Fr\u00e9nay and Michel Verleysen. Classification in the presence of label noise: a survey. IEEE transactions on neural networks and learning systems, 25(5):845--869, 2013."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487575.2487612"},{"key":"e_1_3_2_1_17_1","volume-title":"Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural networks, 18(5--6):602--610","author":"Graves Alex","year":"2005","unstructured":"Alex Graves and J\u00fcrgen Schmidhuber . Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural networks, 18(5--6):602--610 , 2005 . Alex Graves and J\u00fcrgen Schmidhuber. Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural networks, 18(5--6):602--610, 2005."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718487.1718515"},{"key":"e_1_3_2_1_19_1","volume-title":"NeurIPS","author":"Hendrycks Dan","year":"2018","unstructured":"Dan Hendrycks , Mantas Mazeika , Duncan Wilson , and Kevin Gimpel . Using trusted data to train deep networks on labels corrupted by severe noise . In NeurIPS , 2018 . Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. Using trusted data to train deep networks on labels corrupted by severe noise. In NeurIPS, 2018."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/302979.303030"},{"key":"e_1_3_2_1_21_1","volume-title":"UAI","author":"Horvitz Eric","year":"1999","unstructured":"Eric Horvitz , Andy Jacobs , and David Hovel . Attention-sensitive alerting . In UAI , 1999 . Eric Horvitz, Andy Jacobs, and David Hovel. Attention-sensitive alerting. In UAI, 1999."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2396761.2396771"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1229179.1229181"},{"key":"e_1_3_2_1_24_1","volume-title":"Smart reply: Automated response suggestion for email. arXiv preprint arXiv:1606.04870","author":"Kannan Anjuli","year":"2016","unstructured":"Anjuli Kannan , Karol Kurach , Sujith Ravi , Tobias Kaufmann , Andrew Tomkins , Balint Miklos , Greg Corrado , L\u00e1szl\u00f3 Luk\u00e1cs , Marina Ganea , Peter Young , Smart reply: Automated response suggestion for email. arXiv preprint arXiv:1606.04870 , 2016 . Anjuli Kannan, Karol Kurach, Sujith Ravi, Tobias Kaufmann, Andrew Tomkins, Balint Miklos, Greg Corrado, L\u00e1szl\u00f3 Luk\u00e1cs, Marina Ganea, Peter Young, et al. Smart reply: Automated response suggestion for email. arXiv preprint arXiv:1606.04870, 2016."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30115-8_22"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2736277.2741130"},{"key":"e_1_3_2_1_27_1","volume-title":"NeurIPS","author":"Kumar M Pawan","year":"2010","unstructured":"M Pawan Kumar , Benjamin Packer , and Daphne Koller . Self-paced learning for latent variable models . In NeurIPS , 2010 . M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. In NeurIPS, 2010."},{"key":"e_1_3_2_1_28_1","volume-title":"HLT","author":"Lampert Andrew","year":"2010","unstructured":"Andrew Lampert , Robert Dale , and Cecile Paris . Detecting emails containing requests for action . In HLT , 2010 . Andrew Lampert, Robert Dale, and Cecile Paris. Detecting emails containing requests for action. In HLT, 2010."},{"key":"e_1_3_2_1_29_1","volume-title":"NAACL","author":"Lampert Andrew","year":"2010","unstructured":"Andrew Lampert , Robert Dale , and Cecile Paris . Detecting emails containing requests for action . In NAACL , 2010 . Andrew Lampert, Robert Dale, and Cecile Paris. Detecting emails containing requests for action. In NAACL, 2010."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.211"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11931"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-68155-9_16"},{"key":"e_1_3_2_1_33_1","volume-title":"WWW","author":"Mackenzie Joel","year":"2019","unstructured":"Joel Mackenzie , Kshitiz Gupta , Fang Qiao , Ahmed Hassan Awadallah , and Milad Shokouhi . Exploring user behavior in email re-finding tasks . In WWW , 2019 . Joel Mackenzie, Kshitiz Gupta, Fang Qiao, Ahmed Hassan Awadallah, and Milad Shokouhi. Exploring user behavior in email re-finding tasks. In WWW, 2019."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33016826"},{"key":"e_1_3_2_1_35_1","volume-title":"WWW","author":"Mitra Bhaskar","year":"2017","unstructured":"Bhaskar Mitra , Fernando Diaz , and Nick Craswell . Learning to match using local and distributed representations of text for web search . In WWW , 2017 . Bhaskar Mitra, Fernando Diaz, and Nick Craswell. Learning to match using local and distributed representations of text for web search. In WWW, 2017."},{"key":"e_1_3_2_1_36_1","volume-title":"NeurIPS","author":"Natarajan Nagarajan","year":"2013","unstructured":"Nagarajan Natarajan , Inderjit S Dhillon , Pradeep K Ravikumar , and Ambuj Tewari . Learning with noisy labels . In NeurIPS , 2013 . Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. In NeurIPS, 2013."},{"key":"e_1_3_2_1_37_1","volume-title":"A study of the effect of different types of noise on the precision of supervised learning techniques. Artificial intelligence review, 33(4):275--306","author":"Nettleton David F","year":"2010","unstructured":"David F Nettleton , Albert Orriols-Puig , and Albert Fornells . A study of the effect of different types of noise on the precision of supervised learning techniques. Artificial intelligence review, 33(4):275--306 , 2010 . David F Nettleton, Albert Orriols-Puig, and Albert Fornells. A study of the effect of different types of noise on the precision of supervised learning techniques. Artificial intelligence review, 33(4):275--306, 2010."},{"key":"e_1_3_2_1_38_1","volume-title":"Avocado research email collection","author":"Oard Douglas","year":"2015","unstructured":"Douglas Oard , William Webber , David Kirsch , and Sergey Golitsynskiy . Avocado research email collection . Philadelphia : Linguistic Data Consortium , 2015 . Douglas Oard, William Webber, David Kirsch, and Sergey Golitsynskiy. Avocado research email collection. Philadelphia: Linguistic Data Consortium, 2015."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.299"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.240"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_1_42_1","unstructured":"Alexander Ratner Stephen H Bach Henry Ehrenberg Jason Fries Sen Wu and Christopher R\u00e9. Snorkel: Rapid training data creation with weak supervision. VLDB.  Alexander Ratner Stephen H Bach Henry Ehrenberg Jason Fries Sen Wu and Christopher R\u00e9. Snorkel: Rapid training data creation with weak supervision. VLDB."},{"key":"e_1_3_2_1_43_1","volume-title":"Training complex models with multi-task weak supervision. arXiv preprint arXiv:1810.02840","author":"Ratner Alexander","year":"2018","unstructured":"Alexander Ratner , Braden Hancock , Jared Dunnmon , Frederic Sala , Shreyash Pandey , and Christopher R\u00e9 . Training complex models with multi-task weak supervision. arXiv preprint arXiv:1810.02840 , 2018 . Alexander Ratner, Braden Hancock, Jared Dunnmon, Frederic Sala, Shreyash Pandey, and Christopher R\u00e9. Training complex models with multi-task weak supervision. arXiv preprint arXiv:1810.02840, 2018."},{"key":"e_1_3_2_1_44_1","volume-title":"Training deep neural networks on noisy labels with bootstrapping. arXiv preprint arXiv:1412.6596","author":"Reed Scott","year":"2014","unstructured":"Scott Reed , Honglak Lee , Dragomir Anguelov , Christian Szegedy , Dumitru Erhan , and Andrew Rabinovich . Training deep neural networks on noisy labels with bootstrapping. arXiv preprint arXiv:1412.6596 , 2014 . Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. Training deep neural networks on noisy labels with bootstrapping. arXiv preprint arXiv:1412.6596, 2014."},{"key":"e_1_3_2_1_45_1","volume-title":"Learning to reweight examples for robust deep learning. arXiv preprint arXiv:1803.09050","author":"Ren Mengye","year":"2018","unstructured":"Mengye Ren , Wenyuan Zeng , Bin Yang , and Raquel Urtasun . Learning to reweight examples for robust deep learning. arXiv preprint arXiv:1803.09050 , 2018 . Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. arXiv preprint arXiv:1803.09050, 2018."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2016.03.002"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3291028"},{"key":"e_1_3_2_1_48_1","volume-title":"Training convolutional networks with noisy labels. arXiv preprint arXiv:1406.2080","author":"Sukhbaatar Sainbayar","year":"2014","unstructured":"Sainbayar Sukhbaatar , Joan Bruna , Manohar Paluri , Lubomir Bourdev , and Rob Fergus . Training convolutional networks with noisy labels. arXiv preprint arXiv:1406.2080 , 2014 . Sainbayar Sukhbaatar, Joan Bruna, Manohar Paluri, Lubomir Bourdev, and Rob Fergus. Training convolutional networks with noisy labels. arXiv preprint arXiv:1406.2080, 2014."},{"key":"e_1_3_2_1_49_1","volume-title":"ICML","author":"Varma Paroma","year":"2019","unstructured":"Paroma Varma , Frederic Sala , Ann He , Alexander Ratner , and Christopher R\u00e9 . Learning dependency structures for weak supervision models . ICML , 2019 . Paroma Varma, Frederic Sala, Ann He, Alexander Ratner, and Christopher R\u00e9. Learning dependency structures for weak supervision models. ICML, 2019."},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331260"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331260"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080782"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210046"},{"key":"e_1_3_2_1_54_1","volume-title":"Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701","author":"Zeiler Matthew D","year":"2012","unstructured":"Matthew D Zeiler . Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 , 2012 . Matthew D Zeiler. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701, 2012."},{"key":"e_1_3_2_1_55_1","volume-title":"Understanding deep learning requires rethinking generalization","author":"Zhang Chiyuan","year":"2016","unstructured":"Chiyuan Zhang , Samy Bengio , Moritz Hardt , Benjamin Recht , and Oriol Vinyals . Understanding deep learning requires rethinking generalization , 2016 . Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization, 2016."}],"event":{"name":"SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval","location":"Virtual Event China","acronym":"SIGIR '20","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397271.3401121","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3397271.3401121","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:38Z","timestamp":1750195898000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397271.3401121"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,25]]},"references-count":55,"alternative-id":["10.1145\/3397271.3401121","10.1145\/3397271"],"URL":"https:\/\/doi.org\/10.1145\/3397271.3401121","relation":{},"subject":[],"published":{"date-parts":[[2020,7,25]]},"assertion":[{"value":"2020-07-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}