{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T06:47:50Z","timestamp":1770274070078,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":41,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,4,14]],"date-time":"2021-04-14T00:00:00Z","timestamp":1618358400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,4,14]]},"DOI":"10.1145\/3397481.3450698","type":"proceedings-article","created":{"date-parts":[[2021,4,14]],"date-time":"2021-04-14T06:07:18Z","timestamp":1618380438000},"page":"392-401","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":36,"title":["Increasing the Speed and Accuracy of Data Labeling Through an AI Assisted Interface"],"prefix":"10.1145","author":[{"given":"Michael","family":"Desmond","sequence":"first","affiliation":[{"name":"IBM Research, United States"}]},{"given":"Michael","family":"Muller","sequence":"additional","affiliation":[{"name":"IBM Research, United States"}]},{"given":"Zahra","family":"Ashktorab","sequence":"additional","affiliation":[{"name":"IBM Research, United States"}]},{"given":"Casey","family":"Dugan","sequence":"additional","affiliation":[{"name":"IBM Research, United States"}]},{"given":"Evelyn","family":"Duesterwald","sequence":"additional","affiliation":[{"name":"IBM Research, United States"}]},{"given":"Kristina","family":"Brimijoin","sequence":"additional","affiliation":[{"name":"IBM Research, United States"}]},{"given":"Catherine","family":"Finegan-Dollak","sequence":"additional","affiliation":[{"name":"IBM Research, United States"}]},{"given":"Michelle","family":"Brachman","sequence":"additional","affiliation":[{"name":"IBM Research, United States"}]},{"given":"Aabhas","family":"Sharma","sequence":"additional","affiliation":[{"name":"IBM Research, United States"}]},{"given":"Narendra Nath","family":"Joshi","sequence":"additional","affiliation":[{"name":"IBM Research, United States"}]},{"given":"Qian","family":"Pan","sequence":"additional","affiliation":[{"name":"IBM Research, United States"}]}],"member":"320","published-online":{"date-parts":[[2021,4,14]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Mahmoud\u00a0J Abu\u00a0Ghali Abdullah Abu\u00a0Ayyad Samy\u00a0S Abu-Naser and Mousa Abu\u00a0Laban. 2018. An intelligent tutoring system for teaching english grammar. (2018).  Mahmoud\u00a0J Abu\u00a0Ghali Abdullah Abu\u00a0Ayyad Samy\u00a0S Abu-Naser and Mousa Abu\u00a0Laban. 2018. An intelligent tutoring system for teaching english grammar. (2018)."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753534"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN48605.2020.9207304"},{"key":"e_1_3_2_1_4_1","volume-title":"Understanding information systems continuance: an expectation-confirmation model. MIS quarterly","author":"Bhattacherjee Anol","year":"2001","unstructured":"Anol Bhattacherjee . 2001. Understanding information systems continuance: an expectation-confirmation model. MIS quarterly ( 2001 ), 351\u2013370. Anol Bhattacherjee. 2001. Understanding information systems continuance: an expectation-confirmation model. MIS quarterly (2001), 351\u2013370."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300234"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359206"},{"key":"e_1_3_2_1_7_1","volume-title":"Universal Sentence Encoder. arXiv preprint abs\/1803.11175","author":"Cer Daniel","year":"2018","unstructured":"Daniel Cer , Yinfei Yang , Sheng-yi Kong, Nan Hua , Nicole Limtiaco , Rhomni\u00a0 St. John , Noah Constant , Mario Guajardo-Cespedes , Steve Yuan , Chris Tar , Yun-Hsuan Sung , Brian Strope , and Ray Kurzweil . 2018. Universal Sentence Encoder. arXiv preprint abs\/1803.11175 ( 2018 ). arxiv:1803.11175http:\/\/arxiv.org\/abs\/1803.11175 Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni\u00a0St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Universal Sentence Encoder. arXiv preprint abs\/1803.11175 (2018). arxiv:1803.11175http:\/\/arxiv.org\/abs\/1803.11175"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300789"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300460"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240723"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1177\/0963721409359277"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159661"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3185517"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2011-000089"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1807184115"},{"key":"e_1_3_2_1_16_1","volume-title":"A comparison of the general linear mixed model and repeated measures ANOVA using a dataset with multiple missing data points. Biological research for nursing 6, 2","author":"Krueger Charlene","year":"2004","unstructured":"Charlene Krueger and Lili Tian . 2004. A comparison of the general linear mixed model and repeated measures ANOVA using a dataset with multiple missing data points. Biological research for nursing 6, 2 ( 2004 ), 151\u2013157. Charlene Krueger and Lili Tian. 2004. A comparison of the general linear mixed model and repeated measures ANOVA using a dataset with multiple missing data points. Biological research for nursing 6, 2 (2004), 151\u2013157."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557238"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287590"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359297"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0043158"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174223"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.14778\/3157794.3157797"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-019-00552-1"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1719970.1720006"},{"key":"e_1_3_2_1_28_1","volume-title":"Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. 1467\u20131478","author":"Settles Burr","year":"2011","unstructured":"Burr Settles . 2011 . Closing the loop: Fast, interactive semi-supervised annotation with queries on features and instances . In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. 1467\u20131478 . Burr Settles. 2011. Closing the loop: Fast, interactive semi-supervised annotation with queries on features and instances. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. 1467\u20131478."},{"key":"e_1_3_2_1_29_1","volume-title":"Proceedings of the NIPS Workshop on Cost-Sensitive Learning.","author":"Settles Burr","year":"2008","unstructured":"Burr Settles , Mark Craven , and Lewis Friedland . 2008 . Active Learning with Real Annotation Costs . In Proceedings of the NIPS Workshop on Cost-Sensitive Learning. Burr Settles, Mark Craven, and Lewis Friedland. 2008. Active Learning with Real Annotation Costs. In Proceedings of the NIPS Workshop on Cost-Sensitive Learning."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"crossref","unstructured":"Ben Shneiderman. 2020. Human-Centered Artificial Intelligence: Reliable Safe & Trustworthy. International Journal of Human\u2013Computer Interaction 36 6(2020) 495\u2013504.  Ben Shneiderman. 2020. Human-Centered Artificial Intelligence: Reliable Safe & Trustworthy. International Journal of Human\u2013Computer Interaction 36 6(2020) 495\u2013504.","DOI":"10.1080\/10447318.2020.1741118"},{"key":"e_1_3_2_1_31_1","volume-title":"Healthy Data: Policy solutions for big data and AI innovation in health","author":"Stinson Catherine","year":"2018","unstructured":"Catherine Stinson . 2018 . Healthy Data: Policy solutions for big data and AI innovation in health . Mowat Centre for Policy Innovation . Catherine Stinson. 2018. Healthy Data: Policy solutions for big data and AI innovation in health. Mowat Centre for Policy Innovation."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025171.3025208"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-019-05855-6"},{"key":"e_1_3_2_1_34_1","unstructured":"Oriol Vinyals Charles Blundell Timothy Lillicrap Daan Wierstra 2016. Matching networks for one shot learning. In Advances in neural information processing systems. 3630\u20133638.  Oriol Vinyals Charles Blundell Timothy Lillicrap Daan Wierstra 2016. Matching networks for one shot learning. In Advances in neural information processing systems. 3630\u20133638."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359313"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300831"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"crossref","unstructured":"Jennifer Watts-Perotti and David\u00a0D Woods. 1999. How experienced users avoid getting lost in large display networks. International Journal of Human\u2013Computer Interaction 11 4(1999) 269\u2013299.  Jennifer Watts-Perotti and David\u00a0D Woods. 1999. How experienced users avoid getting lost in large display networks. International Journal of Human\u2013Computer Interaction 11 4(1999) 269\u2013299.","DOI":"10.1207\/S15327590IJHC1104_1"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290607.3312817"},{"key":"e_1_3_2_1_39_1","volume-title":"Mouni Reddy, and Geoff Zweig.","author":"Williams D.","year":"2015","unstructured":"Jason\u00a0 D. Williams , Nobal\u00a0 B. Niraula , Pradeep Dasigi , Aparna Lakshmiratan , Carlos Garcia\u00a0Jurado Suarez , Mouni Reddy, and Geoff Zweig. 2015 . Rapidly scaling dialog systems with interactive learning. In Natural Language Dialog Systems and Intelligent Assistants. Springer , 1\u201313. Jason\u00a0D. Williams, Nobal\u00a0B. Niraula, Pradeep Dasigi, Aparna Lakshmiratan, Carlos Garcia\u00a0Jurado Suarez, Mouni Reddy, and Geoff Zweig. 2015. Rapidly scaling dialog systems with interactive learning. In Natural Language Dialog Systems and Intelligent Assistants. Springer, 1\u201313."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359245"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2930238.2930290"},{"key":"e_1_3_2_1_42_1","volume-title":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","author":"Zhang Yunfeng","unstructured":"Yunfeng Zhang , Q.\u00a0 Vera Liao , and Rachel K . \u00a0E. Bellamy. 2020. Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making . In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency ( Barcelona, Spain) (FAT* \u201920). Association for Computing Machinery, New York, NY, USA, 295\u2013305. https:\/\/doi.org\/10.1145\/3351095.3372852 10.1145\/3351095.3372852 Yunfeng Zhang, Q.\u00a0Vera Liao, and Rachel K.\u00a0E. Bellamy. 2020. Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* \u201920). Association for Computing Machinery, New York, NY, USA, 295\u2013305. https:\/\/doi.org\/10.1145\/3351095.3372852"},{"key":"e_1_3_2_1_43_1","unstructured":"Dengyong Zhou Olivier Bousquet Thomas\u00a0N Lal Jason Weston and Bernhard Sch\u00f6lkopf. 2004. Learning with local and global consistency. In Advances in neural information processing systems. 321\u2013328.  Dengyong Zhou Olivier Bousquet Thomas\u00a0N Lal Jason Weston and Bernhard Sch\u00f6lkopf. 2004. Learning with local and global consistency. In Advances in neural information processing systems. 321\u2013328."}],"event":{"name":"IUI '21: 26th International Conference on Intelligent User Interfaces","location":"College Station TX USA","acronym":"IUI '21","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence","SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["26th International Conference on Intelligent User Interfaces"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397481.3450698","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3397481.3450698","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:47:32Z","timestamp":1750193252000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397481.3450698"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,14]]},"references-count":41,"alternative-id":["10.1145\/3397481.3450698","10.1145\/3397481"],"URL":"https:\/\/doi.org\/10.1145\/3397481.3450698","relation":{},"subject":[],"published":{"date-parts":[[2021,4,14]]},"assertion":[{"value":"2021-04-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}