{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T16:50:06Z","timestamp":1772902206522,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":51,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,10,16]],"date-time":"2021-10-16T00:00:00Z","timestamp":1634342400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,10,16]]},"DOI":"10.1145\/3490355.3490358","type":"proceedings-article","created":{"date-parts":[[2022,2,7]],"date-time":"2022-02-07T23:22:11Z","timestamp":1644276131000},"page":"22-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Think-Aloud Verbalizations for Identifying User Experience Problems: Effects of Language Proficiency with Chinese Non-Native English Speakers"],"prefix":"10.1145","author":[{"given":"Mingming","family":"Fan","sequence":"first","affiliation":[{"name":"The Hong Kong University of Science and Technology, China"}]},{"given":"Lingyun","family":"Zhu","sequence":"additional","affiliation":[{"name":"Rochester Institute of Technology, United States"}]}],"member":"320","published-online":{"date-parts":[[2022,2,7]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/3190862.3190865"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240838"},{"key":"e_1_3_2_1_3_1","volume-title":"Thinking aloud: Reconciling theory and practice","author":"Boren Ted","year":"2000","unstructured":"Ted Boren and Judith Ramey . 2000. Thinking aloud: Reconciling theory and practice . IEEE transactions on professional communication 43, 3( 2000 ), 261\u2013278. Ted Boren and Judith Ramey. 2000. Thinking aloud: Reconciling theory and practice. IEEE transactions on professional communication 43, 3(2000), 261\u2013278."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518948"},{"key":"e_1_3_2_1_6_1","volume-title":"Conference on Human Factors in Computing Systems - Proceedings, 2225\u20132234","author":"Chalil\u00a0Madathil Kapil","year":"2011","unstructured":"Kapil Chalil\u00a0Madathil and Joel Greenstein . 2011 . Synchronous remote usability testing - A new approach facilitated by virtual worlds . Conference on Human Factors in Computing Systems - Proceedings, 2225\u20132234 . https:\/\/doi.org\/10.1145\/1978942.1979267 Kapil Chalil\u00a0Madathil and Joel Greenstein. 2011. Synchronous remote usability testing - A new approach facilitated by virtual worlds. Conference on Human Factors in Computing Systems - Proceedings, 2225\u20132234. https:\/\/doi.org\/10.1145\/1978942.1979267"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Torkil Clemmensen. 2011. Templates for cross-cultural and culturally specific usability testing: results from field studies and ethnographic interviewing in three countries. Intl. Journal of Human\u2013Computer Interaction 27 7(2011) 634\u2013669.  Torkil Clemmensen. 2011. Templates for cross-cultural and culturally specific usability testing: results from field studies and ethnographic interviewing in three countries. Intl. Journal of Human\u2013Computer Interaction 27 7(2011) 634\u2013669.","DOI":"10.1080\/10447318.2011.555303"},{"key":"e_1_3_2_1_8_1","volume-title":"Cultural cognition in usability evaluation. Interacting with computers 21, 3","author":"Clemmensen Torkil","year":"2009","unstructured":"Torkil Clemmensen , Morten Hertzum , Kasper Hornb\u00e6k , Qingxin Shi , and Pradeep Yammiyavar . 2009. Cultural cognition in usability evaluation. Interacting with computers 21, 3 ( 2009 ), 212\u2013220. Torkil Clemmensen, Morten Hertzum, Kasper Hornb\u00e6k, Qingxin Shi, and Pradeep Yammiyavar. 2009. Cultural cognition in usability evaluation. Interacting with computers 21, 3 (2009), 212\u2013220."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPC.2010.2052859"},{"key":"e_1_3_2_1_10_1","unstructured":"American\u00a0Immigration Council. 2020. The H-1B Visa Program: A Primer on the Program and Its Impact on Jobs Wages and the Economy. https:\/\/www.americanimmigrationcouncil.org\/research\/h1b-visa-program-fact-sheet.  American\u00a0Immigration Council. 2020. The H-1B Visa Program: A Primer on the Program and Its Impact on Jobs Wages and the Economy. https:\/\/www.americanimmigrationcouncil.org\/research\/h1b-visa-program-fact-sheet."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"crossref","unstructured":"David Crystal. 2003. English as a global language. Ernst Klett Sprachen.  David Crystal. 2003. English as a global language. Ernst Klett Sprachen.","DOI":"10.1017\/CBO9780511486999"},{"key":"e_1_3_2_1_12_1","unstructured":"Joseph\u00a0S Dumas Joseph\u00a0S Dumas and Janice Redish. 1999. A practical guide to usability testing. Intellect books.  Joseph\u00a0S Dumas Joseph\u00a0S Dumas and Janice Redish. 1999. A practical guide to usability testing. Intellect books."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPC.2012.2206190"},{"key":"e_1_3_2_1_14_1","unstructured":"K\u00a0Anders Ericsson and Herbert\u00a0A Simon. 1984. Protocol analysis: Verbal reports as data.the MIT Press.  K\u00a0Anders Ericsson and Herbert\u00a0A Simon. 1984. Protocol analysis: Verbal reports as data.the MIT Press."},{"key":"e_1_3_2_1_15_1","unstructured":"Ethnologue. 2020. What is the most spoken language?https:\/\/www.ethnologue.com\/guides\/most-spoken-languages.  Ethnologue. 2020. What is the most spoken language?https:\/\/www.ethnologue.com\/guides\/most-spoken-languages."},{"key":"e_1_3_2_1_16_1","article-title":"Automatic Detection of Usability Problem Encounters in Think-aloud Sessions","volume":"10","author":"Fan Mingming","year":"2020","unstructured":"Mingming Fan , Yue Li , and Khai\u00a0 N Truong . 2020 . Automatic Detection of Usability Problem Encounters in Think-aloud Sessions . ACM Transactions on Interactive Intelligent Systems (TiiS) 10 , 2, Article 16 (May 2020), 24\u00a0pages. https:\/\/doi.org\/10.1145\/3385732 Mingming Fan, Yue Li, and Khai\u00a0N Truong. 2020. Automatic Detection of Usability Problem Encounters in Think-aloud Sessions. ACM Transactions on Interactive Intelligent Systems (TiiS) 10, 2, Article 16 (May 2020), 24\u00a0pages. https:\/\/doi.org\/10.1145\/3385732","journal-title":"ACM Transactions on Interactive Intelligent Systems (TiiS)"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3325281"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/3532708.3532711"},{"key":"e_1_3_2_1_19_1","first-page":"343","article-title":"VisTA: Integrating Machine Intelligence with Visualization to Support the Investigation of Think-Aloud Sessions","volume":"26","author":"Fan Mingming","year":"2020","unstructured":"Mingming Fan , Ke Wu , Jian Zhao , Yue Li , Winter Wei , and Khai\u00a0 N Truong . 2020 . VisTA: Integrating Machine Intelligence with Visualization to Support the Investigation of Think-Aloud Sessions . IEEE Transactions on Visualization and Computer Graphics (TVCG) 26 , 1(2020), 343 \u2013 352 . https:\/\/doi.org\/10.1109\/TVCG.2019.2934797 Mingming Fan, Ke Wu, Jian Zhao, Yue Li, Winter Wei, and Khai\u00a0N Truong. 2020. VisTA: Integrating Machine Intelligence with Visualization to Support the Investigation of Think-Aloud Sessions. IEEE Transactions on Visualization and Computer Graphics (TVCG) 26, 1(2020), 343\u2013352. https:\/\/doi.org\/10.1109\/TVCG.2019.2934797","journal-title":"IEEE Transactions on Visualization and Computer Graphics (TVCG)"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445680"},{"key":"e_1_3_2_1_21_1","unstructured":"Center for Immigration\u00a0Studies. 2018. Almost Half Speak a Foreign Language in America\u2019s Largest Cities. https:\/\/cis.org\/Report\/Almost-Half-Speak-Foreign-Language-Americas-Largest-Cities. (Accessed on 08\/28\/2020).  Center for Immigration\u00a0Studies. 2018. Almost Half Speak a Foreign Language in America\u2019s Largest Cities. https:\/\/cis.org\/Report\/Almost-Half-Speak-Foreign-Language-Americas-Largest-Cities. (Accessed on 08\/28\/2020)."},{"key":"e_1_3_2_1_22_1","unstructured":"FullStory. 2021. FullStory | Robust Analytics Session Replay Heatmaps Dev Tools. https:\/\/www.fullstory.com.  FullStory. 2021. FullStory | Robust Analytics Session Replay Heatmaps Dev Tools. https:\/\/www.fullstory.com."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2016.09.009"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1124772.1124961"},{"key":"e_1_3_2_1_25_1","volume-title":"Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: A methodological comparison. Interacting with Computers 16 (09","author":"Haak Maaike","year":"2004","unstructured":"Maaike Haak , Menno De\u00a0Jong , and Peter Schellens . 2004. Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: A methodological comparison. Interacting with Computers 16 (09 2004 ). https:\/\/doi.org\/10.1016\/j.intcom.2004.07.007 Maaike Haak, Menno De\u00a0Jong, and Peter Schellens. 2004. Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: A methodological comparison. Interacting with Computers 16 (09 2004). https:\/\/doi.org\/10.1016\/j.intcom.2004.07.007"},{"key":"e_1_3_2_1_26_1","volume-title":"Cultural differences and usability evaluation: Individualistic and collectivistic participants compared. Technical communication 51, 4","author":"Hall Marinda","year":"2004","unstructured":"Marinda Hall , Menno De\u00a0Jong , and Michael Steehouder . 2004. Cultural differences and usability evaluation: Individualistic and collectivistic participants compared. Technical communication 51, 4 ( 2004 ), 489\u2013503. Marinda Hall, Menno De\u00a0Jong, and Michael Steehouder. 2004. Cultural differences and usability evaluation: Individualistic and collectivistic participants compared. Technical communication 51, 4 (2004), 489\u2013503."},{"key":"e_1_3_2_1_27_1","unstructured":"Melanie Hanson. 2021. International Student Population & Enrollment Statistics [2021]. https:\/\/educationdata.org\/international-student-enrollment-statistics.  Melanie Hanson. 2021. International Student Population & Enrollment Statistics [2021]. https:\/\/educationdata.org\/international-student-enrollment-statistics."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301423"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2015.1065691"},{"key":"e_1_3_2_1_30_1","volume-title":"Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload?Behaviour & Information Technology 28, 2","author":"Hertzum Morten","year":"2009","unstructured":"Morten Hertzum , Kristin\u00a0 D Hansen , and Hans\u00a0 HK Andersen . 2009. Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload?Behaviour & Information Technology 28, 2 ( 2009 ), 165\u2013181. Morten Hertzum, Kristin\u00a0D Hansen, and Hans\u00a0HK Andersen. 2009. Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload?Behaviour & Information Technology 28, 2 (2009), 165\u2013181."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3240167.3240181"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2019.10.001"},{"key":"e_1_3_2_1_33_1","volume-title":"Thinking About Thinking Aloud: A Comparison of Two Verbal Protocols for Usability Testing. Professional Communication","author":"Krahmer Emiel","year":"2004","unstructured":"Emiel Krahmer and Nicole Ummelen . 2004. Thinking About Thinking Aloud: A Comparison of Two Verbal Protocols for Usability Testing. Professional Communication , IEEE Transactions on 47 (07 2004 ), 105\u2013117. https:\/\/doi.org\/10.1109\/TPC.2004.828205 Emiel Krahmer and Nicole Ummelen. 2004. Thinking About Thinking Aloud: A Comparison of Two Verbal Protocols for Usability Testing. Professional Communication, IEEE Transactions on 47 (07 2004), 105\u2013117. https:\/\/doi.org\/10.1109\/TPC.2004.828205"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPC.2011.2182569"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2481407"},{"key":"e_1_3_2_1_36_1","unstructured":"Jakob Nielsen. 1994. Severity Ratings for Usability Problems. https:\/\/www.nngroup.com\/articles\/how-to-rate-the-severity-of-usability-problems\/.  Jakob Nielsen. 1994. Severity Ratings for Usability Problems. https:\/\/www.nngroup.com\/articles\/how-to-rate-the-severity-of-usability-problems\/."},{"key":"e_1_3_2_1_37_1","volume-title":"Usability engineering","author":"Nielsen Jakob","unstructured":"Jakob Nielsen . 1994. Usability engineering . Elsevier . Jakob Nielsen. 1994. Usability engineering. Elsevier."},{"key":"e_1_3_2_1_38_1","unstructured":"NNGroup. 2014. Demonstrate Thinking Aloud by Showing Users a Video. https:\/\/www.nngroup.com\/articles\/thinking-aloud-demo-video\/  NNGroup. 2014. Demonstrate Thinking Aloud by Showing Users a Video. https:\/\/www.nngroup.com\/articles\/thinking-aloud-demo-video\/"},{"key":"e_1_3_2_1_39_1","unstructured":"Don Norman. 2013. The design of everyday things: Revised and expanded edition. Basic books.  Don Norman. 2013. The design of everyday things: Revised and expanded edition. Basic books."},{"key":"e_1_3_2_1_40_1","unstructured":"The\u00a0Council of Europe. 2021. Common European Framework of Reference for Languages: Learning Teaching Assessment (CEFR). https:\/\/www.coe.int\/en\/web\/common-european-framework-reference-languages.  The\u00a0Council of Europe. 2021. Common European Framework of Reference for Languages: Learning Teaching Assessment (CEFR). https:\/\/www.coe.int\/en\/web\/common-european-framework-reference-languages."},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753685"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753685"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPCC.2010.5529815"},{"key":"e_1_3_2_1_44_1","volume-title":"Otter Voice Meeting Notes - Otter.ai. https:\/\/otter.ai\/.","year":"2020","unstructured":"Otter.ai. 2020 . Otter Voice Meeting Notes - Otter.ai. https:\/\/otter.ai\/. Otter.ai. 2020. Otter Voice Meeting Notes - Otter.ai. https:\/\/otter.ai\/."},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098279.3098558"},{"key":"e_1_3_2_1_46_1","volume-title":"Handbook of usability testing: how to plan, design and conduct effective tests","author":"Rubin Jeffrey","unstructured":"Jeffrey Rubin and Dana Chisnell . 2008. Handbook of usability testing: how to plan, design and conduct effective tests . John Wiley & Sons . Jeffrey Rubin and Dana Chisnell. 2008. Handbook of usability testing: how to plan, design and conduct effective tests. John Wiley & Sons."},{"key":"e_1_3_2_1_47_1","unstructured":"Congressional\u00a0Research Service. 2020. H-2A and H-2B Temporary Worker Visas: Policy and Related Issues. https:\/\/fas.org\/sgp\/crs\/homesec\/R44849.pdf.  Congressional\u00a0Research Service. 2020. H-2A and H-2B Temporary Worker Visas: Policy and Related Issues. https:\/\/fas.org\/sgp\/crs\/homesec\/R44849.pdf."},{"key":"e_1_3_2_1_48_1","volume-title":"From \u201dthought and language","author":"Slobin Dan","unstructured":"Dan Slobin . 1996. From \u201dthought and language \u201d to \u201dthinking for speaking\u201d. Rethinking linguistic relativity. Cambridge : Cambridge University Press . Dan Slobin. 1996. From \u201dthought and language\u201d to \u201dthinking for speaking\u201d. Rethinking linguistic relativity. Cambridge: Cambridge University Press."},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1029533.1029567"},{"key":"e_1_3_2_1_50_1","unstructured":"UserTesting. 2021. UserTesting: The Human Insight Platform. https:\/\/www.usertesting.com\/.  UserTesting. 2021. UserTesting: The Human Insight Platform. https:\/\/www.usertesting.com\/."},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1868914.1868979"},{"key":"e_1_3_2_1_52_1","volume-title":"The impact of two different think-aloud instructions in a usability test: a case of just following orders?Behaviour & Information Technology 33, 2","author":"Zhao Tingting","year":"2014","unstructured":"Tingting Zhao , Sharon McDonald , and Helen\u00a0 M Edwards . 2014. The impact of two different think-aloud instructions in a usability test: a case of just following orders?Behaviour & Information Technology 33, 2 ( 2014 ), 163\u2013183. Tingting Zhao, Sharon McDonald, and Helen\u00a0M Edwards. 2014. The impact of two different think-aloud instructions in a usability test: a case of just following orders?Behaviour & Information Technology 33, 2 (2014), 163\u2013183."}],"event":{"name":"Chinese CHI 2021: The Ninth International Symposium of Chinese CHI","location":"Online Hong Kong","acronym":"Chinese CHI 2021"},"container-title":["The Ninth International Symposium of Chinese CHI"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3490355.3490358","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3490355.3490358","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:43Z","timestamp":1750188643000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3490355.3490358"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,16]]},"references-count":51,"alternative-id":["10.1145\/3490355.3490358","10.1145\/3490355"],"URL":"https:\/\/doi.org\/10.1145\/3490355.3490358","relation":{},"subject":[],"published":{"date-parts":[[2021,10,16]]},"assertion":[{"value":"2022-02-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}