{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,8]],"date-time":"2026-06-08T23:34:52Z","timestamp":1780961692031,"version":"3.54.1"},"reference-count":55,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T00:00:00Z","timestamp":1718323200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001700","name":"MEXT \u201cInnovation Platform for Society 5.0\u201d Program","doi-asserted-by":"publisher","award":["JPMXP0518071489"],"award-info":[{"award-number":["JPMXP0518071489"]}],"id":[{"id":"10.13039\/501100001700","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Subtitles play a crucial role in facilitating the understanding of visual content when watching films and television programs. In this study, we propose a method for presenting subtitles in a way that considers cognitive load when viewing video content in a non-native language. Subtitles are generally displayed at the bottom of the screen, which causes frequent eye focus switching between subtitles and video, increasing the cognitive load. In our proposed method, we focused on the position, display time, and amount of information contained in the subtitles to reduce the cognitive load and to avoid disturbing the viewer\u2019s concentration. We conducted two experiments to investigate the effects of our proposed subtitle method on gaze distribution, comprehension, and cognitive load during English-language video viewing. Twelve non-native English-speaking subjects participated in the first experiment. The results show that participants\u2019 gazes were more focused around the center of the screen when using our proposed subtitles compared to regular subtitles. Comprehension levels recorded using LightSub were similar, but slightly inferior to those recorded using regular subtitles. However, it was confirmed that most of the participants were viewing the video with a higher cognitive load using the proposed subtitle method. In the second experiment, we investigated subtitles considering connected speech form in English with 18 non-native English speakers. The results revealed that the proposed method, considering connected speech form, demonstrated an improvement in cognitive load during video viewing but it remained higher than that of regular subtitles.<\/jats:p>","DOI":"10.3390\/mti8060051","type":"journal-article","created":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T08:02:26Z","timestamp":1718352146000},"page":"51","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["LightSub: Unobtrusive Subtitles with Reduced Information and Decreased Eye Movement"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-2001-7290","authenticated-orcid":false,"given":"Yuki","family":"Nishi","sequence":"first","affiliation":[{"name":"Graduate School of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yugo","family":"Nakamura","sequence":"additional","affiliation":[{"name":"Graduate School of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shogo","family":"Fukushima","sequence":"additional","affiliation":[{"name":"Graduate School of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7156-9160","authenticated-orcid":false,"given":"Yutaka","family":"Arakawa","sequence":"additional","affiliation":[{"name":"Graduate School of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,14]]},"reference":[{"key":"ref_1","first-page":"52","article-title":"Language learning with Netflix: Exploring the effects of dual subtitles on vocabulary learning and listening comprehension","volume":"22","author":"Dizon","year":"2021","journal-title":"Comput. Assist. Lang. Learn."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1111\/j.1468-2958.2005.tb00874.x","article-title":"How attention partitions itself during simultaneous message presentations","volume":"31","author":"Bergen","year":"2005","journal-title":"Hum. Commun. Res."},{"key":"ref_3","unstructured":"Baker, R.G., Lambourne, A.D., and Rowston, G. (1982). Handbook for Television Subtitlers, Independent Broadcasting Authority. Engineering Division."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kurzhals, K., Cetinkaya, E., Hu, Y., Wang, W., and Weiskopf, D. (2017, January 6\u201311). Close to the action: Eye-tracking evaluation of speaker-following subtitles. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.","DOI":"10.1145\/3025453.3025772"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Potter, M.C. (2018). Rapid serial visual presentation (RSVP): A method for studying language processing. New Methods in Reading Comprehension Research, Routledge.","DOI":"10.4324\/9780429505379-5"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Rzayev, R., Wo\u017aniak, P.W., Dingler, T., and Henze, N. (2018, January 21\u201327). Reading on smart glasses: The effect of text position, presentation type and walking. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada.","DOI":"10.1145\/3173574.3173619"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1130","DOI":"10.1177\/1541931213601265","article-title":"RSVP reading on a smart watch","volume":"Volume 60","author":"Gannon","year":"2016","journal-title":"Human Factors and Ergonomics Society Annual Meeting"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"d\u2019Ydewalle, G., Van Rensbergen, J., and Pollet, J. (1987). Reading a message when the same message is available auditorily in another language: The case of subtitling. Eye Movements from Physiology to Cognition, Elsevier.","DOI":"10.1016\/B978-0-444-70113-8.50047-3"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1017\/S0142716412000434","article-title":"Processing of native and foreign language subtitles in films: An eye tracking study","volume":"35","author":"Bisson","year":"2014","journal-title":"Appl. Psycholinguist."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"650","DOI":"10.1177\/009365091018005005","article-title":"Watching subtitled television: Automatic reading behavior","volume":"18","author":"Praet","year":"1991","journal-title":"Commun. Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1167\/13.4.1","article-title":"Eye movements while viewing narrated, captioned, and silent videos","volume":"13","author":"Ross","year":"2013","journal-title":"J. Vis."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Trudgill, P., and Hannah, J. (2013). International English: A Guide to the Varieties of Standard English, Routledge.","DOI":"10.4324\/9780203785225"},{"key":"ref_13","unstructured":"Webb, S. (2015). Extensive viewing: Language learning through watching television. Language Learning Beyond the Classroom, Routledge."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1080\/09571736.2019.1616806","article-title":"Extensive viewing of captioned and subtitled TV series: A study of L2 vocabulary learning by adolescents","volume":"47","author":"Pujadas","year":"2019","journal-title":"Lang. Learn. J."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"590","DOI":"10.17507\/jltr.1303.15","article-title":"Comparing L2 incidental vocabulary learning through viewing, listening, and reading","volume":"13","author":"Aldukhayel","year":"2022","journal-title":"J. Lang. Teach. Res."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Renandya, W.A., and Jacobs, G.M. (2016). Extensive Reading and Listening in the L2 Classroom, Springer.","DOI":"10.1007\/978-3-319-38834-2_8"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1016\/j.sbspro.2016.10.068","article-title":"The role of video materials in EFL classrooms","volume":"232","author":"Bajrami","year":"2016","journal-title":"Procedia-Soc. Behav. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chang, A.C.S. (2016). Teaching L2 listening: In and outside the classroom. English Language Teaching Today: Linking Theory and Practice, Springer.","DOI":"10.1007\/978-3-319-38834-2_9"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1017\/S0272263119000494","article-title":"Learning vocabulary through reading, listening, and viewing: Which mode of input is most effective?","volume":"42","author":"Feng","year":"2020","journal-title":"Stud. Second. Lang. Acquis."},{"key":"ref_20","first-page":"79","article-title":"Effective approaches to teaching listening: Chinese EFL teachers\u2019 perspectives","volume":"9","author":"Li","year":"2012","journal-title":"J. Asia Tefl"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1080\/10904018.2019.1582346","article-title":"Can L2 phonological vocabulary knowledge and listening comprehension be developed through extensive movie viewing? The case of Arab EFL learners","volume":"34","author":"Masrai","year":"2020","journal-title":"Int. J. List."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.sbspro.2014.07.516","article-title":"An investigation of using video vs. audio for teaching vocabulary","volume":"143","year":"2014","journal-title":"Procedia-Soc. Behav. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1093\/elt\/ccq015","article-title":"\u2018Teacher, the tape is too fast!\u2019 Extensive listening in ELT","volume":"65","author":"Renandya","year":"2011","journal-title":"ELT J."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Graham, S., and Santos, D. (2015). Strategies for Second Language Listening: Current Scenarios and Improved Pedagogy, Springer.","DOI":"10.1057\/9781137410528"},{"key":"ref_25","first-page":"25","article-title":"The Application of Video Clips with Small Group and Individual Activities to Improve Young Learners\u2019 Speaking Performance","volume":"17","author":"Muslem","year":"2017","journal-title":"Teach. Engl. Technol."},{"key":"ref_26","first-page":"167","article-title":"Authentic Captioned Sitcom as Listening Comprehension Material in English Language Teaching","volume":"9","year":"2020","journal-title":"ELT Res. J."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1080\/15213269.2013.826119","article-title":"The impact of subtitles on comprehension of narrative film","volume":"16","author":"Lee","year":"2013","journal-title":"Media Psychol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/j.system.2016.10.002","article-title":"Learning vocabulary through audiovisual input: The differential effect of L1 subtitles and captions","volume":"63","author":"Peters","year":"2016","journal-title":"System"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"237","DOI":"10.15639\/teflinjournal.v30i2\/237-256","article-title":"Extensive listening and viewing in ELT","volume":"30","author":"Ivone","year":"2019","journal-title":"Teflin J."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1111\/j.1467-8535.2009.01004.x","article-title":"The effect of films with and without subtitles on listening comprehension of EFL learners","volume":"42","author":"Hayati","year":"2011","journal-title":"Br. J. Educ. Technol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1093\/elt\/42.4.272","article-title":"The value of teletext sub-titles in language learning","volume":"42","author":"Vanderplank","year":"1988","journal-title":"ELT J."},{"key":"ref_32","first-page":"10","article-title":"A very verbal medium: Language learning through closed captions","volume":"3","author":"Vanderplank","year":"1993","journal-title":"TESOL J."},{"key":"ref_33","first-page":"9","article-title":"Closed-Captioned Television: A New Technology for Enhancing Reading Skills of Learning Disabled Students","volume":"4","author":"Koskinen","year":"1986","journal-title":"Spectrum"},{"key":"ref_34","first-page":"1","article-title":"Effects of English captions in relation to learner proficiency level and type of information","volume":"30","author":"Hirose","year":"1993","journal-title":"Lang. Lab."},{"key":"ref_35","first-page":"38","article-title":"The effects of captioned television videotapes on the listening comprehension of beginning, intermediate, and advanced ESL students","volume":"29","author":"Markham","year":"1989","journal-title":"Educ. Technol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1017\/CBO9780511816819.004","article-title":"Cognitive theory of multimedia learning","volume":"41","author":"Mayer","year":"2005","journal-title":"Camb. Handb. Multimed. Learn."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1037\/0022-0663.94.1.156","article-title":"Verbal redundancy in multimedia learning: When reading helps listening","volume":"94","author":"Moreno","year":"2002","journal-title":"J. Educ. Psychol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1002\/(SICI)1099-0720(199908)13:4<351::AID-ACP589>3.0.CO;2-6","article-title":"Managing split-attention and redundancy in multimedia instruction","volume":"13","author":"Kalyuga","year":"1999","journal-title":"Appl. Cogn. Psychol."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"262","DOI":"10.3758\/BF03196973","article-title":"Conceptual processing of text during skimming and rapid sequential reading","volume":"11","author":"Masson","year":"1983","journal-title":"Mem. Cogn."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"215","DOI":"10.3758\/BF03210208","article-title":"Visual perception of rapidly presented word sequences of varying complexity","volume":"8","author":"Forster","year":"1970","journal-title":"Percept. Psychophys."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Juola, J.F., Haugh, D., Trast, S., Ferraro, F.R., and Liebhaber, M. (1987). Reading with and without eye movements. Eye Movements from Physiology to Cognition, Elsevier.","DOI":"10.1016\/B978-0-444-70113-8.50071-0"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"372","DOI":"10.1037\/0033-2909.124.3.372","article-title":"Eye movements in reading and information processing: 20 years of research","volume":"124","author":"Rayner","year":"1998","journal-title":"Psychol. Bull."},{"key":"ref_43","first-page":"27","article-title":"Effects of RSVP display design on visual performance in accomplishing dual tasks with small screens","volume":"1","author":"Chen","year":"2007","journal-title":"Int. J. Des."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Kosch, T., Schmidt, A., Thanheiser, S., and Chuang, L.L. (2020, January 25\u201330). One does not simply RSVP: Mental workload to select speed reading parameters using electroencephalography. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3313831.3376766"},{"key":"ref_45","first-page":"153","article-title":"RU ready 4 new subtitles? Investigating the potential of social translation practices and creative spellings","volume":"10","year":"2011","journal-title":"Linguist. Antverp. New-Ser. Themes Transl. Stud."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Vy, Q.V., and Fels, D.I. (2010, January 14\u201316). Using Placement and Name for Speaker Identification in Captioning. Proceedings of the 12th International Conference on Computers Helping People with Special Needs: Part I, Vienna, Austria.","DOI":"10.1007\/978-3-642-14097-6_40"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Foerster, A. (2010). Towards a creative approach in subtitling: A case study. New Insights into Audiovisual Translation and Media Accessibility, Brill.","DOI":"10.1163\/9789042031814_008"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Brown, A., Jones, R., Crabb, M., Sandford, J., Brooks, M., Armstrong, M., and Jay, C. (2015, January 3\u20135). Dynamic subtitles: The user experience. Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, Brussels, Belgium.","DOI":"10.1145\/2745197.2745204"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1037\/h0055392","article-title":"The information capacity of the human motor system in controlling the amplitude of movement","volume":"47","author":"Fitts","year":"1954","journal-title":"J. Exp. Psychol."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1002\/9781118976005.ch17","article-title":"Fitts\u2019 law","volume":"1","author":"MacKenzie","year":"2018","journal-title":"Wiley Handb. Hum. Comput. Interact."},{"key":"ref_51","first-page":"49","article-title":"Evaluating the lexical difficulty of teaching materials with NWLC","volume":"2","author":"Milliner","year":"2022","journal-title":"ELF 2"},{"key":"ref_52","unstructured":"Council of Europe. Council for Cultural Co-operation. Education Committee. Modern Languages Division (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment, Cambridge University Press."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"904","DOI":"10.1177\/154193120605000909","article-title":"NASA-task load index (NASA-TLX); 20 years later","volume":"Volume 50","author":"Hart","year":"2006","journal-title":"Human Factors and Ergonomics Society Annual Meeting"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Carley, P., and Mees, I. (2019). American English Phonetics and Pronunciation Practice, Routledge.","DOI":"10.4324\/9780429492228"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Carley, P., and Mees, I.M. (2021). British English Phonetic Transcription, Routledge.","DOI":"10.4324\/9781003007890"}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/8\/6\/51\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:58:38Z","timestamp":1760108318000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/8\/6\/51"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,14]]},"references-count":55,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["mti8060051"],"URL":"https:\/\/doi.org\/10.3390\/mti8060051","relation":{},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,14]]}}}