{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,17]],"date-time":"2026-01-17T20:18:22Z","timestamp":1768681102519,"version":"3.49.0"},"reference-count":104,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2025,5,22]],"date-time":"2025-05-22T00:00:00Z","timestamp":1747872000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Department of Research and Universities of the Generalitat de Catalunya","award":["AGAUR 2023 DI060"],"award-info":[{"award-number":["AGAUR 2023 DI060"]}]},{"name":"Horizon Europe's European Innovation Council","award":["101071147"],"award-info":[{"award-number":["101071147"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,5,22]]},"abstract":"<jats:p>While Large Language Models (LLMs) have significantly advanced natural language processing, aligning them with human preferences remains an open challenge. Although current alignment methods rely primarily on explicit feedback, eye-tracking (ET) data offers insights into real-time cognitive processing during reading. In this paper, we present OASST-ETC, a novel eye-tracking corpus capturing reading patterns from 24 participants, while evaluating LLM-generated responses from the OASST1 dataset. Our analysis reveals distinct reading patterns between preferred and non-preferred responses, which we compare with synthetic eye-tracking data. Furthermore, we examine the correlation between human reading measures and attention patterns from various transformer-based models, discovering stronger correlations in preferred responses. This work introduces a unique resource for studying human cognitive processing in LLM evaluation and suggests promising directions for incorporating eye-tracking data into alignment methods. The dataset and analysis code are publicly available.<\/jats:p>","DOI":"10.1145\/3725840","type":"journal-article","created":{"date-parts":[[2025,5,22]],"date-time":"2025-05-22T18:18:53Z","timestamp":1747937933000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-9785-6740","authenticated-orcid":false,"given":"Angela","family":"Lopez-Cardona","sequence":"first","affiliation":[{"name":"Telef\u00f3nica Scientific Research, Barcelona, Spain and Universitat Polit\u00e8cnica de Catalunya, Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8158-1673","authenticated-orcid":false,"given":"Sebastian","family":"Idesis","sequence":"additional","affiliation":[{"name":"Telef\u00f3nica Scientific Research, Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5056-7633","authenticated-orcid":false,"given":"Miguel","family":"Barreda-\u00c1ngeles","sequence":"additional","affiliation":[{"name":"Telef\u00f3nica Scientific Research, Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0941-0260","authenticated-orcid":false,"given":"Sergi","family":"Abadal","sequence":"additional","affiliation":[{"name":"Universitat Polit\u00e8cnica de Catalunya, Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4212-0890","authenticated-orcid":false,"given":"Ioannis","family":"Arapakis","sequence":"additional","affiliation":[{"name":"Telef\u00f3nica Scientific Research, Barcelona, Spain"}]}],"member":"320","published-online":{"date-parts":[[2025,5,22]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3379156.3391335"},{"key":"e_1_2_1_2_1","unstructured":"Meta AI. 2024. llama3\/MODEL_CARD.md at main \u00b7 meta-llama\/llama3. https:\/\/github.com\/meta-llama\/llama3\/blob\/main\/MODEL_CARD.md"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3660795"},{"key":"e_1_2_1_4_1","first-page":"129","volume-title":"Behavioural Decision Theory: George Wright","author":"Anand Paul","year":"1984","unstructured":"Paul Anand. 1988. Behavioural Decision Theory: George Wright, Penguin Books, Harmondsworth, 1984. pp. 129. \u00a33.95. Journal of Economic Psychology, Vol. 9, 1 (1988), 115--117."},{"key":"e_1_2_1_5_1","unstructured":"Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell Jackson Kernion Andy Jones Anna Chen Anna Goldie Azalia Mirhoseini Cameron McKinnon Carol Chen Catherine Olsson Christopher Olah Danny Hernandez Dawn Drain Deep Ganguli Dustin Li Eli Tran-Johnson Ethan Perez Jamie Kerr Jared Mueller Jeffrey Ladish Joshua Landau Kamal Ndousse Kamile Lukosuite Liane Lovitt Michael Sellitto Nelson Elhage Nicholas Schiefer Noemi Mercado Nova DasSarma Robert Lasenby Robin Larson Sam Ringer Scott Johnston Shauna Kravec Sheer El Showk Stanislav Fort Tamera Lanham Timothy Telleen-Lawton Tom Conerly Tom Henighan Tristan Hume Samuel R. Bowman Zac Hatfield-Dodds Ben Mann Dario Amodei Nicholas Joseph Sam McCandlish Tom Brown and Jared Kaplan. 2022. Constitutional AI: Harmlessness from AI Feedback. http:\/\/arxiv.org\/abs\/2212.08073 arXiv:2212.08073 [cs]."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1037\/0003-066X.54.7.462"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K18-1030"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.cmcl-1.9"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1162\/opmi_a_00054"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.960"},{"key":"e_1_2_1_11_1","volume-title":"David Robert Reich, Tannon Kew, and Lena Ann J\u00e4ger.","author":"Bolliger Lena Sophia","year":"2024","unstructured":"Lena Sophia Bolliger, Patrick Haller, Isabelle Caroline Rose Cretton, David Robert Reich, Tannon Kew, and Lena Ann J\u00e4ger. 2024. EMTeC: A Corpus of Eye Movements on Machine-Generated Texts. http:\/\/arxiv.org\/abs\/2408.04289 arXiv:2408.04289 [cs]."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.2307\/2334029"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-021-01554-0"},{"key":"e_1_2_1_14_1","unstructured":"Stephen Casper Xander Davies Claudia Shi Thomas Krendl Gilbert J\u00e9r\u00e9my Scheurer Javier Rando Rachel Freedman Tomasz Korbak David Lindner Pedro Freire Tony Wang Samuel Marks Charbel-Rapha\u00ebl Segerie Micah Carroll Andi Peng Phillip Christoffersen Mehul Damani Stewart Slocum Usman Anwar Anand Siththaranjan Max Nadeau Eric J Michaud Jacob Pfau Dmitrii Krasheninnikov Xin Chen Lauro Langosco Peter Hase Erdem Biyik Anca Dragan David Krueger Dorsa Sadigh and Dylan Hadfield-Menell. 2023. Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback. Transactions on Machine Learning Research (2023)."},{"key":"e_1_2_1_15_1","unstructured":"Toon Colman Margot Fonteyne Joke Daems Nicolas Dirix and Lieve Macken. 2022. GECO-MT: The Ghent Eye-tracking Corpus of Machine Translation. In Proceedings of the Thirteenth Language Resources and Evaluation Conference Nicoletta Calzolari Fr\u00e9d\u00e9ric B\u00e9chet Philippe Blache Khalid Choukri Christopher Cieri Thierry Declerck Sara Goggi Hitoshi Isahara Bente Maegaard Joseph Mariani H\u00e9l\u00e8ne Mazo Jan Odijk and Stelios Piperidis (Eds.). European Language Resources Association Marseille France 29--38. https:\/\/aclanthology.org\/2022.lrec-1.4"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-016-0734-0"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 41st International Conference on Machine Learning (ICML'24","volume":"9744","author":"Cui Ganqu","year":"2024","unstructured":"Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024. ULTRAFEEDBACK: boosting language models with scaled AI feedback. In Proceedings of the 41st International Conference on Machine Learning (ICML'24, Vol. 235). JMLR.org, Vienna, Austria, 9722--9744."},{"key":"e_1_2_1_18_1","volume-title":"Validation of Gazepoint low-cost eye-tracking and psychophysiology bundle. Behavior research methods","author":"Cuve H\u00e9lio Clemente","year":"2022","unstructured":"H\u00e9lio Clemente Cuve, Jelka Stojanov, Xavier Roberts-Gaal, Caroline Catmur, and Geoffrey Bird. 2022. Validation of Gazepoint low-cost eye-tracking and psychophysiology bundle. Behavior research methods, Vol. 54, 2 (2022), 1027--1049."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.400"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-short.21"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3591131"},{"key":"e_1_2_1_22_1","unstructured":"Tim Dettmers Artidoro Pagnoni Ari Holtzman and Luke Zettlemoyer. 2023. QLoRA: Efficient Finetuning of Quantized LLMs. https:\/\/openreview.net\/forum?id=OUIFPHEgJU&referrer=%5Bthe%20profile%20of%20Ari%20Holtzman%5D(%2Fprofile%3Fid%3D Ari_Holtzman1)"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19--1423"},{"key":"e_1_2_1_24_1","unstructured":"Xiao Ding Bowen Chen Li Du Bing Qin and Ting Liu. 2022. CogBERT: Cognition-Guided Pre-trained Language Models. In Proceedings of the 29th International Conference on Computational Linguistics Nicoletta Calzolari Chu-Ren Huang Hansaem Kim James Pustejovsky Leo Wanner Key-Sun Choi Pum-Mo Ryu Hsin-Hsi Chen Lucia Donatelli Heng Ji Sadao Kurohashi Patrizia Paggio Nianwen Xue Seokhwan Kim Younggyun Hahm Zhong He Tony Kyungil Lee Enrico Santus Francis Bond and Seung-Hoon Na (Eds.). International Committee on Computational Linguistics Gyeongju Republic of Korea 3210--3225. https:\/\/aclanthology.org\/2022.coling-1.284"},{"key":"e_1_2_1_25_1","volume-title":"RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment","author":"Dong Hanze","year":"2023","unstructured":"Hanze Dong, Wei Xiong, Deepanshu Goyal, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. 2023b. RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment. http:\/\/arxiv.org\/abs\/2304.06767 arXiv:2304.06767 [cs, stat]."},{"key":"e_1_2_1_26_1","volume-title":"Xianchao Wu, and Oleksii Kuchaiev.","author":"Dong Yi","year":"2023","unstructured":"Yi Dong, Zhilin Wang, Makesh Narsimhan Sreedhar, Xianchao Wu, and Oleksii Kuchaiev. 2023a. SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF. http:\/\/arxiv.org\/abs\/2310.05344 arXiv:2310.05344 [cs]."},{"key":"e_1_2_1_27_1","unstructured":"Nicolai Dorka. 2024. Quantile Regression for Distributional Reward Models in RLHF. http:\/\/arxiv.org\/abs\/2409.10164 arXiv:2409.10164."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","unstructured":"Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Amy Yang Angela Fan and et al. 2024. The Llama 3 Herd of Models. https:\/\/doi.org\/10.48550\/arXiv.2407.21783 arXiv:2407.21783.","DOI":"10.48550\/arXiv.2407.21783"},{"key":"e_1_2_1_29_1","volume-title":"Eye tracking methodology: Theory and practice","author":"Duchowski Andrew T","unstructured":"Andrew T Duchowski. 2017. Eye tracking methodology: Theory and practice. Springer."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-023-02187--1"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.296"},{"key":"e_1_2_1_32_1","unstructured":"Google. 2023. Google Bard - Herramienta de IA Generativa y Bot Conversacional. https:\/\/bard.google.com"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.31234\/osf.io\/muv4q"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the Second Workshop on Linguistic and Neurocognitive Resources, Emmanuele Chersoni, Barry Devereux, and Chu-Ren Huang (Eds.). European Language Resources Association","author":"Hollenstein Nora","year":"2020","unstructured":"Nora Hollenstein, Maria Barrett, and Lisa Beinborn. 2020a. Towards Best Practices for Leveraging Human Language Processing Signals for Natural Language Processing. In Proceedings of the Second Workshop on Linguistic and Neurocognitive Resources, Emmanuele Chersoni, Barry Devereux, and Chu-Ren Huang (Eds.). European Language Resources Association, Marseille, France, 15--27. https:\/\/aclanthology.org\/2020.lincr-1.3"},{"key":"e_1_2_1_35_1","unstructured":"Nora Hollenstein Maria Barrett and Marina Bj\u00f6rnsd\u00f3ttir. 2022a. The Copenhagen Corpus of Eye Tracking Recordings from Natural Reading of Danish Texts. In Proceedings of the Thirteenth Language Resources and Evaluation Conference Nicoletta Calzolari Fr\u00e9d\u00e9ric B\u00e9chet Philippe Blache Khalid Choukri Christopher Cieri Thierry Declerck Sara Goggi Hitoshi Isahara Bente Maegaard Joseph Mariani H\u00e9l\u00e8ne Mazo Jan Odijk and Stelios Piperidis (Eds.). European Language Resources Association Marseille France 1712--1720. https:\/\/aclanthology.org\/2022.lrec-1.182"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","unstructured":"Nora Hollenstein Maria Barrett Marius Troendle Francesco Bigiolli Nicolas Langer and Ce Zhang. 2019. Advancing NLP with Cognitive Language Processing Signals. https:\/\/doi.org\/10.48550\/arXiv.1904.02682 arXiv:1904.02682 [cs].","DOI":"10.48550\/arXiv.1904.02682"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-short.19"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.cmcl-1.14"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.cmcl-1.7"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.10"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2018.291"},{"key":"e_1_2_1_42_1","unstructured":"Nora Hollenstein Marius Troendle Ce Zhang and Nicolas Langer. 2020b. ZuCo 2.0: A Dataset of Physiological Recordings During Natural Reading and Annotation. In Proceedings of the Twelfth Language Resources and Evaluation Conference Nicoletta Calzolari Fr\u00e9d\u00e9ric B\u00e9chet Philippe Blache Khalid Choukri Christopher Cieri Thierry Declerck Sara Goggi Hitoshi Isahara Bente Maegaard Joseph Mariani H\u00e9l\u00e8ne Mazo Asuncion Moreno Jan Odijk and Stelios Piperidis (Eds.). European Language Resources Association Marseille France 138--146. https:\/\/aclanthology.org\/2020.lrec-1.18"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19--1001"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.321"},{"key":"e_1_2_1_45_1","volume-title":"Longer Fixations","author":"Huang Xinting","unstructured":"Xinting Huang, Jiajing Wan, Ioannis Kritikos, and Nora Hollenstein. 2023. Longer Fixations, More Computation: Gaze-Guided Recurrent Neural Networks. http:\/\/arxiv.org\/abs\/2311.00159 arXiv:2311.00159 [cs]."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2403.00506"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3649902.3655658"},{"key":"e_1_2_1_48_1","volume-title":"Juntao Dai, Xuehai Pan, Aidan O'Gara, Yingshan Lei, Hua Xu, Brian Tse, Jie Fu, Stephen McAleer, Yaodong Yang, Yizhou Wang, Song-Chun Zhu, Yike Guo, and Wen Gao.","author":"Ji Jiaming","year":"2024","unstructured":"Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang, Fanzhi Zeng, Kwan Yee Ng, Juntao Dai, Xuehai Pan, Aidan O'Gara, Yingshan Lei, Hua Xu, Brian Tse, Jie Fu, Stephen McAleer, Yaodong Yang, Yizhou Wang, Song-Chun Zhu, Yike Guo, and Wen Gao. 2024. AI Alignment: A Comprehensive Survey. http:\/\/arxiv.org\/abs\/2310.19852 arXiv:2310.19852 [cs]."},{"key":"e_1_2_1_49_1","unstructured":"Albert Q. Jiang Alexandre Sablayrolles Arthur Mensch Chris Bamford Devendra Singh Chaplot Diego de las Casas Florian Bressand Gianna Lengyel Guillaume Lample Lucile Saulnier L\u00e9lio Renard Lavaud Marie-Anne Lachaux Pierre Stock Teven Le Scao Thibaut Lavril Thomas Wang Timoth\u00e9e Lacroix and William El Sayed. 2023. Mistral 7B. http:\/\/arxiv.org\/abs\/2310.06825 arXiv:2310.06825 [cs]."},{"key":"e_1_2_1_50_1","unstructured":"Tianhao Wu * Hanlin Zhu and Jiantao Banghua Zhu * Jiao Evan Frick *. 2023. Starling-7B: Increasing LLM Helpfulness & Harmlessness with RLAIF. https:\/\/starling.cs.berkeley.edu"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.eacl-main.139"},{"key":"e_1_2_1_52_1","volume-title":"Ryan Cotterell, Lena Ann J\u00e4ger, and Ethan Wilcox.","author":"Kiegeland Samuel","year":"2024","unstructured":"Samuel Kiegeland, David Robert Reich, Ryan Cotterell, Lena Ann J\u00e4ger, and Ethan Wilcox. 2024. The Pupil Becomes the Master: Eye-Tracking Feedback for Tuning LLMs. https:\/\/openreview.net\/forum?id=8oLUcBgKua"},{"key":"e_1_2_1_53_1","volume-title":"Oliver Stanley, Rich\u00e1rd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick.","author":"K\u00f6pf Andreas","year":"2023","unstructured":"Andreas K\u00f6pf, Yannic Kilcher, Dimitri von R\u00fctte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Rich\u00e1rd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. 2023. OpenAssistant Conversations -- Democratizing Large Language Model Alignment. http:\/\/arxiv.org\/abs\/2304.07327 arXiv:2304.07327 [cs]."},{"key":"e_1_2_1_54_1","volume-title":"Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi.","author":"Lambert Nathan","year":"2024","unstructured":"Nathan Lambert, Valentina Pyatkin, Jacob Morrison, L. J. Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. 2024. RewardBench: Evaluating Reward Models for Language Modeling. http:\/\/arxiv.org\/abs\/2403.13787 arXiv:2403.13787 [cs]."},{"key":"e_1_2_1_55_1","volume-title":"RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback","author":"Lee Harrison","year":"2023","unstructured":"Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, and Abhinav Rastogi. 2023. RLAIF: Scaling Reinforcement Learning from Human Feedback with AI Feedback. http:\/\/arxiv.org\/abs\/2309.00267 arXiv:2309.00267 [cs]."},{"key":"e_1_2_1_56_1","volume-title":"Modeling User Preferences via Brain-Computer Interfacing. arXiv preprint arXiv:2405.09691","author":"Leiva Luis A","year":"2024","unstructured":"Luis A Leiva, Javier Ttraver, Alexandra Kawala-Sterniuk, and Tuukka Ruotsalo. 2024. Modeling User Preferences via Brain-Computer Interfacing. arXiv preprint arXiv:2405.09691 (2024)."},{"key":"e_1_2_1_57_1","volume-title":"HRLAIF: Improvements in Helpfulness and Harmlessness in Open-domain Reinforcement Learning From AI Feedback","author":"Li Ang","year":"2024","unstructured":"Ang Li, Qiugen Xiao, Peng Cao, Jian Tang, Yi Yuan, Zijie Zhao, Xiaoyuan Chen, Liang Zhang, Xiangyang Li, Kaitong Yang, Weidong Guo, Yukang Gan, Xu Yu, Daniell Wang, and Ying Shan. 2024b. HRLAIF: Improvements in Helpfulness and Harmlessness in Open-domain Reinforcement Learning From AI Feedback. http:\/\/arxiv.org\/abs\/2403.08309 arXiv:2403.08309 [cs]."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.cmcl-1.9"},{"key":"e_1_2_1_59_1","unstructured":"Peizhao Li Junfeng He Gang Li Rachit Bhargava Shaolei Shen Nachiappan Valliappan Youwei Liang Hongxiang Gu Venky Ramachandran Golnaz Farhadi Yang Li Kai J. Kohlhoff and Vidhya Navalpakkam. 2024a. UniAR: A Unified model for predicting human Attention and Responses on visual content. https:\/\/openreview.net\/forum?id=FjssnGuHih&noteId=n31VtGnAO9"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2309.05463"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. https:\/\/doi.org\/10.48550\/arXiv.1907.11692 arXiv:1907.11692 [cs].","DOI":"10.48550\/arXiv.1907.11692"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","unstructured":"Angela Lopez-Cardona Carlos Segura Alexandros Karatzoglou Sergi Abadal and Ioannis Arapakis. 2024. Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models. https:\/\/doi.org\/10.48550\/arXiv.2410.01532 arXiv:2410.01532 version: 1.","DOI":"10.48550\/arXiv.2410.01532"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-017-0908--4"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.764"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.0956--7976.2005.01549.x"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/683"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1219"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.aacl-main.86"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.21428\/594757db.90170c50"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.9884"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/K16-1016"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12068"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.84.3.231"},{"key":"e_1_2_1_74_1","unstructured":"OpenAI. 2023. GPT-4 Technical Report. https:\/\/doi.org\/10.48550\/arXiv.2303.08774 arXiv:2303.08774 [cs]."},{"key":"e_1_2_1_75_1","volume-title":"Proceedings of the 36th International Conference on Neural Information Processing Systems (NIPS '22)","author":"Ouyang Long","year":"2024","unstructured":"Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2024. Training language models to follow instructions with human feedback. In Proceedings of the 36th International Conference on Neural Information Processing Systems (NIPS '22). Curran Associates Inc., Red Hook, NY, USA, 27730--27744."},{"key":"e_1_2_1_76_1","unstructured":"Aliz\u00e9e Pace Jonathan Mallinson Eric Malmi Sebastian Krause and Aliaksei Severyn. 2024. West-of-N: Synthetic Preference Generation for Improved Reward Modeling. http:\/\/arxiv.org\/abs\/2401.12086 arXiv:2401.12086 [cs]."},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-018-01193-y"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588015.3588410"},{"key":"e_1_2_1_79_1","volume-title":"Advances in Neural Information Processing Systems","volume":"36","author":"Rafailov Rafael","year":"2023","unstructured":"Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. Advances in Neural Information Processing Systems, Vol. 36 (Dec. 2023), 53728--53741. https:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/3517031.3529639"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.291"},{"key":"e_1_2_1_82_1","unstructured":"Tiago Ribeiro Stephanie Brandl Anders S\u00f8gaard and Nora Hollenstein. 2023. WebQAmGaze: A Multilingual Webcam Eye-Tracking-While-Reading Dataset. https:\/\/arxiv.org\/abs\/2303.17876v3"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","unstructured":"John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. https:\/\/doi.org\/10.48550\/arXiv.1707.06347 arXiv:1707.06347 [cs].","DOI":"10.48550\/arXiv.1707.06347"},{"key":"e_1_2_1_84_1","unstructured":"Lingfeng Shen Sihao Chen Linfeng Song Lifeng Jin Baolin Peng Haitao Mi Daniel Khashabi and Dong Yu. 2023. The Trickle-down Impact of Reward Inconsistency on RLHF. https:\/\/openreview.net\/forum?id=MeHmwCDifc"},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-021-01772--6"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.conll-1.2"},{"key":"e_1_2_1_87_1","volume-title":"Advances in Neural Information Processing Systems","volume":"33","author":"Sood Ekta","year":"2020","unstructured":"Ekta Sood, Simon Tannert, Philipp Mueller, and Andreas Bulling. 2020b. Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., 6327--6341. https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/460191c72f67e90150a093b4585e7eb4-Abstract.html"},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale Dan Bikel Lukas Blecher Cristian Canton Ferrer Moya Chen Guillem Cucurull David Esiobu Jude Fernandes Jeremy Fu Wenyin Fu Brian Fuller Cynthia Gao Vedanuj Goswami Naman Goyal Anthony Hartshorn Saghar Hosseini Rui Hou Hakan Inan Marcin Kardas Viktor Kerkez Madian Khabsa Isabel Kloumann Artem Korenev Punit Singh Koura Marie-Anne Lachaux Thibaut Lavril Jenya Lee Diana Liskovich Yinghai Lu Yuning Mao Xavier Martinet Todor Mihaylov Pushkar Mishra Igor Molybog Yixin Nie Andrew Poulton Jeremy Reizenstein Rashi Rungta Kalyan Saladi Alan Schelten Ruan Silva Eric Michael Smith Ranjan Subramanian Xiaoqing Ellen Tan Binh Tang Ross Taylor Adina Williams Jian Xiang Kuan Puxin Xu Zheng Yan Iliyan Zarov Yuchen Zhang Angela Fan Melanie Kambadur Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov and Thomas Scialom. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. https:\/\/doi.org\/10.48550\/arXiv.2307.09288 arXiv:2307.09288 [cs].","DOI":"10.48550\/arXiv.2307.09288"},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-024-09725--8"},{"key":"e_1_2_1_90_1","volume-title":"Proceedings of the Workshop: Bridging Neurons and Symbols for Natural Language Processing and Knowledge Graphs Reasoning (NeusymBridge) @ LREC-COLING-2024","author":"Wang Xintong","year":"2024","unstructured":"Xintong Wang, Xiaoyu Li, Xingshan Li, and Chris Biemann. 2024b. Probing Large Language Models from a Human Behavioral Perspective. In Proceedings of the Workshop: Bridging Neurons and Symbols for Natural Language Processing and Knowledge Graphs Reasoning (NeusymBridge) @ LREC-COLING-2024, Tiansi Dong, Erhard Hinrichs, Zhen Han, Kang Liu, Yangqiu Song, Yixin Cao, Christian F. Hempelmann, and Rafet Sifa (Eds.). ELRA and ICCL, Torino, Italia, 1--7. https:\/\/aclanthology.org\/2024.neusymbridge-1.1"},{"key":"e_1_2_1_91_1","volume-title":"Makesh Narsimhan Sreedhar, and Oleksii Kuchaiev","author":"Wang Zhilin","year":"2024","unstructured":"Zhilin Wang, Yi Dong, Olivier Delalleau, Jiaqi Zeng, Gerald Shen, Daniel Egert, Jimmy J. Zhang, Makesh Narsimhan Sreedhar, and Oleksii Kuchaiev. 2024a. HelpSteer2: Open-source dataset for training top-performing reward models. http:\/\/arxiv.org\/abs\/2406.08673 arXiv:2406.08673 [cs]."},{"key":"e_1_2_1_92_1","volume-title":"Daniel Egert, Olivier Delalleau, Jane Polak Scowcroft, Neel Kant, Aidan Swope, and Oleksii Kuchaiev.","author":"Wang Zhilin","year":"2023","unstructured":"Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Polak Scowcroft, Neel Kant, Aidan Swope, and Oleksii Kuchaiev. 2023. HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM. http:\/\/arxiv.org\/abs\/2311.09528 arXiv:2311.09528 [cs]."},{"key":"e_1_2_1_93_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.cmcl-1.22"},{"key":"e_1_2_1_94_1","unstructured":"Tianhao Wu Weizhe Yuan Olga Golovneva Jing Xu Yuandong Tian Jiantao Jiao Jason Weston and Sainbayar Sukhbaatar. 2024b. Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge. http:\/\/arxiv.org\/abs\/2407.19594 arXiv:2407.19594 [cs]."},{"key":"e_1_2_1_95_1","unstructured":"Tianhao Wu Banghua Zhu Ruoyu Zhang Zhaojin Wen Kannan Ramchandran and Jiantao Jiao. 2023b. Pairwise Proximal Policy Optimization: Harnessing Relative Feedback for LLM Alignment. http:\/\/arxiv.org\/abs\/2310.00212 arXiv:2310.00212 [cs]."},{"key":"e_1_2_1_96_1","unstructured":"Zeqiu Wu Yushi Hu Weijia Shi Nouha Dziri Alane Suhr Prithviraj Ammanabrolu Noah A. Smith Mari Ostendorf and Hannaneh Hajishirzi. 2023a. Fine-Grained Human Feedback Gives Better Rewards for Language Model Training. http:\/\/arxiv.org\/abs\/2306.01693 arXiv:2306.01693 [cs]."},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.7557\/18.6797"},{"key":"e_1_2_1_98_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.12950"},{"key":"e_1_2_1_99_1","volume-title":"CeER: A Nested Name Entity Recognition Model Incorporating Gaze Feature","author":"Yu Jie","unstructured":"Jie Yu, Wenya Kong, and Fangfang Liu. 2024. CeER: A Nested Name Entity Recognition Model Incorporating Gaze Feature. In Web and Big Data, Wenjie Zhang, Anthony Tung, Zhonglong Zheng, Zhengyi Yang, Xiaoyang Wang, and Hongjie Guo (Eds.). Springer Nature Singapore, Singapore, 32--45."},{"key":"e_1_2_1_100_1","doi-asserted-by":"publisher","unstructured":"Lifan Yuan Ganqu Cui Hanbin Wang Ning Ding Xingyao Wang Jia Deng Boji Shan Huimin Chen Ruobing Xie Yankai Lin Zhenghao Liu Bowen Zhou Hao Peng Zhiyuan Liu and Maosong Sun. 2024. Advancing LLM Reasoning Generalists with Preference Trees. https:\/\/doi.org\/10.48550\/arXiv.2404.02078 arXiv:2404.02078.","DOI":"10.48550\/arXiv.2404.02078"},{"key":"e_1_2_1_101_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.05302"},{"key":"e_1_2_1_102_1","volume-title":"Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024","author":"Zermiani Francesca","year":"2024","unstructured":"Francesca Zermiani, Prajit Dhar, Ekta Sood, Fabian K\u00f6gel, Andreas Bulling, and Maria Wirzberger. 2024. InteRead: An Eye Tracking Dataset of Interrupted Reading. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (Eds.). ELRA and ICCL, Torino, Italia, 9154--9169. https:\/\/aclanthology.org\/2024.lrec-main.802"},{"key":"e_1_2_1_103_1","volume-title":"Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024","author":"Zhang Leran","year":"2024","unstructured":"Leran Zhang and Nora Hollenstein. 2024. Eye-Tracking Features Masking Transformer Attention in Question-Answering Tasks. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (Eds.). ELRA and ICCL, Torino, Italy, 7057--7070. https:\/\/aclanthology.org\/2024.lrec-main.619"},{"key":"e_1_2_1_104_1","doi-asserted-by":"publisher","DOI":"10.1145\/3643732"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3725840","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3725840","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,23]],"date-time":"2025-08-23T01:50:59Z","timestamp":1755913859000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3725840"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,22]]},"references-count":104,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,5,22]]}},"alternative-id":["10.1145\/3725840"],"URL":"https:\/\/doi.org\/10.1145\/3725840","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,22]]},"assertion":[{"value":"2025-05-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}