{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T10:34:29Z","timestamp":1777113269920,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":61,"publisher":"ACM","license":[{"start":{"date-parts":[[2027,4,26]],"date-time":"2027-04-26T00:00:00Z","timestamp":1808697600000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2331379"],"award-info":[{"award-number":["2331379"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,4,27]]},"DOI":"10.1145\/3785022.3785088","type":"proceedings-article","created":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T09:39:01Z","timestamp":1777109941000},"page":"405-414","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["How to Assess AI Literacy: Misalignment Between Self-Reported and Objective-Based Measures"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-3532-0661","authenticated-orcid":false,"given":"Shan","family":"Zhang","sequence":"first","affiliation":[{"name":"University of Florida, Gainesville, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6461-7611","authenticated-orcid":false,"given":"Ruiwei","family":"Xiao","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7373-4959","authenticated-orcid":false,"given":"Anthony F.","family":"Botelho","sequence":"additional","affiliation":[{"name":"University of Florida, Gainesville, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5758-3396","authenticated-orcid":false,"given":"Guanze","family":"Liao","sequence":"additional","affiliation":[{"name":"National Tsing Hua University, Hsinchu, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2887-5477","authenticated-orcid":false,"given":"Thomas K. F.","family":"Chiu","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, Hong Kong"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2291-1468","authenticated-orcid":false,"given":"John","family":"Stamper","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5850-4768","authenticated-orcid":false,"given":"Kenneth R.","family":"Koedinger","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"}]}],"member":"320","published-online":{"date-parts":[[2026,4,26]]},"reference":[{"key":"e_1_3_3_2_2_2","doi-asserted-by":"crossref","unstructured":"Allison\u00a0J Ames and Randall\u00a0D Penfield. 2015. An NCME instructional module on item-fit statistics for Item Response Theory models. Educational Measurement: Issues and Practice 34 3 (2015) 39\u201348.","DOI":"10.1111\/emip.12067"},{"key":"e_1_3_3_2_3_2","volume-title":"The basics of item response theory","author":"Baker Frank\u00a0B","year":"2001","unstructured":"Frank\u00a0B Baker. 2001. The basics of item response theory. ERIC."},{"key":"e_1_3_3_2_4_2","doi-asserted-by":"crossref","unstructured":"Arne Bewersdorff Claudia Nerdel and Xiaoming Zhai. 2025. How AI literacy correlates with affective behavioral cognitive and contextual variables: A systematic review. Computers and Education: Artificial Intelligence (2025) 100493.","DOI":"10.1016\/j.caeai.2025.100493"},{"key":"e_1_3_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i21.30351"},{"key":"e_1_3_3_2_6_2","doi-asserted-by":"crossref","unstructured":"William\u00a0J Boone. 2016. Rasch analysis for instrument development: why when and how? CBE\u2014Life Sciences Education 15 4 (2016) rm4.","DOI":"10.1187\/cbe.16-04-0148"},{"key":"e_1_3_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-99-7947-9_20"},{"key":"e_1_3_3_2_8_2","volume-title":"Confirmatory factor analysis for applied research","author":"Brown Timothy\u00a0A","year":"2015","unstructured":"Timothy\u00a0A Brown. 2015. Confirmatory factor analysis for applied research. Guilford publications."},{"key":"e_1_3_3_2_9_2","doi-asserted-by":"crossref","unstructured":"Astrid Carolus Martin\u00a0J Koch Samantha Straka Marc\u00a0Erich Latoschik and Carolin Wienrich. 2023. MAILS-Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change-and meta-competencies. Computers in Human Behavior: Artificial Humans 1 2 (2023) 100014.","DOI":"10.1016\/j.chbah.2023.100014"},{"key":"e_1_3_3_2_10_2","unstructured":"Ching\u00a0Sing Chai Pei-Yi Lin Morris Siu-Yung Jong Yun Dai Thomas\u00a0KF Chiu and Jianjun Qin. 2021. Perceptions of and behavioral intentions towards learning artificial intelligence in primary school students. Educational Technology & Society 24 3 (2021) 89\u2013101."},{"key":"e_1_3_3_2_11_2","doi-asserted-by":"crossref","unstructured":"Thomas\u00a0KF Chiu. 2021. A holistic approach to the design of artificial intelligence (AI) education for K-12 schools. TechTrends 65 5 (2021) 796\u2013807.","DOI":"10.1007\/s11528-021-00637-1"},{"key":"e_1_3_3_2_12_2","doi-asserted-by":"crossref","unstructured":"Thomas\u00a0KF Chiu. 2025. Responsible digital citizen: building AI ethics awareness across subjects. 2759\u20132761\u00a0pages.","DOI":"10.1080\/10494820.2025.2494914"},{"key":"e_1_3_3_2_13_2","doi-asserted-by":"crossref","unstructured":"Thomas\u00a0KF Chiu Yifan Chen King\u00a0Woon Yau Ching-sing Chai Helen Meng Irwin King Savio Wong and Yeung Yam. 2024. Developing and validating measures for AI literacy tests: From self-reported to objective measures. Computers and Education: Artificial Intelligence 7 (2024) 100282.","DOI":"10.1016\/j.caeai.2024.100282"},{"key":"e_1_3_3_2_14_2","unstructured":"Code.org. 2025. Teachers \u2013 Free K\u201312 Computer Science & AI Curriculum and Training. Code.org website. https:\/\/code.org\/en-US\/teachers"},{"key":"e_1_3_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781003398172-33"},{"key":"e_1_3_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781410605269"},{"key":"e_1_3_3_2_17_2","unstructured":"European Commission. 2021. Digital Education Action Plan (2021\u20132027). https:\/\/education.ec.europa.eu\/focus-topics\/digital-education\/action-plan. Policy Framework."},{"key":"e_1_3_3_2_18_2","volume-title":"AI Literacy Programs in Europe","author":"Figueiredo T\u00e1nia","year":"2025","unstructured":"T\u00e1nia Figueiredo. 2025. AI Literacy Programs in Europe. Future of Life Institute. EU Artificial Intelligence Act website."},{"key":"e_1_3_3_2_19_2","unstructured":"Google. [n. d.]. AI for Educators \u2013 Grow with Google. https:\/\/grow.google\/ai-for-educators\/. Accessed: 2025-12-09."},{"key":"e_1_3_3_2_20_2","doi-asserted-by":"crossref","unstructured":"Joe\u00a0F Hair Christian\u00a0M Ringle and Marko Sarstedt. 2011. PLS-SEM: Indeed a silver bullet. Journal of Marketing theory and Practice 19 2 (2011) 139\u2013152.","DOI":"10.2753\/MTP1069-6679190202"},{"key":"e_1_3_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v39i28.35186"},{"key":"e_1_3_3_2_22_2","volume-title":"i2 Learning","author":"Learning i2","year":"2025","unstructured":"i2 Learning. 2025. i2 Learning. https:\/\/i2learning.org\/"},{"key":"e_1_3_3_2_23_2","volume-title":"Key Initiatives","author":"(ISTE) International Society for Technology in Education","unstructured":"International Society for Technology in Education (ISTE). [n. d.]. Key Initiatives. Accessed: 2025-09-22."},{"key":"e_1_3_3_2_24_2","doi-asserted-by":"crossref","unstructured":"Mehtab Iqbal Khushbu Singh Sushmita Khan Oluwafemi Osho Emily Sidnam-Mauch Nicole Bannister Kelly Caine and Bart Knijnenburg. 2025. Teaching AI Awareness in Middle School Classrooms: Design Implementation and Evaluation of Two Education Modules on Algorithmic Bias and Filter Bubbles. Computers and Education: Artificial Intelligence (2025) 100425.","DOI":"10.1016\/j.caeai.2025.100425"},{"key":"e_1_3_3_2_25_2","doi-asserted-by":"crossref","unstructured":"Yueqiao Jin Roberto Martinez-Maldonado Dragan Ga\u0161evi\u0107 and Lixiang Yan. 2025. GLAT: The generative AI literacy assessment test. Computers and Education: Artificial Intelligence (2025) 100436.","DOI":"10.1016\/j.caeai.2025.100436"},{"key":"e_1_3_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Henry\u00a0F Kaiser. 1974. An index of factorial simplicity. psychometrika 39 1 (1974) 31\u201336.","DOI":"10.1007\/BF02291575"},{"key":"e_1_3_3_2_27_2","doi-asserted-by":"crossref","unstructured":"Kateryna\u00a0V Keefer. 2015. Self-report assessments of emotional competencies: A critical look at methods and meanings. Journal of Psychoeducational Assessment 33 1 (2015) 3\u201323.","DOI":"10.1177\/0734282914550381"},{"key":"e_1_3_3_2_28_2","doi-asserted-by":"crossref","unstructured":"G\u00a0Frederic Kuder and Marion\u00a0W Richardson. 1937. The theory of the estimation of test reliability. Psychometrika 2 3 (1937) 151\u2013160.","DOI":"10.1007\/BF02288391"},{"key":"e_1_3_3_2_29_2","doi-asserted-by":"crossref","unstructured":"Matthias\u00a0Carl Laupichler Alexandra Aster Nicolas Haverkamp and Tobias Raupach. 2023. Development of the \u201cScale for the assessment of non-experts\u2019 AI literacy\u201d\u2013An exploratory factor analysis. Computers in Human Behavior Reports 12 (2023) 100338.","DOI":"10.1016\/j.chbr.2023.100338"},{"key":"e_1_3_3_2_30_2","doi-asserted-by":"crossref","unstructured":"Matthias\u00a0Carl Laupichler Alexandra Aster and Tobias Raupach. 2023. Delphi study for the development and preliminary validation of an item set for the assessment of non-experts\u2019 AI literacy. Computers and Education: Artificial Intelligence 4 (2023) 100126.","DOI":"10.1016\/j.caeai.2023.100126"},{"key":"e_1_3_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376727"},{"key":"e_1_3_3_2_32_2","unstructured":"Shuai Ma and Zhenzhen Chen. 2024. The development and validation of the artificial intelligence literacy scale for Chinese college students (AILS-CCS). Ieee Access (2024)."},{"key":"e_1_3_3_2_33_2","doi-asserted-by":"crossref","unstructured":"Andr\u00e9 Markus Astrid Carolus and Carolin Wienrich. 2025. Objective Measurement of AI Literacy: Development and Validation of the AI Competency Objective Scale (AICOS). arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2503.12921 (2025).","DOI":"10.1016\/j.caeai.2025.100485"},{"key":"e_1_3_3_2_34_2","doi-asserted-by":"crossref","unstructured":"Alberto Maydeu-Olivares. 2013. Goodness-of-fit assessment of item response theory models. Measurement: Interdisciplinary Research and Perspectives 11 3 (2013) 71\u2013101.","DOI":"10.1080\/15366367.2013.831680"},{"key":"e_1_3_3_2_35_2","volume-title":"AI Competency Framework for Teachers","author":"Miao Fengchun","year":"2024","unstructured":"Fengchun Miao and Mutlu Cukurova. 2024. AI Competency Framework for Teachers. UNESCO. https:\/\/www.unesco.org\/en\/articles\/ai-competency-framework-teachers Published 8 Aug 2024; last updated 18 Aug 2025."},{"key":"e_1_3_3_2_36_2","doi-asserted-by":"crossref","unstructured":"Tamar Mikeladze Paulien\u00a0C Meijer and Roald\u00a0P Verhoeff. 2024. A comprehensive exploration of artificial intelligence competence frameworks for educators: A critical review. European Journal of Education 59 3 (2024) e12663.","DOI":"10.1111\/ejed.12663"},{"key":"e_1_3_3_2_37_2","unstructured":"Kelly Mills Pati Ruiz Keun-woo Lee Merijke Coenraad Judi Fusco Jeremy Roschelle and Josh Weisgrau. 2024. AI Literacy: A Framework to Understand Evaluate and Use Emerging Technology. Digital Promise (2024)."},{"key":"e_1_3_3_2_38_2","unstructured":"Ministry of Education of the People\u2019s Republic of China. 2025. Education Modernization Plan 2024\u20132035. https:\/\/www.gov.cn\/zhengce\/202501\/content_7000579.htm"},{"key":"e_1_3_3_2_39_2","unstructured":"National Academy for Educational Research. 2024. 108 Curriculum Guidelines. https:\/\/www.naer.edu.tw\/eng\/PageSyllabus?fid=148."},{"key":"e_1_3_3_2_40_2","doi-asserted-by":"crossref","unstructured":"Tanya Nazaretsky Moriah Ariely Mutlu Cukurova and Giora Alexandron. 2022. Teachers\u2019 trust in AI-powered educational technology and a professional development program to improve it. British journal of educational technology 53 4 (2022) 914\u2013931.","DOI":"10.1111\/bjet.13232"},{"key":"e_1_3_3_2_41_2","doi-asserted-by":"crossref","unstructured":"Davy Tsz\u00a0Kit Ng Jac Ka\u00a0Lok Leung Samuel Kai\u00a0Wah Chu and Maggie\u00a0Shen Qiao. 2021. Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence 2 (2021) 100041.","DOI":"10.1016\/j.caeai.2021.100041"},{"key":"e_1_3_3_2_42_2","doi-asserted-by":"crossref","unstructured":"Davy Tsz\u00a0Kit Ng Jac Ka\u00a0Lok Leung Jiahong Su Ross Chi\u00a0Wui Ng and Samuel Kai\u00a0Wah Chu. 2023. Teachers\u2019 AI digital competencies and twenty-first century skills in the post-pandemic world. Educational technology research and development 71 1 (2023) 137\u2013161.","DOI":"10.1007\/s11423-023-10203-6"},{"key":"e_1_3_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-18880-0"},{"key":"e_1_3_3_2_44_2","doi-asserted-by":"crossref","unstructured":"Yimin Ning Cheng Zhang Binyan Xu Ying Zhou and Tommy\u00a0Tanu Wijaya. 2024. Teachers\u2019 AI-TPACK: Exploring the relationship between knowledge elements. Sustainability 16 3 (2024) 978.","DOI":"10.3390\/su16030978"},{"key":"e_1_3_3_2_45_2","unstructured":"Yimin Ning Wenjun Zhang Dengming Yao Bowen Fang Binyan Xu and Tommy\u00a0Tanu Wijaya. 2025. Development and validation of the artificial intelligence literacy scale for teachers (AILST). Education and Information Technologies (2025) 1\u201335."},{"key":"e_1_3_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3519866"},{"key":"e_1_3_3_2_47_2","doi-asserted-by":"crossref","unstructured":"Bonnie\u00a0R Rush David\u00a0C Rankin and Brad\u00a0J White. 2016. The impact of item-writing flaws and item complexity on examination item difficulty and discrimination value. BMC medical education 16 1 (2016) 250.","DOI":"10.1186\/s12909-016-0773-3"},{"key":"e_1_3_3_2_48_2","doi-asserted-by":"crossref","unstructured":"John Sabatini Tenaha O\u2019Reilly Jonathan Weeks and Zuowei Wang. 2020. Engineering a twenty-first century reading comprehension assessment system utilizing scenario-based assessment techniques. International Journal of Testing 20 1 (2020) 1\u201323.","DOI":"10.1080\/15305058.2018.1551224"},{"key":"e_1_3_3_2_49_2","doi-asserted-by":"crossref","unstructured":"Yukyeong Song Lauren\u00a0R Weisberg Shan Zhang Xiaoyi Tian Kristy\u00a0Elizabeth Boyer and Maya Israel. 2024. A framework for inclusive AI learning design for diverse learners. Computers and Education: Artificial Intelligence 6 (2024) 100212.","DOI":"10.1016\/j.caeai.2024.100212"},{"key":"e_1_3_3_2_50_2","volume-title":"AI Guidance for Schools Toolkit","year":"2023","unstructured":"TeachAI. 2023. AI Guidance for Schools Toolkit. https:\/\/www.teachai.org\/toolkit"},{"key":"e_1_3_3_2_51_2","doi-asserted-by":"crossref","unstructured":"Ieva Tenberga and Linda Daniela. 2024. Artificial intelligence literacy competencies for teachers through self-assessment tools. Sustainability 16 23 (2024) 10386.","DOI":"10.3390\/su162310386"},{"key":"e_1_3_3_2_52_2","unstructured":"The White House. 2025. Executive Order on AI Education. https:\/\/www.whitehouse.gov\/presidential-actions\/2025\/04\/advancing-artificial-intelligence-education-for-american-youth\/. Executive Order."},{"key":"e_1_3_3_2_53_2","doi-asserted-by":"crossref","unstructured":"Arzu\u00a0Deveci Topal Asiye\u00a0Toker G\u00f6k\u00e7e Canan\u00a0Dilek Eren and Aynur\u00a0Kolburan Ge\u00e7er. 2025. Artificial intelligence literacy scale: A study of reliability and validity in Turkish university students. Journal of Learning and Teaching in Digital Age 10 1 (2025) 58\u201367.","DOI":"10.53850\/joltida.1440845"},{"key":"e_1_3_3_2_54_2","unstructured":"UNESCO. 2023. Guidance for Generative AI in Education and Research. https:\/\/www.unesco.org\/en\/articles\/guidance-generative-ai-education-and-research. Ethical Guidance Report."},{"key":"e_1_3_3_2_55_2","first-page":"898","volume-title":"Society for Information Technology & Teacher Education International Conference","author":"Vyortkina Dina","year":"2024","unstructured":"Dina Vyortkina. 2024. AI Literacy Framework for Educators: Challenges and Opportunities. In Society for Information Technology & Teacher Education International Conference. Association for the Advancement of Computing in Education (AACE), 898\u2013903."},{"key":"e_1_3_3_2_56_2","doi-asserted-by":"crossref","unstructured":"Ning Wang and James Lester. 2023. K-12 Education in the Age of AI: A Call to Action for K-12 AI Literacy. International journal of artificial intelligence in education 33 2 (2023) 228\u2013232.","DOI":"10.1007\/s40593-023-00358-x"},{"key":"e_1_3_3_2_57_2","volume-title":"Proceedings of the 19th International Conference of the Learning Sciences-ICLS 2025, pp. 2554-2556","author":"Wu Ying-Tien","year":"2025","unstructured":"Ying-Tien Wu, Li-Jen Wang, and Tsun-Hui Shih. 2025. Enhancing Elementary Teachers\u2019 AI-TPACK through a Professional Development Workshop. In Proceedings of the 19th International Conference of the Learning Sciences-ICLS 2025, pp. 2554-2556. International Society of the Learning Sciences."},{"key":"e_1_3_3_2_58_2","unstructured":"Ruiwei Xiao Xinying Hou Runlong Ye Majeed Kazemitabaar Nicholas Diana Michael Liut and John Stamper. 2025. Improving Student-AI Interaction Through Pedagogical Prompting: An Example in Computer Science Education. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2506.19107 (2025)."},{"key":"e_1_3_3_2_59_2","doi-asserted-by":"crossref","unstructured":"Bilal Younis. 2025. The artificial intelligence literacy (AIL) scale for teachers: A tool for enhancing AI education. Journal of Digital Learning in Teacher Education 41 1 (2025) 37\u201356.","DOI":"10.1080\/21532974.2024.2441682"},{"key":"e_1_3_3_2_60_2","doi-asserted-by":"crossref","unstructured":"Miao Yue Morris Siu-Yung Jong and Davy Tsz\u00a0Kit Ng. 2024. Understanding K\u201312 teachers\u2019 technological pedagogical content knowledge readiness and attitudes toward artificial intelligence education. Education and information technologies 29 15 (2024) 19505\u201319536.","DOI":"10.1007\/s10639-024-12621-2"},{"key":"e_1_3_3_2_61_2","doi-asserted-by":"crossref","unstructured":"Helen Zhang Anthony Perry and Irene Lee. 2025. Developing and validating the artificial intelligence literacy concept inventory: An instrument to assess artificial intelligence literacy among middle school students. International Journal of Artificial Intelligence in Education 35 1 (2025) 398\u2013438.","DOI":"10.1007\/s40593-024-00398-x"},{"key":"e_1_3_3_2_62_2","doi-asserted-by":"crossref","unstructured":"Shan Zhang Priyadharshini Ganapathy\u00a0Prasad and Noah\u00a0L Schroeder. 2025. Learning About AI: A Systematic Review of Reviews on AI Literacy. Journal of Educational Computing Research (2025) 07356331251342081.","DOI":"10.1177\/07356331251342081"}],"event":{"name":"LAK 2026: LAK26: 16th International Learning Analytics and Knowledge Conference","location":"Bergen Norway","acronym":"LAK 2026"},"container-title":["Proceedings of the LAK26: 16th International Learning Analytics and Knowledge Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3785022.3785088","content-type":"text\/html","content-version":"vor","intended-application":"syndication"}],"deposited":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T09:41:12Z","timestamp":1777110072000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3785022.3785088"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,26]]},"references-count":61,"alternative-id":["10.1145\/3785022.3785088","10.1145\/3785022"],"URL":"https:\/\/doi.org\/10.1145\/3785022.3785088","relation":{},"subject":[],"published":{"date-parts":[[2026,4,26]]},"assertion":[{"value":"2026-04-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}