{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T04:27:12Z","timestamp":1773808032521,"version":"3.50.1"},"reference-count":0,"publisher":"Association for the Advancement of Artificial Intelligence (AAAI)","issue":"42","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["AAAI"],"abstract":"<jats:p>Smartphones bring significant convenience to users but also enable devices to extensively record various types of personal information.\nExisting smartphone agents powered by Multimodal Large Language Models (MLLMs) have achieved remarkable performance in automating different tasks.\nHowever, as the cost, these agents are granted substantial access to sensitive users' personal information during this operation. \nTo gain a thorough understanding of the privacy awareness of these agents, we present the first large-scale benchmark encompassing 7,138 scenarios to the best of our knowledge.\nIn addition, for privacy context in scenarios, we annotate its type (e.g., Account Credentials), sensitivity level, and location.\nWe then carefully benchmark seven available mainstream smartphone agents. \nOur results demonstrate that almost all benchmarked agents show unsatisfying privacy awareness (RA), with performance remaining below 60% even with explicit hints. \nOverall, closed-source agents show better privacy ability than open-source ones, and Gemini 2.0-flash achieves the best, achieving an RA of 67%.\nWe also find that the agents\u2019 privacy detection capability is highly related to scenario sensitivity level, i.e., the scenario with a higher sensitivity level is typically more identifiable. \nWe hope the findings enlighten the research community to rethink the unbalanced utility-privacy tradeoff about smartphone agents.<\/jats:p>","DOI":"10.1609\/aaai.v40i42.40874","type":"journal-article","created":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T03:30:56Z","timestamp":1773804656000},"page":"35626-35634","source":"Crossref","is-referenced-by-count":0,"title":["Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered Smartphone Agents"],"prefix":"10.1609","volume":"40","author":[{"given":"Zhixin","family":"Lin","sequence":"first","affiliation":[]},{"given":"Jungang","family":"Li","sequence":"additional","affiliation":[]},{"given":"Shidong","family":"Pan","sequence":"additional","affiliation":[]},{"given":"Yibo","family":"Shi","sequence":"additional","affiliation":[]},{"given":"Yue","family":"Yao","sequence":"additional","affiliation":[]},{"given":"Dongliang","family":"Xu","sequence":"additional","affiliation":[]}],"member":"9382","published-online":{"date-parts":[[2026,3,14]]},"container-title":["Proceedings of the AAAI Conference on Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/ojs.aaai.org\/index.php\/AAAI\/article\/download\/40874\/44835","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/ojs.aaai.org\/index.php\/AAAI\/article\/download\/40874\/44835","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T03:30:57Z","timestamp":1773804657000},"score":1,"resource":{"primary":{"URL":"https:\/\/ojs.aaai.org\/index.php\/AAAI\/article\/view\/40874"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,14]]},"references-count":0,"journal-issue":{"issue":"42","published-online":{"date-parts":[[2026,3,17]]}},"URL":"https:\/\/doi.org\/10.1609\/aaai.v40i42.40874","relation":{},"ISSN":["2374-3468","2159-5399"],"issn-type":[{"value":"2374-3468","type":"electronic"},{"value":"2159-5399","type":"print"}],"subject":[],"published":{"date-parts":[[2026,3,14]]}}}