{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T23:38:02Z","timestamp":1761176282415,"version":"build-2065373602"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643686318","type":"electronic"}],"license":[{"start":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T00:00:00Z","timestamp":1761004800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,10,21]]},"abstract":"<jats:p>Membership Inference Attacks (MIAs) have recently been employed to determine whether a specific text was part of the pre-training data of Large Language Models (LLMs). However, existing methods often misinfer non-members as members, leading to a high false positive rate, or depend on additional reference models for probability calibration, which limits their practicality. To overcome these challenges, we introduce a novel framework called Automatic Calibration Membership Inference Attack (ACMIA), which utilizes a tunable temperature to calibrate output probabilities effectively. This approach is inspired by our theoretical insights into maximum likelihood estimation during the pre-training of LLMs. We introduce ACMIA in three configurations designed to accommodate different levels of model access and increase the probability gap between members and non-members, improving the reliability and robustness of membership inference. Extensive experiments on various open-source LLMs demonstrate that our proposed attack is highly effective, robust, and generalizable, surpassing state-of-the-art baselines across three widely used benchmarks. The source code is publicly available.<\/jats:p>","DOI":"10.3233\/faia251364","type":"book-chapter","created":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T09:59:30Z","timestamp":1761127170000},"source":"Crossref","is-referenced-by-count":0,"title":["Automatic Calibration for Membership Inference Attack on Large Language Models"],"prefix":"10.3233","author":[{"given":"Saleh","family":"Zare Zade","sequence":"first","affiliation":[{"name":"Department of Computer Science, Wayne State University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yao","family":"Qiang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Oakland University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiangyu","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Wayne State University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hui","family":"Zhu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Wayne State University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohammad Amin","family":"Roshani","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Wayne State University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Prashant","family":"Khanduri","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Wayne State University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dongxiao","family":"Zhu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Wayne State University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","ECAI 2025"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA251364","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T09:59:30Z","timestamp":1761127170000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA251364"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,21]]},"ISBN":["9781643686318"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia251364","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"value":"0922-6389","type":"print"},{"value":"1879-8314","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,21]]}}}