{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T09:18:54Z","timestamp":1784107134363,"version":"3.55.0"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643685489","type":"electronic"}],"license":[{"start":{"date-parts":[[2024,10,16]],"date-time":"2024-10-16T00:00:00Z","timestamp":1729036800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,10,16]]},"abstract":"<jats:p>Despite being prevalent in the general field of Natural Language Processing (NLP), pre-trained language models inherently carry privacy and copyright concerns due to their nature of training on large-scale web-scraped data. In this paper, we pioneer a systematic exploration of such risks associated with pre-trained language encoders, specifically focusing on the membership leakage of pre-training data exposed through downstream models adapted from pre-trained language encoders\u2013an aspect largely overlooked in existing literature. Our study encompasses comprehensive experiments across four types of pre-trained encoder architectures, three representative downstream tasks, and five benchmark datasets. Intriguingly, our evaluations reveal, for the first time, the existence of membership leakage even when only the black-box output of the downstream model is exposed, highlighting a privacy risk far greater than previously assumed. Alongside, we present in-depth analysis and insights toward guiding future researchers and practitioners in addressing the privacy considerations in developing pre-trained language models.<\/jats:p>","DOI":"10.3233\/faia240960","type":"book-chapter","created":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T13:48:20Z","timestamp":1729172900000},"source":"Crossref","is-referenced-by-count":2,"title":["Inside the Black Box: Detecting Data Leakage in Pre-Trained Language Encoders"],"prefix":"10.3233","author":[{"given":"Yuan","family":"Xin","sequence":"first","affiliation":[{"name":"CISPA Helmholtz Center for Information Security"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zheng","family":"Li","sequence":"additional","affiliation":[{"name":"CISPA Helmholtz Center for Information Security"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ning","family":"Yu","sequence":"additional","affiliation":[{"name":"Netflix Eyeline Studios"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dingfan","family":"Chen","sequence":"additional","affiliation":[{"name":"CISPA Helmholtz Center for Information Security"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mario","family":"Fritz","sequence":"additional","affiliation":[{"name":"CISPA Helmholtz Center for Information Security"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Backes","sequence":"additional","affiliation":[{"name":"CISPA Helmholtz Center for Information Security"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yang","family":"Zhang","sequence":"additional","affiliation":[{"name":"CISPA Helmholtz Center for Information Security"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","ECAI 2024"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA240960","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,17]],"date-time":"2024-10-17T13:48:21Z","timestamp":1729172901000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA240960"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,16]]},"ISBN":["9781643685489"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia240960","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"value":"0922-6389","type":"print"},{"value":"1879-8314","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,16]]}}}