{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,2]],"date-time":"2026-03-02T21:45:40Z","timestamp":1772487940882,"version":"3.50.1"},"reference-count":48,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T00:00:00Z","timestamp":1757462400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Commun. Netw."],"abstract":"<jats:p>The framework of Input and Output Privacy aids in conceptualization of data privacy protections, providing considerations for situations where multiple parties are collaborating in a compute system (Input Privacy) as well as considerations when releasing data from a compute process (Output Privacy). Similar frameworks for conceptualization of privacy protections at a systems design level are lacking within the Artificial Intelligence space, which can lead to mischaracterizations and incorrect implementations of privacy protections. In this paper, we apply the Input and Output Privacy framework to Artificial Intelligence (AI) systems, establishing parallels between traditional data systems and newer AI systems to help privacy professionals and AI developers and deployers conceptualize and determine the places in those systems where privacy protections have the greatest effect. We discuss why the Input and Output Privacy framework is useful when evaluating privacy protections for AI systems, examine the similarities and differences of Input and Output privacy between traditional data systems and AI systems, and provide considerations on how to protect Input and Output Privacy for systems utilizing AI models. This framework offers developers and deployers of AI systems common ground for conceptualizing where and how privacy protections can be applied in their systems and for minimizing risk of misaligned implementations of privacy protection.<\/jats:p>","DOI":"10.3389\/frcmn.2025.1600750","type":"journal-article","created":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T05:27:14Z","timestamp":1757482034000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Privacy considerations for LLMs and other AI models: an input and output privacy approach"],"prefix":"10.3389","volume":"6","author":[{"given":"Zixin","family":"Nie","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Leena","family":"Dave","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rashonda","family":"Lewis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,9,10]]},"reference":[{"key":"B1","unstructured":"Agencia Espa\u00f1ola de Protecci\u00f3n de Datos (AEPD)\n          \n          \n          2023"},{"key":"B4","year":"2017"},{"key":"B2","doi-asserted-by":"publisher","first-page":"e230212","DOI":"10.1148\/ryai.230212","article-title":"Securing collaborative medical AI by using differential privacy: domain transfer for classification of chest radiographs","volume":"6","author":"Arasteh","year":"2024","journal-title":"Radiol. Artif. Intell."},{"key":"B3","volume-title":"Sharing sensitive Department of Education data across organizational Boundaries using secure Multiparty computation","author":"Archer","year":"2021"},{"key":"B8","doi-asserted-by":"publisher","first-page":"226544","DOI":"10.1109\/access.2020.3045465","article-title":"PrivFT: private and Fast text classification with homomorphic encryption","volume":"8","author":"Badawi","year":"2020","journal-title":"IEEE Access"},{"key":"B9","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1080\/01621459.2023.2270795","article-title":"A Feasibility study of differentially private summary Statistics and Regression analyses with Evaluations on administrative and survey data","volume":"119","author":"Barrientos","year":"2023","journal-title":"arXiv"},{"key":"B10","doi-asserted-by":"publisher","first-page":"2280","DOI":"10.1145\/3531146.3534642","article-title":"What does it mean for a language model to Preserve privacy?","author":"Brown","year":"2022","journal-title":"arXiv"},{"key":"B11","unstructured":"Extracting training data from large language models\n          \n          2633\n          2650\n          \n            \n              Carlini\n              N.\n            \n            \n              Tram\u00e8r\n              F.\n            \n            \n              Wallace\n              E.\n            \n            \n              Jagielski\n              M.\n            \n            \n              Herbert-Voss\n              A.\n            \n            \n              Lee\n              K.\n            \n          \n          \n          2021"},{"key":"B12","article-title":"Quantifying Memorization across neural language models","volume-title":"The Eleventh International Conference on learning Representations","author":"Carlini","year":"2023"},{"key":"B5","article-title":"Making FHE Faster for ML: beating our previous paper benchmarks with concrete ML. Zama.ai","author":"Chevallier-Mames","year":"2024"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-3-030-78086-9_1","article-title":"Programmable Bootstrapping enables Efficient homomorphic inference of deep neural networks","author":"Chillotti","year":"2021"},{"key":"B14","volume-title":"Data management Body of Knowledge","author":"Dama-Dmbok","year":"2017"},{"key":"B15","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1007\/978-3-031-40837-3_16","article-title":"Memorization of named entities in fine-tuned BERT models","author":"Diera","year":"2023"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.29012\/jpc.689","article-title":"Differential privacy in practice: Expose your epsilons","volume":"9","author":"Dwork","year":"2019","journal-title":"J. Priv. Confidentiality"},{"key":"B17","doi-asserted-by":"publisher","first-page":"103605","DOI":"10.1016\/j.cose.2023.103605","article-title":"Preserving data privacy in machine learning systems","volume":"137","author":"El Mestari","year":"2024","journal-title":"Comput. and Secur."},{"key":"B18","doi-asserted-by":"crossref","DOI":"10.1201\/b14764","volume-title":"Guide to the de-identification of Personal Health Information","author":"El Emam","year":"2013"},{"key":"B19","article-title":"A Unified framework for Quantifying privacy risk in synthetic data","author":"Giomi","year":"2022","journal-title":"arXiv"},{"key":"B20","doi-asserted-by":"publisher","first-page":"4799","DOI":"10.1038\/s41467-022-32168-5","article-title":"Secure human action recognition by encrypted neural network inference","volume":"13","author":"Kim","year":"2022","journal-title":"Nat. Commun."},{"key":"B21","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1109\/mitp.2023.3275489","article-title":"Cybercrime and privacy threats of large language models","volume":"25","author":"Kshetri","year":"2023","journal-title":"IT Prof."},{"key":"B22","article-title":"Does BERT Pretrained on Clinical Notes reveal sensitive data?","author":"Lehman","year":"2021","journal-title":"arXiv"},{"key":"B23","article-title":"Backdoor attacks on pre-trained models by Layerwise Weight poisoning","author":"Li","year":"2021","journal-title":"arXiv"},{"key":"B24","article-title":"A survey on private transformer inference","author":"Li","year":"2024","journal-title":"arXiv"},{"key":"B25","article-title":"LLMs can understand encrypted prompt: towards privacy-computing Friendly Transformers","author":"Liu","year":"2023","journal-title":"arXiv"},{"key":"B26","article-title":"Differentially private Decoding in large language models","author":"Majmudar","year":"2022","journal-title":"arXiv"},{"key":"B27","article-title":"Federated learning: Collaborative machine learning without Centralized training data","author":"McMahan","year":"2017","journal-title":"Google Res"},{"key":"B28","volume-title":"Privacy attacks in Federated learning","author":"Near","year":"2024"},{"key":"B29","unstructured":"An Efficient approach for securing Audio data in AI training with fully homomorphic encryption\n          \n          \n            \n              Nguyen\n              L.\n            \n            \n              Phan\n              B.\n            \n            \n              Zhang\n              L.\n            \n            \n              Nguyen\n              T.\n            \n          \n          \n          2025"},{"key":"B30","first-page":"69","article-title":"America\u2019s DataHub Consortium: privacy preserving Technology phase 1 \u2013 Environmental scan","author":"Nie","year":"2024","journal-title":"RTI Int. Natl. Cent. Sci. Eng. Statistics"},{"key":"B31","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-75178-4","volume-title":"Synthetic data for deep learning","author":"Nikolenko","year":"2021"},{"key":"B32","doi-asserted-by":"publisher","first-page":"404","DOI":"10.1016\/j.jiixd.2024.02.001","article-title":"A survey on membership inference attacks and defenses in machine learning","volume":"2","author":"Niu","year":"2024","journal-title":"J. Inf. Intell."},{"key":"B33","doi-asserted-by":"crossref","first-page":"1314","DOI":"10.1109\/SP40000.2020.00095","article-title":"Privacy risks of General-Purpose language models","volume-title":"2020 IEEE Symposium on security and privacy (SP)","author":"Pan","year":"2020"},{"key":"B34","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1007\/978-3-031-63592-2_15","article-title":"MedBlindTuner: towards privacy-preserving fine-tuning on Biomedical Images with Transformers and fully homomorphic encryption","volume-title":"AI for health Equity and Fairness: leveraging AI to Address social Determinants of health","author":"Panzade","year":"2024"},{"key":"B35","article-title":"Encryption-friendly LLM Architecture","author":"Rho","year":"2024","journal-title":"arXiv"},{"key":"B36","unstructured":"A reflection on privacy and data Confidentiality in Official Statistics\n          \n          \n            \n              Ricciato\n              F.\n            \n            \n              Bujnowska\n              A.\n            \n            \n              Wirthmann\n              A.\n            \n            \n              Hahn\n              M.\n            \n            \n              Barredo-Capelot\n              E.\n            \n          \n          \n          2020"},{"key":"B37","unstructured":"Privacy preserving technologies, part three: private statistical analysis and private text classification based on homomorphic encryption\n          \n          \n            \n              Santos\n              B.\n            \n            \n              Zanussi\n              Z.\n            \n          \n          \n          2022"},{"key":"B38","article-title":"Identifying and mitigating privacy risks Stemming from language models: a survey","author":"Smith","year":"2024","journal-title":"arXiv"},{"key":"B39","article-title":"Beyond Memorization: Violating privacy via inference with large language models","author":"Staab","year":"2024","journal-title":"arXiv"},{"key":"B40","article-title":"Structured Transparency: Ensuring input and output privacy","author":"Stutz","year":"2021","journal-title":"OpenMined Blog Priv. AI Ser"},{"key":"B41","unstructured":"Guide on privacy-Enhancing technologies for Official Statistics\n          \n          \n          2023"},{"key":"B42","unstructured":"Blueprint for an AI Bill of Rights | OSTP\n          \n          \n          2025"},{"key":"B49","year":"2022"},{"key":"B6","year":"2025"},{"key":"B43","article-title":"Memorization in deep learning: a survey","author":"Wei","year":"2024","journal-title":"arXiv"},{"key":"B44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/icmc60390.2024.00008","article-title":"On protecting the data privacy of large language models (LLMs): a survey","author":"Yan","year":"2024","journal-title":"arXiv"},{"key":"B45","article-title":"PoisonPrompt: backdoor attack on Prompt-based large language models","author":"Yao","year":"2023","journal-title":"arXiv"},{"key":"B46","article-title":"Text revealer: private text reconstruction via model Inversion attacks against Transformers","author":"Zhang","year":"2022","journal-title":"arXiv"},{"key":"B47","article-title":"\u201cGhost of the past\u201d: identifying and resolving privacy leakage from LLM\u2019s memory through proactive user interaction","author":"Zhang","year":"2024","journal-title":"arXiv"},{"key":"B48","doi-asserted-by":"publisher","first-page":"2824","DOI":"10.1109\/tkde.2020.3014246","article-title":"More than privacy: applying differential privacy in Key Areas of Artificial Intelligence","volume":"34","author":"Zhu","year":"2022","journal-title":"IEEE Trans. Knowl. Data Eng."}],"container-title":["Frontiers in Communications and Networks"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frcmn.2025.1600750\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T05:27:19Z","timestamp":1757482039000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frcmn.2025.1600750\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,10]]},"references-count":48,"alternative-id":["10.3389\/frcmn.2025.1600750"],"URL":"https:\/\/doi.org\/10.3389\/frcmn.2025.1600750","relation":{},"ISSN":["2673-530X"],"issn-type":[{"value":"2673-530X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,10]]},"article-number":"1600750"}}