{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T01:24:23Z","timestamp":1777339463586,"version":"3.51.4"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,2,28]],"date-time":"2023-02-28T00:00:00Z","timestamp":1677542400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100004410","name":"Scientific and Technological Research Council of Turkey","doi-asserted-by":"crossref","award":["T\u00dcB\u0130TAK"],"award-info":[{"award-number":["T\u00dcB\u0130TAK"]}],"id":[{"id":"10.13039\/501100004410","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Turkish Directorate of Strategy and Budget","award":["2007K12\u2013873"],"award-info":[{"award-number":["2007K12\u2013873"]}]},{"name":"Hybrid Intelligence Center"},{"name":"Dutch Ministry of Education"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Internet Technol."],"published-print":{"date-parts":[[2023,2,28]]},"abstract":"<jats:p>\n            Many software systems, such as online social networks, enable users to share information about themselves. Although the action of sharing is simple, it requires an elaborate thought process on privacy: what to share, with whom to share, and for what purposes. Thinking about these for each piece of content to be shared is tedious. Recent approaches to tackle this problem build personal assistants that can help users by learning what is private over time and recommending privacy labels such as private or public to individual content that a user considers sharing. However, privacy is inherently\n            <jats:italic>ambiguous<\/jats:italic>\n            and highly\n            <jats:italic>personal<\/jats:italic>\n            . Existing approaches to recommend privacy decisions do not address these aspects of privacy sufficiently. Ideally, a personal assistant should be able to adjust its recommendation based on a given user, considering that user\u2019s privacy understanding. Moreover, the personal assistant should be able to assess when its recommendation would be uncertain and let the user make the decision on her own. Accordingly, this article proposes a personal assistant that uses evidential deep learning to classify content based on its privacy label. An important characteristic of the personal assistant is that it can model its uncertainty in its decisions explicitly, determine that it does not know the answer, and delegate from making a recommendation when its uncertainty is high. By factoring in the user\u2019s own understanding of privacy, such as risk factors or own labels, the personal assistant can personalize its recommendations per user. We evaluate our proposed personal assistant using a well-known dataset. Our results show that our personal assistant can accurately identify uncertain cases, personalize them to its user\u2019s needs, and thus helps users preserve their privacy well.\n          <\/jats:p>","DOI":"10.1145\/3561820","type":"journal-article","created":{"date-parts":[[2022,9,16]],"date-time":"2022-09-16T09:29:18Z","timestamp":1663320558000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Uncertainty-Aware Personal Assistant for Making Personalized Privacy Decisions"],"prefix":"10.1145","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9200-8848","authenticated-orcid":false,"given":"Gonul","family":"Ayci","sequence":"first","affiliation":[{"name":"Bogazici University, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8806-4508","authenticated-orcid":false,"given":"Murat","family":"Sensoy","sequence":"additional","affiliation":[{"name":"Amazon Alexa AI, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8376-1056","authenticated-orcid":false,"given":"Arzucan","family":"\u00d6zg\u00fcr","sequence":"additional","affiliation":[{"name":"Bogazici University, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7848-1834","authenticated-orcid":false,"given":"Pinar","family":"Yolum","sequence":"additional","affiliation":[{"name":"Utrecht University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,3,23]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2005.22"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376389"},{"key":"e_1_3_2_4_2","unstructured":"Preetam Prabhu Srikar Dammu Srinivasa Rao Chalamala and Ajeet Kumar Singh. 2021. Explainable and personalized privacy prediction. In Proceedings of the 3rd International Workshop on Privacy Security and Trust in Computational Intelligence (PSTCI\u201921) . 1\u201310."},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Arthur P. Dempster. 2008. A generalization of Bayesian inference. In Classic Works of the Dempster-Shafer Theory of Belief Functions Roland R. Yager and Liping Liu (Eds.). Studies in Fuzziness and Soft Computing Vol. 219. Springer 73\u2013104.","DOI":"10.1007\/978-3-540-44792-4_4"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.5555\/1791834.1791836"},{"key":"e_1_3_2_7_2","first-page":"36","article-title":"Smart home personal assistants: A security and privacy review","volume":"53","author":"Edu Jide S.","year":"2020","unstructured":"Jide S. Edu, Jose M. Such, and Guillermo Suarez-Tangil. 2020. Smart home personal assistants: A security and privacy review. ACM Computing Surveys 53, 6 (Dec. 2020), Article 116, 36 pages.","journal-title":"ACM Computing Surveys"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772727"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2017.4180836"},{"key":"e_1_3_2_10_2","first-page":"1050","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Gal Yarin","year":"2016","unstructured":"Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the International Conference on Machine Learning. 1050\u20131059."},{"key":"e_1_3_2_11_2","article-title":"Explaining and harnessing adversarial examples","author":"Goodfellow Ian J.","year":"2014","unstructured":"Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).","journal-title":"arXiv preprint arXiv:1412.6572"},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","first-page":"2259","DOI":"10.1007\/s11042-021-11667-5","article-title":"Learning multi-level and multi-scale deep representations for privacy image classification","author":"Han Yahui","year":"2022","unstructured":"Yahui Han, Yonggang Huang, Lei Pan, and Yunbo Zheng. 2022. Learning multi-level and multi-scale deep representations for privacy image classification. Multimedia Tools and Applications 81, 2 (2022), 2259\u20132274.","journal-title":"Multimedia Tools and Applications"},{"key":"e_1_3_2_13_2","first-page":"223","volume-title":"Proceedings of the 20th International Conference on Architectural Support for Programming Languages and Operating Systems","author":"Hauswald Johann","year":"2015","unstructured":"Johann Hauswald, Michael A. Laurenzano, Yunqi Zhang, Cheng Li, Austin Rovinski, Arjun Khurana, Ronald G. Dreslinski, et\u00a0al. 2015. Sirius: An open end-to-end voice and vision personal assistant and its implications for future warehouse scale computers. In Proceedings of the 20th International Conference on Architectural Support for Programming Languages and Operating Systems. 223\u2013238."},{"key":"e_1_3_2_14_2","first-page":"770","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He Kaiming","year":"2016","unstructured":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770\u2013778."},{"key":"e_1_3_2_15_2","article-title":"Aleatoric and epistemic uncertainty in machine learning: A tutorial introduction","author":"H\u00fcllermeier Eyke","year":"2019","unstructured":"Eyke H\u00fcllermeier and Willem Waegeman. 2019. Aleatoric and epistemic uncertainty in machine learning: A tutorial introduction. arXiv preprint arXiv:1910.09457 (2019).","journal-title":"arXiv preprint arXiv:1910.09457"},{"key":"e_1_3_2_16_2","first-page":"91","volume-title":"Proceedings of the 2020 6th International Conference on Big Data Computing and Communications (BIGCOM\u201920)","author":"Jiao Rui","year":"2020","unstructured":"Rui Jiao, Lan Zhang, and Anran Li. 2020. IEye: Personalized image privacy detection. In Proceedings of the 2020 6th International Conference on Big Data Computing and Communications (BIGCOM\u201920). IEEE, Los Alamitos, CA, 91\u201395."},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-42337-1","volume-title":"Subjective Logic","author":"J\u00f8sang Audun","year":"2016","unstructured":"Audun J\u00f8sang. 2016. Subjective Logic. Springer."},{"key":"e_1_3_2_18_2","first-page":"1","volume-title":"Proceedings of the 17th International Conference on Information Fusion (FUSION\u201914)","author":"Kaplan Lance","year":"2014","unstructured":"Lance Kaplan, Murat \u015eensoy, and Geeth de Mel. 2014. Trust estimation and fusion of uncertain information by exploiting consistency. In Proceedings of the 17th International Conference on Information Fusion (FUSION\u201914). IEEE, Los Alamitos, CA, 1\u20138."},{"key":"e_1_3_2_19_2","first-page":"22","article-title":"Preserving privacy as social responsibility in online social networks","volume":"18","author":"Kekulluoglu Dilara","year":"2018","unstructured":"Dilara Kekulluoglu, Nadin K\u00f6kciyan, and P\u0131nar Yolum. 2018. Preserving privacy as social responsibility in online social networks. ACM Transactions on Internet Technology 18, 4 (April 2018), Article 42, 22 pages.","journal-title":"ACM Transactions on Internet Technology"},{"key":"e_1_3_2_20_2","first-page":"3","volume-title":"Proceedings of the 1st International Workshop on AI for Privacy and Security","author":"Kepez Berkant","year":"2016","unstructured":"Berkant Kepez and P\u0131nar Yolum. 2016. Learning privacy rules cooperatively in online social networks. In Proceedings of the 1st International Workshop on AI for Privacy and Security. ACM, New York, NY, 3."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2016.2583425"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2020.3020006"},{"key":"e_1_3_2_23_2","article-title":"ImageNet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 (2012).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-020-09488-1"},{"key":"e_1_3_2_25_2","article-title":"Simple and scalable predictive uncertainty estimation using deep ensembles","volume":"30","author":"Lakshminarayanan Balaji","year":"2017","unstructured":"Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems 30 (2017).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3436755"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3465171"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2017.4180831"},{"key":"e_1_3_2_29_2","volume-title":"Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201921)","author":"Mosca Francesca","year":"2021","unstructured":"Francesca Mosca and Jose Such. 2021. ELVIRA: An explainable agent for value and utility-driven multiuser privacy. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201921)."},{"issue":"2","key":"e_1_3_2_30_2","first-page":"47","article-title":"An intelligent personal assistant for task and time management","volume":"28","author":"Myers Karen","year":"2007","unstructured":"Karen Myers, Pauline Berry, Jim Blythe, Ken Conley, Melinda Gervasio, Deborah L. McGuinness, David Morley, Avi Pfeffer, Martha Pollack, and Milind Tambe. 2007. An intelligent personal assistant for task and time management. AI Magazine 28, 2 (2007), 47.","journal-title":"AI Magazine"},{"key":"e_1_3_2_31_2","volume-title":"Computer-Intensive Methods for Testing Hypotheses","author":"Noreen Eric W.","year":"1989","unstructured":"Eric W. Noreen. 1989. Computer-Intensive Methods for Testing Hypotheses. Wiley, New York, NY."},{"key":"e_1_3_2_32_2","first-page":"3179","volume-title":"Advances in Neural Information Processing Systems","author":"Sensoy Murat","year":"2018","unstructured":"Murat Sensoy, Lance Kaplan, and Melih Kandemir. 2018. Evidential deep learning to quantify classification uncertainty. In Advances in Neural Information Processing Systems. 3179\u20133189."},{"key":"e_1_3_2_33_2","first-page":"2484","volume-title":"Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision","author":"Sensoy Murat","year":"2021","unstructured":"Murat Sensoy, Maryam Saleki, Simon Julier, Reyhan Aydogan, and John Reid. 2021. Misclassification risk and uncertainty quantification in deep classifiers. In Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision. 2484\u20132492."},{"key":"e_1_3_2_34_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).","journal-title":"arXiv preprint arXiv:1409.1556"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/2983644"},{"issue":"1","key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1109\/TKDE.2014.2320729","article-title":"Privacy policy inference of user-uploaded images on content sharing sites","volume":"27","author":"Squicciarini Anna Cinzia","year":"2014","unstructured":"Anna Cinzia Squicciarini, Dan Lin, Smitha Sundareswaran, and Joshua Wede. 2014. Privacy policy inference of user-uploaded images on content sharing sites. IEEE Transactions on Knowledge and Data Engineering 27, 1 (2014), 193\u2013206.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1109\/PST.2017.00047","volume-title":"Proceedings of the 2017 15th Annual Conference on Privacy, Security and Trust (PST\u201917)","author":"Squicciarini Anna Cinzia","year":"2017","unstructured":"Anna Cinzia Squicciarini, Andrea Novelli, Dan Lin, Cornelia Caragea, and Haoti Zhong. 2017. From tag to protect: A tag-driven policy recommender system for image sharing. In Proceedings of the 2017 15th Annual Conference on Privacy, Security and Trust (PST\u201917). IEEE, Los Alamitos, CA, 337\u201333709."},{"key":"e_1_3_2_38_2","first-page":"29","article-title":"Privacy policy negotiation in social media","volume":"11","author":"Such Jose M.","year":"2016","unstructured":"Jose M. Such and Michael Rovatsos. 2016. Privacy policy negotiation in social media. ACM Transactions on Autonomous and Adaptive Systems 11, 1 (Feb. 2016), Article 4, 29 pages.","journal-title":"ACM Transactions on Autonomous and Adaptive Systems"},{"key":"e_1_3_2_39_2","first-page":"1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Szegedy Christian","year":"2015","unstructured":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1\u20139."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3386082"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10169"},{"key":"e_1_3_2_43_2","first-page":"32","article-title":"PANOLA: A personal assistant for supporting users in preserving privacy","volume":"22","author":"Ulusoy Onuralp","year":"2021","unstructured":"Onuralp Ulusoy and P\u0131nar Yolum. 2021. PANOLA: A personal assistant for supporting users in preserving privacy. ACM Transactions on Internet Technology 22, 1 (2021), Article 27, 32 pages.","journal-title":"ACM Transactions on Internet Technology"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2017.2787986"},{"key":"e_1_3_2_45_2","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1007\/978-3-319-58469-0_7","volume-title":"Proceedings of the IFIP International Conference on ICT Systems Security and Privacy Protection","author":"Yuan Lin","year":"2017","unstructured":"Lin Yuan, Jo\u00ebl Theytaz, and Touradj Ebrahimi. 2017. Context-dependent privacy-aware photo sharing based on machine learning. In Proceedings of the IFIP International Conference on ICT Systems Security and Privacy Protection. 93\u2013107."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/2348283.2348292"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/PST.2008.16"},{"key":"e_1_3_2_48_2","first-page":"3952","volume-title":"Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI\u201917)","volume":"17","author":"Zhong Haoti","year":"2017","unstructured":"Haoti Zhong, Anna Cinzia Squicciarini, David J. Miller, and Cornelia Caragea. 2017. A group-based personalized model for image privacy classification and labeling. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI\u201917), Vol. 17. 3952\u20133958."}],"container-title":["ACM Transactions on Internet Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3561820","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3561820","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:24Z","timestamp":1750186944000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3561820"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,28]]},"references-count":47,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,2,28]]}},"alternative-id":["10.1145\/3561820"],"URL":"https:\/\/doi.org\/10.1145\/3561820","relation":{},"ISSN":["1533-5399","1557-6051"],"issn-type":[{"value":"1533-5399","type":"print"},{"value":"1557-6051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,28]]},"assertion":[{"value":"2022-04-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-08-29","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-03-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}