{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T10:57:42Z","timestamp":1780657062259,"version":"3.54.1"},"reference-count":373,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2024,6,17]],"date-time":"2024-06-17T00:00:00Z","timestamp":1718582400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,6,17]],"date-time":"2024-06-17T00:00:00Z","timestamp":1718582400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"European Union\u2019s Horizon 2020 research and innovation programme","award":["956123"],"award-info":[{"award-number":["956123"]}]},{"name":"U.K. EPSRC","award":["EP\/T026995\/1"],"award-info":[{"award-number":["EP\/T026995\/1"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Intell Rev"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Large language models (LLMs) have exploded a new heatwave of AI for their ability to engage end-users in human-level conversations with detailed and articulate answers across many knowledge domains. In response to their fast adoption in many industrial applications, this survey concerns their safety and trustworthiness. First, we review known vulnerabilities and limitations of the LLMs, categorising them into inherent issues, attacks, and unintended bugs. Then, we consider if and how the Verification and Validation (V&amp;V) techniques, which have been widely developed for traditional software and deep learning models such as convolutional neural networks as independent processes to check the alignment of their implementations against the specifications, can be integrated and further extended throughout the lifecycle of the LLMs to provide rigorous analysis to the safety and trustworthiness of LLMs and their applications. Specifically, we consider four complementary techniques: falsification and evaluation, verification, runtime monitoring, and regulations and ethical use. In total, 370+ references are considered to support the quick understanding of the safety and trustworthiness issues from the perspective of V&amp;V. While intensive research has been conducted to identify the safety and trustworthiness issues, rigorous yet practical methods are called for to ensure the alignment of LLMs with safety and trustworthiness requirements.<\/jats:p>","DOI":"10.1007\/s10462-024-10824-0","type":"journal-article","created":{"date-parts":[[2024,6,17]],"date-time":"2024-06-17T07:01:46Z","timestamp":1718607706000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":103,"title":["A survey of safety and trustworthiness of large language models through the lens of verification and validation"],"prefix":"10.1007","volume":"57","author":[{"given":"Xiaowei","family":"Huang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenjie","family":"Ruan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gaojie","family":"Jin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"Dong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Changshun","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saddek","family":"Bensalem","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronghui","family":"Mu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"Qi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xingyu","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaiwen","family":"Cai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanghao","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sihao","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peipei","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dengyu","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andre","family":"Freitas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mustafa A.","family":"Mustafa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,6,17]]},"reference":[{"key":"10824_CR1","unstructured":"(2004) Quality management systems\u2014process validation guidance. https:\/\/www.imdrf.org\/sites\/default\/files\/docs\/ghtf\/final\/sg3\/technical-docs\/ghtf-sg3-n99-10-2004-qms-process-guidance-04010.pdf. GHTF. Accessed 20 Aug 2023"},{"key":"10824_CR2","unstructured":"(2018) Ethics guidelines for trustworthy AI. https:\/\/ec.europa.eu\/futurium\/en\/ai-alliance-consultation.1.html. European Commission. Accessed 20 Aug 2023"},{"key":"10824_CR3","unstructured":"(2018) The data protection act. https:\/\/www.legislation.gov.uk\/ukpga\/2018\/12\/contents\/enacted. Accessed 20 Aug 2023"},{"key":"10824_CR4","unstructured":"(2021) China\u2019s regulations on the administration of deep synthesis internet information services. https:\/\/www.chinalawtranslate.com\/en\/deep-synthesis\/. Accessed 20 Aug 2023"},{"key":"10824_CR5","unstructured":"(2022) AI risk management framework. https:\/\/www.nist.gov\/itl\/ai-risk-management-framework. Accessed 20 Aug 2023"},{"key":"10824_CR6","unstructured":"(2022) China\u2019s regulations on recommendation algorithms. http:\/\/www.cac.gov.cn\/2022-01\/04\/c_1642894606258238.htm. Accessed 20 Aug 2023"},{"key":"10824_CR7","unstructured":"(2022) Content at scale. https:\/\/contentatscale.ai\/ai-content-detector\/. Accessed 20 Aug 2023"},{"key":"10824_CR8","unstructured":"(2022) Copyleaks. https:\/\/copyleaks.com\/ai-content-detector. Accessed 20 Aug 2023"},{"key":"10824_CR9","unstructured":"(2022) New meta AI demo writes racist and inaccurate scientific literature, gets pulled. https:\/\/arstechnica.com\/information-technology\/2022\/11\/after-controversy-meta-pulls-demo-of-ai-model-that-writes-scientific-papers\/. Accessed 20 Aug 2023"},{"key":"10824_CR10","unstructured":"(2022) Originality AI. https:\/\/originality.ai. Accessed 20 Aug 2023"},{"key":"10824_CR11","unstructured":"(2022) Prompt injection attacks against GPT-3. https:\/\/simonwillison.net\/2022\/Sep\/12\/prompt-injection\/. Accessed 20 Aug 2023"},{"key":"10824_CR12","unstructured":"(2023) \u2018He would still be here\u2019: man dies by suicide after talking with AI chatbot, widow says. https:\/\/www.vice.com\/en\/article\/pkadgm\/man-dies-by-suicide-after-talking-with-ai-chatbot-widow-says. Accessed 23 Aug 2023"},{"key":"10824_CR13","unstructured":"(2023) A pro-innovation approach to AI regulation. https:\/\/assets.publishing.service.gov.uk\/government\/uploads\/system\/uploads\/attachment_data\/file\/1146542\/a_pro-innovation_approach_to_AI_regulation.pdf. Accessed 20 Aug 2023"},{"key":"10824_CR14","doi-asserted-by":"crossref","unstructured":"(2023) Blueprint for an AI bill of rights. https:\/\/www.whitehouse.gov\/ostp\/ai-bill-of-rights\/. Accessed 20 Aug 2023","DOI":"10.4324\/9781003415091-4"},{"key":"10824_CR15","unstructured":"(2023) ChatGPT: get instant answers, find creative inspiration, and learn something new. https:\/\/openai.com\/chatgpt. Accessed 20 Aug 2023"},{"key":"10824_CR16","unstructured":"(2023) ChatGPT: US lawyer admits using AI for case research. https:\/\/www.bbc.co.uk\/news\/world-us-canada-65735769. Accessed 23 Aug 2023"},{"key":"10824_CR17","unstructured":"(2023) China\u2019s algorithm registry. https:\/\/beian.cac.gov.cn\/#\/index. Accessed 20 Aug 2023"},{"key":"10824_CR18","unstructured":"(2023) EU AI act. https:\/\/artificialintelligenceact.eu. Accessed 20 Aug 2023"},{"key":"10824_CR19","unstructured":"(2023) EU data act. https:\/\/ec.europa.eu\/commission\/presscorner\/detail\/en\/ip_22_1113. Accessed 20 Aug 2023"},{"key":"10824_CR20","unstructured":"(2023) Prompt leaking. https:\/\/learnprompting.org\/docs\/prompt_hacking\/leaking. Accessed 20 Aug 2023"},{"key":"10824_CR21","unstructured":"(2023) Responsible AI principles from Microsoft. https:\/\/www.microsoft.com\/en-us\/ai\/responsible-ai. Accessed 20 Aug 2023"},{"key":"10824_CR22","unstructured":"(2023) Three Samsung employees reportedly leaked sensitive data to ChatGPT. https:\/\/www.engadget.com\/three-samsung-employees-reportedly-leaked-sensitive-data-to-chatgpt-190221114.html. Accessed 20 Aug 2023"},{"key":"10824_CR23","unstructured":"(2023) Understanding artificial intelligence ethics and safety: a guide for the responsible design and implementation of AI systems in the public sector.\u00a0https:\/\/www.turing.ac.uk\/news\/publications\/understanding-artificial-intelligence-ethics-and-safety. Accessed 20 Aug 2023"},{"key":"10824_CR24","unstructured":"Aghakhani H, Dai W, Manoel A, Fernandes X, Kharkar A, Kruegel C, Vigna G, Evans D, Zorn B, Sim R (2023) TrojanPuzzle: covertly poisoning code-suggestion models. arXiv Preprint http:\/\/arxiv.org\/abs\/2301.02344"},{"key":"10824_CR25","doi-asserted-by":"crossref","unstructured":"Agrawal M, Hegselmann S, Lang H, Kim Y, Sontag D (2022) Large language models are zero-shot clinical information extractors. arXiv Preprint http:\/\/arxiv.org\/abs\/2205.12689","DOI":"10.18653\/v1\/2022.emnlp-main.130"},{"key":"10824_CR26","doi-asserted-by":"crossref","unstructured":"Aiyappa, R An J, Kwak H, Ahn Y-Y (2023) Can we trust the evaluation on ChatGPT? arXiv Preprint http:\/\/arxiv.org\/abs\/2303.12767","DOI":"10.18653\/v1\/2023.trustnlp-1.5"},{"issue":"10","key":"10824_CR27","doi-asserted-by":"crossref","first-page":"1537","DOI":"10.1109\/TCAD.2015.2474396","volume":"34","author":"F Akopyan","year":"2015","unstructured":"Akopyan F, Sawada J, Cassidy A, Alvarez-Icaza R, Arthur J, Merolla P, Imam N, Nakamura Y, Datta P, Nam G-J et al (2015) TrueNorth: design and tool flow of a 65 MW 1 million neuron programmable neurosynaptic chip. IEEE Trans Comput Aided Des Integr Circuits Syst 34(10):1537\u20131557","journal-title":"IEEE Trans Comput Aided Des Integr Circuits Syst"},{"key":"10824_CR28","doi-asserted-by":"crossref","unstructured":"Alshiekh M, Bloem R, Ehlers R, K\u00f6nighofer B, Niekum S, Topcu U (2018) Safe reinforcement learning via shielding. In: Proceedings of the AAAI conference on artificial intelligence, vol 32","DOI":"10.1609\/aaai.v32i1.11797"},{"key":"10824_CR29","doi-asserted-by":"crossref","unstructured":"Alzantot M, Sharma Y, Elgohary A, Ho B-J, Srivastava M, Chang K-W (2018) Generating natural language adversarial examples. arXiv Preprint http:\/\/arxiv.org\/abs\/1804.07998","DOI":"10.18653\/v1\/D18-1316"},{"key":"10824_CR30","doi-asserted-by":"crossref","unstructured":"Arora U, Huang W, He H (2021) Types of out-of-distribution texts and how to detect them. arXiv Preprint http:\/\/arxiv.org\/abs\/2109.06827","DOI":"10.18653\/v1\/2021.emnlp-main.835"},{"key":"10824_CR31","unstructured":"Bai Y, Jones A, Ndousse K, Askell A, Chen A, DasSarma N, Drain D, Fort S, Ganguli D, Henighan T et\u00a0al (2022a) Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv Preprint http:\/\/arxiv.org\/abs\/2204.05862"},{"key":"10824_CR32","unstructured":"Bai Y, Kadavath S, Kundu S, Askell A, Kernion J, Jones A, Chen A, Goldie A, Mirhoseini A, McKinnon C et\u00a0al (2022b) Constitutional AI: harmlessness from AI feedback. arXiv Preprint http:\/\/arxiv.org\/abs\/2212.08073"},{"key":"10824_CR33","unstructured":"Balaji Y, Nah S, Huang X, Vahdat A, Song J, Kreis K, Aittala M, Aila T, Laine S, Catanzaro B et\u00a0al (2022) eDiff-I: text-to-image diffusion models with an ensemble of expert denoisers. arXiv Preprint http:\/\/arxiv.org\/abs\/2211.01324"},{"key":"10824_CR34","doi-asserted-by":"crossref","unstructured":"Balakrishnan A, Puranic AG, Qin X, Dokhanchi A, Deshmukh JV, Ben Amor H, Fainekos G (2019) Specifying and evaluating quality metrics for vision-based perception systems. In: Design, automation & test in Europe conference & exhibition (DATE). pp 1433\u20131438","DOI":"10.23919\/DATE.2019.8715114"},{"key":"10824_CR35","doi-asserted-by":"crossref","unstructured":"Bang Y, Cahyawijaya S, Lee N, Dai W, Su D, Wilie B, Lovenia H, Ji Z, Yu T, Chung W et\u00a0al (2023) A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.04023","DOI":"10.18653\/v1\/2023.ijcnlp-main.45"},{"key":"10824_CR36","doi-asserted-by":"crossref","unstructured":"Bartocci E, Falcone Y (2018) Lectures on runtime verification. Springer","DOI":"10.1007\/978-3-319-75632-5"},{"issue":"4","key":"10824_CR37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2000799.2000800","volume":"20","author":"A Bauer","year":"2011","unstructured":"Bauer A, Leucker M, Schallhart C (2011) Runtime verification for LTL and TLTL. ACM Trans Softw Eng Methodol 20(4):1\u201364","journal-title":"ACM Trans Softw Eng Methodol"},{"key":"10824_CR38","unstructured":"Belinkov Y, Bisk Y (2017) Synthetic and natural noise both break neural machine translation. arXiv Preprint http:\/\/arxiv.org\/abs\/1711.02173"},{"key":"10824_CR39","doi-asserted-by":"crossref","unstructured":"Bensalem S, Lakhnech Y, Saidi H (1996) Powerful techniques for the automatic generation of invariants. In: Computer aided verification: 8th international conference, CAV\u201996 New Brunswick, NJ, USA, July 31\u2013August 3, 1996 proceedings 8. Springer, pp 323\u2013335","DOI":"10.1007\/3-540-61474-5_80"},{"key":"10824_CR40","doi-asserted-by":"crossref","unstructured":"Bensalem S, Lakhnech Y, Owre S (1998) Invest: a tool for the verification of invariants. In: Computer aided verification: 10th international conference, CAV\u201998 Vancouver, BC, Canada, June 28\u2013July 2, 1998 proceedings 10. Springer, pp 505\u2013510","DOI":"10.1007\/BFb0028771"},{"key":"10824_CR41","doi-asserted-by":"crossref","unstructured":"Bensalem S, Cheng C-H, Huang X, Katsaros P, Molin A, Nickovic D, Peled D (2022) Formal specification for learning-enabled autonomous systems. In: International workshop on numerical software verification. Springer, pp 131\u2013143","DOI":"10.1007\/978-3-031-21222-2_8"},{"key":"10824_CR42","doi-asserted-by":"crossref","unstructured":"Bensalem S, Cheng C-H, Huang W, Huang X, Wu C, Zhao X (2023) What, indeed, is an achievable provable guarantee for learning-enabled safety critical systems. In: ISoLA 2023","DOI":"10.1007\/978-3-031-46002-9_4"},{"key":"10824_CR43","unstructured":"Berthier N, Alshareef A, Sharp J, Schewe S, Huang X (2021) Abstraction and symbolic execution of deep neural networks with Bayesian approximation of hidden features. arXiv Preprint http:\/\/arxiv.org\/abs\/2103.03704"},{"key":"10824_CR44","volume-title":"Automated theorem proving","author":"W Bibel","year":"2013","unstructured":"Bibel W (2013) Automated theorem proving. Springer Science & Business Media, Berlin"},{"key":"10824_CR45","unstructured":"Bitcoin energy consumption index. https:\/\/digiconomist.net\/bitcoin-energy-consumption. Accessed 17 Aug 2023"},{"key":"10824_CR46","doi-asserted-by":"crossref","unstructured":"Black S, Biderman S, Hallahan E, Anthony Q, Gao L, Golding L, He H, Leahy C, McDonell K, Phang J et\u00a0al (2022) GPT-Neox-20B: an open-source autoregressive language model. arXiv Preprint http:\/\/arxiv.org\/abs\/2204.06745","DOI":"10.18653\/v1\/2022.bigscience-1.9"},{"key":"10824_CR47","doi-asserted-by":"crossref","unstructured":"Bonaert G, Dimitrov DI, Baader M, Vechev M (2021) Fast and precise certification of transformers. In: Proceedings of the 42nd ACM SIGPLAN international conference on programming language design and implementation. pp 466\u2013481","DOI":"10.1145\/3453483.3454056"},{"key":"10824_CR48","doi-asserted-by":"crossref","unstructured":"Borji A (2023) A categorical archive of ChatGPT failures. CoRR. http:\/\/arxiv.org\/abs\/2302.03494","DOI":"10.21203\/rs.3.rs-2895792\/v1"},{"key":"10824_CR49","doi-asserted-by":"crossref","unstructured":"Botacin M (2023) GPThreats-3: is automatic malware generation a threat? In: 2023 IEEE security and privacy workshops (SPW). pp 238\u2013254","DOI":"10.1109\/SPW59333.2023.00027"},{"key":"#cr-split#-10824_CR50.1","unstructured":"Brants T, Popat AC, Xu P, Och FJ, Dean J (2007) Large language models in machine translation. In: Eisner J"},{"key":"#cr-split#-10824_CR50.2","unstructured":"(ed) EMNLP-CoNLL 2007, proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning, June 28-30, 2007, Prague, Czech Republic. ACL, pp 858-867"},{"key":"10824_CR51","unstructured":"Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler DM, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D (2020a) Language models are few-shot learners. In: Proceedings of the 34th international conference on neural information processing systems, NIPS\u201920, Red Hook, NY, USA, 2020. Curran Associates Inc"},{"key":"10824_CR52","first-page":"1877","volume":"33","author":"T Brown","year":"2020","unstructured":"Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A et al (2020b) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877\u20131901","journal-title":"Adv Neural Inf Process Syst"},{"key":"10824_CR53","unstructured":"Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler D, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D (2020c) Language models are few-shot learners. In: Larochelle H, Ranzato M, Hadsell R, Balcan M, Lin H (eds) Advances in neural information processing systems, vol\u00a033. Curran Associates, Inc., pp 1877\u20131901"},{"key":"10824_CR54","unstructured":"Bullwinkle M, Urban E (2023) Introduction to red teaming large language models (LLMS). https:\/\/learn.microsoft.com\/en-us\/azure\/ai-services\/openai\/concepts\/red-teaming. Accessed 20 Aug 2023"},{"key":"10824_CR55","unstructured":"Bursztein E (2018) Attacks against machine learning\u2014an overview. https:\/\/elie.net\/blog\/ai\/attacks-against-machine-learning-an-overview\/. Accessed 20 Aug 2023"},{"key":"10824_CR56","unstructured":"Cambiaso E, Caviglione L (2023) Scamming the scammers: using ChatGPT to reply mails for wasting time and resources. arXiv Preprint http:\/\/arxiv.org\/abs\/2303.13521"},{"key":"10824_CR57","doi-asserted-by":"crossref","unstructured":"Cao Y, Li D, Fang M, Zhou T, Gao J, Zhan Y, Tao D (2022) TASA: deceiving question answering models by twin answer sentences attack. arXiv Preprint http:\/\/arxiv.org\/abs\/2210.15221","DOI":"10.18653\/v1\/2022.emnlp-main.821"},{"key":"10824_CR58","unstructured":"Carlini N, Jagielski M, Choquette-Choo CA, Paleka D, Pearce W, Anderson H, Terzis A, Thomas K, Tram\u00e8r F (2023) Poisoning web-scale training datasets is practical. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.10149"},{"key":"10824_CR59","unstructured":"Chen B, Carvalho W, Baracaldo N, Ludwig H, Edwards B, Lee T, Molloy I, Srivastava B (2019) Detecting backdoor attacks on deep neural networks by activation clustering. In: SafeAI@ AAAI"},{"key":"10824_CR60","unstructured":"Chen M, Tworek J, Jun H, Yuan Q, de Oliveira Pinto HP, Kaplan J, Edwards H, Burda Y, Joseph N, Brockman G et\u00a0al (2021a) Evaluating large language models trained on code. arXiv Preprint http:\/\/arxiv.org\/abs\/2107.03374"},{"key":"10824_CR61","doi-asserted-by":"crossref","unstructured":"Chen X, Salem A, Chen D, Backes M, Ma S, Shen Q, Wu Z, Zhang Y (2021b) BadNL: backdoor attacks against NLP models with semantic-preserving improvements. In: Annual computer security applications conference. pp 554\u2013569","DOI":"10.1145\/3485832.3485837"},{"key":"10824_CR62","doi-asserted-by":"crossref","unstructured":"Chen S, Bi X, Gao R, Sun X (2022) Holistic sentence embeddings for better out-of-distribution detection. In: Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics, pp 6676\u20136686","DOI":"10.18653\/v1\/2022.findings-emnlp.497"},{"key":"10824_CR63","unstructured":"Chen L, Zaharia M, Zou J (2023a) How is ChatGPT\u2019s behavior changing over time? arXiv Preprint http:\/\/arxiv.org\/abs\/2307.09009"},{"key":"10824_CR64","doi-asserted-by":"crossref","unstructured":"Chen S, Yang W, Bi X, Sun X (2023b) Fine-tuning deteriorates general textual out-of-distribution detection by distorting task-agnostic features. In: Findings of the Association for Computational Linguistics: EACL 2023. pp 552\u2013567","DOI":"10.18653\/v1\/2023.findings-eacl.41"},{"key":"10824_CR65","doi-asserted-by":"crossref","unstructured":"Chen S, Kann BH, Foote MB, Aerts HJ, Savova GK, Mak RH, Bitterman DS (2023c) The utility of ChatGPT for cancer treatment information. medRxiv, pp 2023\u201303","DOI":"10.1101\/2023.03.16.23287316"},{"key":"10824_CR66","doi-asserted-by":"crossref","unstructured":"Cheng Y, Jiang L, Macherey W (2019a) Robust neural machine translation with doubly adversarial inputs. arXiv Preprint http:\/\/arxiv.org\/abs\/1906.02443","DOI":"10.18653\/v1\/P19-1425"},{"key":"10824_CR67","doi-asserted-by":"crossref","unstructured":"Cheng C, N\u00fchrenberg G, Yasuoka H (2019b) Runtime monitoring neuron activation patterns. In: DATE2019. pp 300\u2013303","DOI":"10.23919\/DATE.2019.8714971"},{"key":"10824_CR68","doi-asserted-by":"crossref","unstructured":"Cheng M, Yi J, Chen P-Y, Zhang H, Hsieh C-J (2020) Seq2Sick: evaluating the robustness of sequence-to-sequence models with adversarial examples. In: Proceedings of the AAAI conference on artificial intelligence, vol 34. pp 3601\u20133608","DOI":"10.1609\/aaai.v34i04.5767"},{"key":"10824_CR69","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1007\/978-3-031-19992-9_26","volume-title":"Automated technology for verification and analysis","author":"C-H Cheng","year":"2022","unstructured":"Cheng C-H, Wu C, Seferis E, Bensalem S (2022) Prioritizing corners in OOD detectors via symbolic string manipulation. In: Bouajjani A, Hol\u00edk L, Wu Z (eds) Automated technology for verification and analysis. Springer International Publishing, Cham, pp 397\u2013413"},{"key":"10824_CR70","unstructured":"Chiang W-L, Li Z, Lin Z, Sheng Y, Wu Z, Zhang H, Zheng L, Zhuang S, Zhuang Y, Gonzalez JE et\u00a0al (2023) Vicuna: an open-source chatbot impressing GPT-4 with 90%* ChatGPT quality. See https:\/\/vicuna.lmsys.org. Accessed 14 Apr 2023"},{"key":"10824_CR71","doi-asserted-by":"crossref","unstructured":"Cho JH, Hariharan B (2019) On the efficacy of knowledge distillation. In: Proceedings of the IEEE\/CVF international conference on computer vision. pp 4794\u20134802","DOI":"10.1109\/ICCV.2019.00489"},{"key":"10824_CR72","doi-asserted-by":"crossref","unstructured":"Cho H, Park C, Kang J, Yoo KM, Kim T, Lee S-G (2022) Enhancing out-of-distribution detection in natural language understanding via implicit layer ensemble. In: Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics, pp 783\u2013798","DOI":"10.18653\/v1\/2022.findings-emnlp.55"},{"key":"10824_CR73","unstructured":"Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S et\u00a0al (2022) PaLM: scaling language modeling with pathways. arXiv Preprint http:\/\/arxiv.org\/abs\/2204.02311"},{"key":"10824_CR74","unstructured":"Christiano PF, Leike J, Brown T, Martic M, Legg S, Amodei D (2017) Deep reinforcement learning from human preferences. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30. Curran Associates, Inc"},{"key":"10824_CR75","unstructured":"Clark K, Luong M-T, Le QV, Manning CD (2020) Electra: pre-training text encoders as discriminators rather than generators. arXiv Preprint http:\/\/arxiv.org\/abs\/2003.10555"},{"key":"10824_CR76","unstructured":"Cobbe K, Kosaraju V, Bavarian M, Chen M, Jun H, Kaiser L, Plappert M, Tworek J, Hilton J, Nakano R et\u00a0al (2021) Training verifiers to solve math word problems. arXiv Preprint http:\/\/arxiv.org\/abs\/2110.14168"},{"key":"10824_CR77","unstructured":"Cohen J, Rosenfeld E, Kolter Z (2019) Certified adversarial robustness via randomized smoothing. In: International conference on machine learning. PMLR, pp 1310\u20131320"},{"key":"10824_CR78","unstructured":"Croce F, Hein M (2020) Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In: International conference on machine learning. PMLR, pp 2206\u20132216"},{"key":"10824_CR79","doi-asserted-by":"crossref","first-page":"138872","DOI":"10.1109\/ACCESS.2019.2941376","volume":"7","author":"J Dai","year":"2019","unstructured":"Dai J, Chen C, Li Y (2019) A backdoor attack against LSTM-based text classification systems. IEEE Access 7:138872\u2013138878","journal-title":"IEEE Access"},{"key":"10824_CR80","doi-asserted-by":"crossref","unstructured":"Dan S, Roth D (2021) On the effects of transformer size on in-and out-of-domain calibration. In: Findings of the Association for Computational Linguistics: EMNLP 2021. pp 2096\u20132101","DOI":"10.18653\/v1\/2021.findings-emnlp.180"},{"issue":"1","key":"10824_CR81","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MM.2018.112130359","volume":"38","author":"M Davies","year":"2018","unstructured":"Davies M, Srinivasa N, Lin T-H, Chinya G, Cao Y, Choday SH, Dimou G, Joshi P, Imam N, Jain S et al (2018) Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 38(1):82\u201399","journal-title":"IEEE Micro"},{"key":"10824_CR82","doi-asserted-by":"crossref","unstructured":"De\u00a0Moura L, Bj\u00f8rner N (2008) Z3: an efficient SMT solver. In: Tools and algorithms for the construction and analysis of systems: 14th international conference, TACAS 2008, held as part of the joint European conferences on theory and practice of software, ETAPS 2008, Budapest, Hungary, March 29\u2013April 6, 2008. Proceedings 14. Springer, pp 337\u2013340","DOI":"10.1007\/978-3-540-78800-3_24"},{"issue":"3","key":"10824_CR83","doi-asserted-by":"crossref","first-page":"498","DOI":"10.1016\/j.joule.2022.02.005","volume":"6","author":"A De Vries","year":"2022","unstructured":"De Vries A, Gallersd\u00f6rfer U, Klaa\u00dfen L, Stoll C (2022) Revisiting bitcoin\u2019s carbon footprint. Joule 6(3):498\u2013502","journal-title":"Joule"},{"key":"10824_CR84","doi-asserted-by":"crossref","unstructured":"Desai S, Durrett G (2020) Calibration of pre-trained transformers. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), Online, Nov. 2020. Association for Computational Linguistics, pp 295\u2013302","DOI":"10.18653\/v1\/2020.emnlp-main.21"},{"key":"10824_CR85","doi-asserted-by":"crossref","unstructured":"Deshpande A, Murahari V, Rajpurohit T, Kalyan A, Narasimhan K (2023) Toxicity in ChatGPT: analyzing persona-assigned language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.05335","DOI":"10.18653\/v1\/2023.findings-emnlp.88"},{"key":"10824_CR86","unstructured":"Dettmers T, Lewis M, Belkada Y, Zettlemoyer L (2022) GPT3. int8 (): 8-bit matrix multiplication for transformers at scale. In: Advances in neural information processing systems, vol 35. pp 30318\u201330332"},{"key":"10824_CR87","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K (2018) BERT: pre-training of deep bidirectional transformers for language understanding. arXiv Preprint http:\/\/arxiv.org\/abs\/1810.04805"},{"key":"10824_CR88","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the Association for Computational Linguistics: human language technologies, volume 1 (long and short papers), Minneapolis, Minnesota, June 2019. Association for Computational Linguistics, pp 4171\u20134186"},{"key":"10824_CR89","unstructured":"DeVries T, Taylor GW (2018) Learning confidence for out-of-distribution detection in neural networks. arXiv Preprint http:\/\/arxiv.org\/abs\/1802.04865"},{"key":"10824_CR90","unstructured":"Dey N (2023) GPT: a family of open, compute-efficient, large language models. https:\/\/www.cerebras.net\/blog\/cerebras-gpt-a-family-of-open-compute-efficient-large-language-models\/. Accessed 20 Aug 2023"},{"key":"10824_CR91","unstructured":"Dodge J, Ilharco G, Schwartz R, Farhadi A, Hajishirzi H, Smith N (2020) Fine-tuning pretrained language models: weight initializations, data orders, and early stopping. arXiv Preprint http:\/\/arxiv.org\/abs\/2002.06305"},{"issue":"2021","key":"10824_CR92","first-page":"15","volume":"21","author":"T Du","year":"2021","unstructured":"Du T, Ji S, Shen L, Zhang Y, Li J, Shi J, Fang C, Yin J, Beyah R, Wang T (2021) CERT-RNN: towards certifying the robustness of recurrent neural networks. CCS 21(2021):15\u201319","journal-title":"CCS"},{"key":"10824_CR93","unstructured":"Du N, Huang Y, Dai AM, Tong S, Lepikhin D, Xu Y, Krikun M, Zhou Y, Yu AW, Firat O et\u00a0al (2022) GLaM: efficient scaling of language models with mixture-of-experts. In: International conference on machine learning. PMLR, pp 5547\u20135569"},{"key":"10824_CR94","doi-asserted-by":"crossref","unstructured":"Duan H, Yang Y, Abbasi A, Tam KY (2022) BARLE: background-aware representation learning for background shift out-of-distribution detection. In: Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, Dec. 2022. Association for Computational Linguistics, pp 750\u2013764","DOI":"10.18653\/v1\/2022.findings-emnlp.53"},{"key":"10824_CR95","unstructured":"Duan J, Kong F, Wang S, Shi X, Xu K (2023) Are diffusion models vulnerable to membership inference attacks? arXiv Preprint http:\/\/arxiv.org\/abs\/2302.01316"},{"issue":"2","key":"10824_CR96","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3185517","volume":"8","author":"JJ Dudley","year":"2018","unstructured":"Dudley JJ, Kristensson PO (2018) A review of user interface design for interactive machine learning. ACM Trans Interact Intell Syst 8(2):1\u201337","journal-title":"ACM Trans Interact Intell Syst"},{"key":"10824_CR97","unstructured":"E2Analyst (2023) GPT-4: everything you want to know about OpenAI\u2019s new AI model. https:\/\/medium.com\/predict\/gpt-4-everything-you-want-to-know-about-openais-new-ai-model-a5977b42e495. Accessed 20 Aug 2023"},{"key":"10824_CR98","doi-asserted-by":"crossref","unstructured":"Ebrahimi J, Rao A, Lowd D, Dou D (2017) HotFlip: white-box adversarial examples for text classification. arXiv Preprint http:\/\/arxiv.org\/abs\/1712.06751","DOI":"10.18653\/v1\/P18-2006"},{"key":"10824_CR99","unstructured":"Edwards B (2023) Study claims ChatGPT is losing capability, but some experts aren\u2019t convinced. https:\/\/arstechnica.com\/information-technology\/2023\/07\/is-chatgpt-getting-worse-over-time-study-claims-yes-but-others-arent-sure\/. Accessed 20 Aug 2023"},{"issue":"4","key":"10824_CR100","first-page":"15","volume":"5","author":"D Eppstein","year":"1996","unstructured":"Eppstein D (1996) Zonohedra and zonotopes. Math Educ Res 5(4):15\u201321","journal-title":"Math Educ Res"},{"key":"10824_CR101","doi-asserted-by":"crossref","unstructured":"Esser P, Rombach R, Ommer B (2021) Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. pp 12873\u201312883","DOI":"10.1109\/CVPR46437.2021.01268"},{"key":"10824_CR102","unstructured":"EU GDPR (2016). https:\/\/gdpr-info.eu. Accessed 20 Aug 2023"},{"key":"10824_CR103","doi-asserted-by":"crossref","first-page":"06","DOI":"10.1080\/23311916.2023.2222988","volume":"10","author":"F Farhat","year":"2023","unstructured":"Farhat F, Sohail S, Madsen D (2023) How trustworthy is ChatGPT? The case of bibliometric analyses. Cogent Eng 10:06","journal-title":"Cogent Eng"},{"key":"10824_CR104","first-page":"1","volume":"23","author":"W Fedus","year":"2021","unstructured":"Fedus W, Zoph B, Shazeer N (2021) Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. J Mach Learn Res 23:1\u201340","journal-title":"J Mach Learn Res"},{"key":"10824_CR105","unstructured":"Feinman R, Curtin RR, Shintre S, Gardner AB (2017) Detecting adversarial samples from artifacts. arXiv Preprint http:\/\/arxiv.org\/abs\/1703.00410"},{"key":"10824_CR106","doi-asserted-by":"crossref","unstructured":"Fitting M (1996) First-order logic and automated theorem proving. Graduate texts in computer science, second edn. Springer","DOI":"10.1007\/978-1-4612-2360-3"},{"key":"10824_CR107","unstructured":"Frantar E, Alistarh D (2022) Optimal brain compression: a framework for accurate post-training quantization and pruning. arXiv Preprint http:\/\/arxiv.org\/abs\/2208.11580"},{"key":"10824_CR108","unstructured":"Frantar E, Ashkboos S, Hoefler T, Alistarh D (2023) GPTQ: accurate quantization for generative pre-trained transformers. In: International conference on learning representations"},{"key":"10824_CR109","unstructured":"Frieder S, Pinchetti L, Griffiths R-R, Salvatori T, Lukasiewicz T, Petersen PC, Chevalier A, Berner J (2023) Mathematical capabilities of ChatGPT. arXiv Preprint http:\/\/arxiv.org\/abs\/2301.13867"},{"key":"10824_CR110","doi-asserted-by":"crossref","unstructured":"Gangal V, Arora A, Einolghozati A, Gupta S (2020) Likelihood ratios and generative classifiers for unsupervised out-of-domain detection in task oriented dialog. In: Proceedings of the AAAI conference on artificial intelligence, vol 34. pp 7764\u20137771","DOI":"10.1609\/aaai.v34i05.6280"},{"key":"10824_CR111","unstructured":"Ganguli D, Askell A, Schiefer N, Liao T, Luko\u0161i\u016bt\u0117 K, Chen A, Goldie A, Mirhoseini A, Olsson C, Hernandez D et\u00a0al (2023) The capacity for moral self-correction in large language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.07459"},{"key":"10824_CR112","doi-asserted-by":"crossref","unstructured":"Gao J, Lanchantin J, Soffa ML, Qi Y (2018) Black-box generation of adversarial text sequences to evade deep learning classifiers. In: 2018 IEEE security and privacy workshops (SPW). IEEE, pp 50\u201356","DOI":"10.1109\/SPW.2018.00016"},{"key":"10824_CR113","unstructured":"Gao L, Madaan A, Zhou S, Alon U, Liu P, Yang Y, Callan J, Neubig G (2023) PAL: program-aided language models"},{"issue":"1","key":"10824_CR114","first-page":"1437","volume":"16","author":"J Garc\u0131a","year":"2015","unstructured":"Garc\u0131a J, Fern\u00e1ndez F (2015) A comprehensive survey on safe reinforcement learning. J Mach Learn Res 16(1):1437\u20131480","journal-title":"J Mach Learn Res"},{"key":"10824_CR115","unstructured":"Goodfellow I, Papernot N (2017) The challenge of verification and testing of machine learning. Cleverhans-blog"},{"key":"10824_CR116","unstructured":"Goodfellow IJ, Shlens J, Szegedy C (2014) Explaining and harnessing adversarial examples. arXiv Preprint http:\/\/arxiv.org\/abs\/1412.6572"},{"issue":"11","key":"10824_CR117","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1145\/3422622","volume":"63","author":"I Goodfellow","year":"2020","unstructured":"Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun ACM 63(11):139\u2013144","journal-title":"Commun ACM"},{"key":"10824_CR118","unstructured":"Goodin D (2023) Hackers are selling a service that bypasses ChatGPT restrictions on malware. https:\/\/arstechnica.com\/information-technology\/2023\/02\/now-open-fee-based-telegram-service-that-uses-chatgpt-to-generate-malware\/. Accessed 20 Aug 2023"},{"key":"10824_CR119","unstructured":"Gopinath D, Wang K, Zhang M, Pasareanu CS, Khurshid S (2018) Symbolic execution for deep neural networks. arXiv Preprint http:\/\/arxiv.org\/abs\/1807.10439"},{"key":"10824_CR120","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1007\/s11263-021-01453-z","volume":"129","author":"J Gou","year":"2021","unstructured":"Gou J, Yu B, Maybank SJ, Tao D (2021) Knowledge distillation: a survey. Int J Comput Vis 129:1789\u20131819","journal-title":"Int J Comput Vis"},{"key":"10824_CR121","unstructured":"Gowal S, Dvijotham K, Stanforth R, Bunel R, Qin C, Uesato J, Arandjelovic R, Mann T, Kohli P (2018) On the effectiveness of interval bound propagation for training verifiably robust models. arXiv Preprint http:\/\/arxiv.org\/abs\/1810.12715"},{"key":"10824_CR122","unstructured":"Goyal S, Doddapaneni S, Khapra MM, Ravindran B (2022) A survey in adversarial defences and robustness in NLP. arXiv Preprint http:\/\/arxiv.org\/abs\/2203.06414"},{"key":"10824_CR123","unstructured":"GPT-4\u2019s details are leaked. https:\/\/archive.md\/2RQ8X. Accessed 17 Aug 2023"},{"key":"10824_CR124","unstructured":"Greshake K, Abdelnabi S, Mishra S, Endres C, Holz T, Fritz M (2023) More than you\u2019ve asked for: a comprehensive analysis of novel prompt injection threats to application-integrated large language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.12173"},{"key":"10824_CR125","doi-asserted-by":"crossref","first-page":"47230","DOI":"10.1109\/ACCESS.2019.2909068","volume":"7","author":"T Gu","year":"2019","unstructured":"Gu T, Liu K, Dolan-Gavitt B, Garg S (2019) BadNets: evaluating backdooring attacks on deep neural networks. IEEE Access 7:47230\u201347244","journal-title":"IEEE Access"},{"key":"10824_CR126","doi-asserted-by":"crossref","unstructured":"Gu J-C, Li T, Liu Q, Ling Z-H, Su Z, Wei S, Zhu X (2020) Speaker-aware BERT for multi-turn response selection in retrieval-based chatbots. In: Proceedings of the 29th ACM international conference on information & knowledge management, CIKM \u201920, New York, NY, USA, 2020. Association for Computing Machinery, pp 2041\u20132044","DOI":"10.1145\/3340531.3412330"},{"key":"10824_CR127","unstructured":"Gu S, Yang L, Du Y, Chen G, Walter F, Wang J, Yang Y, Knoll A (2022) A review of safe reinforcement learning: methods, theory and applications. arXiv Preprint http:\/\/arxiv.org\/abs\/2205.10330"},{"key":"10824_CR128","unstructured":"Gu Y, Dong L, Wei F, Huang M (2023a) Knowledge distillation of large language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2306.08543"},{"key":"10824_CR129","doi-asserted-by":"crossref","unstructured":"Gu S, Kshirsagar A, Du Y, Chen G, Yang Y, Peters J, Knoll A (2023b) A human-centered safe robot reinforcement learning framework with interactive behaviors. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.13137","DOI":"10.3389\/fnbot.2023.1280341"},{"issue":"37","key":"10824_CR130","doi-asserted-by":"crossref","first-page":"eaay7120","DOI":"10.1126\/scirobotics.aay7120","volume":"4","author":"D Gunning","year":"2019","unstructured":"Gunning D, Stefik M, Choi J, Miller T, Stumpf S, Yang G-Z (2019) XAI\u2014explainable artificial intelligence. Sci Robot 4(37):eaay7120","journal-title":"Sci Robot"},{"key":"10824_CR131","unstructured":"Guo B, Zhang X, Wang Z, Jiang M, Nie J, Ding Y, Yue J, Wu Y (2023) How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection. CoRR. abs\/2301.07597"},{"key":"10824_CR132","doi-asserted-by":"crossref","unstructured":"He R, Sun S, Yang J, Bai S, Qi X (2022) Knowledge distillation as efficient pre-training: faster convergence, higher data-efficiency, and better transferability. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. pp 9161\u20139171","DOI":"10.1109\/CVPR52688.2022.00895"},{"key":"10824_CR133","unstructured":"Hendrycks D, Gimpel K (2016) A baseline for detecting misclassified and out-of-distribution examples in neural networks. In: International conference on learning representations"},{"key":"10824_CR134","doi-asserted-by":"crossref","unstructured":"Hendrycks D, Liu X, Wallace E, Dziedzic A, Krishnan R, Song D (2020) Pretrained transformers improve out-of-distribution robustness. In: Proceedings of the 58th annual meeting of the association for computational linguistics. pp 2744\u20132751","DOI":"10.18653\/v1\/2020.acl-main.244"},{"key":"10824_CR135","unstructured":"Henzinger TA, Lukina A, Schilling C (2020) Outside the box: abstraction-based monitoring of neural networks. In: ECAI2020"},{"key":"10824_CR136","unstructured":"Hinton G, Vinyals O, Dean J (2015) Distilling the knowledge in a neural network. arXiv Preprint http:\/\/arxiv.org\/abs\/1503.02531"},{"key":"10824_CR137","unstructured":"Hintze A (2023) ChatGPT believes it is conscious. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.12898"},{"key":"10824_CR138","unstructured":"Hoffmann J, Borgeaud S, Mensch A, Buchatskaya E, Cai T, Rutherford E, de Las Casas D, Hendricks LA, Welbl J, Clark A et\u00a0al (2022) Training compute-optimal large language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2203.15556"},{"key":"10824_CR139","doi-asserted-by":"crossref","unstructured":"Holmes J, Liu Z, Zhang L, Ding Y, Sio TT, McGee LA, Ashman JB, Li X, Liu T, Shen J et\u00a0al (2023) Evaluating large language models on a highly-specialized topic, radiation oncology physics. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.01938","DOI":"10.3389\/fonc.2023.1219326"},{"key":"10824_CR140","unstructured":"Hosseini H, Kannan S, Zhang B, Poovendran R (2017) Deceiving Google\u2019s perspective API built for detecting toxic comments. arXiv Preprint http:\/\/arxiv.org\/abs\/1702.08138"},{"key":"10824_CR141","unstructured":"Houlsby N, Giurgiu A, Jastrzebski S, Morrone B, De\u00a0Laroussilhe Q, Gesmundo A, Attariyan M, Gelly S (2019) Parameter-efficient transfer learning for NLP. In: International conference on machine learning. PMLR, pp 2790\u20132799"},{"key":"10824_CR142","doi-asserted-by":"crossref","unstructured":"Hrinchuk O, Popova M, Ginsburg B (2020) Correction of automatic speech recognition with transformer sequence-to-sequence model. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 7074\u20137078","DOI":"10.1109\/ICASSP40776.2020.9053051"},{"key":"10824_CR143","unstructured":"Hu Z, Yang Z, Liang X, Salakhutdinov R, Xing EP (2017) Toward controlled generation of text. In: International conference on machine learning. PMLR, pp 1587\u20131596"},{"key":"10824_CR144","unstructured":"Hu EJ, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W (2021) LoRA: low-rank adaptation of large language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2106.09685"},{"key":"10824_CR145","unstructured":"Hu EJ, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W (2022) LoRA: low-rank adaptation of large language models. In: International conference on learning representations"},{"key":"10824_CR146","doi-asserted-by":"crossref","unstructured":"Huang X, Jin G, Ruan W (2012) Machine learning basics. In: Machine learning safety. Springer, pp 3\u201313","DOI":"10.1007\/978-981-19-6814-3_1"},{"key":"10824_CR147","doi-asserted-by":"crossref","unstructured":"Huang X, Kwiatkowska M, Wang S, Wu M (2017) Safety verification of deep neural networks. In: Majumdar R, Kuncak V (eds) Computer aided verification\u201429th international conference, CAV 2017, Heidelberg, Germany, July 24\u201328, 2017, proceedings, part I, volume 10426 of lecture notes in computer science. Springer, pp 3\u201329","DOI":"10.1007\/978-3-319-63387-9_1"},{"key":"10824_CR148","doi-asserted-by":"crossref","unstructured":"Huang P-S, Stanforth R, Welbl J, Dyer C, Yogatama D, Gowal S, Dvijotham K, Kohli P (2019a) Achieving verified robustness to symbol substitutions via interval bound propagation. arXiv Preprint http:\/\/arxiv.org\/abs\/1909.01492","DOI":"10.18653\/v1\/D19-1419"},{"key":"10824_CR149","unstructured":"Huang X, Alzantot M, Srivastava M (2019b) NeuronInspect: detecting backdoors in neural networks via output explanations. arXiv Preprint http:\/\/arxiv.org\/abs\/1911.07399"},{"key":"10824_CR150","doi-asserted-by":"crossref","first-page":"100270","DOI":"10.1016\/j.cosrev.2020.100270","volume":"37","author":"X Huang","year":"2020","unstructured":"Huang X, Kroening D, Ruan W, Sharp J, Sun Y, Thamo E, Wu M, Yi X (2020a) A survey of safety and trustworthiness of deep neural networks: verification, testing, adversarial attack and defence, and interpretability. Comput Sci Rev 37:100270","journal-title":"Comput Sci Rev"},{"key":"10824_CR151","unstructured":"Huang H, Li Z, Wang L, Chen S, Dong B, Zhou X (2020b) Feature space singularity for out-of-distribution detection. arXiv Preprint http:\/\/arxiv.org\/abs\/2011.14654"},{"issue":"3","key":"10824_CR152","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1109\/TR.2021.3080664","volume":"71","author":"W Huang","year":"2021","unstructured":"Huang W, Sun Y, Zhao X, Sharp J, Ruan W, Meng J, Huang X (2021) Coverage-guided testing for recurrent neural networks. IEEE Trans Reliab 71(3):1191\u20131206","journal-title":"IEEE Trans Reliab"},{"key":"10824_CR153","doi-asserted-by":"crossref","unstructured":"Huang X, Ruan W, Tang Q, Zhao X (2022a) Bridging formal methods and machine learning with global optimisation. In: Formal methods and software engineering: 23rd international conference on formal engineering methods, ICFEM 2022, Madrid, Spain, October 24\u201327, 2022, proceedings. Springer-Verlag, Berlin, Heidelberg, pp 1\u201319","DOI":"10.1007\/978-3-031-17244-1_1"},{"key":"10824_CR154","unstructured":"Huang W, Zhao X, Banks A, Cox V, Huang X (2022b) Hierarchical distribution-aware testing of deep learning. arXiv Preprint http:\/\/arxiv.org\/abs\/2205.08589"},{"key":"10824_CR155","doi-asserted-by":"crossref","unstructured":"Huang W, Zhao X, Jin G, Huang X (2022c) Safari: versatile and efficient evaluations for robustness of interpretability. arXiv Preprint http:\/\/arxiv.org\/abs\/2208.09418","DOI":"10.1109\/ICCV51070.2023.00190"},{"key":"10824_CR156","unstructured":"Ilyas A, Santurkar S, Tsipras D, Engstrom L, Tran B, Madry A (2019) Adversarial examples are not bugs, they are features. In: Advances in neural information processing systems, vol 32"},{"key":"10824_CR157","unstructured":"Italy became the first western country to ban ChatGPT. https:\/\/www.cnbc.com\/2023\/04\/04\/italy-has-banned-chatgpt-heres-what-other-countries-are-doing.html. Accessed 17 Aug 2023"},{"key":"10824_CR158","unstructured":"Ivankay A, Girardi I, Marchiori C, Frossard P (2022) Fooling explanations in text classifiers. arXiv Preprint http:\/\/arxiv.org\/abs\/2206.03178"},{"key":"10824_CR159","doi-asserted-by":"crossref","unstructured":"Iyyer, M Wieting J, Gimpel K, Zettlemoyer L (2018) Adversarial example generation with syntactically controlled paraphrase networks. arXiv Preprint http:\/\/arxiv.org\/abs\/1804.06059","DOI":"10.18653\/v1\/N18-1170"},{"issue":"1","key":"10824_CR160","doi-asserted-by":"crossref","first-page":"2","DOI":"10.3390\/technologies9010002","volume":"9","author":"A Jaiswal","year":"2020","unstructured":"Jaiswal A, Babu AR, Zadeh MZ, Banerjee D, Makedon F (2020) A survey on contrastive self-supervised learning. Technologies 9(1):2","journal-title":"Technologies"},{"key":"10824_CR161","doi-asserted-by":"crossref","unstructured":"Jang M, Lukasiewicz T (2023) Consistency analysis of ChatGPT. arXiv Preprint http:\/\/arxiv.org\/abs\/2303.06273","DOI":"10.18653\/v1\/2023.emnlp-main.991"},{"key":"10824_CR162","unstructured":"Jansen N, K\u00f6nighofer B, Junges S, Bloem R (2018) Shielded decision-making in MDPs. arXiv Preprint http:\/\/arxiv.org\/abs\/1807.06096"},{"key":"10824_CR163","volume-title":"Safe reinforcement learning using probabilistic shields","author":"N Jansen","year":"2020","unstructured":"Jansen N, K\u00f6nighofer B, Junges J, Serban A, Bloem R (2020) Safe reinforcement learning using probabilistic shields. Schloss Dagstuhl, Dagstuhl"},{"key":"10824_CR164","unstructured":"Ji Y, Gong Y, Peng Y, Ni C, Sun P, Pan D, Ma B, Li X (2023) Exploring ChatGPT\u2019s ability to rank content: a preliminary study on consistency with human preferences"},{"key":"10824_CR165","doi-asserted-by":"crossref","unstructured":"Jia R, Liang P (2017) Adversarial examples for evaluating reading comprehension systems. arXiv Preprint http:\/\/arxiv.org\/abs\/1707.07328","DOI":"10.18653\/v1\/D17-1215"},{"key":"10824_CR166","doi-asserted-by":"crossref","unstructured":"Jia R, Raghunathan A, G\u00f6ksel K, Liang P (2019) Certified robustness to adversarial word substitutions. arXiv Preprint http:\/\/arxiv.org\/abs\/1909.00986","DOI":"10.18653\/v1\/D19-1423"},{"key":"10824_CR167","unstructured":"Jiang AQ, Welleck S, Zhou JP, Li W, Liu J, Jamnik M, Lacroix T, Wu Y, Lample G (2022) Draft, sketch, and prove: guiding formal theorem provers with informal proofs. arXiv Preprint http:\/\/arxiv.org\/abs\/2210.12283"},{"key":"10824_CR168","unstructured":"Jiao W, Wang W, Huang J-t, Wang X, Tu Z (2023) Is ChatGPT a good translator? A preliminary study. arXiv Preprint http:\/\/arxiv.org\/abs\/2301.08745"},{"key":"10824_CR169","doi-asserted-by":"crossref","unstructured":"Jin D, Jin Z, Zhou JT, Szolovits P (2020) Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In: Proceedings of the AAAI conference on artificial intelligence, vol 34. pp 8018\u20138025","DOI":"10.1609\/aaai.v34i05.6311"},{"key":"10824_CR170","unstructured":"Kalyan KS, Rajasekharan A, Sangeetha S (2021) AMMUS: a survey of transformer-based pretrained models in natural language processing. arXiv Preprint http:\/\/arxiv.org\/abs\/2108.05542"},{"issue":"9","key":"10824_CR171","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1145\/3546954","volume":"65","author":"S Kambhampati","year":"2022","unstructured":"Kambhampati S (2022) Changing the nature of AI research. Commun ACM 65(9):8\u20139","journal-title":"Commun ACM"},{"key":"10824_CR172","unstructured":"Kande R, Pearce H, Tan B, Dolan-Gavitt B, Thakur S, Karri R, Rajendran J (2023) LLM-assisted generation of hardware assertions. CoRR. abs\/2306.14027"},{"key":"10824_CR173","doi-asserted-by":"crossref","unstructured":"Kang D, Li X, Stoica I, Guestrin C, Zaharia M, Hashimoto T (2023a) Exploiting programmatic behavior of LLMS: dual-use through standard security attacks. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.05733","DOI":"10.1109\/SPW63631.2024.00018"},{"key":"10824_CR174","unstructured":"Kang Y, Zhang Q, Roth R (2023b) The ethics of AI-generated maps: a study of DALLE 2 and implications for cartography. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.10743"},{"key":"10824_CR175","unstructured":"Kaplan J, McCandlish S, Henighan T, Brown TB, Chess B, Child R, Gray S, Radford A, Wu J, Amodei D (2020) Scaling laws for neural language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2001.08361"},{"key":"10824_CR176","doi-asserted-by":"crossref","unstructured":"Katz DM, Bommarito MJ, Gao S, Arredondo P (2023) GPT-4 passes the bar exam. Available at SSRN 4389233","DOI":"10.2139\/ssrn.4389233"},{"key":"10824_CR177","doi-asserted-by":"crossref","unstructured":"Khoury R, Avila AR, Brunelle J, Camara BM (2023) How secure is code generated by ChatGPT? arXiv Preprint http:\/\/arxiv.org\/abs\/2304.09655","DOI":"10.1109\/SMC53992.2023.10394237"},{"key":"10824_CR178","doi-asserted-by":"crossref","first-page":"5","DOI":"10.30582\/kdps.2023.36.1.5","volume":"141","author":"Y-M Kim","year":"2023","unstructured":"Kim Y-M (2023) Data and fair use. Korea Copyright Commission 141:5\u201353","journal-title":"Korea Copyright Commission"},{"key":"10824_CR179","unstructured":"Ko C-Y, Lyu Z, Weng L, Daniel L, Wong N, Lin D (2019) POPQORN: quantifying robustness of recurrent neural networks. In: International conference on machine learning. PMLR, pp 3468\u20133477"},{"key":"10824_CR180","unstructured":"Koh JY, Fried D, Salakhutdinov R (2023) Generating images with multimodal language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2305.17216"},{"key":"10824_CR181","unstructured":"Kuleshov V, Thakoor S, Lau T, Ermon S (2018) Adversarial examples for natural language classification problems. arXiv Preprint"},{"key":"10824_CR182","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1162\/tacl_a_00318","volume":"8","author":"A Kumar","year":"2020","unstructured":"Kumar A, Ahuja K, Vadapalli R, Talukdar P (2020) Syntax-guided controlled generation of paraphrases. Trans Assoc Comput Linguist 8:330\u2013345","journal-title":"Trans Assoc Comput Linguist"},{"key":"10824_CR183","doi-asserted-by":"crossref","unstructured":"Kurita K, Michel P, Neubig G (2020) Weight poisoning attacks on pretrained models. In: Proceedings of the 58th annual meeting of the association for computational linguistics. pp 2793\u20132806","DOI":"10.18653\/v1\/2020.acl-main.249"},{"key":"10824_CR184","doi-asserted-by":"crossref","unstructured":"La\u00a0Malfa E, Wu M, Laurenti L, Wang B, Hartshorn A, Kwiatkowska M (2020) Assessing robustness of text classification through maximal safe radius computation. arXiv Preprint http:\/\/arxiv.org\/abs\/2010.02004","DOI":"10.18653\/v1\/2020.findings-emnlp.266"},{"key":"10824_CR185","unstructured":"Lam M, Sethi R, Ullman JD, Aho A (2006) Compilers: principles, techniques, and tools. Pearson Education"},{"key":"10824_CR186","unstructured":"Lambert N, Castricato L, von Werra L, Havrilla A (2022) Illustrating reinforcement learning from human feedback (RLHF). Hugging Face Blog. https:\/\/huggingface.co\/blog\/rlhf"},{"key":"10824_CR187","unstructured":"Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) Albert: a lite BERT for self-supervised learning of language representations. arXiv Preprint http:\/\/arxiv.org\/abs\/1909.11942"},{"key":"10824_CR188","unstructured":"Lee P (2016) Learning from Tay\u2019s introduction. https:\/\/blogs.microsoft.com\/blog\/2016\/03\/25\/learning-tays-introduction\/. Accessed 20 Aug 2023"},{"key":"10824_CR189","doi-asserted-by":"crossref","first-page":"6","DOI":"10.3352\/jeehp.2023.20.6","volume":"20","author":"JY Lee","year":"2023","unstructured":"Lee JY (2023) Can an artificial intelligence chatbot be the author of a scholarly article? J Educ Eval Health Prof 20:6","journal-title":"J Educ Eval Health Prof"},{"key":"10824_CR190","unstructured":"Lee C, Cho K, Kang W (2019) Mixout: effective regularization to finetune large-scale pretrained language models. arXiv Preprint http:\/\/arxiv.org\/abs\/1909.11299"},{"key":"10824_CR191","unstructured":"Lee N, Bang Y, Madotto A, Fung P (2020) Misinformation has high perplexity. arXiv Preprint http:\/\/arxiv.org\/abs\/2006.04666"},{"key":"10824_CR192","unstructured":"Lee K, Liu H, Ryu M, Watkins O, Du Y, Boutilier C, Abbeel P, Ghavamzadeh M, Gu SS (2023) Aligning text-to-image models using human feedback. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.12192"},{"key":"10824_CR193","doi-asserted-by":"crossref","unstructured":"Lei Y, Cao Y, Li D, Zhou T, Fang M, Pechenizkiy M (2022) Phrase-level textual adversarial attack with label preservation. arXiv Preprint http:\/\/arxiv.org\/abs\/2205.10710","DOI":"10.18653\/v1\/2022.findings-naacl.83"},{"key":"10824_CR194","unstructured":"Lepikhin D, Lee H, Xu Y, Chen D, Firat O, Huang Y, Krikun M, Shazeer N, Chen Z (2020) GShard: scaling giant models with conditional computation and automatic sharding. arXiv Preprint http:\/\/arxiv.org\/abs\/2006.16668"},{"key":"10824_CR195","doi-asserted-by":"crossref","unstructured":"Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O, Stoyanov V, Zettlemoyer L (2020) BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of the 58th annual meeting of the association for computational linguistics, Online, July 2020. Association for Computational Linguistics, pp 7871\u20137880","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"10824_CR196","doi-asserted-by":"crossref","unstructured":"Li J, Ji S, Du T, Li B, Wang T (2018a) TextBugger: generating adversarial text against real-world applications. arXiv Preprint http:\/\/arxiv.org\/abs\/1812.05271","DOI":"10.14722\/ndss.2019.23138"},{"key":"10824_CR197","unstructured":"Li Y, Ding L, Gao X (2018b) On the decision boundary of deep neural networks. arXiv Preprint http:\/\/arxiv.org\/abs\/1808.05385"},{"key":"10824_CR198","doi-asserted-by":"crossref","unstructured":"Li S, Liu H, Dong T, Zhao BZH, Xue M, Zhu H, Lu J (2021a) Hidden backdoors in human-centric language models. In: CCS \u201921: 2021 ACM SIGSAC conference on computer and communications security, virtual event, Republic of Korea, November 15\u201319, 2021. ACM, pp 3123\u20133140","DOI":"10.1145\/3460120.3484576"},{"key":"10824_CR199","doi-asserted-by":"crossref","unstructured":"Li X, Li J, Sun X, Fan C, Zhang T, Wu F, Meng Y, Zhang J (2021b) kFolden: k-fold ensemble for out-of-distribution detection-fold ensemble for out-of-distribution detection. In: Proceedings of the 2021 conference on empirical methods in natural language processing. pp 3102\u20133115","DOI":"10.18653\/v1\/2021.emnlp-main.248"},{"key":"10824_CR200","doi-asserted-by":"crossref","unstructured":"Li J, Tang T, Zhao WX, Nie JY, Wen J-R (2022) Pretrained language models for text generation: a survey. arXiv Preprint http:\/\/arxiv.org\/abs\/2201.05273","DOI":"10.24963\/ijcai.2021\/612"},{"key":"10824_CR201","doi-asserted-by":"crossref","unstructured":"Li J, Cheng X, Zhao WX, Nie J-Y, Wen J-R (2023a) HaluEval: a large-scale hallucination evaluation benchmark for large language models. arXiv e-prints, p arXiv\u20132305","DOI":"10.18653\/v1\/2023.emnlp-main.397"},{"key":"10824_CR202","doi-asserted-by":"crossref","unstructured":"Li H, Guo D, Fan W, Xu M, Song Y (2023b) Multi-step jailbreaking privacy attacks on ChatGPT. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.05197","DOI":"10.18653\/v1\/2023.findings-emnlp.272"},{"key":"10824_CR203","doi-asserted-by":"crossref","unstructured":"Liang B, Li H, Su M, Bian P, Li X, Shi W (2017) Deep text classification can be fooled. arXiv Preprint http:\/\/arxiv.org\/abs\/1704.08006","DOI":"10.24963\/ijcai.2018\/585"},{"key":"10824_CR204","unstructured":"Liang S, Li Y, Srikant R (2018) Enhancing the reliability of out-of-distribution image detection in neural networks. In: 6th international conference on learning representations, ICLR 2018"},{"key":"10824_CR205","doi-asserted-by":"crossref","unstructured":"Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan, D Doll\u00e1r P, Zitnick CL (2014) Microsoft COCO: common objects in context. In: Computer vision\u2013ECCV 2014: 13th European conference, Zurich, Switzerland, September 6\u201312, 2014, proceedings, part V 13. Springer, pp 740\u2013755","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"10824_CR206","unstructured":"Lin Z, Xu P, Winata GI, Siddique FB, Liu Z, Shin J, Fung P (2019) CAiRE: an empathetic neural chatbot. arXiv Preprint http:\/\/arxiv.org\/abs\/1907.12108"},{"key":"10824_CR207","unstructured":"Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized BERT pretraining approach. arXiv Preprint http:\/\/arxiv.org\/abs\/1907.11692"},{"key":"10824_CR208","first-page":"21464","volume":"33","author":"W Liu","year":"2020","unstructured":"Liu W, Wang X, Owens J, Li Y (2020) Energy-based out-of-distribution detection. Adv Neural Inf Process Syst 33:21464\u201321475","journal-title":"Adv Neural Inf Process Syst"},{"issue":"3\u20134","key":"10824_CR209","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1561\/2400000035","volume":"4","author":"C Liu","year":"2021","unstructured":"Liu C, Arnon T, Lazarus C, Strong C, Barrett C, Kochenderfer MJ et al (2021a) Algorithms for verifying deep neural networks. Found Trends Optim 4(3\u20134):244\u2013404","journal-title":"Found Trends Optim"},{"issue":"1","key":"10824_CR210","first-page":"857","volume":"35","author":"X Liu","year":"2021","unstructured":"Liu X, Zhang F, Hou Z, Mian L, Wang Z, Zhang J, Tang J (2021b) Self-supervised learning: generative or contrastive. IEEE Trans Knowl Data Eng 35(1):857\u2013876","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"10824_CR211","first-page":"28092","volume":"34","author":"Z Liu","year":"2021","unstructured":"Liu Z, Wang Y, Han K, Zhang W, Ma S, Gao W (2021c) Post-training quantization for vision transformer. Adv Neural Inf Process Syst 34:28092\u201328103","journal-title":"Adv Neural Inf Process Syst"},{"key":"10824_CR212","unstructured":"Liu Y, Han T, Ma S, Zhang J, Yang Y, Tian J, He H, Li A, He M, Liu Z et\u00a0al (2023a) Summary of ChatGPT\/GPT-4 research and perspective towards the future of large language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.01852"},{"key":"10824_CR213","unstructured":"Liu H, Ning R, Teng Z, Liu J, Zhou Q, Zhang Y (2023b) Evaluating the logical reasoning ability of ChatGPT and GPT-4. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.03439"},{"key":"10824_CR214","unstructured":"Liu J, Xia CS, Wang Y, Zhang L (2023c) Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. arXiv Preprint http:\/\/arxiv.org\/abs\/2305.01210"},{"key":"10824_CR215","unstructured":"Liu Z, Yu X, Zhang L, Wu Z, Cao C, Dai H, Zhao L, Liu W, Shen D, Li Q et\u00a0al (2023d) DeID-GPT: zero-shot medical text de-identification by GPT-4. arXiv Preprint http:\/\/arxiv.org\/abs\/2303.11032"},{"key":"10824_CR216","unstructured":"Lou R, Zhang K, Yin W (2023) Is prompt all you need? No. A comprehensive and broader view of instruction learning. arXiv Preprint http:\/\/arxiv.org\/abs\/2303.10475"},{"key":"10824_CR217","doi-asserted-by":"crossref","unstructured":"Madaan N, Padhi I, Panwar N, Saha D (2021) Generate your counterfactuals: towards controlled counterfactual generation for text. In: Proceedings of the AAAI conference on artificial intelligence, vol 35. pp 13516\u201313524","DOI":"10.1609\/aaai.v35i15.17594"},{"key":"10824_CR218","unstructured":"Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2017) Towards deep learning models resistant to adversarial attacks. arXiv Preprint http:\/\/arxiv.org\/abs\/1706.06083"},{"key":"10824_CR219","doi-asserted-by":"crossref","unstructured":"Malinka K, Peres\u00edni M, Firc A, Hujn\u00e1k O, Janus F (2023) On the educational impact of ChatGPT: is artificial intelligence ready to obtain a university degree? In: Proceedings of the 2023 conference on innovation and technology in computer science education V. 1. pp 47\u201353","DOI":"10.1145\/3587102.3588827"},{"key":"10824_CR220","volume-title":"The temporal logic of reactive and concurrent systems: specification","author":"Z Manna","year":"2012","unstructured":"Manna Z, Pnueli A (2012) The temporal logic of reactive and concurrent systems: specification. Springer Science & Business Media, Berlin"},{"key":"10824_CR221","unstructured":"March 20 ChatGPT outage: here\u2019s what happened. https:\/\/openai.com\/blog\/march-20-chatgpt-outage. OpenAI. Accessed 20 Aug 2023"},{"key":"10824_CR222","unstructured":"Maus N, Chao P, Wong E, Gardner J (2023) Adversarial prompting for black box foundation models. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.04237"},{"key":"10824_CR223","unstructured":"McCune W (2005) Prover9 and Mace4. https:\/\/www.cs.unm.edu\/~mccune\/prover9\/. Accessed 20 Aug 2023"},{"key":"10824_CR224","unstructured":"Mehdi Y (2023) Announcing the next wave of AI innovation with Microsoft Bing and Edge"},{"key":"10824_CR225","doi-asserted-by":"crossref","unstructured":"Min S, Lyu X, Holtzman A, Artetxe M, Lewis M, Hajishirzi H, Zettlemoyer L (2022) Rethinking the role of demonstrations: what makes in-context learning work? arXiv Preprint http:\/\/arxiv.org\/abs\/2202.12837","DOI":"10.18653\/v1\/2022.emnlp-main.759"},{"key":"10824_CR226","unstructured":"Mirman M, Gehr T, Vechev M (2018) Differentiable abstract interpretation for provably robust neural networks. In: Dy J, Krause A (eds) Proceedings of the 35th international conference on machine learning, volume\u00a080 of proceedings of machine learning research, 10\u201315 July 2018. PMLR, pp 3578\u20133586"},{"key":"10824_CR227","unstructured":"Mitrovi\u0107 S, Andreoletti D, Ayoub O (2023) ChatGPT or human? Detect and explain. Explaining decisions of machine learning model for detecting short ChatGPT-generated text"},{"key":"10824_CR228","doi-asserted-by":"crossref","unstructured":"Monteiro J, Albuquerque I, Akhtar Z, Falk TH (2019) Generalizable adversarial examples detection based on bi-model decision mismatch. In: 2019 IEEE international conference on systems, man and cybernetics (SMC). IEEE, pp 2839\u20132844","DOI":"10.1109\/SMC.2019.8913861"},{"key":"10824_CR229","unstructured":"Nagel M, Amjad RA, Van\u00a0Baalen M, Louizos C, Blankevoort T (2020) Up or down? Adaptive rounding for post-training quantization. In: International conference on machine learning. PMLR, pp 7197\u20137206"},{"key":"10824_CR230","unstructured":"Nelson B, Barreno M, Chi FJ, Joseph AD, Rubinstein BIP, Saini U, Sutton C, Tygar JD, Xia K (2008) Exploiting machine learning to subvert your spam filter. In: Proceedings of the 1st Usenix workshop on large-scale exploits and emergent threats, LEET\u201908, USA, 2008. USENIX Association"},{"key":"10824_CR231","unstructured":"News TH (2023) WormGPT: new AI tool allows cybercriminals to launch sophisticated cyber attacks. https:\/\/thehackernews.com\/2023\/07\/wormgpt-new-ai-tool-allows.html. Accessed 20 Aug 2023"},{"key":"10824_CR232","unstructured":"Ni A, Iyer S, Radev D, Stoyanov V, Yih W-t, Wang S, Lin XV (2023) Lever: learning to verify language-to-code generation with execution. In: International conference on machine learning. PMLR, pp 26106\u201326128"},{"key":"10824_CR233","unstructured":"Nichol A, Dhariwal P, Ramesh A, Shyam P, Mishkin P, McGrew B, Sutskever I, Chen M (2021) Glide: towards photorealistic image generation and editing with text-guided diffusion models. arXiv Preprint http:\/\/arxiv.org\/abs\/2112.10741"},{"key":"10824_CR234","doi-asserted-by":"crossref","unstructured":"Nie Y, Williams A, Dinan E, Bansal M, Weston J, Kiela D (2019) Adversarial NLI: a new benchmark for natural language understanding. arXiv Preprint http:\/\/arxiv.org\/abs\/1910.14599","DOI":"10.18653\/v1\/2020.acl-main.441"},{"key":"10824_CR235","unstructured":"OpenAI (2023) GPT-4 technical report. arXiv e-prints http:\/\/arxiv.org\/abs\/2303.08774"},{"key":"10824_CR236","unstructured":"OpenAI says a bug leaked sensitive ChatGPT user data. https:\/\/www.engadget.com\/chatgpt-briefly-went-offline-after-a-bug-revealed-user-chat-histories-115632504.html. Engadget. Accessed 20 Aug 2023"},{"key":"10824_CR237","first-page":"27730","volume":"35","author":"L Ouyang","year":"2022","unstructured":"Ouyang L, Wu J, Jiang X, Almeida D, Wainwright C, Mishkin P, Zhang C, Agarwal S, Slama K, Ray A et al (2022) Training language models to follow instructions with human feedback. Adv Neural Inf Process Syst 35:27730\u201327744","journal-title":"Adv Neural Inf Process Syst"},{"key":"10824_CR238","doi-asserted-by":"crossref","unstructured":"Pan S, Luo L, Wang Y, Chen C, Wang J, Wu X (2023) Unifying large language models and knowledge graphs: a roadmap","DOI":"10.1109\/TKDE.2024.3352100"},{"issue":"2","key":"10824_CR239","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3439950","volume":"54","author":"G Pang","year":"2021","unstructured":"Pang G, Shen C, Cao L, Hengel AVD (2021) Deep learning for anomaly detection: a review. ACM Comput Surv (CSUR) 54(2):1\u201338","journal-title":"ACM Comput Surv (CSUR)"},{"key":"10824_CR240","unstructured":"Park G, Park B, Kwon SJ, Kim B, Lee Y, Lee D (2022) nuQmm: quantized MatMul for efficient inference of large-scale generative language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2206.09557"},{"issue":"7","key":"10824_CR241","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/MC.2022.3148714","volume":"55","author":"D Patterson","year":"2022","unstructured":"Patterson D, Gonzalez J, Holzle U, Le Q, Liang C, Munguia L-M, Rothchild D, So DR, Texier M, Dean J (2022) The carbon footprint of machine learning training will plateau, then shrink. Computer 55(7):18\u201328","journal-title":"Computer"},{"key":"10824_CR242","unstructured":"Pause giant AI experiments: an open letter. https:\/\/futureoflife.org\/open-letter\/pause-giant-ai-experiments\/. Accessed 20 Aug 2023"},{"key":"10824_CR243","doi-asserted-by":"crossref","unstructured":"Pearce H, Tan B, Ahmad B, Karri R, Dolan-Gavitt B (2023) Examining zero-shot vulnerability repair with large language models. In: 2023 IEEE symposium on security and privacy (SP). IEEE, pp 2339\u20132356","DOI":"10.1109\/SP46215.2023.10179324"},{"key":"10824_CR244","unstructured":"Pegoraro A, Kumari K, Fereidooni H, Sadeghi A-R (2023) To ChatGPT, or not to ChatGPT: that is the question! arXiv Preprint http:\/\/arxiv.org\/abs\/2304.01487"},{"key":"10824_CR245","unstructured":"Peng B, Li C, He P, Galley M, Gao J (2023) Instruction tuning with GPT-4. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.03277"},{"key":"10824_CR246","doi-asserted-by":"crossref","unstructured":"Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). pp 1532\u20131543","DOI":"10.3115\/v1\/D14-1162"},{"key":"10824_CR247","unstructured":"Perez F, Ribeiro I (2022) Ignore previous prompt: attack techniques for language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2211.09527"},{"key":"10824_CR248","doi-asserted-by":"crossref","unstructured":"Podolskiy A, Lipin D, Bout A, Artemova E, Piontkovskaya I (2021) Revisiting Mahalanobis distance for transformer-based out-of-domain detection. In: Proceedings of the AAAI conference on artificial intelligence, vol 35. pp 13675\u201313682","DOI":"10.1609\/aaai.v35i15.17612"},{"key":"10824_CR249","unstructured":"Prompt engineering guide. https:\/\/github.com\/dair-ai\/Prompt-Engineering-Guide\/tree\/main\/guides. Accessed 20 Aug 2023"},{"key":"10824_CR250","unstructured":"Qi Y, Zhao X, Huang X (2023) Safety analysis in the era of large language models: a case study of STPA using ChatGPT. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.01246"},{"key":"10824_CR251","unstructured":"Radford A, Jozefowicz R, Sutskever I (2017) Learning to generate reviews and discovering sentiment. arXiv Preprint http:\/\/arxiv.org\/abs\/1704.01444"},{"key":"10824_CR252","unstructured":"Radford A, Narasimhan K, Salimans T, Sutskever I et\u00a0al (2018) Improving language understanding by generative pre-training. OpenAI"},{"key":"10824_CR253","unstructured":"Rae JW, Borgeaud S, Cai T, Millican K, Hoffmann J, Song F, Aslanides J, Henderson S, Ring R, Young S et\u00a0al (2021) Scaling language models: methods, analysis & insights from training Gopher. arXiv Preprint http:\/\/arxiv.org\/abs\/2112.11446"},{"issue":"1","key":"10824_CR254","first-page":"5485","volume":"21","author":"C Raffel","year":"2020","unstructured":"Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu PJ (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21(1):5485\u20135551","journal-title":"J Mach Learn Res"},{"key":"10824_CR255","unstructured":"Ramamurthy R, Ammanabrolu P, Brantley K, Hessel J, Sifa R, Bauckhage C, Hajishirzi H, Choi Y (2022) Is reinforcement learning (not) for natural language processing?: benchmarks, baselines, and building blocks for natural language policy optimization. arXiv Preprint http:\/\/arxiv.org\/abs\/2210.01241"},{"key":"10824_CR256","unstructured":"Ramesh A, Pavlov M, Goh G, Gray S, Voss C, Radford A, Chen M, Sutskever I (2021) Zero-shot text-to-image generation. In: International conference on machine learning. PMLR, pp 8821\u20138831"},{"key":"10824_CR257","unstructured":"Ramesh A, Dhariwal P, Nichol A, Chu C, Chen M (2022) Hierarchical text-conditional image generation with clip latents. arXiv Preprint http:\/\/arxiv.org\/abs\/2204.06125"},{"key":"10824_CR258","doi-asserted-by":"crossref","unstructured":"Reiss MV (2023) Testing the reliability of ChatGPT for text annotation and classification: a cautionary remark. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.11085","DOI":"10.31219\/osf.io\/rvy5p"},{"key":"10824_CR259","doi-asserted-by":"crossref","unstructured":"Ren S, Deng Y, He K, Che W (2019a) Generating natural language adversarial examples through probability weighted word saliency. In: Proceedings of the 57th annual meeting of the association for computational linguistics. pp 1085\u20131097","DOI":"10.18653\/v1\/P19-1103"},{"key":"10824_CR260","unstructured":"Ren J, Liu PJ, Fertig E, Snoek J, Poplin R, Depristo M, Dillon J, Lakshminarayanan B (2019b) Likelihood ratios for out-of-distribution detection. In: Advances in neural information processing systems, vol 32"},{"key":"10824_CR261","unstructured":"Ren X, Zhou P, Meng X, Huang X, Wang Y, Wang W, Li P, Zhang X, Podolskiy A, Arshinov G et\u00a0al (2023) Pangu-$$\\sigma$$: towards trillion parameter language model with sparse heterogeneous computing. arXiv Preprint http:\/\/arxiv.org\/abs\/2303.10845"},{"key":"10824_CR262","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C (2016) \u201cWhy should I trust you?\u201d: explaining the predictions of any classifier. In: HLT-NAACL demos","DOI":"10.1145\/2939672.2939778"},{"key":"10824_CR263","unstructured":"Rolfe JT (2016) Discrete variational autoencoders. arXiv Preprint http:\/\/arxiv.org\/abs\/1609.02200"},{"key":"10824_CR264","doi-asserted-by":"crossref","unstructured":"Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B (2022) High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. pp 10684\u201310695","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"10824_CR265","doi-asserted-by":"crossref","unstructured":"Ruan W, Huang X, Kwiatkowska M (2018) Reachability analysis of deep neural networks with provable guarantees. In: IJCAI2018. pp 2651\u20132659","DOI":"10.24963\/ijcai.2018\/368"},{"key":"10824_CR266","doi-asserted-by":"crossref","unstructured":"Ruan W, Wu M, Sun Y, Huang X, Kroening D, Kwiatkowska M (2019) Global robustness evaluation of deep neural networks with provable guarantees for the hamming distance. In: IJCAI2019. pp 5944\u20135952","DOI":"10.24963\/ijcai.2019\/824"},{"key":"10824_CR267","doi-asserted-by":"crossref","unstructured":"Ruder S, Peters ME, Swayamdipta S, Wolf T (2019) Transfer learning in natural language processing. In: Proceedings of the 2019 conference of the North American chapter of the Association for Computational Linguistics: tutorials. pp 15\u201318","DOI":"10.18653\/v1\/N19-5004"},{"key":"10824_CR268","doi-asserted-by":"crossref","first-page":"682","DOI":"10.3389\/fnins.2017.00682","volume":"11","author":"B Rueckauer","year":"2017","unstructured":"Rueckauer B, Lungu I-A, Hu Y, Pfeiffer M, Liu S-C (2017) Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Front Neurosci 11:682","journal-title":"Front Neurosci"},{"key":"10824_CR269","unstructured":"Rutinowski J, Franke S, Endendyk J, Dormuth I, Pauly M (2023) The self-perception and political biases of ChatGPT. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.07333"},{"key":"10824_CR270","doi-asserted-by":"crossref","unstructured":"Ryou W, Chen J, Balunovic M, Singh G, Dan A, Vechev M (2021) Scalable polyhedral verification of recurrent neural networks. In: International conference on computer aided verification. Springer, pp 225\u2013248","DOI":"10.1007\/978-3-030-81685-8_10"},{"key":"10824_CR271","first-page":"36479","volume":"35","author":"C Saharia","year":"2022","unstructured":"Saharia C, Chan W, Saxena S, Li L, Whang J, Denton EL, Ghasemipour K, Gontijo Lopes R, Karagol Ayan B, Salimans T et al (2022) Photorealistic text-to-image diffusion models with deep language understanding. Adv Neural Inf Process Syst 35:36479\u201336494","journal-title":"Adv Neural Inf Process Syst"},{"key":"10824_CR272","unstructured":"Samanta S, Mehta S (2017) Towards crafting text adversarial samples. arXiv Preprint http:\/\/arxiv.org\/abs\/1707.02812"},{"key":"10824_CR273","unstructured":"Sandoval G, Pearce H, Nys T, Karri R, Garg S, Dolan-Gavitt B (2023) Lost at C: a user study on the security implications of large language model code assistants. arXiv Preprint http:\/\/arxiv.org\/abs\/2208.09727"},{"key":"10824_CR274","unstructured":"Scao TL, Fan A, Akiki C, Pavlick E, Ili\u0107 S, Hesslow D, Castagn\u00e9 R, Luccioni AS, Yvon F, Gall\u00e9 M et\u00a0al (2022) Bloom: a 176B-parameter open-access multilingual language model. arXiv Preprint http:\/\/arxiv.org\/abs\/2211.05100"},{"key":"10824_CR275","unstructured":"Schulman J, Wolski F, Dhariwal P, Radford A, Klimov O (2017) Proximal policy optimization algorithms. arXiv Preprint http:\/\/arxiv.org\/abs\/1707.06347"},{"key":"10824_CR276","unstructured":"Senate U (2023) Senate judiciary subcommittee hearing on oversight of AI. https:\/\/techpolicy.press\/transcript-senate-judiciary-subcommittee-hearing-on-oversight-of-ai\/. Accessed 20 Aug 2023"},{"key":"10824_CR277","unstructured":"Seshia SA, Sadigh D, Sastry SS (2016) Towards verified artificial intelligence. arXiv Preprint http:\/\/arxiv.org\/abs\/1606.08514"},{"key":"10824_CR278","unstructured":"Shanahan M (2022) Talking about large language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2212.03551"},{"key":"10824_CR279","doi-asserted-by":"crossref","unstructured":"Shen Y, Hsu Y-C, Ray A, Jin H (2021a) Enhancing the generalization for intent classification and out-of-domain detection in SLU. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers). pp 2443\u20132453","DOI":"10.18653\/v1\/2021.acl-long.190"},{"key":"10824_CR280","doi-asserted-by":"crossref","unstructured":"Shen L, Ji S, Zhang X, Li J, Chen J, Shi J, Fang C, Yin J, Wang T (2021b) Backdoor pre-trained models can transfer to all. In: Proceedings of the 2021 ACM SIGSAC conference on computer and communications security. pp 3141\u20133158","DOI":"10.1145\/3460120.3485370"},{"key":"10824_CR281","unstructured":"Shen X, Chen Z, Backes M, Zhang Y (2023) In ChatGPT we trust? Measuring and characterizing the reliability of ChatGPT. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.08979"},{"key":"10824_CR282","unstructured":"Shi Z, Zhang H, Chang K-W, Huang M, Hsieh C-J (2019) Robustness verification for transformers. In: International conference on learning representations"},{"key":"10824_CR283","doi-asserted-by":"crossref","unstructured":"Shuster K, Poff S, Chen M, Kiela D, Weston J (2021) Retrieval augmentation reduces hallucination in conversation. arXiv Preprint http:\/\/arxiv.org\/abs\/2104.07567","DOI":"10.18653\/v1\/2021.findings-emnlp.320"},{"key":"10824_CR284","doi-asserted-by":"crossref","unstructured":"Shuster K, Komeili M, Adolphs L, Roller S, Szlam A, Weston J (2022) Language models that seek for knowledge: modular search & generation for dialogue and prompt completion. arXiv Preprint http:\/\/arxiv.org\/abs\/2203.13224","DOI":"10.18653\/v1\/2022.findings-emnlp.27"},{"key":"10824_CR285","unstructured":"Sinha A, Namkoong H, Volpi R, Duchi J (2017) Certifying some distributional robustness with principled adversarial training. arXiv Preprint http:\/\/arxiv.org\/abs\/1710.10571"},{"key":"10824_CR286","unstructured":"Smith L, Gal Y (2018) Understanding measures of uncertainty for adversarial example detection. arXiv Preprint http:\/\/arxiv.org\/abs\/1803.08533"},{"key":"10824_CR287","unstructured":"Smith S, Patwary M, Norick B, LeGresley P, Rajbhandari S, Casper J, Liu Z, Prabhumoye S, Zerveas G, Korthikanti V et\u00a0al (2022) Using deepspeed and megatron to train megatron-turing NLG 530B, a large-scale generative language model. arXiv Preprint http:\/\/arxiv.org\/abs\/2201.11990"},{"key":"10824_CR288","doi-asserted-by":"crossref","unstructured":"Sobania D, Briesch M, Hanna C, Petke J (2023) An analysis of the automatic bug fixing performance of ChatGPT. arXiv Preprint http:\/\/arxiv.org\/abs\/2301.08653","DOI":"10.1109\/APR59189.2023.00012"},{"key":"10824_CR289","unstructured":"Soltan S, Ananthakrishnan S, FitzGerald J, Gupta R, Hamza W, Khan H, Peris C, Rawls S, Rosenbaum A, Rumshisky A et\u00a0al (2022) AlexaTM 20B: few-shot learning using a large-scale multilingual seq2seq model. arXiv Preprint http:\/\/arxiv.org\/abs\/2208.01448"},{"key":"10824_CR290","doi-asserted-by":"crossref","unstructured":"Struppek L, Hintersdorf D, Kersting K (2022) Rickrolling the artist: injecting invisible backdoors into text-guided image generation models. arXiv Preprint http:\/\/arxiv.org\/abs\/2211.02408","DOI":"10.1109\/ICCV51070.2023.00423"},{"key":"10824_CR291","unstructured":"Sun Y, Huang X, Kroening D, Sharp J, Hill M, Ashmore R (2018a) Testing deep neural networks. arXiv Preprint http:\/\/arxiv.org\/abs\/1803.04792"},{"key":"10824_CR292","doi-asserted-by":"crossref","unstructured":"Sun Y, Wu M, Ruan W, Huang X, Kwiatkowska M, Kroening D (2018b) Concolic testing for deep neural networks. In: ASE2018","DOI":"10.1145\/3238147.3238172"},{"issue":"5s","key":"10824_CR293","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3358233","volume":"18","author":"Y Sun","year":"2019","unstructured":"Sun Y, Huang X, Kroening D, Sharp J, Hill M, Ashmore R (2019) Structural test coverage criteria for deep neural networks. ACM Trans Embed Comput Syst 18(5s):1\u201323","journal-title":"ACM Trans Embed Comput Syst"},{"key":"10824_CR294","unstructured":"Sun Y, Wang S, Feng S, Ding S, Pang C, Shang J, Liu J, Chen X, Zhao Y, Lu Y et\u00a0al (2021) ERNIE 3.0: large-scale knowledge enhanced pre-training for language understanding and generation. arXiv Preprint http:\/\/arxiv.org\/abs\/2107.02137"},{"key":"10824_CR295","unstructured":"Sun H, Zhang Z, Deng J, Cheng J, Huang M (2023) Safety assessment of Chinese large language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.10436"},{"key":"10824_CR296","unstructured":"Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R (2013) Intriguing properties of neural networks. arXiv Preprint http:\/\/arxiv.org\/abs\/1312.6199"},{"key":"10824_CR297","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.compind.2015.09.005","volume":"78","author":"L Tanguy","year":"2016","unstructured":"Tanguy L, Tulechki N, Urieli A, Hermann E, Raynal C (2016) Natural language processing for aviation safety reports: from classification to interactive analysis. Comput Ind 78:80\u201395","journal-title":"Comput Ind"},{"key":"10824_CR298","unstructured":"Taori R, Gulrajani I, Zhang T, Dubois Y, Li X, Guestrin C, Liang P, Hashimoto TB (2023) Stanford Alpaca: an instruction-following LLaMa model"},{"key":"10824_CR299","unstructured":"Taylor R, Kardas M, Cucurull G, Scialom T, Hartshorn A, Saravia E, Poulton A, Kerkez V, Stojnic R (2022) Galactica: a large language model for science. arXiv Preprint http:\/\/arxiv.org\/abs\/2211.09085"},{"key":"10824_CR300","doi-asserted-by":"crossref","unstructured":"Tejankar A, Sanjabi M, Wang Q, Wang S, Firooz H, Pirsiavash H, Tan L (2023) Defending against patch-based backdoor attacks on self-supervised learning. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition. pp 12239\u201312249","DOI":"10.1109\/CVPR52729.2023.01178"},{"key":"10824_CR301","doi-asserted-by":"crossref","unstructured":"Thakur S, Ahmad B, Fan Z, Pearce H, Tan B, Karri R, Dolan-Gavitt B, Garg S (2023) Benchmarking large language models for automated Verilog RTL code generation. In: 2023 design, automation & test in Europe conference & exhibition (DATE). IEEE, pp 1\u20136","DOI":"10.23919\/DATE56975.2023.10137086"},{"key":"10824_CR302","unstructured":"The carbon footprint of GPT-4. https:\/\/towardsdatascience.com\/the-carbon-footprint-of-gpt-4-d6c676eb21ae. Accessed 17 Aug 2023"},{"key":"10824_CR303","unstructured":"Thoppilan R, De\u00a0Freitas D, Hall J, Shazeer N, Kulshreshtha A, Cheng H-T, Jin A, Bos T, Baker L, Du Y et\u00a0al (2022) LaMDA: language models for dialog applications. arXiv Preprint http:\/\/arxiv.org\/abs\/2201.08239"},{"key":"10824_CR304","doi-asserted-by":"crossref","unstructured":"Thorne J, Vlachos A, Christodoulopoulos C, Mittal A (2018) FEVER: a large-scale dataset for fact extraction and verification. In: 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, NAACL HLT 2018. Association for Computational Linguistics (ACL), pp 809\u2013819","DOI":"10.18653\/v1\/N18-1074"},{"key":"10824_CR305","unstructured":"Tools such as ChatGPT threaten transparent science; here are our ground rules for their use. https:\/\/www.nature.com\/articles\/d41586-023-00191-1. Accessed 20 Aug 2023"},{"key":"10824_CR306","unstructured":"Touvron H, Lavril T, Izacard G, Martinet X, Lachaux M-A, Lacroix T, Rozi\u00e8re B, Goyal N, Hambro E, Azhar F et\u00a0al (2023) LLaMA: open and efficient foundation language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.13971"},{"key":"10824_CR307","doi-asserted-by":"crossref","unstructured":"Tulshan AS, Dhage SN (2019) Survey on virtual assistant: Google assistant, Siri, Cortana, Alexa. In: Advances in signal processing and intelligent recognition systems: 4th international symposium SIRS 2018, Bangalore, India, September 19\u201322, 2018, revised selected papers 4. Springer, pp 190\u2013201","DOI":"10.1007\/978-981-13-5758-9_17"},{"key":"10824_CR308","doi-asserted-by":"crossref","unstructured":"Tung F, Mori G (2019) Similarity-preserving knowledge distillation. In: Proceedings of the IEEE\/CVF international conference on computer vision. pp 1365\u20131374","DOI":"10.1109\/ICCV.2019.00145"},{"key":"10824_CR309","unstructured":"Uchendu A, Lee J, Shen H, Le T, Huang TK, Lee D (2023) Understanding individual and team-based human factors in detecting deepfake texts. CoRR. abs\/2304.01002"},{"key":"10824_CR310","unstructured":"Vardi MY, Wolper P (1986) An automata-theoretic approach to automatic program verification. In: 1st symposium in logic in computer science (LICS). IEEE Computer Society"},{"key":"10824_CR311","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Lu, Polosukhin I (2017) Attention is all you need. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30. Curran Associates, Inc"},{"key":"10824_CR312","unstructured":"Wallace M, Khandelwal R, Tang B (2022) Does IBP scale? arXiv Preprint"},{"key":"10824_CR313","doi-asserted-by":"crossref","unstructured":"Wang Y, Bansal M (2018) Robust machine comprehension models via adversarial training. arXiv Preprint http:\/\/arxiv.org\/abs\/1804.06473","DOI":"10.18653\/v1\/N18-2091"},{"key":"10824_CR314","doi-asserted-by":"crossref","unstructured":"Wang G, Lin Y, Yi W (2010) Kernel fusion: an effective method for better power efficiency on multithreaded GPU. In: 2010 IEEE\/ACM Int\u2019l conference on green computing and communications & Int\u2019l conference on cyber, physical and social computing. IEEE, pp 344\u2013350","DOI":"10.1109\/GreenCom-CPSCom.2010.102"},{"key":"10824_CR315","doi-asserted-by":"crossref","unstructured":"Wang W, Tang P, Lou J, Xiong L (2021a) Certified robustness to word substitution attack with differential privacy. In: Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies. pp 1102\u20131112","DOI":"10.18653\/v1\/2021.naacl-main.87"},{"key":"10824_CR316","unstructured":"Wang B, Xu C, Wang S, Gan Z, Cheng Y, Gao J, Awadallah AH, Li B (2021b) Adversarial glue: a multi-task benchmark for robustness evaluation of language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2111.02840"},{"key":"10824_CR317","unstructured":"Wang J, Hu X, Hou W, Chen H, Zheng R, Wang Y, Yang L, Huang H, Ye W, Geng X, Jiao B, Zhang Y, Xie X (2023a) On the robustness of ChatGPT: an adversarial and out-of-distribution perspective. arXiv e-prints http:\/\/arxiv.org\/abs\/2302.12095"},{"key":"10824_CR318","unstructured":"Wang X, Wei J, Schuurmans D, Le QV, Chi EH, Narang S, Chowdhery A, Zhou D (2023b) Self-consistency improves chain of thought reasoning in language models. In: The eleventh international conference on learning representations"},{"key":"10824_CR319","unstructured":"Wang F, Xu P, Ruan W, Huang X (2023c) Towards verifying the geometric robustness of large-scale neural networks. arXiv Preprint http:\/\/arxiv.org\/abs\/2301.12456"},{"key":"10824_CR320","unstructured":"Wei J, Wang X, Schuurmans D, Bosma M, Brian Ichter, Xia F, Chi EH, Le QV, Zhou D (2022) Chain of thought prompting elicits reasoning in large language models. In: Oh AH, Agarwal A, Belgrave D, Cho K (eds) Advances in neural information processing systems"},{"key":"10824_CR321","unstructured":"Wei J, Kim S, Jung H, Kim Y-H (2023) Leveraging large language models to power chatbots for collecting user self-reported data. arXiv Preprint http:\/\/arxiv.org\/abs\/2301.05843"},{"key":"10824_CR322","unstructured":"Weng T-W, Zhang H, Chen P-Y, Yi J, Su D, Gao Y, Hsieh C-J, Daniel L (2018) Evaluating the robustness of neural networks: an extreme value theory approach. arXiv Preprint http:\/\/arxiv.org\/abs\/1801.10578"},{"key":"10824_CR323","doi-asserted-by":"crossref","unstructured":"Weng Y, Zhu M, He S, Liu K, Zhao J (2022) Large language models are reasoners with self-verification. arXiv Preprint http:\/\/arxiv.org\/abs\/2212.09561","DOI":"10.18653\/v1\/2023.findings-emnlp.167"},{"key":"10824_CR324","unstructured":"Weng Y, Zhu M, Xia F, Li B, He S, Liu K, Zhao J (2023) Neural comprehension: language models with compiled neural networks. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.01665"},{"key":"10824_CR325","doi-asserted-by":"crossref","unstructured":"Wicker M, Huang X, Kwiatkowska M (2018) Feature-guided black-box safety testing of deep neural networks. In: Tools and algorithms for the construction and analysis of systems: 24th international conference, TACAS 2018, held as part of the European joint conferences on theory and practice of software, ETAPS 2018, Thessaloniki, Greece, April 14\u201320, 2018, proceedings, part I 24. pp 408\u2013426","DOI":"10.1007\/978-3-319-89960-2_22"},{"key":"10824_CR326","unstructured":"Wolf Y, Wies N, Levine Y, Shashua A (2023) Fundamental limitations of alignment in large language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.11082"},{"key":"10824_CR327","unstructured":"Wong E, Rice L, Kolter JZ (2020) Fast is better than free: revisiting adversarial training. arXiv Preprint http:\/\/arxiv.org\/abs\/2001.03994"},{"key":"10824_CR328","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1016\/j.tcs.2019.05.046","volume":"807","author":"M Wu","year":"2020","unstructured":"Wu M, Wicker M, Ruan W, Huang X, Kwiatkowska M (2020) A game-based approximate verification of deep neural networks with provable guarantees. Theor Comput Sci 807:298\u2013329","journal-title":"Theor Comput Sci"},{"key":"10824_CR329","unstructured":"Wu Y, Jiang AQ, Li W, Rabe MN, Staats CE, Jamnik M, Szegedy C (2022a) Autoformalization with large language models. In: Oh AH, Agarwal A, Belgrave D, Cho K (eds) Advances in neural information processing systems"},{"key":"10824_CR330","doi-asserted-by":"crossref","first-page":"759900","DOI":"10.3389\/fnins.2022.759900","volume":"16","author":"D Wu","year":"2022","unstructured":"Wu D, Yi X, Huang X (2022b) A little energy goes a long way: build an energy-efficient, accurate spiking neural network from convolutional neural network. Front Neurosci 16:759900","journal-title":"Front Neurosci"},{"key":"10824_CR331","unstructured":"Wu S, Irsoy O, Lu S, Dabravolski V, Dredze M, Gehrmann S, Kambadur P, Rosenberg D, Mann G (2023a) BloombergGPT: a large language model for finance. arXiv Preprint http:\/\/arxiv.org\/abs\/2303.17564"},{"key":"10824_CR332","unstructured":"Wu D, Jin G, Yu H, Yi X, Huang X (2023b) Optimising event-driven spiking neural network with regularisation and cutoff. arXiv Preprint http:\/\/arxiv.org\/abs\/2301.09522"},{"key":"10824_CR333","doi-asserted-by":"crossref","unstructured":"Wu X, Sun K, Zhu F, Zhao R, Li H (2023c) Better aligning text-to-image models with human preference. arXiv Preprint http:\/\/arxiv.org\/abs\/2303.14420","DOI":"10.1109\/ICCV51070.2023.00200"},{"key":"10824_CR334","unstructured":"Wu M, Waheed A, Zhang C, Abdul-Mageed M, Aji AF (2023d) LaMini-LM: a diverse herd of distilled models from large-scale instructions. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.14402"},{"key":"10824_CR335","unstructured":"Wu H, Wang W, Wan Y, Jiao W, Lyu M (2023e) ChatGPT or grammarly? Evaluating ChatGPT on grammatical error correction benchmark. arXiv Preprint http:\/\/arxiv.org\/abs\/2303.13648"},{"key":"10824_CR336","doi-asserted-by":"crossref","unstructured":"Xu F, Uszkoreit H, Du Y, Fan W, Zhao D, Zhu J (2019) Explainable AI: a brief survey on history, research areas, approaches and challenges. In: Natural language processing and Chinese computing: 8th CCF international conference, NLPCC 2019, Dunhuang, China, October 9\u201314, 2019, proceedings, part II 8. Springer, pp 563\u2013574","DOI":"10.1007\/978-3-030-32236-6_51"},{"key":"10824_CR337","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/s11633-019-1211-x","volume":"17","author":"H Xu","year":"2020","unstructured":"Xu H, Ma Y, Liu H-C, Deb D, Liu H, Tang J-L, Jain AK (2020a) Adversarial attacks and defenses in images, graphs and text: a review. Int J Autom Comput 17:151\u2013178","journal-title":"Int J Autom Comput"},{"key":"10824_CR338","doi-asserted-by":"crossref","unstructured":"Xu H, He K, Yan Y, Liu S, Liu Z, Xu W (2020b) A deep generative distance-based classifier for out-of-domain detection with Mahalanobis space. In: Proceedings of the 28th international conference on computational linguistics. pp 1452\u20131460","DOI":"10.18653\/v1\/2020.coling-main.125"},{"issue":"4","key":"10824_CR339","doi-asserted-by":"crossref","first-page":"3801","DOI":"10.1007\/s40747-022-00790-x","volume":"9","author":"P Xu","year":"2022","unstructured":"Xu P, Ruan W, Huang X (2022) Quantifying safety risks of deep neural networks. Complex Intell Syst 9(4):3801\u20133818","journal-title":"Complex Intell Syst"},{"key":"10824_CR340","unstructured":"Xu J, Liu X, Wu Y, Tong Y, Li Q, Ding M, Tang J, Dong Y (2023) ImageReward: learning and evaluating human preferences for text-to-image generation. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.05977"},{"key":"10824_CR341","unstructured":"Yandex. Yandex\/YaLM-100B: pretrained language model with 100B parameters. https:\/\/github.com\/yandex\/YaLM-100B. Accessed 20 Aug 2023"},{"key":"10824_CR342","unstructured":"Yang Z (2023) Chinese tech giant Baidu just released its answer to ChatGPT"},{"key":"10824_CR343","unstructured":"Yang Z, Dai Z, Yang Y, Carbonell J, Salakhutdinov RR, Le QV (2019) XLNet: generalized autoregressive pretraining for language understanding. In: Advances in neural information processing systems, vol 32"},{"key":"10824_CR344","unstructured":"Yang J, Zhou K, Li Y, Liu Z (2021a) Generalized out-of-distribution detection: a survey. arXiv Preprint http:\/\/arxiv.org\/abs\/2110.11334"},{"key":"10824_CR345","doi-asserted-by":"crossref","unstructured":"Yang W, Li L, Zhang Z, Ren X, Sun X, He B (2021b) Be careful about poisoned word embeddings: exploring the vulnerability of the embedding layers in NLP models. In: Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies. pp 2048\u20132058","DOI":"10.18653\/v1\/2021.naacl-main.165"},{"key":"10824_CR346","unstructured":"Yang J, Jin H, Tang R, Han X, Feng Q, Jiang H, Yin B, Hu X (2023) Harnessing the power of LLMs in practice: a survey on ChatGPT and beyond. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.13712"},{"key":"10824_CR347","unstructured":"Yao Z, Yazdani Aminabadi R, Zhang M, Wu X, Li C, He Y (2022) ZeroQuant: efficient and affordable post-training quantization for large-scale transformers. In: Advances in neural information processing systems, vol 35. pp 27168\u201327183"},{"key":"10824_CR348","unstructured":"Yao S, Zhao J, Yu D, Du N, Shafran I, Narasimhan KR, Cao Y (2023) ReAct: synergizing reasoning and acting in language models. In: The eleventh international conference on learning representations"},{"key":"10824_CR349","doi-asserted-by":"crossref","unstructured":"Ye M, Gong C, Liu Q (2020) Safer: a structure-free approach for certified robustness to adversarial word substitutions. arXiv Preprint http:\/\/arxiv.org\/abs\/2005.14424","DOI":"10.18653\/v1\/2020.acl-main.317"},{"key":"10824_CR350","doi-asserted-by":"crossref","unstructured":"Ye X, Iyer S, Celikyilmaz A, Stoyanov V, Durrett G, Pasunuru R (2022) Complementary explanations for effective in-context learning. arXiv Preprint http:\/\/arxiv.org\/abs\/2211.13892","DOI":"10.18653\/v1\/2023.findings-acl.273"},{"key":"10824_CR351","doi-asserted-by":"crossref","unstructured":"Yilmaz E, Toraman C (2022) D2U: distance-to-uniform learning for out-of-scope detection. In: Proceedings of the 2022 conference of the North American chapter of the association for computational linguistics: human language technologies. pp 2093\u20132108","DOI":"10.18653\/v1\/2022.naacl-main.152"},{"key":"10824_CR352","unstructured":"Yu J, Xu Y, Koh JY, Luong T, Baid G, Wang Z, Vasudevan V, Ku A, Yang Y, Ayan BK et\u00a0al (2022) Scaling autoregressive models for content-rich text-to-image generation. arXiv Preprint http:\/\/arxiv.org\/abs\/2206.10789"},{"key":"10824_CR353","doi-asserted-by":"crossref","unstructured":"Zeng Z, He K, Yan Y, Liu Z, Wu Y, Xu H, Jiang H, Xu W (2021a) Modeling discriminative representations for out-of-domain detection with supervised contrastive learning. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 2: short papers). pp 870\u2013878","DOI":"10.18653\/v1\/2021.acl-short.110"},{"key":"10824_CR354","unstructured":"Zeng W, Ren X, Su T, Wang H, Liao Y, Wang Z, Jiang X, Yang Z, Wang K, Zhang X et\u00a0al (2021b) Pangu-$$\\alpha$$: large-scale autoregressive pretrained Chinese language models with auto-parallel computation. arXiv Preprint http:\/\/arxiv.org\/abs\/2104.12369"},{"key":"10824_CR356","unstructured":"Zeng J, Zheng X, Xu J, Li L, Yuan L, Huang X (2021c) Certified robustness to text adversarial attacks by randomized [mask]. arXiv Preprint http:\/\/arxiv.org\/abs\/2105.03743"},{"key":"10824_CR357","unstructured":"Zhang J, Zhao Y, Saleh M, Liu P (2020) PEGASUS: pre-training with extracted gap-sentences for abstractive summarization. In: III HD, Singh A (eds) Proceedings of the 37th international conference on machine learning, volume 119 of proceedings of machine learning research, 13\u201318 July 2020. PMLR, pp 11328\u201311339"},{"key":"10824_CR358","doi-asserted-by":"crossref","unstructured":"Zhang Y, Albarghouthi A, D\u2019Antoni L (2021) Certified robustness to programmable transformations in LSTMS. arXiv Preprint http:\/\/arxiv.org\/abs\/2102.07818","DOI":"10.18653\/v1\/2021.emnlp-main.82"},{"key":"10824_CR359","unstructured":"Zhang S, Roller S, Goyal N, Artetxe M, Chen M, Chen S, Dewan C, Diab M, Li X, Lin XV et\u00a0al (2022) OPT: open pre-trained transformer language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2205.01068"},{"key":"10824_CR360","unstructured":"Zhang T, Ladhak F, Durmus E, Liang P, McKeown K, Hashimoto TB (2023a) Benchmarking large language models for news summarization. arXiv Preprint http:\/\/arxiv.org\/abs\/2301.13848"},{"key":"10824_CR361","doi-asserted-by":"crossref","unstructured":"Zhang C, Ruan W, Wang F, Xu P, Min G, Huang X (2023b) Model-agnostic reachability analysis on deep neural networks. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.00813","DOI":"10.1007\/978-3-031-33374-3_27"},{"key":"10824_CR362","unstructured":"Zhang C, Ruan W, Xu P (2023c) Reachability analysis of neural network control systems. arXiv Preprint http:\/\/arxiv.org\/abs\/2301.12100"},{"key":"10824_CR363","unstructured":"Zhao Z, Dua D, Singh S (2017) Generating natural adversarial examples. arXiv Preprint http:\/\/arxiv.org\/abs\/1710.11342"},{"key":"10824_CR364","unstructured":"Zhao X, Huang W, Huang X, Robu V, Flynn D (2021a) BayLIME: Bayesian local interpretable model-agnostic explanations. In: de\u00a0Campos C, Maathuis MH (eds) Proceedings of the thirty-seventh conference on uncertainty in artificial intelligence, volume 161 of proceedings of machine learning research, 27\u201330 July 2021. PMLR, pp 887\u2013896"},{"key":"10824_CR365","doi-asserted-by":"crossref","unstructured":"Zhao X, Huang W, Schewe S, Dong Y, Huang X (2021b) Detecting operational adversarial examples for reliable deep learning. In: 2021 51st annual IEEE\/IFIP international conference on dependable systems and networks\u2014supplemental volume (DSN-S). pp 5\u20136","DOI":"10.1109\/DSN-S52858.2021.00013"},{"key":"10824_CR366","unstructured":"Zhao WX, Zhou K, Li J, Tang T, Wang X, Hou Y, Min Y, Zhang B, Zhang J, Dong Z et\u00a0al (2023a) A survey of large language models. arXiv Preprint http:\/\/arxiv.org\/abs\/2303.18223"},{"key":"10824_CR367","unstructured":"Zhao R, Li X, Chia YK, Ding B, Bing L (2023b) Can ChatGPT-like generative models guarantee factual accuracy? On the mistakes of new generation search engines. arXiv Preprint http:\/\/arxiv.org\/abs\/2304.11076"},{"key":"10824_CR368","unstructured":"Zhong Q, Ding L, Liu J, Du B, Tao D (2023) Can ChatGPT understand too? A comparative study on ChatGPT and fine-tuned BERT. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.10198"},{"key":"10824_CR369","doi-asserted-by":"crossref","unstructured":"Zhou W, Liu F, Chen M (2021) Contrastive out-of-distribution detection for pretrained transformers. In: Proceedings of the 2021 conference on empirical methods in natural language processing (EMNLP)","DOI":"10.18653\/v1\/2021.emnlp-main.84"},{"key":"10824_CR370","doi-asserted-by":"crossref","unstructured":"Zhou Y, Liu P, Qiu X (2022) KNN-contrastive learning for out-of-domain intent classification. In: Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers). pp 5129\u20135141","DOI":"10.18653\/v1\/2022.acl-long.352"},{"key":"10824_CR371","unstructured":"Zhou C, Li Q, Li C, Yu J, Liu Y, Wang G, Zhang K, Ji C, Yan Q, He L et\u00a0al (2023) A comprehensive survey on pretrained foundation models: a history from BERT to ChatGPT. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.09419"},{"key":"10824_CR372","unstructured":"Zhu RJ, Zhao Q, Eshraghian JK (2023) SpikeGPT: generative pre-trained language model with spiking neural networks. arXiv Preprint http:\/\/arxiv.org\/abs\/2302.13939"},{"key":"10824_CR373","unstructured":"Ziegler DM, Stiennon N, Wu J, Brown TB, Radford A, Amodei D, Christiano P, Irving G (2019) Fine-tuning language models from human preferences. arXiv Preprint http:\/\/arxiv.org\/abs\/1909.08593"}],"container-title":["Artificial Intelligence Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-024-10824-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10462-024-10824-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-024-10824-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,15]],"date-time":"2024-07-15T10:21:05Z","timestamp":1721038865000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10462-024-10824-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,17]]},"references-count":373,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["10824"],"URL":"https:\/\/doi.org\/10.1007\/s10462-024-10824-0","relation":{},"ISSN":["1573-7462"],"issn-type":[{"value":"1573-7462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,17]]},"assertion":[{"value":"5 June 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 June 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"175"}}