{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T19:02:31Z","timestamp":1779217351131,"version":"3.51.4"},"reference-count":65,"publisher":"Emerald","issue":"3","license":[{"start":{"date-parts":[[2023,11,23]],"date-time":"2023-11-23T00:00:00Z","timestamp":1700697600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ICS"],"published-print":{"date-parts":[[2024,6,11]]},"abstract":"<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title>\n<jats:p>The purpose of this paper is to highlight the key technical challenges that derive from the recently proposed European Artificial Intelligence Act and specifically, to investigate the applicability of the requirements that the AI Act mandates to high-risk AI systems from the perspective of AI security.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title>\n<jats:p>This paper presents the main points of the proposed AI Act, with emphasis on the compliance requirements of high-risk systems. It matches known AI security threats with the relevant technical requirements, it demonstrates the impact that these security threats can have to the AI Act technical requirements and evaluates the applicability of these requirements based on the effectiveness of the existing security protection measures. Finally, the paper highlights the necessity for an integrated framework for AI system evaluation.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Findings<\/jats:title>\n<jats:p>The findings of the EU AI Act technical assessment highlight the gap between the proposed requirements and the available AI security countermeasures as well as the necessity for an AI security evaluation framework.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title>\n<jats:p>AI Act, high-risk AI systems, security threats, security countermeasures.<\/jats:p>\n<\/jats:sec>","DOI":"10.1108\/ics-10-2022-0165","type":"journal-article","created":{"date-parts":[[2023,11,21]],"date-time":"2023-11-21T07:11:28Z","timestamp":1700550688000},"page":"265-281","source":"Crossref","is-referenced-by-count":25,"title":["European Artificial Intelligence Act: an AI security approach"],"prefix":"10.1108","volume":"32","author":[{"given":"Konstantinos","family":"Kalodanis","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Panagiotis","family":"Rizomiliotis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dimosthenis","family":"Anagnostopoulos","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","published-online":{"date-parts":[[2023,11,23]]},"reference":[{"key":"key2024060713480370100_ref001","unstructured":"Adam, K. (2020), \u201cThe U.K. used an algorithm to estimate exam results. The calculations favored elites\u201d, available at: www.washingtonpost.com\/world\/europe\/the-uk-used-an-algorithm-to-estimate-exam-results-the-calculations-favored-elites\/2020\/08\/17\/2b116d48-e091-11ea-82d8-5e55d47e90ca_story.html (accessed 10 February 2022)."},{"issue":"1\/4","key":"key2024060713480370100_ref002","first-page":"27","article-title":"Multiple classifier systems for robust classifier design in adversarial environments","volume":"1","year":"2010","journal-title":"International Journal of Machine Learning and Cybernetics"},{"key":"key2024060713480370100_ref003","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1007\/978-3-642-21557-5_37","article-title":"Bagging classifiers for fighting poisoning attacks in adversarial classification tasks","year":"2011","journal-title":"Multiple Classifier Systems (MCS) 2011"},{"key":"key2024060713480370100_ref004","article-title":"Notes from the AI frontier. modeling the impact of AI on the world economy","year":"2018"},{"key":"key2024060713480370100_ref005","unstructured":"Carlini, N., et al. (2016), \u201cDefensive distillation is not robust to adversarial examples\u201d, unpublished manuscript, arXiv:1607.04311, available at: arxiv.org\/abs\/1607.04311 (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref006","first-page":"39","article-title":"Towards evaluating the robustness of neural networks","volume-title":"proceedings of\u2019, 2017 IEEE Symposium on Security and Privacy (SP)","year":"2017"},{"issue":"2013","key":"key2024060713480370100_ref007","first-page":"2905","article-title":"Near-optimal algorithms for differentially-private principal components","volume":"14","year":"2013","journal-title":"Journal of Machine Learning Research"},{"key":"key2024060713480370100_ref008","first-page":"1310","article-title":"Certified adversarial robustness via randomized smoothing","year":"2019"},{"key":"key2024060713480370100_ref009","unstructured":"Dziugaite, G.K., et al. (2016), \u201cA study of the effect of JPG compression on adversarial images\u201d, unpublished manuscript, arXiv:1608.00853, available at: arxiv.org\/abs\/1608.00853 (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref010","unstructured":"ENISA (2016), \u201cLatest version of ENISA\u2019s threat taxonomy\u201d, available at: www.enisa.europa.eu\/topics\/threat-risk-management\/threats-and-trends\/enisa-threat-landscape\/threat-taxonomy\/view (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref011","article-title":"AI cybersecurity challenges. Threat landscape for artificial intelligence","author":"ENISA","year":"2020"},{"key":"key2024060713480370100_ref012","unstructured":"ENISA (2021a), \u201cCybersecurity challenges in the uptake of artificial intelligence in autonomous driving\u201d, available at: www.enisa.europa.eu\/publications\/enisa-jrc-cybersecurity-challenges-in-the-uptake-of-artificial-intelligence-in-autonomous-driving (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref013","volume-title":"Securing Machine Learning Algorithms","author":"ENISA","year":"2021"},{"key":"key2024060713480370100_ref014","volume-title":"Cybersecurity of AI and Standardisation","author":"ENISA","year":"2023"},{"issue":"452","key":"key2024060713480370100_ref015","article-title":"The ethics of artificial intelligence: issues and initiatives\u2019, scientific foresight unit (STOA)","volume":"634","author":"European Parliamentary Research Service","year":"2020","journal-title":"PE"},{"key":"key2024060713480370100_ref016","first-page":"1","article-title":"Masking deep neural network models for robustness against adversarial samples","year":"2019"},{"key":"key2024060713480370100_ref017","unstructured":"Goodfellow, I.J., et al. (2015), \u201cExplaining and harnessing adversarial examples\u201d, unpublished manuscript, arXiv:1412.6572v3, available at: arxiv.org\/abs\/1412.6572v3 (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref018","first-page":"3","article-title":"Deepsafe: a data-driven approach for assessing robustness of neural networks","year":"2018"},{"key":"key2024060713480370100_ref019","unstructured":"Grosse, K., et al (2017), \u201cOn the (statistical) detection of adversarial examples\u201d, unpublished manuscript, arXiv:1702.06280, available at: arxiv.org\/abs\/1702.06280 (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref020","unstructured":"Gu, S. and Rigazio, L. (2015), \u201cTowards deep neural network architectures robust to adversarial examples\u201d, unpublished manuscript, arXiv:1412.5068, available at: arxiv.org\/abs\/1412.5068 (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref021","article-title":"Gutachten der datenethikkommission","volume-title":"Bundensministerium Der Justiz Und Fur Verbraucherschutz","year":"2019"},{"key":"key2024060713480370100_ref022","unstructured":"Hinton, G., et al. (2015), \u201cDistilling the knowledge in a neural network\u201d, unpublished manuscript, arXiv:1503.02531 [stat.ML], available at: arxiv.org\/abs\/1503.02531 (accessed 19 May 2022)."},{"issue":"1","key":"key2024060713480370100_ref023","first-page":"20","article-title":"Artificial intelligence security: threats and countermeasures","volume":"55","year":"2021","journal-title":"ACM Computing Surveys"},{"key":"key2024060713480370100_ref024","first-page":"19","article-title":"Manipulating machine learning: poisoning attacks and countermeasures for regression learning","year":"2021"},{"key":"key2024060713480370100_ref025","first-page":"97","article-title":"An efficient SMT solver for verifying deep neural networks","year":"2017"},{"key":"key2024060713480370100_ref026","unstructured":"Knight, W. (2019), \u201cThe apple card didn\u2019t \u2018see\u2019 gender - and that\u2019s the problem\u201d, available at: www.wired.com\/story\/the-apple-card-didnt-see-genderand-thats-the-problem\/ (accessed 10 February 2022)."},{"key":"key2024060713480370100_ref027","article-title":"Understanding artificial intelligence ethics and safety: a guide for the responsible design and implementation of AI systems in the public sector","year":"2019"},{"key":"key2024060713480370100_ref028","doi-asserted-by":"crossref","first-page":"12103","DOI":"10.1109\/ACCESS.2018.2805680","article-title":"A survey on security threats and defensive techniques of machine learning: a data driven view","volume":"6","year":"2018","journal-title":"IEEE Access"},{"key":"key2024060713480370100_ref029","first-page":"446","article-title":"SafetyNet: detecting and rejecting adversarial examples robustly","year":"2017"},{"key":"key2024060713480370100_ref030","unstructured":"Luo, Y., et al. (2015), \u201cFoveation-based mechanisms alleviate adversarial examples\u201d, unpublished manuscript, arXiv:1511.06292, available at: arxiv.org\/abs\/1511.06292 (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref031","first-page":"301","article-title":"A unified gradient regularization family for adversarial examples","year":"2016"},{"issue":"8","key":"key2024060713480370100_ref032","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1109\/MC.2019.2909955","article-title":"Security engineering for machine learning","volume":"52","year":"2019","journal-title":"Computer"},{"key":"key2024060713480370100_ref033","first-page":"135","article-title":"Magnet: a two-pronged defense against adversarial examples","year":"2017"},{"key":"key2024060713480370100_ref034","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2022.103006","article-title":"The threat of offensive AI to organizations","volume":"124","year":"2023","journal-title":"Computers and Security"},{"key":"key2024060713480370100_ref035","first-page":"1765","article-title":"Universal adversarial perturbations","year":"2017"},{"key":"key2024060713480370100_ref036","first-page":"9078","article-title":"Robustness via curvature regularization, and vice versa","year":"2019"},{"key":"key2024060713480370100_ref037","unstructured":"Muller, N., et al. (2021), \u201cDefending against adversarial denial-of-service data poisoning attacks\u201d, working paper, ACSAC Dynamics, available at: arxiv.org\/pdf\/2104.06744.pdf (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref038","first-page":"1","article-title":"Exploiting machine learning to subvert your spam filter","year":"2008"},{"key":"key2024060713480370100_ref039","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1007\/978-0-387-88735-7_2","article-title":"Misleading learners: co-opting your spam filter","volume-title":"Machine Learning in Cyber Trust: Security, Privacy, and Reliability","year":"2009"},{"key":"key2024060713480370100_ref040","first-page":"582","article-title":"Distillation as a defense to adversarial perturbations against deep neural networks","year":"2016"},{"key":"key2024060713480370100_ref041","unstructured":"Said, C. (2018), \u201cVideo shows Uber robot car in fatal accident did not try to avoid woman\u201d, available at: www.sfgate.com\/business\/article\/Uber-video-shows-robot-car-in-fatal-accident-did-12771938.php (accessed 20 September 2022)."},{"key":"key2024060713480370100_ref042","unstructured":"Samangouei, P., et al. (2018), \u201cDefense-GAN: protecting classifiers against adversarial attacks using generative models\u201d, unpublished manuscript, arXiv:1805.06605, available at: arxiv.org\/abs\/1805.06605 (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref043","article-title":"Differentially private model selection via stability arguments and the robustness of the lasso","year":"2013"},{"key":"key2024060713480370100_ref044","unstructured":"Song, Y., et al. (2017), \u201cPixelDefend: leveraging generative models to understand and defend against adversarial examples\u201d, unpublished manuscript, arXiv:1710.10766, available at: arxiv.org\/abs\/1710.10766 (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref045","unstructured":"Tatwadarshi, P. (2021), \u201cSecurity threats to machine learning systems\u201d, available at: www.analyticsvidhya.com\/blog\/2021\/01\/security-threats-to-machine-learning-systems (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref046","unstructured":"The EU Artificial Intelligence Act (2023), available at: www.artificial-intelligence-act.com (accessed 20 November 2023)."},{"key":"key2024060713480370100_ref047","unstructured":"The White House (2019), \u201cMaintaining American leadership in artificial intelligence, executive order 13859\u201d, available at: www.federalregister.gov\/documents\/2019\/02\/14\/2019-02544\/maintaining-american-leadership-in-artificial-intelligence (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref048","article-title":"Ensemble adversarial training: attacks and defenses","year":"2018"},{"key":"key2024060713480370100_ref049","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.jpdc.2019.03.003","article-title":"The security of machine learning in an adversarial setting: a survey","volume":"130","year":"2019","journal-title":"Journal of Parallel and Distributed Computing"},{"key":"key2024060713480370100_ref050","unstructured":"Xie, C., et al. (2017), \u201cMitigating adversarial effects through randomization\u201d, unpublished manuscript, arXiv:1711.01991, available at: arxiv.org\/abs\/1711.01991 (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref051","volume-title":"Towards a Robust and Trustworthy Machine Learning System Development","year":"2021"},{"key":"key2024060713480370100_ref052","unstructured":"Xu, W., et al. (2017), \u201cFeature squeezing: detecting adversarial examples in deep neural networks\u201d, unpublished manuscript, arXiv:1704.01155, available at: arxiv.org\/abs\/1704.01155 (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref053","article-title":"A survey of artificial intelligence risk assessment methodologies. The global state of play and leading practices identified","year":"2021"},{"key":"key2024060713480370100_ref054","volume-title":"Secure, Robust and Transparent Application of AI. Problems Measures and Need for Action","author":"Bundensamt fur Sicherheit in der Informationstechnik","year":"2021"},{"key":"key2024060713480370100_ref055","article-title":"sentiNet: detecting localized universal attack against deep learning systems","year":"2020"},{"issue":"16","key":"key2024060713480370100_ref056","article-title":"Robustness evaluations of sustainable machine learning models against data poisoning attacks in the internet of things","volume":"12","year":"2020","journal-title":"Sustainability"},{"key":"key2024060713480370100_ref057","unstructured":"European Commission (2021a), \u201cAnnexes to the proposal for a regulation of the European parliament and of the council laying down harmonized rules on artificial intelligence (artificial intelligence ACT) and amending certain union legislative acts\u201d, Brussels, available at: https:\/\/eur-lex.europa.eu\/resource.html?uri=cellar:0649735-a372-11eb-9585-01aa75ed71a1.0001.02\/DOC2&format=PDF (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref058","unstructured":"European Commission (2021b), \u201cProposal for a regulation of the European parliament and of the council laying down harmonized rules on artificial intelligence (artificial intelligence ACT) and amending certain union legislative acts\u201d, Brussels, available at: eur-lex.europa.eu\/legal-content\/EN\/TXT\/?uri=celex%3A52021PC0206 (accessed 19 May 2022)."},{"issue":"7","key":"key2024060713480370100_ref059","article-title":"Semantics-aligned representation learning for person re-identification","volume":"34","year":"2020","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"key2024060713480370100_ref060","first-page":"21","article-title":"Decamouflage: a framework to detect image-scaling attacks on convolutional neural networks","year":"2020"},{"key":"key2024060713480370100_ref061","unstructured":"Labaca-Castro, R., et al. (2022), \u201cRealizable universal adversarial perturbations for malware\u201d, unpublished manuscript, arXiv:2102.06747v2 [cs.CR], available at: https:\/\/arxiv.org\/pdf\/2102.06747.pdf (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref062","unstructured":"NIST (2019), \u201cA taxonomy and terminology of adversarial machine learning\u201d, available at: https:\/\/csrc.nist.gov\/publications\/detail\/nistir\/8269\/draft (accessed 19 May 2022)."},{"key":"key2024060713480370100_ref063","first-page":"1363","article-title":"Adversarial preprocessing: understanding and preventing image-scaling attacks in machine learning","year":"2020"},{"key":"key2024060713480370100_ref064","article-title":"Case studies on AI skills capacity building and AI in workforce development in Africa","year":"2021"},{"key":"key2024060713480370100_ref065","first-page":"1004","article-title":"PyCRA: Physical challenge-response authentication for active sensors under spoofing attacks categories and subject descriptors","year":"2015"}],"container-title":["Information &amp; Computer Security"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/ICS-10-2022-0165\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/ICS-10-2022-0165\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T01:23:16Z","timestamp":1753406596000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/ics\/article\/32\/3\/265-281\/1237025"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,23]]},"references-count":65,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,11,23]]},"published-print":{"date-parts":[[2024,6,11]]}},"alternative-id":["10.1108\/ICS-10-2022-0165"],"URL":"https:\/\/doi.org\/10.1108\/ics-10-2022-0165","relation":{},"ISSN":["2056-4961","2056-4961"],"issn-type":[{"value":"2056-4961","type":"print"},{"value":"2056-4961","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,23]]}}}