{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T01:04:53Z","timestamp":1774400693548,"version":"3.50.1"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2023,7,17]],"date-time":"2023-07-17T00:00:00Z","timestamp":1689552000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,17]],"date-time":"2023-07-17T00:00:00Z","timestamp":1689552000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007176","name":"Scuola Superiore Sant'Anna","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007176","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Real-Time Syst"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The heavy use of machine learning algorithms in safety-critical systems poses serious questions related to safety, security, and predictability issues, requiring novel architectural approaches to guarantee such properties. This paper presents an architecture solution that leverages heterogeneous platforms and virtualization technologies to support AI-powered applications consisting of modules with mixed criticalities and safety requirements. The hypervisor exploits the security features of the Xilinx ZCU104 MPSoCs to create two isolated execution environments: a high performance domain running deep learning algorithms under the Linux operating system and a safety-critical domain running control and monitoring functions under the freeRTOS real-time operating system. The proposed approach is validated by a use case consisting of an unmanned aerial vehicle capable of tracking moving targets using a deep neural network accelerated on the FGPA available on the platform.<\/jats:p>","DOI":"10.1007\/s11241-023-09402-4","type":"journal-article","created":{"date-parts":[[2023,7,17]],"date-time":"2023-07-17T17:02:12Z","timestamp":1689613332000},"page":"609-635","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Supporting AI-powered real-time cyber-physical systems on heterogeneous platforms via hypervisor technology"],"prefix":"10.1007","volume":"59","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1714-8960","authenticated-orcid":false,"given":"Edoardo","family":"Cittadini","sequence":"first","affiliation":[]},{"given":"Mauro","family":"Marinoni","sequence":"additional","affiliation":[]},{"given":"Alessandro","family":"Biondi","sequence":"additional","affiliation":[]},{"given":"Giorgiomaria","family":"Cicero","sequence":"additional","affiliation":[]},{"given":"Giorgio","family":"Buttazzo","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,7,17]]},"reference":[{"key":"9402_CR1","unstructured":"Accelerat: The CLARE Software Stack. https:\/\/accelerat.eu\/clare"},{"key":"9402_CR2","unstructured":"Almeida J, Prochazka M (2009) Safe and Secure Partitioning with Pikeos: Towards Integrated Modular Avionics in Space. In: Proceedings of DASIA 2009 Data Systems in Aerospace, p. 27"},{"key":"9402_CR3","unstructured":"AMD Xilinx: DPU - Deep Learning Processing Unit. https:\/\/www.xilinx.com\/products\/intellectual-property\/dpu.html"},{"key":"9402_CR4","unstructured":"AMD Xilinx: FINN Framework. https:\/\/xilinx.github.io\/finn\/"},{"key":"9402_CR5","unstructured":"AMD Xilinx: Vitis AI - Adaptable and Real-Time AI Inference Acceleration. https:\/\/www.xilinx.com\/products\/design-tools\/vitis\/vitis-ai.html"},{"issue":"3","key":"9402_CR6","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1109\/LES.2019.2953253","volume":"12","author":"A Biondi","year":"2020","unstructured":"Biondi A, Nesti F, Cicero G, Casini D, Buttazzo G (2020) A Safe, Secure, and Predictable Software Architecture for Deep Learning in Safety-Critical Systems. IEEE Embedded Systems Letters 12(3):78\u201382","journal-title":"IEEE Embedded Systems Letters"},{"key":"9402_CR7","doi-asserted-by":"publisher","first-page":"75446","DOI":"10.1109\/ACCESS.2021.3080842","volume":"9","author":"A Biondi","year":"2021","unstructured":"Biondi A, Casini D, Cicero G, Borgioli N, Buttazzo G, Patti G, Leonardi L, Bello LL, Solieri M, Burgio P, Olmedo IS, Ruocco A, Palazzi L, Bertogna M, Cilardo A, Mazzocca N, Mazzeo A (2021) Sphere: a multi-soc architecture for next-generation cyber-physical systems based on heterogeneous platforms. IEEE Access 9:75446\u201375459","journal-title":"IEEE Access"},{"key":"9402_CR8","doi-asserted-by":"crossref","unstructured":"Biondi A, Balsini A, Pagani M, Rossi E, Marinoni M, Buttazzo G (2016) A Framework for Supporting Real-Time Applications on Dynamic Reconfigurable FPGAs. In: Proc. of the IEEE Real-Time Systems Symposium (RTSS 2016), Porto, Portugal","DOI":"10.1109\/RTSS.2016.010"},{"key":"9402_CR9","doi-asserted-by":"crossref","unstructured":"Cavicchioli R, Capodieci N, Bertogna M (2017) Memory Interference Characterization Between CPU Cores and Integrated GPUs in Mixed-Criticality Platforms. In: Proc. of the 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA 2017)","DOI":"10.1109\/ETFA.2017.8247615"},{"key":"9402_CR10","unstructured":"Cordts M, Omran M, Ramos S, Scharw\u00e4chter T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2015) The Cityscapes Dataset. In: CVPR Workshop on The Future of Datasets in Vision"},{"key":"9402_CR11","unstructured":"Craveiro J, Rufino J, Schoofs T, Windsor J (2009) Flexible Operating System Integration in Partitioned Aerospace Systems. In: Actas do INForum - Simposio de Informatica, p 49\u201360"},{"key":"9402_CR12","doi-asserted-by":"publisher","first-page":"50627","DOI":"10.1109\/ACCESS.2018.2869094","volume":"6","author":"A Crespo","year":"2018","unstructured":"Crespo A, Balbastre P, Sim\u00f3 J, Coronel J, P\u00e9rez DG, Bonnot P (2018) Hypervisor-based multicore feedback control of mixed-criticality systems. IEEE Access 6:50627\u201350640","journal-title":"IEEE Access"},{"key":"9402_CR13","unstructured":"Crespo A, Ripoll I, Masmano M, Arberet P, Jean-Jacques M (2009) XtratuM: an Open Source Hypervisor for TSP Embedded Systems in Aerospace. In: Proceedings of DASIA 2009 Data Systems in Aerospace"},{"key":"9402_CR14","unstructured":"Crespo A, Ripoll I, Masmano M, Arberet P, Metge JJ (2009) XtratuM: An open source hypervisor for TSP embedded systems in aerospace. In: Data Systems In Aerospace (DASIA), Istanbul, Turkey (May 26-29)"},{"key":"9402_CR15","unstructured":"CubePilot: The Cube Autopilot. https:\/\/www.cubepilot.com\/"},{"key":"9402_CR16","doi-asserted-by":"crossref","unstructured":"Ding C, Wang S, Liu N, Xu K, Wang Y, Liang Y (2019) REQ-YOLO: A Resource-Aware, Efficient Quantization Framework for Object Detection on FPGAs. arXiv:1909.13396","DOI":"10.1145\/3289602.3293904"},{"key":"9402_CR17","unstructured":"Fahim F, Hawks B, Herwig C, Hirschauer J, Jindariani S, Tran N, Carloni LP, Di Guglielmo G, Harris P, Krupa J, Rankin D, Valentin MB, Hester J, Luo Y, Mamish J, Orgrenci-Memik S, Aarrestad T, Javed H, Loncar V, Pierini M, Pol AA, Summers S, Duarte J, Hauck S, Hsu S-C, Ngadiuba J, Liu M, Hoang D, Kreinar E, Wu Z (2021) hls4ml: an Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices. arXiv. arXiv:2103.05579"},{"key":"9402_CR18","doi-asserted-by":"crossref","unstructured":"Farrukh A, West R (2022) FLYOS: Integrated Modular Avionics for Autonomous Multicopters. In: 28th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS 2022)","DOI":"10.1109\/RTAS54340.2022.00014"},{"key":"9402_CR19","doi-asserted-by":"crossref","unstructured":"Gaska T, Werner B, Flagg D (2011) Applying virtualization to avionics systems - the integration challenges. IEEE Aerospace and Electronic Systems Magazine 26","DOI":"10.1109\/DASC.2010.5655297"},{"key":"9402_CR20","doi-asserted-by":"crossref","unstructured":"Gholami A, Kim S, Zhen D, Yao Z, Mahoney M, Keutzer K (2022) A Survey of Quantization Methods for Efficient Neural Network Inference. Low-Power Computer Vision, p 291\u2013326","DOI":"10.1201\/9781003162810-13"},{"key":"9402_CR21","unstructured":"Guti\u00e9rrez CSV, Juan LUS, Ugarte IZ, Vilches VM (2018) Towards a distributed and real-time framework for robots: Evaluation of ROS 2.0 communications for real-time robotic applications. arXiv:1809.02595"},{"key":"9402_CR22","doi-asserted-by":"crossref","unstructured":"Happe M, Traber A, Keller A (2015) Preemptive hardware multitasking in reconos. In: International Workshop on Applied Reconfigurable Computing","DOI":"10.1007\/978-3-319-16214-0_7"},{"key":"9402_CR23","unstructured":"Intel Corporation: Overview of Intel Ready to Fly Drone. https:\/\/www.intel.it\/content\/www\/it\/it\/support\/articles\/000023272\/drones\/development-drones.html"},{"key":"9402_CR24","doi-asserted-by":"crossref","unstructured":"Ji Q, Dai C, Hou C, Li X (2021) Real-time embedded object detection and tracking system in zynq soc. EURASIP Journal on Image and Video Processing 21","DOI":"10.1186\/s13640-021-00561-7"},{"issue":"10","key":"9402_CR25","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1145\/3230627","volume":"61","author":"G Klein","year":"2018","unstructured":"Klein G, Andronick J, Fernandez M, Kuz I, Murray T, Heiser G (2018) Formally Verified Software in the Real World. Communications of the ACM 61(10):68\u201377","journal-title":"Communications of the ACM"},{"key":"9402_CR26","doi-asserted-by":"crossref","unstructured":"Leiner B, Schlager M, Obermaisser R, Huber B (2007) A Comparison of Partitioning Operating Systems for Integrated Systems. In: Computer Safety, Reliability, and Security. Springer, Berlin p 342\u2013355","DOI":"10.1007\/978-3-540-75101-4_33"},{"issue":"6","key":"9402_CR27","first-page":"821","volume":"46","author":"J Lelli","year":"2016","unstructured":"Lelli J, Scordino C, Abeni L, Faggioli D (2016) Deadline scheduling in the Linux kernel. Software: Pract Exper 46(6):821\u2013839","journal-title":"Software: Practice and Experience"},{"key":"9402_CR28","doi-asserted-by":"publisher","first-page":"370","DOI":"10.1016\/j.neucom.2021.07.045","volume":"461","author":"T Liang","year":"2021","unstructured":"Liang T, Glossner J, Wang L, Shi S, Zhang X (2021) Pruning and quantization for deep neural network acceleration: a survey. Neurocomputing 461:370\u2013403","journal-title":"Neurocomputing"},{"key":"9402_CR29","doi-asserted-by":"crossref","unstructured":"Liu M, Niu J, Wang X (2017) An autopilot system based on ros distributed architecture and deep learning. In: 2017 IEEE 15th International Conference on Industrial Informatics (INDIN), pp 1229\u20131234","DOI":"10.1109\/INDIN.2017.8104950"},{"key":"9402_CR30","unstructured":"LYNX Software Technologies: LynxSecure Embedded Hypervisor and Separation Kernel. http:\/\/www.lynx.com\/products\/hypervisors\/"},{"key":"9402_CR31","doi-asserted-by":"crossref","unstructured":"Meier L, Honegger D, Pollefeys M (2015) Px4: A node-based multithreaded open source robotics framework for deeply embedded platforms. In: 2015 IEEE International Conference on Robotics and Automation (ICRA), p 6235\u20136240","DOI":"10.1109\/ICRA.2015.7140074"},{"key":"9402_CR32","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1109\/2.362628","volume":"28","author":"D Musliner","year":"1995","unstructured":"Musliner D, Hendler J, Agrawala A, Durfee E, Strosnider J, Paul CJ (1995) Challenges of real-time ai. Computer 28:58\u201366","journal-title":"Computer"},{"key":"9402_CR33","unstructured":"O\u2019Kelly M, Sukhil V, Abbas H, Harkins J, Kao C, Pant YV, Mangharam R, Agarwal D, Behl M, Burgio P, Bertogna M (2019) F1\/10: An Open-Source Autonomous Cyber-Physical Platform. arXiv. arXiv:1901.08567"},{"key":"9402_CR34","doi-asserted-by":"crossref","unstructured":"P\u00e9rez H, Guti\u00e9rrez JJ (2016) Handling heterogeneous partitioned systems through ARINC-653 and DDS. Comput. Stand. Interfaces 50:258-268","DOI":"10.1016\/j.csi.2016.10.012"},{"key":"9402_CR35","doi-asserted-by":"crossref","unstructured":"P\u00e9rez H, Guti\u00e9rrez JJ, Peir\u00f3 S, Crespo A (2016) Distributed architecture for developing mixed-criticality systems in multi-core platforms. J Syst Soft 123:145\u2013159","DOI":"10.1016\/j.jss.2016.08.088"},{"key":"9402_CR36","doi-asserted-by":"crossref","unstructured":"Qasaimeh M, Denolf K, Lo J, Vissers K, Zambreno J, Jones PH (2019) Comparing energy efficiency of cpu, gpu and fpga implementations for vision kernels. In: 2019 IEEE International Conference on Embedded Software and Systems (ICESS), p 1\u20138","DOI":"10.1109\/ICESS.2019.8782524"},{"key":"9402_CR37","doi-asserted-by":"publisher","first-page":"783","DOI":"10.1007\/s00146-020-01049-0","volume":"36","author":"P Radanliev","year":"2020","unstructured":"Radanliev P, De Roure D, Van Kleek M, Santos O, Ani U (2020) Artificial intelligence in cyber physical systems. AI & Sociaty 36:783\u2013796","journal-title":"AI & Sociaty"},{"key":"9402_CR38","unstructured":"Redmon J, Farhadi A (2018) YOLOv3: An Incremental Improvement. arXiv"},{"key":"9402_CR39","doi-asserted-by":"crossref","unstructured":"Reke M, Peter D, Schulte-Tigges J, Schiffer S, Ferrein A, Walter T, Matheis D (2020) A Self-Driving Car Architecture in ROS2, p 1\u20136","DOI":"10.1109\/SAUPEC\/RobMech\/PRASA48453.2020.9041020"},{"key":"9402_CR40","doi-asserted-by":"crossref","unstructured":"Rupnow K, Fu W, Compton K (2009) Block, drop or roll(back): alternative preemption methods for rh multi-tasking. In: 2009 17th IEEE Symposium on Field Programmable Custom Computing Machines, p 63\u201370","DOI":"10.1109\/FCCM.2009.30"},{"key":"9402_CR41","doi-asserted-by":"crossref","unstructured":"Sciangula G, Restuccia F, Biondi A, Buttazzo G (2022) Hardware Acceleration of Deep Neural Networks for Autonomous Driving on FPGA-based SoC. In: the 25th Euromicro Conference on Digital System Design (DSD), Maspalomas, Gran Canaria, Spain","DOI":"10.1109\/DSD57027.2022.00061"},{"key":"9402_CR42","doi-asserted-by":"crossref","unstructured":"Scordino C, Savino IM, Cuomo L, Miccio L, Tagliavini A, Bertogna M, Solieri M (2020) Real-time virtualization for industrial automation. In: 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), vol. 1, p 353\u2013360","DOI":"10.1109\/ETFA46521.2020.9211890"},{"key":"9402_CR43","doi-asserted-by":"crossref","unstructured":"Seng KP, Lee PJ, Ang LM (2021) Embedded intelligence on fpga: survey, applications and challenges. Electronics 10(8):895","DOI":"10.3390\/electronics10080895"},{"issue":"11","key":"9402_CR44","doi-asserted-by":"publisher","first-page":"1988","DOI":"10.1109\/TC.2020.3033730","volume":"70","author":"B Seyoum","year":"2021","unstructured":"Seyoum B, Pagani M, Biondi A, Balleri S, Buttazzo G (2021) Spatio-temporal optimization of deep neural networks for reconfigurable FPGA SoCs. IEEE Transactions on Computers 70(11):1988\u20132000","journal-title":"IEEE Transactions on Computers"},{"key":"9402_CR45","unstructured":"SYSGO: PikeOS Hypervisor. http:\/\/www.sysgo.com\/products\/pikeos-rtos-and-virtualization-concept"},{"key":"9402_CR46","doi-asserted-by":"crossref","unstructured":"Vaishnav A, Pham KD, Koch D (2018) A survey on fpga virtualization. In: 2018 28th International Conference on Field Programmable Logic and Applications (FPL), pp 131\u20131317","DOI":"10.1109\/FPL.2018.00031"},{"key":"9402_CR47","doi-asserted-by":"crossref","unstructured":"Wang C, Luo Z (2022) A review of the optimal design of neural networks based on fpga. Appl Sci 12(21):10771","DOI":"10.3390\/app122110771"},{"key":"9402_CR48","doi-asserted-by":"crossref","unstructured":"Wojke N, Bewley A, Paulus D (2017) Simple Online and Realtime Tracking with a Deep Association Metric. In 2017 IEEE international conference on image processing (ICIP), 3645\u20133649 Sep 2017","DOI":"10.1109\/ICIP.2017.8296962"},{"key":"9402_CR49","doi-asserted-by":"crossref","unstructured":"Zhang Y, Sun P, Jiang Y, Yu D, Weng F, Yuan Z, Luo P, Liu W, Wang X (2021) ByteTrack: multi-Object Tracking by Associating Every Detection Box. arXiv","DOI":"10.1007\/978-3-031-20047-2_1"},{"key":"9402_CR50","doi-asserted-by":"crossref","unstructured":"Zhou S, Wang Y, Wen H, He Q, Zou Y (2017) Balanced quantization: an effective and efficient approach to quantized neural networks. Journal of Computer Science and Technology 32","DOI":"10.1007\/s11390-017-1750-y"}],"container-title":["Real-Time Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11241-023-09402-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11241-023-09402-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11241-023-09402-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,25]],"date-time":"2024-01-25T14:30:02Z","timestamp":1706193002000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11241-023-09402-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,17]]},"references-count":50,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["9402"],"URL":"https:\/\/doi.org\/10.1007\/s11241-023-09402-4","relation":{},"ISSN":["0922-6443","1573-1383"],"issn-type":[{"value":"0922-6443","type":"print"},{"value":"1573-1383","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,17]]},"assertion":[{"value":"16 June 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 July 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}