{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T05:55:12Z","timestamp":1763358912082,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T00:00:00Z","timestamp":1679616000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"H2020 Humane-AI-Net","award":["952026","CHIST-ERA-19-XAI-010-MUR-484-22","PE0000013"],"award-info":[{"award-number":["952026","CHIST-ERA-19-XAI-010-MUR-484-22","PE0000013"]}]},{"DOI":"10.13039\/501100001942","name":"CHIST-ERA","doi-asserted-by":"publisher","award":["952026","CHIST-ERA-19-XAI-010-MUR-484-22","PE0000013"],"award-info":[{"award-number":["952026","CHIST-ERA-19-XAI-010-MUR-484-22","PE0000013"]}],"id":[{"id":"10.13039\/501100001942","id-type":"DOI","asserted-by":"publisher"}]},{"name":"European Union under the Italian National Recovery and Resilience Plan (NRRP) of partnership on \u201cArtificial Intelligence: Foundational Aspects\u201d","award":["952026","CHIST-ERA-19-XAI-010-MUR-484-22","PE0000013"],"award-info":[{"award-number":["952026","CHIST-ERA-19-XAI-010-MUR-484-22","PE0000013"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Given the increasing prevalence of intelligent systems capable of autonomous actions or augmenting human activities, it is important to consider scenarios in which the human, autonomous system, or both can exhibit failures as a result of one of several contributing factors (e.g., perception). Failures for either humans or autonomous agents can lead to simply a reduced performance level, or a failure can lead to something as severe as injury or death. For our topic, we consider the hybrid human\u2013AI teaming case where a managing agent is tasked with identifying when to perform a delegated assignment and whether the human or autonomous system should gain control. In this context, the manager will estimate its best action based on the likelihood of either (human, autonomous) agent\u2019s failure as a result of their sensing capabilities and possible deficiencies. We model how the environmental context can contribute to, or exacerbate, these sensing deficiencies. These contexts provide cases where the manager must learn to identify agents with capabilities that are suitable for decision-making. As such, we demonstrate how a reinforcement learning manager can correct the context\u2013delegation association and assist the hybrid team of agents in outperforming the behavior of any agent working in isolation.<\/jats:p>","DOI":"10.3390\/s23073409","type":"journal-article","created":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T03:16:46Z","timestamp":1679627806000},"page":"3409","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Compensating for Sensing Failures via Delegation in Human\u2013AI Hybrid Systems"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7191-8781","authenticated-orcid":false,"given":"Andrew","family":"Fuchs","sequence":"first","affiliation":[{"name":"Department of Computer Science, Universit\u00e1 di Pisa, 56124 Pisa, Italy"},{"name":"Institute for Informatics and Telematics (IIT), National Research Council (CNR), 56124 Pisa, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1694-612X","authenticated-orcid":false,"given":"Andrea","family":"Passarella","sequence":"additional","affiliation":[{"name":"Institute for Informatics and Telematics (IIT), National Research Council (CNR), 56124 Pisa, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4097-4064","authenticated-orcid":false,"given":"Marco","family":"Conti","sequence":"additional","affiliation":[{"name":"Institute for Informatics and Telematics (IIT), National Research Council (CNR), 56124 Pisa, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,24]]},"reference":[{"key":"ref_1","unstructured":"(2023, February 06). Fatality and Injury Reporting System Tool (FIRST), Available online: https:\/\/cdan.dot.gov\/query."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Fuchs, A., Passarella, A., and Conti, M. (2023). Modeling, Replicating, and Predicting Human Behavior: A Survey. ACM Trans. Auton. Adapt. Syst., just accepted.","DOI":"10.1145\/3580492"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1002\/rob.20255","article-title":"Autonomous driving in urban environments: Boss and the urban challenge","volume":"25","author":"Urmson","year":"2008","journal-title":"J. Field Robot."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1046","DOI":"10.1109\/TIV.2022.3223131","article-title":"Milestones in autonomous driving and intelligent vehicles: Survey of surveys","volume":"8","author":"Chen","year":"2022","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"16983","DOI":"10.1109\/TITS.2022.3160932","article-title":"State estimation and motion prediction of vehicles and vulnerable road users for cooperative autonomous driving: A survey","volume":"23","author":"Ghorai","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Badrloo, S., Varshosaz, M., Pirasteh, S., and Li, J. (2022). Image-Based Obstacle Detection Methods for the Safe Navigation of Unmanned Vehicles: A Review. Remote Sens., 14.","DOI":"10.3390\/rs14153824"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.isprsjprs.2022.12.021","article-title":"Perception and sensing for autonomous vehicles under adverse weather conditions: A survey","volume":"196","author":"Zhang","year":"2023","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yeong, D.J., Velasco-Hernandez, G., Barry, J., and Walsh, J. (2021). Sensor and sensor fusion technology in autonomous vehicles: A review. Sensors, 21.","DOI":"10.20944\/preprints202102.0459.v1"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Kuhn, C.B., Hofbauer, M., Petrovic, G., and Steinbach, E. (November, January 19). Introspective black box failure prediction for autonomous driving. Proceedings of the 2020 IEEE Intelligent Vehicles Symposium (IV), Las Vegas, NV, USA.","DOI":"10.1109\/IV47402.2020.9304844"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hecker, S., Dai, D., and Van Gool, L. (2018, January 26\u201330). Failure prediction for autonomous driving. Proceedings of the 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China.","DOI":"10.1109\/IVS.2018.8500495"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kuhn, C.B., Hofbauer, M., Petrovic, G., and Steinbach, E. (2021, January 11\u201317). Trajectory-based failure prediction for autonomous driving. Proceedings of the 2021 IEEE Intelligent Vehicles Symposium (IV), Nagoya, Japan.","DOI":"10.1109\/IV48863.2021.9575937"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Besnier, V., Bursuc, A., Picard, D., and Briot, A. (2021, January 11\u201317). Triggering Failures: Out-Of-Distribution detection by learning from local adversarial attacks in Semantic Segmentation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Virtual.","DOI":"10.1109\/ICCV48922.2021.01541"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kuhn, C.B., Hofbauer, M., Petrovic, G., and Steinbach, E. (2020, January 2\u20134). Better look twice-improving visual scene perception using a two-stage approach. Proceedings of the 2020 IEEE International Symposium on Multimedia (ISM), Naples, Italy.","DOI":"10.1109\/ISM.2020.00013"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1007\/s11219-020-09519-w","article-title":"An adaptive system for autonomous driving","volume":"28","author":"Zimmermann","year":"2020","journal-title":"Softw. Qual. J."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3135","DOI":"10.1109\/TITS.2019.2926042","article-title":"Is it safe to drive? An overview of factors, metrics, and datasets for driveability assessment in autonomous driving","volume":"21","author":"Guo","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Rosique, F., Navarro, P.J., Fern\u00e1ndez, C., and Padilla, A. (2019). A systematic review of perception system and simulators for autonomous vehicles research. Sensors, 19.","DOI":"10.3390\/s19030648"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Secci, F., and Ceccarelli, A. (2020, January 12\u201315). On failures of RGB cameras and their effects in autonomous driving applications. Proceedings of the 2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE), Coimbra, Portugal.","DOI":"10.1109\/ISSRE5003.2020.00011"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., and Song, D. (2018, January 18\u201322). Robust physical-world attacks on deep learning visual classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00175"},{"key":"ref_19","first-page":"3914","article-title":"Adversarial examples that fool both computer vision and time-limited humans","volume":"32","author":"Elsayed","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"155161","DOI":"10.1109\/ACCESS.2021.3127960","article-title":"Advances in adversarial attacks and defenses in computer vision: A survey","volume":"9","author":"Akhtar","year":"2021","journal-title":"IEEE Access"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Deng, Y., Zheng, X., Zhang, T., Chen, C., Lou, G., and Kim, M. (2020, January 23\u201327). An analysis of adversarial attacks and defenses on autonomous driving models. Proceedings of the 2020 IEEE International Conference on Pervasive Computing and Communications (PerCom), Austin, TX, USA.","DOI":"10.1109\/PerCom45495.2020.9127389"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Cao, Y., Xiao, C., Cyr, B., Zhou, Y., Park, W., Rampazzi, S., Chen, Q.A., Fu, K., and Mao, Z.M. (2019, January 11\u201315). Adversarial sensor attack on lidar-based perception in autonomous driving. Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, London, UK.","DOI":"10.1145\/3319535.3339815"},{"key":"ref_23","unstructured":"Cao, H., Zou, W., Wang, Y., Song, T., and Liu, M. (2022). Emerging Threats in Deep Learning-Based Autonomous Driving: A Comprehensive Survey. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1951","DOI":"10.1109\/TITS.2019.2909066","article-title":"Automated evaluation of semantic segmentation robustness for autonomous driving","volume":"21","author":"Zhou","year":"2019","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Rahman, Q.M., S\u00fcnderhauf, N., and Dayoub, F. (2019, January 3\u20138). Did you miss the sign? A false negative alarm system for traffic sign detectors. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8968525"},{"key":"ref_26","unstructured":"Meresht, V.B., De, A., Singla, A., and Gomez-Rodriguez, M. (2020). Learning to Switch Between Machines and Humans. arXiv."},{"key":"ref_27","unstructured":"Straitouri, E., Singla, A., Meresht, V.B., and Gomez-Rodriguez, M. (2021). Reinforcement Learning Under Algorithmic Triage. arXiv."},{"key":"ref_28","unstructured":"Jacq, A., Ferret, J., Pietquin, O., and Geist, M. (2022, January 9\u201313). Lazy-MDPs: Towards Interpretable RL by Learning When to Act. Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, Online."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1016\/j.apergo.2015.10.011","article-title":"To delegate or not to delegate: A review of control frameworks for autonomous cars","volume":"53","author":"Richards","year":"2016","journal-title":"Appl. Ergon."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Palmer, S., Richards, D., Shelton-Rayner, G., Izzetoglu, K., and Inch, D. (2020, January 19\u201324). Assessing Variable Levels of Delegated Control\u2014A Novel Measure of Trust. Proceedings of the HCI International 2020\u2014Late Breaking Papers: Cognition, Learning and Games: 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark.","DOI":"10.1007\/978-3-030-60128-7_16"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"107308","DOI":"10.1016\/j.chb.2022.107308","article-title":"Rise of the machines: Delegating decisions to autonomous AI","volume":"134","author":"Candrian","year":"2022","journal-title":"Comput. Hum. Behav."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Fuchs, A., Passarella, A., and Conti, M. (2022, January 20\u201324). A Cognitive Framework for Delegation Between Error-Prone AI and Human Agents. Proceedings of the 2022 IEEE International Conference on Smart Computing (SMARTCOMP), Helsinki, Finland.","DOI":"10.1109\/SMARTCOMP55677.2022.00074"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Razak, S.F.A., Yogarayan, S., Abdullah, M.F.A., and Azman, A. (2022). Modeling User Acceptance of In-Vehicle Applications for Safer Road Environment. Future Internet, 14.","DOI":"10.3390\/fi14050148"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Tr\u00f6sterer, S., Meschtscherjakov, A., Mirnig, A.G., Lupp, A., G\u00e4rtner, M., McGee, F., McCall, R., Tscheligi, M., and Engel, T. (2017, January 24\u201327). What we can learn from pilots for handovers and (de) skilling in semi-autonomous driving: An interview study. Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Oldenburg, Germany.","DOI":"10.1145\/3122986.3123020"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2100122","DOI":"10.1002\/aisy.202100122","article-title":"Intelligent In-Vehicle Interaction Technologies","volume":"4","author":"Murali","year":"2022","journal-title":"Adv. Intell. Syst."},{"key":"ref_36","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_37","first-page":"7705","article-title":"Understanding batch normalization","volume":"32","author":"Bjorck","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_38","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel."},{"key":"ref_39","unstructured":"Riemersma, T. (2023, February 03). Colour Metric. Available online: https:\/\/www.compuphase.com\/cmetric.htm."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Cao, Z., Biyik, E., Wang, W.Z., Raventos, A., Gaidon, A., Rosman, G., and Sadigh, D. (2020, January 12\u201316). Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving. Proceedings of the Robotics: Science and Systems (RSS), Virtual.","DOI":"10.15607\/RSS.2020.XVI.039"},{"key":"ref_41","unstructured":"Golnaraghi, F., and Kuo, B.C. (2017). Automatic Control Systems, McGraw-Hill Education."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Korger, U., Del Galdo, G., Grosch, A., and Haardt, M. (2008, January 26\u201327). Quality of service oriented spatial processing in the Manhattan grid. Proceedings of the 2008 International ITG Workshop on Smart Antennas, Darmstadt, Germany.","DOI":"10.1109\/WSA.2008.4475583"},{"key":"ref_43","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13\u201315). CARLA: An Open Urban Driving Simulator. Proceedings of the 1st Annual Conference on Robot Learning, Mountain View, CA, USA."},{"key":"ref_44","unstructured":"Quiter, C., and Ernst, M. (2023, March 09). Deepdrive\/Deepdrive: 2.0. Available online: https:\/\/zenodo.org\/record\/1248998#.ZBrpsHaZOjg."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3409\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:02:02Z","timestamp":1760122922000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3409"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,24]]},"references-count":44,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23073409"],"URL":"https:\/\/doi.org\/10.3390\/s23073409","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,3,24]]}}}