{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T17:40:48Z","timestamp":1780422048207,"version":"3.54.1"},"reference-count":48,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2023,8,3]],"date-time":"2023-08-03T00:00:00Z","timestamp":1691020800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Deep learning has become increasingly common in aerial imagery analysis. As its use continues to grow, it is crucial that we understand and can explain its behavior. One eXplainable AI (XAI) approach is to generate linguistic summarizations of data and\/or models. However, the number of summaries can increase significantly with the number of data attributes, posing a challenge. Herein, we proposed a hierarchical approach for generating and evaluating linguistic statements of black box deep learning models. Our approach scores and ranks statements according to user-specified criteria. A systematic process was outlined for the evaluation of an object detector on a low altitude aerial drone. A deep learning model trained on real imagery was evaluated on a photorealistic simulated dataset with known ground truth across different contexts. The effectiveness and versatility of our approach was demonstrated by showing tailored linguistic summaries for different user types. Ultimately, this process is an efficient human-centric way of identifying successes, shortcomings, and biases in data and deep learning models.<\/jats:p>","DOI":"10.3390\/s23156879","type":"journal-article","created":{"date-parts":[[2023,8,3]],"date-time":"2023-08-03T11:23:03Z","timestamp":1691061783000},"page":"6879","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Linguistic Explanations of Black Box Deep Learning Detectors on Simulated Aerial Drone Imagery"],"prefix":"10.3390","volume":"23","author":[{"given":"Brendan","family":"Alvey","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO 65211, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Derek","family":"Anderson","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO 65211, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0306-7142","authenticated-orcid":false,"given":"James","family":"Keller","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO 65211, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8892-3269","authenticated-orcid":false,"given":"Andrew","family":"Buck","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO 65211, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,3]]},"reference":[{"key":"ref_1","unstructured":"Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_2","unstructured":"Kaplan, J., McCandlish, S., Henighan, T., Brown, T.B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. (2020). Scaling Laws for Neural Language Models. arXiv."},{"key":"ref_3","unstructured":"Larochelle, H., Erhan, D., and Bengio, Y. (2008, January 13\u201317). Zero-Data Learning of New Tasks. Proceedings of the 23rd National Conference on Artificial Intelligence, Chicago, IL, USA."},{"key":"ref_4","unstructured":"Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., and Ray, A. (2022). Training language models to follow instructions with human feedback. arXiv."},{"key":"ref_5","unstructured":"Zhu, W., Liu, H., Dong, Q., Xu, J., Huang, S., Kong, L., Chen, J., and Li, L. (2023). Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis. arXiv."},{"key":"ref_6","unstructured":"Franceschelli, G., and Musolesi, M. (2023). On the Creativity of Large Language Models. arXiv."},{"key":"ref_7","unstructured":"Li, R., Allal, L.B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., and Chim, J. (2023). StarCoder: May the source be with you!. arXiv."},{"key":"ref_8","unstructured":"Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., and Hashimoto, T.B. (2023). Benchmarking Large Language Models for News Summarization. arXiv."},{"key":"ref_9","unstructured":"Chang, Y., Wang, X., Wang, J., Wu, Y., Zhu, K., Chen, H., Yang, L., Yi, X., Wang, C., and Wang, Y. (2023). A Survey on Evaluation of Large Language Models. arXiv."},{"key":"ref_10","unstructured":"Liu, Y., Fabbri, A.R., Liu, P., Radev, D., and Cohan, A. (2023). On Learning to Summarize with Large Language Models as References. arXiv."},{"key":"ref_11","unstructured":"Bills, S., Cammarata, N., Mossing, D., Tillman, H., Gao, L., Goh, G., Sutskever, I., Leike, J., Wu, J., and Saunders, W. (2023, May 03). Language Models Can Explain Neurons in Language Models. Available online: https:\/\/openaipublic.blob.core.windows.net\/neuron-explainer\/paper\/index.html."},{"key":"ref_12","unstructured":"M\u00fcndler, N., He, J., Jenko, S., and Vechev, M. (2023). Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation. arXiv."},{"key":"ref_13","first-page":"8821","article-title":"Zero-Shot Text-to-Image Generation","volume":"Volume 139","author":"Meila","year":"2021","journal-title":"Proceedings of the 38th International Conference on Machine Learning"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 19\u201320). High-Resolution Image Synthesis with Latent Diffusion Models. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"ref_15","unstructured":"Epic Games (2019). Unreal Engine, Epic Games. Available online: https:\/\/www.unrealengine.com."},{"key":"ref_16","first-page":"1211611","article-title":"Procedurally generated simulated datasets for aerial explosive hazard detection","volume":"Volume 12116","author":"Guicheteau","year":"2022","journal-title":"Chemical, Biological, Radiological, Nuclear, and Explosives (CBRNE) Sensing XXIII"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wilbik, A., Keller, J.M., and Alexander, G.L. (2011, January 9\u201312). Linguistic summarization of sensor data for eldercare. Proceedings of the 2011 IEEE International Conference on Systems, Man, and Cybernetics, Anchorage, AK, USA.","DOI":"10.1109\/ICSMC.2011.6084067"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.cviu.2008.07.006","article-title":"Linguistic summarization of video for fall detection using voxel person and fuzzy logic","volume":"113","author":"Anderson","year":"2009","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1016\/j.ins.2018.11.036","article-title":"Application of linguistic summarization methods in time series forecasting","volume":"478","author":"Hryniewicz","year":"2019","journal-title":"Inf. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"103240","DOI":"10.1016\/j.jbi.2019.103240","article-title":"Linguistic summarization of in-home sensor data","volume":"96","author":"Jain","year":"2019","journal-title":"J. Biomed. Inform."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Carvalho, J.P., Lesot, M.J., Kaymak, U., Vieira, S., Bouchon-Meunier, B., and Yager, R.R. (2016). Information Processing and Management of Uncertainty in Knowledge-Based Systems, Springer.","DOI":"10.1007\/978-3-319-40581-0"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jain, A., and Keller, J.M. (2015, January 2\u20135). On the computation of semantically ordered truth values of linguistic protoform summaries. Proceedings of the 2015 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Istanbul, Turkey.","DOI":"10.1109\/FUZZ-IEEE.2015.7337822"},{"key":"ref_23","first-page":"7634","article-title":"Textual summarization of events leading to health alerts","volume":"2015","author":"Jain","year":"2015","journal-title":"Annu. Int. Conf. IEEE Eng. Med. Biol. Soc."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Jain, A., Keller, J.M., and Bezdek, J.C. (2016, January 24\u201327). Quantitative and qualitative comparison of periodic sensor data. Proceedings of the 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Las Vegas, NV, USA.","DOI":"10.1109\/BHI.2016.7455829"},{"key":"ref_25","unstructured":"Juliani, A., Berges, V.P., Vckay, E., Gao, Y., Henry, H., Mattar, M., and Lange, D. (2018). Unity: A General Platform for Intelligent Agents. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Roberts, M., Ramapuram, J., Ranjan, A., Kumar, A., Bautista, M.A., Paczan, N., Webb, R., and Susskind, J.M. (2021). Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding, ICCV.","DOI":"10.1109\/ICCV48922.2021.01073"},{"key":"ref_27","unstructured":"Liang, J., Makoviychuk, V., Handa, A., Chentanez, N., Macklin, M., and Fox, D. (2018). GPU-Accelerated Robotic Simulation for Distributed Reinforcement Learning, PMLR."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Truong, J., Rudolph, M., Yokoyama, N.H., Chernova, S., Batra, D., and Rai, A. (2022, January 14\u201318). Rethinking Sim2Real: Lower Fidelity Simulation Leads to Higher Sim2Real Transfer in Navigation. Proceedings of the 6th Annual Conference on Robot Learning, Auckland, New Zealand.","DOI":"10.1109\/LRA.2021.3062303"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N., and Fox, D. (2019, January 20\u201324). Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793789"},{"key":"ref_30","unstructured":"Chen, X., Hu, J., Jin, C., Li, L., and Wang, L. (2022). Understanding Domain Randomization for Sim-to-real Transfer. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Shrivastava, A., Pfister, T., Tuzel, O., Susskind, J., Wang, W., and Webb, R. (2017, January 21\u201326). Learning from Simulated and Unsupervised Images through Adversarial Training. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.241"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Akers, J., Buck, A., Camaioni, R., Anderson, D., Luke, R., Keller, J., Deardorff, M., and Alvey, B. (2023). Simulated Gold Standard for Quantitative Evaluation of Monocular Vision Algorithms, SPIE.","DOI":"10.1117\/12.2657567"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Georgakis, G., Mousavian, A., Berg, A.C., and Kosecka, J. (2017). Synthesizing Training Data for Object Detection in Indoor Scenes. arXiv.","DOI":"10.15607\/RSS.2017.XIII.043"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Alvey, B.J., Anderson, D.T., Yang, C., Buck, A., Keller, J.M., Yasuda, K.E., and Ryan, H.A. (2021, January 5\u20137). Characterization of Deep Learning-Based Aerial Explosive Hazard Detection using Simulated Data. Proceedings of the 2021 IEEE Symposium Series on Computational Intelligence (SSCI), Orlando, FL, USA.","DOI":"10.1109\/SSCI50451.2021.9659899"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Bouchon-Meunier, B., Yager, R.R., and Zadeh, L.A. (1991). Uncertainty in Knowledge Bases, Springer.","DOI":"10.1007\/BFb0028090"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1016\/j.ins.2005.03.002","article-title":"Linguistic database summaries and their protoforms: Towards natural language based knowledge discovery tools","volume":"173","author":"Kacprzyk","year":"2005","journal-title":"Inf. Sci."},{"key":"ref_37","unstructured":"Xu, Z. (2007). Linguistic Aggregation Operators: An Overview, Springer."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Popek, G., and Katarzyniak, R.P. (2013, January 23\u201325). Interval-based aggregation of fuzzy-linguistic statements. Proceedings of the 2013 10th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), Shenyang, China.","DOI":"10.1109\/FSKD.2013.6816250"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1007\/s11251-009-9110-0","article-title":"Cognitive load theory, educational research, and instructional design: Some food for thought","volume":"38","year":"2010","journal-title":"Instr. Sci."},{"key":"ref_40","unstructured":"Kacprzyk, J., and Zadro\u017cny, S. (2009). Scalable Fuzzy Algorithms for Data Management and Analysis: Methods and Design, IGI Global."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Alvey, B., Anderson, D., and Keller, J. (2023, January 5\u20136). Explainable AI via Linguistic Summarization of Black Box Computer Vision Models. Proceedings of the IEEE Conference on Artifical Intelligence (CAI), Santa Clara, CA, USA.","DOI":"10.1109\/CAI54212.2023.00156"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1018217","DOI":"10.1117\/12.2263005","article-title":"Aggregation of Choquet integrals in GPR and EMI for handheld platform-based explosive hazard detection","volume":"Volume 10182","author":"Bishop","year":"2017","journal-title":"Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XXII"},{"key":"ref_43","first-page":"101820E","article-title":"Fourier features for explosive hazard detection using a wideband electromagnetic induction sensor","volume":"Volume 10182","author":"Bishop","year":"2017","journal-title":"Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XXII"},{"key":"ref_44","first-page":"106280T","article-title":"Generative adversarial networks for ground penetrating radar in hand held explosive hazard detection","volume":"Volume 10628","author":"Bishop","year":"2018","journal-title":"Detection and Sensing of Mines, Explosive Objects, and Obscured Targets XXIII"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Bastian, M., Heymann, S., and Jacomy, M. (2009, January 17\u201320). Gephi: An Open Source Software for Exploring and Manipulating Networks. Proceedings of the International AAAI Conference on Weblogs and Social Media, San Jose, CA, USA.","DOI":"10.1609\/icwsm.v3i1.13937"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Shahriar, S., and Hayawi, K. (2023). Let\u2019s have a chat! A Conversation with ChatGPT: Technology, Applications, and Limitations. arXiv.","DOI":"10.47852\/bonviewAIA3202939"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Lu, Y., Bartolo, M., Moore, A., Riedel, S., and Stenetorp, P. (2022). Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity. arXiv.","DOI":"10.18653\/v1\/2022.acl-long.556"},{"key":"ref_48","unstructured":"White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., and Schmidt, D.C. (2023). A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/15\/6879\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:24:57Z","timestamp":1760127897000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/15\/6879"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,3]]},"references-count":48,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["s23156879"],"URL":"https:\/\/doi.org\/10.3390\/s23156879","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,3]]}}}