{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T05:35:03Z","timestamp":1782279303308,"version":"3.54.5"},"publisher-location":"Cham","reference-count":54,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783031264085","type":"print"},{"value":"9783031264092","type":"electronic"}],"license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,17]],"date-time":"2023-03-17T00:00:00Z","timestamp":1679011200000},"content-version":"vor","delay-in-days":75,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Many methods have been developed to understand complex predictive models and high expectations are placed on post-hoc model explainability. It turns out that such explanations are not robust nor trustworthy, and they can be fooled. This paper presents techniques for attacking Partial Dependence (plots, profiles, PDP), which are among the most popular methods of explaining any predictive model trained on tabular data. We showcase that PD can be manipulated in an adversarial manner, which is alarming, especially in financial or medical applications where auditability became a must-have trait supporting black-box machine learning. The fooling is performed via poisoning the data to bend and shift explanations in the desired direction using genetic and gradient algorithms. We believe this to be the first work using a\u00a0genetic algorithm for manipulating explanations, which is transferable as it generalizes both ways: in a\u00a0model-agnostic and an\u00a0explanation-agnostic manner.<\/jats:p>","DOI":"10.1007\/978-3-031-26409-2_8","type":"book-chapter","created":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T09:04:46Z","timestamp":1678957486000},"page":"121-136","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Fooling Partial Dependence via\u00a0Data Poisoning"],"prefix":"10.1007","author":[{"given":"Hubert","family":"Baniecki","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wojciech","family":"Kretowicz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Przemyslaw","family":"Biecek","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,3,17]]},"reference":[{"key":"8_CR1","unstructured":"Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., Kim, B.: Sanity checks for saliency maps. In: NeurIPS (2018)"},{"key":"8_CR2","unstructured":"Adebayo, J., Muelly, M., Liccardi, I., Kim, B.: Debugging tests for model explanations. In: NeurIPS (2020)"},{"key":"8_CR3","unstructured":"Aivodji, U., Arai, H., Fortineau, O., Gambs, S., Hara, S., Tapp, A.: Fairwashing: the risk of rationalization. In: ICML (2019)"},{"issue":"93","key":"8_CR4","first-page":"1","volume":"20","author":"M Alber","year":"2019","unstructured":"Alber, M., Lapuschkin, S., Seegerer, P., H\u00e4gele, M., Sch\u00fctt, K.T., et al.: iNNvestigate neural networks! J. Mach. Learn. Res. 20(93), 1\u20138 (2019)","journal-title":"J. Mach. Learn. Res."},{"issue":"4","key":"8_CR5","doi-asserted-by":"publisher","first-page":"1059","DOI":"10.1111\/rssb.12377","volume":"82","author":"DW Apley","year":"2020","unstructured":"Apley, D.W., Zhu, J.: Visualizing the effects of predictor variables in black box supervised learning models. J. Roy. Stat. Soc. Ser. B (Stat. Methodol.) 82(4), 1059\u20131086 (2020)","journal-title":"J. Roy. Stat. Soc. Ser. B (Stat. Methodol.)"},{"issue":"7","key":"8_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0130140","volume":"10","author":"S Bach","year":"2015","unstructured":"Bach, S., Binder, A., Montavon, G., Klauschen, F., M\u00fcller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), 1\u201346 (2015)","journal-title":"PLoS ONE"},{"issue":"43","key":"8_CR7","doi-asserted-by":"publisher","first-page":"1798","DOI":"10.21105\/joss.01798","volume":"4","author":"H Baniecki","year":"2019","unstructured":"Baniecki, H., Biecek, P.: modelStudio: interactive studio with explanations for ML predictive models. J. Open Source Softw. 4(43), 1798 (2019)","journal-title":"J. Open Source Softw."},{"issue":"214","key":"8_CR8","first-page":"1","volume":"22","author":"H Baniecki","year":"2021","unstructured":"Baniecki, H., Kretowicz, W., Piatyszek, P., Wisniewski, J., Biecek, P.: dalex: responsible machine learning with interactive explainability and fairness in Python. J. Mach. Learn. Res. 22(214), 1\u20137 (2021)","journal-title":"J. Mach. Learn. Res."},{"key":"8_CR9","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","volume":"58","author":"A Barredo Arrieta","year":"2020","unstructured":"Barredo Arrieta, A., D\u00edaz-Rodr\u00edguez, N., Del Ser, J., Bennetot, A., Tabik, S., et al.: Explainable Artificial Intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58, 82\u2013115 (2020)","journal-title":"Inf. Fusion"},{"key":"8_CR10","doi-asserted-by":"crossref","unstructured":"Bhatt, U., Weller, A., Moura, J.M.F.: Evaluating and aggregating feature-based model explanations. In: IJCAI (2020)","DOI":"10.24963\/ijcai.2020\/417"},{"issue":"84","key":"8_CR11","first-page":"1","volume":"19","author":"P Biecek","year":"2018","unstructured":"Biecek, P.: DALEX: explainers for complex predictive models in R. J. Mach. Learn. Res. 19(84), 1\u20135 (2018)","journal-title":"J. Mach. Learn. Res."},{"key":"8_CR12","doi-asserted-by":"crossref","unstructured":"Biecek, P., Burzykowski, T.: Explanatory Model Analysis. Chapman and Hall\/CRC (2021)","DOI":"10.1201\/9780429027192"},{"key":"8_CR13","unstructured":"Boopathy, A., et al.: Proper network interpretability helps adversarial robustness in classification. In: ICML (2020)"},{"key":"8_CR14","unstructured":"D\u2019Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., et al.: Underspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395 (2020)"},{"key":"8_CR15","unstructured":"Dimanov, B., Bhatt, U., Jamnik, M., Weller, A.: You shouldn\u2019t trust me: learning models which conceal unfairness from multiple explanation methods. In: AAAI SafeAI (2020)"},{"key":"8_CR16","unstructured":"Dombrowski, A.K., Alber, M., Anders, C., Ackermann, M., M\u00fcller, K.R., Kessel, P.: Explanations can be manipulated and geometry is to blame. In: NeurIPS (2019)"},{"key":"8_CR17","unstructured":"Dua, D., Graff, C.: UCI Machine Learning Repository (2017). https:\/\/www.kaggle.com\/ronitf\/heart-disease-uci\/version\/1"},{"issue":"1","key":"8_CR18","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.aei.2005.01.004","volume":"19","author":"E Elbeltagi","year":"2005","unstructured":"Elbeltagi, E., Hegazy, T., Grierson, D.: Comparison among five evolutionary-based optimization algorithms. Adv. Eng. Inform. 19(1), 43\u201353 (2005)","journal-title":"Adv. Eng. Inform."},{"issue":"5","key":"8_CR19","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1214\/aos\/1013203451","volume":"29","author":"JH Friedman","year":"2001","unstructured":"Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29(5), 1189\u20131232 (2001)","journal-title":"Ann. Stat."},{"key":"8_CR20","doi-asserted-by":"crossref","unstructured":"Fukuchi, K., Hara, S., Maehara, T.: Faking fairness via stealthily biased sampling. In: AAAI (2020)","DOI":"10.1609\/aaai.v34i01.5377"},{"key":"8_CR21","doi-asserted-by":"crossref","unstructured":"Ghorbani, A., Abid, A., Zou, J.: Interpretation of neural networks is fragile. In: AAAI (2019)","DOI":"10.1609\/aaai.v33i01.33013681"},{"issue":"3","key":"8_CR22","doi-asserted-by":"publisher","first-page":"137","DOI":"10.3390\/info11030137","volume":"11","author":"N Gill","year":"2020","unstructured":"Gill, N., Hall, P., Montgomery, K., Schmidt, N.: A responsible machine learning workflow with focus on interpretable models, post-hoc explanation, and discrimination testing. Information 11(3), 137 (2020)","journal-title":"Information"},{"issue":"1","key":"8_CR23","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1080\/10618600.2014.907095","volume":"24","author":"A Goldstein","year":"2015","unstructured":"Goldstein, A., Kapelner, A., Bleich, J., Pitkin, E.: Peeking inside the black box: visualizing statistical learning with plots of individual conditional expectation. J. Comput. Graph. Stat. 24(1), 44\u201365 (2015)","journal-title":"J. Comput. Graph. Stat."},{"issue":"1","key":"8_CR24","doi-asserted-by":"publisher","first-page":"421","DOI":"10.32614\/RJ-2017-016","volume":"9","author":"BM Greenwell","year":"2017","unstructured":"Greenwell, B.M.: pdp: an R package for constructing partial dependence plots. R J. 9(1), 421\u2013436 (2017)","journal-title":"R J."},{"key":"8_CR25","unstructured":"Heo, J., Joo, S., Moon, T.: Fooling neural network interpretations via adversarial model manipulation. In: NeurIPS (2019)"},{"issue":"3","key":"8_CR26","doi-asserted-by":"publisher","first-page":"709","DOI":"10.1198\/106186007X237892","volume":"16","author":"G Hooker","year":"2007","unstructured":"Hooker, G.: Generalized functional ANOVA diagnostics for high-dimensional functions of dependent variables. J. Comput. Graph. Stat. 16(3), 709\u2013732 (2007)","journal-title":"J. Comput. Graph. Stat."},{"key":"8_CR27","unstructured":"Hooker, S., Erhan, D., Kindermans, P.J., Kim, B.: A benchmark for interpretability methods in deep neural networks. In: NeurIPS (2019)"},{"key":"8_CR28","unstructured":"Janzing, D., Minorics, L., Bl\u00f6baum, P.: Feature relevance quantification in explainable AI: a causal problem. In: AISTATS (2020)"},{"key":"8_CR29","series-title":"Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence)","doi-asserted-by":"publisher","first-page":"699","DOI":"10.1007\/978-3-030-86520-7_43","volume-title":"Machine Learning and Knowledge Discovery in Databases. Research Track","author":"Y Jia","year":"2021","unstructured":"Jia, Y., Frank, E., Pfahringer, B., Bifet, A., Lim, N.: Studying and exploiting the relationship between model accuracy and explanation\u00a0quality. In: Oliver, N., P\u00e9rez-Cruz, F., Kramer, S., Read, J., Lozano, J.A. (eds.) ECML PKDD 2021. LNCS (LNAI), vol. 12976, pp. 699\u2013714. Springer, Cham (2021). https:\/\/doi.org\/10.1007\/978-3-030-86520-7_43"},{"key":"8_CR30","series-title":"Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence)","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1007\/978-3-030-28954-6_14","volume-title":"Explainable AI: Interpreting, Explaining and Visualizing Deep Learning","author":"P-J Kindermans","year":"2019","unstructured":"Kindermans, P.-J., et al.: The (un)reliability of saliency methods. In: Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., M\u00fcller, K.-R. (eds.) Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. LNCS (LNAI), vol. 11700, pp. 267\u2013280. Springer, Cham (2019). https:\/\/doi.org\/10.1007\/978-3-030-28954-6_14"},{"key":"8_CR31","unstructured":"Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)"},{"key":"8_CR32","doi-asserted-by":"crossref","unstructured":"Lakkaraju, H., Bastani, O.: \u201cHow do i fool you?\u201d: Manipulating user trust via misleading black box explanations. In: AIES (2020)","DOI":"10.1145\/3375627.3375833"},{"key":"8_CR33","doi-asserted-by":"crossref","unstructured":"Lakkaraju, H., Kamar, E., Caruana, R., Leskovec, J.: Faithful and customizable explanations of black box models. In: AIES (2019)","DOI":"10.1145\/3306618.3314229"},{"issue":"7553","key":"8_CR34","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436\u2013444 (2015)","journal-title":"Nature"},{"issue":"3","key":"8_CR35","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1145\/3236386.3241340","volume":"16","author":"ZC Lipton","year":"2018","unstructured":"Lipton, Z.C.: The mythos of model interpretability. Queue 16(3), 31\u201357 (2018)","journal-title":"Queue"},{"key":"8_CR36","unstructured":"Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: NeurIPS (2017)"},{"key":"8_CR37","series-title":"Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence)","doi-asserted-by":"publisher","first-page":"272","DOI":"10.1007\/978-3-030-67661-2_17","volume-title":"Machine Learning and Knowledge Discovery in Databases","author":"P Mangla","year":"2021","unstructured":"Mangla, P., Singh, V., Balasubramanian, V.N.: On saliency maps and adversarial robustness. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds.) ECML PKDD 2020. LNCS (LNAI), vol. 12458, pp. 272\u2013288. Springer, Cham (2021). https:\/\/doi.org\/10.1007\/978-3-030-67661-2_17"},{"key":"8_CR38","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.artint.2018.07.007","volume":"267","author":"T Miller","year":"2019","unstructured":"Miller, T.: Explanation in artificial intelligence: insights from the social sciences. Artif. Intell. 267, 1\u201338 (2019)","journal-title":"Artif. Intell."},{"issue":"26","key":"8_CR39","doi-asserted-by":"publisher","first-page":"786","DOI":"10.21105\/joss.00786","volume":"3","author":"C Molnar","year":"2018","unstructured":"Molnar, C., Casalicchio, G., Bischl, B.: iml: an R package for interpretable machine learning. J. Open Source Softw. 3(26), 786 (2018)","journal-title":"J. Open Source Softw."},{"key":"8_CR40","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., Guestrin, C.: \u201cWhy should i trust you?\u201d: Explaining the predictions of any classifier. In: KDD (2016)","DOI":"10.1145\/2939672.2939778"},{"key":"8_CR41","unstructured":"Rieger, L., Hansen, L.K.: A simple defense against adversarial attacks on heatmap explanations. In: ICML WHI (2020)"},{"key":"8_CR42","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","volume":"1","author":"C Rudin","year":"2019","unstructured":"Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206\u2013215 (2019)","journal-title":"Nat. Mach. Intell."},{"issue":"2","key":"8_CR43","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","volume":"128","author":"RR Selvaraju","year":"2020","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis. 128(2), 336\u2013359 (2020)","journal-title":"Int. J. Comput. Vis."},{"key":"8_CR44","unstructured":"Shrikumar, A., Greenside, P., Kundaje, A.: Learning important features through propagating activation differences. In: ICML (2017)"},{"key":"8_CR45","unstructured":"Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. In: ICLR (2014)"},{"key":"8_CR46","doi-asserted-by":"crossref","unstructured":"Slack, D., Hilgard, S., Jia, E., Singh, S., Lakkaraju, H.: Fooling LIME and SHAP: adversarial attacks on post hoc explanation methods. In: AIES (2020)","DOI":"10.1145\/3375627.3375830"},{"key":"8_CR47","unstructured":"Slack, D., Hilgard, S., Lakkaraju, H., Singh, S.: Counterfactual explanations can be manipulated. In: NeurIPS (2021)"},{"key":"8_CR48","series-title":"Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence)","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1007\/978-3-030-67658-2_10","volume-title":"Machine Learning and Knowledge Discovery in Databases","author":"D Solans","year":"2021","unstructured":"Solans, D., Biggio, B., Castillo, C.: Poisoning attacks on algorithmic fairness. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds.) ECML PKDD 2020. LNCS (LNAI), vol. 12457, pp. 162\u2013177. Springer, Cham (2021). https:\/\/doi.org\/10.1007\/978-3-030-67658-2_10"},{"key":"8_CR49","unstructured":"Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: ICML (2017)"},{"key":"8_CR50","unstructured":"Wang, Z., Wang, H., Ramkumar, S., Mardziel, P., Fredrikson, M., Datta, A.: Smoothed geometry for robust attribution. In: NeurIPS (2020)"},{"key":"8_CR51","doi-asserted-by":"crossref","unstructured":"Warnecke, A., Arp, D., Wressnegger, C., Rieck, K.: Evaluating explanation methods for deep learning in security. In: IEEE EuroS &P (2020)","DOI":"10.1109\/EuroSP48549.2020.00018"},{"key":"8_CR52","first-page":"205","volume":"1","author":"AH Wright","year":"1991","unstructured":"Wright, A.H.: Genetic algorithms for real parameter optimization. Found. Genet. Algorithms 1, 205\u2013218 (1991)","journal-title":"Found. Genet. Algorithms"},{"key":"8_CR53","unstructured":"Zhang, X., Wang, N., Shen, H., Ji, S., Luo, X., Wang, T.: Interpretable deep learning under fire. In: USENIX Security (2020)"},{"issue":"1","key":"8_CR54","doi-asserted-by":"publisher","first-page":"272","DOI":"10.1080\/07350015.2019.1624293","volume":"39","author":"Q Zhao","year":"2019","unstructured":"Zhao, Q., Hastie, T.: Causal interpretations of black-box models. J. Bus. Econ. Stat. 39(1), 272\u2013281 (2019)","journal-title":"J. Bus. Econ. Stat."}],"container-title":["Lecture Notes in Computer Science","Machine Learning and Knowledge Discovery in Databases"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-26409-2_8","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,16]],"date-time":"2023-03-16T09:33:06Z","timestamp":1678959186000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-26409-2_8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"ISBN":["9783031264085","9783031264092"],"references-count":54,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-26409-2_8","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"value":"0302-9743","type":"print"},{"value":"1611-3349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023]]},"assertion":[{"value":"17 March 2023","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"ECML PKDD","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Grenoble","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"France","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2022","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"19 September 2022","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"23 September 2022","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"22","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"ecml2022","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/2022.ecmlpkdd.org\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Double-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"CMT","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"1060","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"236","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"22% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3-4","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3-4","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"No","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"17 demo track papers have been accepted from 28 submissions","order":10,"name":"additional_info_on_review_process","label":"Additional Info on Review Process","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}