{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:23:15Z","timestamp":1750220595935,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":45,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,11,2]],"date-time":"2020-11-02T00:00:00Z","timestamp":1604275200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["830927"],"award-info":[{"award-number":["830927"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,11,2]]},"DOI":"10.1145\/3415958.3433101","type":"proceedings-article","created":{"date-parts":[[2020,11,27]],"date-time":"2020-11-27T23:10:58Z","timestamp":1606518658000},"page":"38-45","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["A Methodology for Non-Functional Property Evaluation of Machine Learning Models"],"prefix":"10.1145","author":[{"given":"Marco","family":"Anisetti","sequence":"first","affiliation":[{"name":"Universit\u00e0 degli Studi di Milano, Milano, Italy"}]},{"given":"Claudio A.","family":"Ardagna","sequence":"additional","affiliation":[{"name":"Universit\u00e0 degli Studi di Milano, Milano, Italy"}]},{"given":"Ernesto","family":"Damiani","sequence":"additional","affiliation":[{"name":"C2PS -- Khalifa University Abu Dhabi, UAE"}]},{"given":"Paolo G.","family":"Panero","sequence":"additional","affiliation":[{"name":"SORIS S.p.A, Torino, Italy"}]}],"member":"320","published-online":{"date-parts":[[2020,11,27]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33012412"},{"key":"e_1_3_2_1_2_1","volume-title":"Automated Test Generation to Detect Individual Discrimination in AI Models","author":"Agarwal A.","year":"2018","unstructured":"A. Agarwal , P. Lohia , S. Nagar , K. Dey , and D. Saha . Automated Test Generation to Detect Individual Discrimination in AI Models . 2018 . A. Agarwal, P. Lohia, S. Nagar, K. Dey, and D. Saha. Automated Test Generation to Detect Individual Discrimination in AI Models. 2018."},{"issue":"1","key":"e_1_3_2_1_3_1","first-page":"1","article-title":"Visual Analytics for Fairness","volume":"26","author":"Ahn Y.","year":"2019","unstructured":"Y. Ahn and Y.-R. Lin . FairSight : Visual Analytics for Fairness in Decision Making. IEEE Transactions on Visualization and Computer Graphics , 26 ( 1 ): 1 - 1 , 2019 . Y. Ahn and Y.-R. Lin. FairSight: Visual Analytics for Fairness in Decision Making. IEEE Transactions on Visualization and Computer Graphics, 26(1):1-1, 2019.","journal-title":"Decision Making. IEEE Transactions on Visualization and Computer Graphics"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSC.2017.2657505"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2460383.2460384"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICWS.2012.53"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2767005"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/795662.796294"},{"issue":"4","key":"e_1_3_2_1_9_1","article-title":"AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias","volume":"63","author":"Bellamy R. K.","year":"2019","unstructured":"R. K. Bellamy , A. Mojsilovic , S. Nagar , K. N. Ramamurthy , J. Richards , D. Saha , P. Sattigeri , M. Singh , K. R. Varshney , Y. Zhang , K. Dey , M. Hind , S. C. Hoffman , S. Houde , K. Kannan , P. Lohia , J. Martino , and S. Mehta . AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias . IBM Journal of Research and Development , 63 ( 4-5 ), 2019 . R. K. Bellamy, A. Mojsilovic, S. Nagar, K. N. Ramamurthy, J. Richards, D. Saha, P. Sattigeri, M. Singh, K. R. Varshney, Y. Zhang, K. Dey, M. Hind, S. C. Hoffman, S. Houde, K. Kannan, P. Lohia, J. Martino, and S. Mehta. AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development, 63(4-5), 2019.","journal-title":"IBM Journal of Research and Development"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2018.07.023"},{"key":"e_1_3_2_1_11_1","volume-title":"FlipTest: Fairness Auditing via Optimal Transport","author":"Black E.","year":"2019","unstructured":"E. Black , S. Yeom , and M. Fredrikson . FlipTest: Fairness Auditing via Optimal Transport . 2019 . E. Black, S. Yeom, and M. Fredrikson. FlipTest: Fairness Auditing via Optimal Transport. 2019."},{"key":"e_1_3_2_1_12_1","volume-title":"An Overview of Advances in Reliability estimation of Individual Predictions in Machine Learning. Technical report","author":"Bosnic Z.","year":"2008","unstructured":"Z. Bosnic and I. Kononenko . An Overview of Advances in Reliability estimation of Individual Predictions in Machine Learning. Technical report , University of Ljubljana , Ljubljana , 2008 . Z. Bosnic and I. Kononenko. An Overview of Advances in Reliability estimation of Individual Predictions in Machine Learning. Technical report, University of Ljubljana, Ljubljana, 2008."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2017.154"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/2986459.2986710"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098095"},{"key":"e_1_3_2_1_16_1","volume-title":"Ai governance: A research agenda. Governance of AI Program","author":"Dafoe A.","year":"2018","unstructured":"A. Dafoe . Ai governance: A research agenda. Governance of AI Program , Future of Humanity Institute , 2018 . A. Dafoe. Ai governance: A research agenda. Governance of AI Program, Future of Humanity Institute, 2018."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-38919-2_1"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2090236.2090255"},{"key":"e_1_3_2_1_19_1","volume-title":"On Formalizing Fairness in Prediction with Machine Learning","author":"Gajane P.","year":"2017","unstructured":"P. Gajane and M. Pechenizkiy . On Formalizing Fairness in Prediction with Machine Learning . 2017 . P. Gajane and M. Pechenizkiy. On Formalizing Fairness in Prediction with Machine Learning. 2017."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106277"},{"key":"e_1_3_2_1_21_1","volume-title":"Machine Learning Explainability vs Interpretability: Two concepts that could help restore trust in AI","author":"Gall R.","year":"2018","unstructured":"R. Gall . Machine Learning Explainability vs Interpretability: Two concepts that could help restore trust in AI , 2018 . R. Gall. Machine Learning Explainability vs Interpretability: Two concepts that could help restore trust in AI, 2018."},{"key":"e_1_3_2_1_22_1","first-page":"361","volume-title":"On Non-Functional Requirements. 15th IEEE International Requirements Engineering Conference","author":"Glinz M.","year":"2007","unstructured":"M. Glinz . On Non-Functional Requirements. 15th IEEE International Requirements Engineering Conference , page 361 , 2007 . M. Glinz. On Non-Functional Requirements. 15th IEEE International Requirements Engineering Conference, page 361, 2007."},{"key":"e_1_3_2_1_24_1","volume-title":"Proc. of the 8th International Conference on Formal Ontology in Information Systems (FOIS 2014","author":"Guizzardi R.","year":"2014","unstructured":"R. Guizzardi , F.-L. Li , A. Borgida , and J. Mylopoulos . An ontological interpretation of non-functional requirements . In Proc. of the 8th International Conference on Formal Ontology in Information Systems (FOIS 2014 ), Rio de Janeiro, Brasil , September 2014 . R. Guizzardi, F.-L. Li, A. Borgida, and J. Mylopoulos. An ontological interpretation of non-functional requirements. In Proc. of the 8th International Conference on Formal Ontology in Information Systems (FOIS 2014), Rio de Janeiro, Brasil, September 2014."},{"key":"e_1_3_2_1_25_1","volume-title":"International Journal of Instruction","author":"Hardt M.","year":"2016","unstructured":"M. Hardt , E. Price , and N. Srebro . Equality of Opportunity in Supervised Learning . International Journal of Instruction , 2016 . M. Hardt, E. Price, and N. Srebro. Equality of Opportunity in Supervised Learning. International Journal of Instruction, 2016."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_47"},{"key":"e_1_3_2_1_27_1","first-page":"410","article-title":"2010 - Systems and software engineering - Vocabulary","volume":"2010","author":"IEC","year":"2010","unstructured":"ISO\/ IEC and IEEE. ISO\/IEC\/IEEE 24765 : 2010 - Systems and software engineering - Vocabulary . Iso\/Iec Ieee , 2010 : 410 , 2010 . ISO\/IEC and IEEE. ISO\/IEC\/IEEE 24765:2010 - Systems and software engineering - Vocabulary. Iso\/Iec Ieee, 2010:410, 2010.","journal-title":"Iso\/Iec Ieee"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0959-4388(00)00067-2"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/3367243.3367417"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295230"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287564"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1774088.1774153"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3317964"},{"key":"e_1_3_2_1_34_1","volume-title":"Los Alamos Science, (Special Issue dedicated to Stanislaw Ulam):125--130","author":"Metropolis N.","year":"1987","unstructured":"N. Metropolis . The beginning of the Monte Carlo method. Los Alamos Science, (Special Issue dedicated to Stanislaw Ulam):125--130 , 1987 . N. Metropolis. The beginning of the Monte Carlo method. Los Alamos Science, (Special Issue dedicated to Stanislaw Ulam):125--130, 1987."},{"key":"e_1_3_2_1_35_1","unstructured":"C. E. Perez. Modularity 2018.  C. E. Perez. Modularity 2018."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3278721.3278742"},{"key":"e_1_3_2_1_37_1","volume-title":"Proc. of Workshop on ML Systems","author":"Schelter S.","year":"2017","unstructured":"S. Schelter , J.-H. Bose , J. Kirschnick , T. Klein , and S. Seufert . Automatically tracking metadata and provenance of machine learning experiments . In Proc. of Workshop on ML Systems , Long Beach, CA, USA , December 2017 . S. Schelter, J.-H. Bose, J. Kirschnick, T. Klein, and S. Seufert. Automatically tracking metadata and provenance of machine learning experiments. In Proc. of Workshop on ML Systems, Long Beach, CA, USA, December 2017."},{"key":"e_1_3_2_1_38_1","volume-title":"Why your Models need Maintenance","author":"Schmitz M.","year":"2017","unstructured":"M. Schmitz . Why your Models need Maintenance , 2017 . M. Schmitz. Why your Models need Maintenance, 2017."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/asmb.2104"},{"key":"e_1_3_2_1_40_1","volume-title":"Environmental & Innovation Law, 10(1)","author":"Spiro M.","year":"2020","unstructured":"M. Spiro . The FTC and AI Governance : A Regulatory Proposal The FTC and AI Governance: A Regulatory Proposal. Seattle Journal of Technology , Environmental & Innovation Law, 10(1) , 2020 . M. Spiro. The FTC and AI Governance: A Regulatory Proposal The FTC and AI Governance: A Regulatory Proposal. Seattle Journal of Technology, Environmental & Innovation Law, 10(1), 2020."},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.5555\/551283"},{"key":"e_1_3_2_1_42_1","volume-title":"European Network of Legal Experts in Anti-Discrimination","author":"Tobler C.","year":"2008","unstructured":"C. Tobler . Limits and potential of the concept of indirect discrimination. Technical report , European Network of Legal Experts in Anti-Discrimination , 2008 . C. Tobler. Limits and potential of the concept of indirect discrimination. Technical report, European Network of Legal Experts in Anti-Discrimination, 2008."},{"key":"e_1_3_2_1_43_1","volume-title":"A Survey","author":"Vanschoren J.","year":"2018","unstructured":"J. Vanschoren . Meta-Learning : A Survey , 2018 . J. Vanschoren. Meta-Learning: A Survey, 2018."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3194770.3194776"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-59569-6_55"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/CESSER-IP.2019.00009"}],"event":{"name":"MEDES '20: 12th International Conference on Management of Digital EcoSystems","sponsor":["SIGAPP ACM Special Interest Group on Applied Computing"],"location":"Virtual Event United Arab Emirates","acronym":"MEDES '20"},"container-title":["Proceedings of the 12th International Conference on Management of Digital EcoSystems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3415958.3433101","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3415958.3433101","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:53Z","timestamp":1750195913000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3415958.3433101"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,2]]},"references-count":45,"alternative-id":["10.1145\/3415958.3433101","10.1145\/3415958"],"URL":"https:\/\/doi.org\/10.1145\/3415958.3433101","relation":{},"subject":[],"published":{"date-parts":[[2020,11,2]]},"assertion":[{"value":"2020-11-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}