{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T20:00:38Z","timestamp":1769630438428,"version":"3.49.0"},"reference-count":37,"publisher":"American Institute of Aeronautics and Astronautics (AIAA)","issue":"3","funder":[{"DOI":"10.13039\/501100001858","name":"VINNOVA","doi-asserted-by":"publisher","award":["0.13039\/501100001858"],"award-info":[{"award-number":["0.13039\/501100001858"]}],"id":[{"id":"10.13039\/501100001858","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001858","name":"VINNOVA","doi-asserted-by":"publisher","award":["2019-02386"],"award-info":[{"award-number":["2019-02386"]}],"id":[{"id":"10.13039\/501100001858","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100018891","name":"Saab","doi-asserted-by":"publisher","award":["0.13039\/501100001858"],"award-info":[{"award-number":["0.13039\/501100001858"]}],"id":[{"id":"10.13039\/501100018891","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100018891","name":"Saab","doi-asserted-by":"publisher","award":["2019-02386"],"award-info":[{"award-number":["2019-02386"]}],"id":[{"id":"10.13039\/501100018891","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["arc.aiaa.org"],"crossmark-restriction":true},"short-container-title":["Journal of Aerospace Information Systems"],"published-print":{"date-parts":[[2023,3]]},"abstract":"<jats:p> Two different metrics quantifying model and simulator predictive capability are formulated and evaluated; both metrics exploit results from conducted validation experiments where simulation results are compared to the corresponding measured quantities. The first metric is inspired by the modified nearest neighbor coverage metric and the second by the Kullback\u2013Liebler divergence. The two different metrics are implemented in Python and in a here-developed general metamodel designed to be applicable for most physics-based simulation models. These two implementations together facilitate both offline and online metric evaluation. Additionally, a connection between the two, here separated, concepts of predictive capability and credibility is established and realized in the metamodel. The two implementations are, finally, evaluated in an aeronautical domain context. <\/jats:p>","DOI":"10.2514\/1.i011153","type":"journal-article","created":{"date-parts":[[2023,1,16]],"date-time":"2023-01-16T07:47:58Z","timestamp":1673855278000},"page":"152-167","update-policy":"https:\/\/doi.org\/10.2514\/aiaa_crossmarkpolicy","source":"Crossref","is-referenced-by-count":4,"title":["Toward Objective Assessment of Simulation Predictive Capability"],"prefix":"10.2514","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5773-3518","authenticated-orcid":false,"given":"Robert","family":"H\u00e4llqvist","sequence":"first","affiliation":[{"name":"Saab AB Aeronautics, 582 54 Link\u00f6ping, Sweden"}]},{"given":"Magnus","family":"Eek","sequence":"additional","affiliation":[{"name":"Swedish National Road and Transport Research Institute (VTI), 583 30 Link\u00f6ping, Sweden"}]},{"given":"Robert","family":"Braun","sequence":"additional","affiliation":[{"name":"Link\u00f6ping University, 581 83 Link\u00f6ping, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2315-0680","authenticated-orcid":false,"given":"Petter","family":"Krus","sequence":"additional","affiliation":[{"name":"Link\u00f6ping University, 581 83 Link\u00f6ping, Sweden"}]}],"member":"1387","reference":[{"key":"r2","doi-asserted-by":"publisher","DOI":"10.2514\/1.I010391"},{"key":"r3","unstructured":"AnderssonH.CarlssonM. \u201cSaab Aeronautics Handbook for Development of Simulation Models: Public Variant,\u201d 12\/00159, Machine Design, Link\u00f6ping Univ., Link\u00f6ping, Sweden, 2012."},{"key":"r5","unstructured":"Standard for Models and Simulations, NASA, Washington, D.C., 2008, pp.\u00a01\u201358."},{"key":"r6","doi-asserted-by":"publisher","DOI":"10.1177\/154851290500200402"},{"key":"r11","doi-asserted-by":"publisher","DOI":"10.1016\/j.compstruc.2010.01.005"},{"key":"r12","doi-asserted-by":"publisher","DOI":"10.1016\/j.ymssp.2014.05.040"},{"key":"r13","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"r14","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729694"},{"key":"r15","unstructured":"\u201cModelica Language-A Unified Object-Oriented Language for Systems Modeling, Language Specification Version 3.5,\u201d Modelica Association, Feb.\u00a02021, https:\/\/modelica.org\/documents\/MLS.pdf."},{"key":"r18","unstructured":"OberkampfW. L.RoyC. J., Verification and Validation in Scientific Computing, Cambridge Univ. Press, Cambridge, MA, 2014, pp.\u00a0378\u2013381, https:\/\/www.ebook.de\/de\/product\/11824763\/william_l_oberkampf_christopher_j_roy_verification_and_validation_in_scientific_computing.html."},{"key":"r19","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-70766-2"},{"key":"r22","doi-asserted-by":"publisher","DOI":"10.1002\/9780470485682.ch6"},{"key":"r23","unstructured":"Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer, American Soc. of Mechanical Engineers, New York, 2009, pp.\u00a01\u201342."},{"key":"r24","doi-asserted-by":"publisher","DOI":"10.1007\/s11831-020-09473-7"},{"key":"r25","doi-asserted-by":"publisher","DOI":"10.1115\/1.4035716"},{"key":"r26","doi-asserted-by":"publisher","DOI":"10.1115\/1.3605079"},{"key":"r27","doi-asserted-by":"publisher","DOI":"10.1049\/ree.1980.0006"},{"key":"r31","unstructured":"DattorroJ., Convex Optimization and Euclidian Distance Geometry, Meboo Publishing, Palo Alto, CA, 2005, pp.\u00a033\u2013134, Chap.\u00a02."},{"key":"r32","doi-asserted-by":"publisher","DOI":"10.7564\/13-COEC10"},{"key":"r33","doi-asserted-by":"publisher","DOI":"10.1109\/18.61115"},{"key":"r34","doi-asserted-by":"publisher","DOI":"10.1016\/j.ecosta.2017.01.006"},{"key":"r35","unstructured":"ForssC.P. \u201cAnalysis and Visualization of Validation Results,\u201d Master\u2019s Thesis, Dept. of Electrical Engineering, Link\u00f6ping Univ., Link\u00f6ping, Sweden, June 2015, http:\/\/www.diva-portal.org\/smash\/record.jsf?pid=diva2:825524&dswid=-869."},{"key":"r36","first-page":"8","volume":"44","author":"Fritzson P.","year":"2005","journal-title":"Simulation News Europe"},{"key":"r37","doi-asserted-by":"publisher","DOI":"10.1016\/S1474-6670(17)43628-7"},{"key":"r38","volume-title":"Dymola Dynamic Modeling Laboratory, User Manual Volume 1","year":"2016","edition":"1"},{"key":"r39","doi-asserted-by":"publisher","DOI":"10.4173\/mic.2020.4.1"},{"key":"r41","doi-asserted-by":"publisher","DOI":"10.3384\/ecp17132653"},{"key":"r42","doi-asserted-by":"publisher","DOI":"10.3384\/ecp1915769"},{"key":"r45","doi-asserted-by":"publisher","DOI":"10.1016\/0959-1524(95)00009-F"},{"key":"r46","doi-asserted-by":"publisher","DOI":"10.1016\/j.cma.2007.07.030"},{"key":"r48","volume-title":"Sannolikhetsteori och statestikteori med till\u00e4mpningar","author":"Blom G.","year":"2009","edition":"5"},{"key":"r49","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.2992473"},{"key":"r50","unstructured":"AnderssonH. \u201cVariability and Customization of Simulator Products: A Product Line Approach in Model Based Systems Engineering,\u201d Ph.D. Thesis, Div. of Machine Design, Dept. of Management and Engineering, Link\u00f6ping Univ., Link\u00f6ping, Sweden, 2012, https:\/\/www.diva-portal.org\/smash\/get\/diva2:474629\/FULLTEXT01.pdf."},{"key":"r51","unstructured":"OchelL. \u201cOMSimulator Documentation,\u201d Techreport Release v2.1.1.post156-g47fd5e0, Open Source Modelica Consortium, March\u00a02022, https:\/\/openmodelica.org\/doc\/OMSimulator\/master\/OMSimulator.pdf."},{"key":"r56","volume-title":"Principles of Object Oriented Modeling and Simulation with Modelica 2.1","author":"Fritzson P.","year":"2004"},{"key":"r57","doi-asserted-by":"publisher","DOI":"10.1115\/1.4051497"},{"key":"r58","first-page":"1","volume-title":"Mathematics: Optimization and Control","author":"Chen Y.","year":"2011"}],"container-title":["Journal of Aerospace Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/arc.aiaa.org\/doi\/pdf\/10.2514\/1.I011153","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,27]],"date-time":"2023-02-27T13:31:44Z","timestamp":1677504704000},"score":1,"resource":{"primary":{"URL":"https:\/\/arc.aiaa.org\/doi\/10.2514\/1.I011153"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3]]},"references-count":37,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,3]]}},"alternative-id":["10.2514\/1.I011153"],"URL":"https:\/\/doi.org\/10.2514\/1.i011153","relation":{},"ISSN":["1940-3151","2327-3097"],"issn-type":[{"value":"1940-3151","type":"print"},{"value":"2327-3097","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3]]},"assertion":[{"value":"2022-05-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-21","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-21","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-01-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}