{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T12:24:06Z","timestamp":1765369446050},"reference-count":114,"publisher":"Tsinghua University Press","issue":"1","license":[{"start":{"date-parts":[[2019,5,10]],"date-time":"2019-05-10T00:00:00Z","timestamp":1557446400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJCS"],"published-print":{"date-parts":[[2019,5,10]]},"abstract":"Purpose<\/jats:title>Simulation is a well-known technique for using computers to imitate or simulate the operations of various kinds of real-world facilities or processes. The facility or process of interest is usually called a system, and to study it scientifically, we often have to make a set of assumptions about how it works. These assumptions, which usually take the form of mathematical or logical relationships, constitute a model that is used to gain some understanding of how the corresponding system behaves, and the quality of these understandings essentially depends on the credibility of given assumptions or models, known as VV&A (verification, validation and accreditation). The main purpose of this paper is to present an in-depth theoretical review and analysis for the application of VV&A in large-scale simulations.<\/jats:p><\/jats:sec>Design\/methodology\/approach<\/jats:title>After summarizing the VV&A of related research studies, the standards, frameworks, techniques, methods and tools have been discussed according to the characteristics of large-scale simulations (such as crowd network simulations).<\/jats:p><\/jats:sec>Findings<\/jats:title>The contributions of this paper will be useful for both academics and practitioners for formulating VV&A in large-scale simulations (such as crowd network simulations).<\/jats:p><\/jats:sec>Originality\/value<\/jats:title>This paper will help researchers to provide support of a recommendation for formulating VV&A in large-scale simulations (such as crowd network simulations).<\/jats:p><\/jats:sec>","DOI":"10.1108\/ijcs-01-2019-0004","type":"journal-article","created":{"date-parts":[[2019,5,7]],"date-time":"2019-05-07T13:53:16Z","timestamp":1557237196000},"page":"63-86","source":"Crossref","is-referenced-by-count":7,"title":["A survey on VV&A of large-scale simulations"],"prefix":"10.26599","volume":"3","author":[{"given":"Yanan","family":"Wang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianqiang","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sun","family":"Hongbo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuan","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Faheem","family":"Akhtar","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Azhar","family":"Imran","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"11138","reference":[{"issue":"3","key":"key2020012411091331700_ref001","first-page":"299","article-title":"Pattern recognition: theory, experiment, computer simulations, and dynamic models of form perception and discovery","volume":"2","year":"1967","journal-title":"Brain Research"},{"issue":"6","key":"key2020012411091331700_ref002","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1177\/003754978003500606","article-title":"Terminology for model credibility","volume":"35","year":"1980","journal-title":"Simulation"},{"key":"key2020012411091331700_ref003","first-page":"5","article-title":"Human computation","year":"2007"},{"key":"key2020012411091331700_ref004","first-page":"126","article-title":"Dynamic modeling and simulation of the anaerobic digestion process","year":"1970"},{"issue":"1","key":"key2020012411091331700_ref005","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1007\/BF02136828","article-title":"Validation, verification and testing through the life cycle of a simulation study","volume":"53","year":"1994","journal-title":"Annals of Operations Research"},{"key":"key2020012411091331700_ref006","first-page":"147","article-title":"Principles and techniques of simulation validation, verification, and testing","volume-title":"Proceedings of the 27th Conference on Winter Simulation","year":"1995"},{"key":"key2020012411091331700_ref007","first-page":"135","article-title":"Verification validation and accreditation of simulation models","volume-title":"Proceedings of the 29th Conference on Winter Simulation","year":"1997"},{"key":"key2020012411091331700_ref008","first-page":"335","article-title":"Verification, validation, and testing","volume":"10","year":"1998","journal-title":"Handbook of Simulation"},{"issue":"2","key":"key2020012411091331700_ref009","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1177\/003754978504500204","article-title":"Formulated problem verification as an explicit requirement of model credibility","volume":"45","year":"1985","journal-title":"Simulation"},{"key":"key2020012411091331700_ref010","first-page":"621","article-title":"Some examples of simulation model validation using hypothesis testing","volume-title":"Proceedings of the 14th Conference on Winter Simulation","year":"1982"},{"issue":"6","key":"key2020012411091331700_ref011","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1177\/003754978203900602","article-title":"Validation of multivariate response models using hoteling two-sample t2 test","volume":"39","year":"1982","journal-title":"Simulation"},{"issue":"3\/4","key":"key2020012411091331700_ref012","first-page":"375","article-title":"Validation of simulation models via simultaneous confidence intervals","volume":"4","year":"1984","journal-title":"American Journal of Mathematical and Management Sciences"},{"key":"key2020012411091331700_ref013","first-page":"829","article-title":"Planning for verification, validation, and accreditation of modeling and simulation applications","volume-title":"Simulation Conference","year":"2000"},{"issue":"1","key":"key2020012411091331700_ref014","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/0377-2217(89)90059-3","article-title":"Multiple tests for validation of system dynamics type of simulation models","volume":"42","year":"1989","journal-title":"European Journal of Operational Research"},{"key":"key2020012411091331700_ref015","article-title":"Wireless sensor network protocol property validation through the system\u2019s simulation in a dedicated framework","volume-title":"10th International Conference on Signal Processing and Communication Systems, ICSPCS 2016 \u2013 Proceedings","year":"2016"},{"issue":"2","key":"key2020012411091331700_ref016","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1198\/004017007000000092","article-title":"A framework for validation of computer models","volume":"49","year":"2007","journal-title":"Technometrics"},{"issue":"1","key":"key2020012411091331700_ref017","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1145\/229493.229511","article-title":"A knowledge-based approach for the validation of simulation models: the foundation","volume":"6","year":"1996","journal-title":"ACM Transactions on Modeling and Computer Simulation"},{"key":"key2020012411091331700_ref018","volume-title":"Computer Applications in the Behavioral Sciences","year":"1962"},{"issue":"1","key":"key2020012411091331700_ref019","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1108\/IJCS-01-2017-0004","article-title":"Crowd science and engineering: concept and research framework","volume":"1","year":"2017","journal-title":"International Journal of Crowd Science"},{"key":"key2020012411091331700_ref020","first-page":"134","article-title":"Agent-based model (ABM) validation considerations","volume-title":"Proceedings of the Third International Conference on Advances in System Simulation (SIMUL 2011)","year":"2011"},{"key":"key2020012411091331700_ref021","first-page":"925","article-title":"Verification and validation in military simulations","volume-title":"Proceedings of the 2004 Winter Simulation Conference","year":"2004"},{"key":"key2020012411091331700_ref023","article-title":"Department of defense verification, validation, and accreditation (W and A) recommended practices guide","year":"1996"},{"key":"key2020012411091331700_ref022","unstructured":"Department of Defense Instruction (DoDI) (2003), \u201cDoD modeling and simulation verification, validation and accreditation\u201d, available at: www.docin.com\/p-116071952.html"},{"issue":"2","key":"key2020012411091331700_ref024","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1177\/003754979105600204","article-title":"An expert system prototype assisting the statistical validation of simulation models","volume":"56","year":"1991","journal-title":"Simulation-Transactions of the Society for Modeling and Simulation International"},{"issue":"3","key":"key2020012411091331700_ref025","article-title":"VALFRAM: validation framework for activity-based models","volume":"19","year":"2016","journal-title":"Jasss"},{"key":"key2020012411091331700_ref026","article-title":"Measuring the predictive capability of computational methods: principles and methods, issues and illustrations","year":"2001"},{"issue":"2","key":"key2020012411091331700_ref027","doi-asserted-by":"crossref","first-page":"1550014","DOI":"10.1142\/S1793962315500142","article-title":"Study of industrially applied methods for verification, validation and uncertainty quantification of simulator models","volume":"6","year":"2015","journal-title":"International Journal of Modeling, Simulation, and Scientific Computing"},{"key":"key2020012411091331700_ref028","article-title":"Simulation using digital computers","year":"1967"},{"issue":"7","key":"key2020012411091331700_ref029","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1287\/mnsc.13.7.525","article-title":"The analysis of simulation\u2013generated time series","volume":"13","year":"1967","journal-title":"Management Science"},{"key":"key2020012411091331700_ref030","article-title":"Verification, validation, and accreditation (VV&A) recommended practice guide","year":"1996"},{"key":"key2020012411091331700_ref031","article-title":"Verification, validation, and accreditation (VV&A) recommended practice guide 2","year":"2000"},{"key":"key2020012411091331700_ref032","article-title":"Verification, Validation, and Accreditation (VV&A) Recommended Practice Guide 2.5","year":"2005"},{"key":"key2020012411091331700_ref033","first-page":"173","article-title":"CAE for the electrical design of urban rail transit systems","year":"1987"},{"key":"key2020012411091331700_ref034","first-page":"635","article-title":"Towards a generic testing framework for agent-based simulation models","year":"2011"},{"key":"key2020012411091331700_ref035","unstructured":"Han, D. (2013), \u201cResearch and implementation on application software hybrid trust measurement\u201d, working paper, Beijing University of Technology, Beijing, 15 June."},{"issue":"11","key":"key2020012411091331700_ref036","first-page":"12","article-title":"Simulation and training: expecting the unexpected","volume":"22","year":"2011","journal-title":"Military and Aerospace Electronics"},{"key":"key2020012411091331700_ref037","article-title":"IEEE Trial-Use recommended practice for distributed interactive simulation verification, validation, accreditation, IEEE trial","year":"1998"},{"key":"key2020012411091331700_ref038","article-title":"IEEE recommended practice for verification, validation, and accreditation of a federation \u2013 an overlay to the High-Level architecture federation development and execution process","year":"2007"},{"issue":"4","key":"key2020012411091331700_ref039","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1177\/003754977803000404","article-title":"On validating simulation models of missile systems","volume":"30","year":"1978","journal-title":"SIMULATION"},{"key":"key2020012411091331700_ref040","article-title":"Flight mechanics panel working group WG-12 on validation of missile system simulation","year":"1986"},{"key":"key2020012411091331700_ref041","article-title":"Validation of models, statistical techniques and data availability","volume-title":"Simulation Conference IEEE","year":"1999"},{"key":"key2020012411091331700_ref042","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1145\/1363686.1363696","article-title":"A validation methodology for agent-based simulations","volume-title":"Proceedings of the 2008 ACM Symposium on Applied Computing","year":"2008"},{"key":"key2020012411091331700_ref043","unstructured":"Kl\u00fcgl, F. (2009), \u201cAgent-based simulation engineering\u201d, PhD thesis, Habilitation Thesis, University of Wurzburg."},{"key":"key2020012411091331700_ref044","volume-title":"Simulation Modeling and Analysis","year":"2007","edition":"4th ed."},{"key":"key2020012411091331700_ref045","article-title":"2002 Defense modeling and simulation office (DMSO) laboratory for human behavior model interchange standards","year":"2003"},{"key":"key2020012411091331700_ref046","article-title":"A parallel multi-agent simulation planning approach to complex logistics system with genetic optimization","volume-title":"International Conference on Wireless Communications","year":"2007"},{"key":"key2020012411091331700_ref047","unstructured":"Li, L. (2012), \u201cResearch on simulation credibility assessment of marine electric propulsion system\u201d, Working paper, Harbin Engineering University, Harbin."},{"issue":"7","key":"key2020012411091331700_ref048","first-page":"1417","article-title":"Research on operational validity evaluation of man-in-the-loop simulation system","volume":"19","year":"2007","journal-title":"Journal of System Simulation"},{"key":"key2020012411091331700_ref049","unstructured":"Liu, M. (2017), \u201cResearch on restful software credibility testing based on behavior declaration\u201d, Working paper, Beijing University of Technology, Beijing, 25 May."},{"key":"key2020012411091331700_ref050","article-title":"Immersive face validation: a new validation technique for agent-based simulation","year":"2012"},{"key":"key2020012411091331700_ref051","unstructured":"Lv, H. (2016), \u201cResearch on software credibility testing method based on behavior declaration\u201d, Working paper, Beijing University of Technology, Beijing, 12 June."},{"key":"key2020012411091331700_ref052","volume-title":"Simulation: The Dynamic Modeling of Ideas and Systems with Computers","year":"1968"},{"issue":"1","key":"key2020012411091331700_ref053","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2307\/1054005","article-title":"The problem of verification in economics","volume":"22","year":"1955","journal-title":"Southern Economic Journal"},{"key":"key2020012411091331700_ref054","volume-title":"Computer Modeling and Simulation","year":"1968"},{"key":"key2020012411091331700_ref055","doi-asserted-by":"crossref","unstructured":"Mehrabadi, N.R., Wen, B., Burgos, R., Boroyevich, D. and Roy, C. (2014), \u201cVerification, validation and uncertainty quantification (VV&UQ) framework applicable to power electronics systems\u201d, SAE Technical Paper 2014-01-2176, available at: https:\/\/doi.org\/10.4271\/2014-01-2176","DOI":"10.4271\/2014-01-2176"},{"issue":"3","key":"key2020012411091331700_ref056","first-page":"A130","article-title":"Computer simulation in Geology","volume":"7","year":"1981","journal-title":"Earth Science Reviews"},{"key":"key2020012411091331700_ref057","first-page":"109","article-title":"Agent-based simulation validation: a case study in a demographic simulation","year":"2011"},{"issue":"2","key":"key2020012411091331700_ref058","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1177\/003754978003400206","article-title":"Comparison of simulation and flight test data for missile systems","volume":"34","year":"1980","journal-title":"SIMULATION: Transactions of the Society for Modeling and Simulation International"},{"key":"key2020012411091331700_ref059","article-title":"A \u2018smart\u2019 approach to VV&A","year":"2001"},{"issue":"2","key":"key2020012411091331700_ref060","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1108\/IJCS-08-2017-0019","article-title":"A study on the MCIN model in intelligent clothing industry","volume":"1","year":"2017","journal-title":"International Journal of Crowd Science"},{"key":"key2020012411091331700_ref061","first-page":"340","article-title":"Verification and validation of agent-based simulations using the VOMAS (virtual overlay multi-agent system) approach","volume-title":"CEUR Workshop Proceedings","year":"2009"},{"key":"key2020012411091331700_ref063","doi-asserted-by":"crossref","unstructured":"Oberkampf, W.L. (1994), \u201cA proposed framework for computational fluid dynamics code calibration\/validation\u201d, Nasa Sti\/recon Technical Report N, 95.","DOI":"10.2514\/6.1994-2540"},{"issue":"1","key":"key2020012411091331700_ref062","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.jcp.2006.03.037","article-title":"Measures of agreement between computation and experiment: validation metrics","volume":"217","year":"2006","journal-title":"Journal of Computational Physics"},{"issue":"12","key":"key2020012411091331700_ref064","first-page":"2658","article-title":"Amelioration and application of similar degree method for simulation credibility evaluation","volume":"19","year":"2017","journal-title":"Journal of System Simulation"},{"key":"key2020012411091331700_ref065","first-page":"1","article-title":"Validation of agent-based simulation through human computation: an example of crowd simulation","volume-title":"International Conference on Multi-Agent-Based Simulation","year":"2011"},{"key":"key2020012411091331700_ref066","volume-title":"Computer Simulation in Management Science","year":"2004"},{"key":"key2020012411091331700_ref067","volume-title":"Agent-Based and Individual-Based Modeling: A Practical Introduction","year":"2011"},{"key":"key2020012411091331700_ref068","first-page":"879","article-title":"The Simulation Team Assessment Tool (STAT): development, reliability and validation","volume-title":"Resuscitation","year":"2012"},{"key":"key2020012411091331700_ref069","article-title":"The nature of modeling","year":"1989"},{"key":"key2020012411091331700_ref070","article-title":"DoD modeling and simulation (M&S) verification, validation, and accreditation (VV&A) (no. DODI-5000.61)","year":"1996"},{"key":"key2020012411091331700_ref074","first-page":"909","article-title":"Strategic directions in verification, validation, and accreditation research","volume-title":"Proceedings of the 2000 Winter Simulation Conference","year":"2000"},{"issue":"4","key":"key2020012411091331700_ref071","first-page":"487","article-title":"Verification, validation, and accreditation of simulation models","volume":"1","year":"1997","journal-title":"Applied System Simulation"},{"key":"key2020012411091331700_ref072","article-title":"Some approaches and paradigms for verifying and validating simulation models","volume-title":"Proceedings of the 2001 Winter Simulation Conference","year":"2001"},{"key":"key2020012411091331700_ref073","first-page":"130","article-title":"Verification and validation of simulation models","volume-title":"Proceedings of the 37th Conference on Winter Simulation, WSC\u201905, Winter Simulation Conference","year":"2005"},{"key":"key2020012411091331700_ref075","article-title":"Verification and evaluation method of cooperative vehicle-infrastructure simulation system based on AHP-GRAP","volume-title":"International Conference on Electromagnetics in Advanced Applications","year":"2014"},{"key":"key2020012411091331700_ref076","volume-title":"The Art and Science","year":"1975"},{"issue":"3","key":"key2020012411091331700_ref077","first-page":"285","article-title":"Modeling and simulation of insulated-gate field-effect transistor switching circuits","volume":"3","year":"2003","journal-title":"IEEE Journal of Solid-State Circuits"},{"issue":"3","key":"key2020012411091331700_ref078","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1162\/artl.2007.13.3.303","article-title":"NetLogo, a multi-agent simulation environment","volume":"13","year":"2007","journal-title":"Artificial Life"},{"issue":"4","key":"key2020012411091331700_ref079","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1016\/0021-9290(68)90029-8","article-title":"Computer modeling of the human systemic arterial tree","volume":"1","year":"1968","journal-title":"Journal of Biomechanics"},{"issue":"16","key":"key2020012411091331700_ref080","doi-asserted-by":"crossref","first-page":"6562","DOI":"10.1073\/pnas.0611677104","article-title":"Algorithm for model validation: theory and applications","volume":"104","year":"2007","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"key2020012411091331700_ref081","first-page":"714","article-title":"An HLA based simulation framework for crowd science","volume-title":"2017 International Conference on Mathematics","year":"2017"},{"key":"key2020012411091331700_ref082","first-page":"269","article-title":"RFID technology applied for validation of an office simulation model","year":"2010"},{"issue":"4","key":"key2020012411091331700_ref083","first-page":"82","article-title":"An overview of the research on VV&A in simulation","volume":"11","year":"2006","journal-title":"Computer Simulation"},{"key":"key2020012411091331700_ref084","first-page":"1","article-title":"Exploring the vast parameter space of multi-agent based simulation","volume-title":"International Conference on Multi-Agent-Based Simulation VII","year":"2006"},{"key":"key2020012411091331700_ref085","first-page":"1","article-title":"Exploring the vast parameter space of multi-agent based simulation","volume-title":"Multiagent-Based Simulation VII. Lecture Notes in Computer Science","year":"2007"},{"key":"key2020012411091331700_ref086","article-title":"A grey theory model using in the simulation credibility evaluation of the integrative avionics system","year":"2012"},{"key":"key2020012411091331700_ref087","unstructured":"Wang, Z., Li, Z., Feng, Y. and Rong, G. (2016), \u201cCrude oil operations under uncertainty: a continuous-time rescheduling framework and a simulation environment for validation\u201d, Industrial and Engineering Chemistry Research, available at: acs.iecr.6b01108"},{"key":"key2020012411091331700_ref088","unstructured":"Weisel, E.W. (2004), \u201cModels, composability, and validity\u201d, Doctoral dissertation, Old Dominion University, Norfolk, VA."},{"key":"key2020012411091331700_ref089","first-page":"68","article-title":"Validity of models and classes of models in semantic composability","volume-title":"Proceeding Fall Simulation Interoperability Workshop 2003","year":"2003"},{"key":"key2020012411091331700_ref090","first-page":"559","article-title":"Guidelines for selecting and using simulation model verification techniques","year":"1989"},{"key":"key2020012411091331700_ref091","first-page":"188","article-title":"Fitting, calibration, and validation of simulation models, simulation","year":"1972"},{"key":"key2020012411091331700_ref092","article-title":"Object-oriented simulation model verification and validation","volume-title":"Proceedings of the 1997 Summer Computer Simulation Conference","year":"1997"},{"issue":"11","key":"key2020012411091331700_ref093","first-page":"17","article-title":"Credibility verification method and calculation based on application behavior declaration","volume":"27","year":"2018","journal-title":"Computer Systems and Applications"},{"key":"key2020012411091331700_ref094","article-title":"Implications of M&S foundations for the V&V of large scale complex simulation models","volume-title":"Proceedings of the Foundations for V&V in the 21st Century Workshop","year":"2002"},{"issue":"1","key":"key2020012411091331700_ref095","first-page":"4","article-title":"A survey on the credibility of system simulation","volume":"9","year":"1997","journal-title":"Journal of System Simulation"},{"key":"key2020012411091331700_ref096","first-page":"534","article-title":"The research review on VV&A working system of complex simulation system","year":"2012"},{"issue":"3","key":"key2020012411091331700_ref097","first-page":"312","article-title":"Simulation credibility","volume":"13","year":"2001","journal-title":"Xitong Fangzhen Xuebao\/Acta Simulata Systematica Sinica"},{"key":"key2020012411091331700_ref098","first-page":"423","article-title":"A multi-agent simulation framework for emergency evacuations incorporating personality and emotions","year":"2010"},{"issue":"3","key":"key2020012411091331700_ref099","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/j.artmed.2006.03.001","article-title":"Knowledge-based data analysis and interpretation","volume":"37","year":"2006","journal-title":"Artificial Intelligence in Medicine"},{"key":"key2020012411091331700_ref0100","article-title":"Weag Thales jp11.20 \u2013 Final state of the REVVA methodology","year":"2005"},{"key":"key2020012411091331700_ref0101","unstructured":"Department of Defense (2000), \u201cVerification validation and accreditation (VV&A) recommended practice guides BUILD 2\u201d, available at: www.docin.com\/p-116071952.html"},{"issue":"3","key":"key2020012411091331700_ref0102","first-page":"118","article-title":"Evaluation approach for network components performance using trustworthiness measurement","volume":"37","year":"2016","journal-title":"Journal on Communications"},{"issue":"8","key":"key2020012411091331700_ref0103","first-page":"43","article-title":"A fuzzy AHP method for credibility evaluation of system simulation","volume":"20","year":"2003","journal-title":"Computer Simulation"},{"key":"key2020012411091331700_ref0104","article-title":"A common verification, validation, and accreditation framework for simulations: Project JP11.20, REVVA","volume-title":"Spring Simulation Interoperability Workshop","year":"2004"},{"issue":"1","key":"key2020012411091331700_ref0105","first-page":"8","article-title":"Credibility evaluation of the C4ISR system simulation test based on the fuzzy AHP method","volume":"21","year":"2018","journal-title":"Software Engineering"},{"key":"key2020012411091331700_ref0106","unstructured":"Li-Ge, Q. (2009), \u201cResearch on the credibility evaluation and its tool of complex simulation system\u201d, Working paper, Harbin Institute of Technology, Harbin, 3 July."},{"issue":"6","key":"key2020012411091331700_ref0107","first-page":"118","article-title":"Research on the simulation credibility evaluation assistant tool based on hierarchical evaluation","volume":"27","year":"2010","journal-title":"Computer Simulation"},{"issue":"3","key":"key2020012411091331700_ref0108","first-page":"209","article-title":"Verification and validation in computational fluid dynamics","volume":"38","year":"2007","journal-title":"Advances in Mechanics"},{"issue":"1","key":"key2020012411091331700_ref0109","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/S0166-3615(00)00071-3","article-title":"Knowledge-based support for simulation analysis of manufacturing cells","volume":"44","year":"2001","journal-title":"Computers in Industry"},{"issue":"5","key":"key2020012411091331700_ref0110","first-page":"666","article-title":"A new method of elevation of confidence level of large-scale perplexing simulation system","volume":"13","year":"2001","journal-title":"Acta Simulata Systematic Sinica"},{"issue":"10","key":"key2020012411091331700_ref0111","first-page":"284","article-title":"Design of the M&S and VV&A management system","volume":"23","year":"2006","journal-title":"Computer Simulation"},{"key":"key2020012411091331700_ref0112","article-title":"A preliminary study of the automatic generation of credibility test cases based on the immune algorithm","year":"2018"},{"issue":"4","key":"key2020012411091331700_ref0113","first-page":"1072","article-title":"Fuzzy evaluation method for Web software trustworthiness","volume":"31","year":"2014","journal-title":"Application Research of Computers"},{"issue":"1","key":"key2020012411091331700_ref0114","first-page":"224","article-title":"VV&A tools for simulation model","volume":"44","year":"2016","journal-title":"Modern Defence Technology"}],"container-title":["International Journal of Crowd Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJCS-01-2019-0004\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJCS-01-2019-0004\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,9,17]],"date-time":"2022-09-17T13:02:27Z","timestamp":1663419747000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJCS-01-2019-0004\/full\/html"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,10]]},"references-count":114,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,5,10]]}},"alternative-id":["10.1108\/IJCS-01-2019-0004"],"URL":"https:\/\/doi.org\/10.1108\/ijcs-01-2019-0004","relation":{},"ISSN":["2398-7294","2398-7294"],"issn-type":[{"value":"2398-7294","type":"print"},{"value":"2398-7294","type":"print"}],"subject":[],"published":{"date-parts":[[2019,5,10]]}}}