{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T07:26:48Z","timestamp":1748071608217},"reference-count":68,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2006,10,31]],"date-time":"2006-10-31T00:00:00Z","timestamp":1162252800000},"content-version":"vor","delay-in-days":4321,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Software Testing Verif &amp; Rel"],"published-print":{"date-parts":[[1995,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Verification and validation are terms that have been used for several years in software engineering, and there are now many verification and validation techniques, plus considerable experience and expertise in using them. However, applying these techniques to knowledge\u2010based system is not straightforward. The essential differences between conventional systems and knowledge\u2010based systems suggest that these techniques must be expanded and adapted, but new techniques are also needed. This article has two major goals: first, it makes some comparisons between verification and validation as found in traditional software engineering and knowledge\u2010based systems, pointing out what is special about the latter as compared with the former; second, it provides a framework for a discussion of the various European work on verification and validation of knowledge\u2010based systems. The perspective put forward in this article allows for a vast amound of work to be surveyed and analysed beyond the implementation level, by differentiating the symbol level and the knowledge level within a knowledge\u2010based system.<\/jats:p>","DOI":"10.1002\/stvr.4370050404","type":"journal-article","created":{"date-parts":[[2006,11,17]],"date-time":"2006-11-17T16:04:38Z","timestamp":1163779478000},"page":"233-271","source":"Crossref","is-referenced-by-count":6,"title":["Techniques for the verification and validation of knowledge\u2010based systems: A survey based on the symbol\/knowledge level distinction"],"prefix":"10.1002","volume":"5","author":[{"given":"Anca I.","family":"Vermesan","sequence":"first","affiliation":[]},{"given":"Trevor","family":"Bench\u2010Capon","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2006,10,31]]},"reference":[{"key":"e_1_2_1_2_1","first-page":"78","volume-title":"Proceedings of the AAA193 Workshop on Validation and Verification of Knowledge\u2010Based Systems","author":"Antoniou G.","year":"1993"},{"key":"e_1_2_1_3_1","first-page":"63","volume-title":"Validation, Verification and Test of Knowledge\u2010Based Systems","author":"Ayel M.","year":"1991"},{"key":"e_1_2_1_4_1","first-page":"357","article-title":"SYCOJET and SACCO, two tools for verifying expert systems","volume":"4","author":"Ayel M.","year":"1993","journal-title":"International Journal of Expert Systems: Research and Applications"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/0957-4174(90)90002-C"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/64.215226"},{"key":"e_1_2_1_7_1","first-page":"151","volume-title":"Lecture Notes in Computer Science","author":"Bezem M.","year":"1987"},{"key":"e_1_2_1_8_1","unstructured":"Bologna S. Ness E.andSivertsen T. (1990) \u2018Dependable knowledge\u2010based systems development and verification: what we can learn from software engineering and what we need\u2019 InProceedings of the 2nd International Conference on Tools for Artificial Intelligence Herndon Virginia U.S.A. IEEE Computer Society Press Los Alamitos California U.S.A. pp.86\u201394."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218194093000045"},{"key":"e_1_2_1_10_1","unstructured":"Bouali F. Loiseau S.andRousset M. C. (1994). \u2018KBS correction: a proposal based on diagnostic theory\u2019 InProceedings of the ECAI\u201094 Workshop on Validation of Knowledge\u2010Based Systems Amsterdam The Netherlands European Conference on Artificial Intelligence pp.21\u201329."},{"key":"e_1_2_1_11_1","unstructured":"Brossier F. Lapicque J. Y.andRouge A.(1992).VITAL\u2014Esprit II Project P5365 SYSECA 315 Bureau de la Colline 92213 Saint\u2010Cloud Cedex France."},{"key":"e_1_2_1_12_1","unstructured":"Cardenosa J.andJuristo N.(1993). \u2018General overview of the VALID project \u2019 in Proceedings of EUROVAV\u203293 European Symposium on the Validation and Verification of Knowledge\u2010Based Systems Palma de Mallorca Spain pp.53\u201367."},{"key":"e_1_2_1_13_1","volume-title":"Maintenance of Knowledge\u2010Based Systems","author":"Coenen F.","year":"1993"},{"key":"e_1_2_1_14_1","first-page":"207","volume-title":"Validation, Verification and Test of Knowledge\u2010based Systems","author":"Craw S.","year":"1991"},{"key":"e_1_2_1_15_1","unstructured":"Craw S. (1993). \u2018Refinement assists quality assurance\u2019 InProceedings of the IJCA193 Workshop on Validation of Knowledge\u2010Based Systems International Joint Conference on Artificial Intelligence Chambery France pp.5\u201315."},{"key":"e_1_2_1_16_1","volume-title":"Verifiable Programming","author":"Dahl O. J.","year":"1992"},{"key":"e_1_2_1_17_1","volume-title":"A Discipline of Programming","author":"Dijkstra E. W.","year":"1976"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0269888900006767"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1016\/0933-3657(93)90015-U"},{"key":"e_1_2_1_20_1","unstructured":"Gamble R. F.(1992). \u2018Developing reliable concurrent rule\u2010based programs\u2019 D. Sc. thesis Washington University U.S.A."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/int.4550090806"},{"key":"e_1_2_1_22_1","unstructured":"Herre H.(1993). \u2018Semantical completeness of model\u2010based diagnosis \u2019 In Proceedings of EUROVAV\u203293 European Symposium on the Validation and Verification of Knowledge\u2010Based Systems Palma de Mallorca Spain pp.217\u2013229."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00289507"},{"key":"e_1_2_1_24_1","first-page":"33","volume-title":"Validation, Verification and Test of Knowledge\u2010based Systems","author":"Hoppe T.","year":"1991"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/64.215222"},{"key":"e_1_2_1_26_1","first-page":"231","volume-title":"Proceedings of EUROVAV \u203293","author":"Hors P.","year":"1993"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/0167-9236(94)90003-5"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/64.215223"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/0957-4174(90)90009-J"},{"key":"e_1_2_1_30_1","volume-title":"Dependability: Basic Concepts and Terminology","author":"Laprie J. C.","year":"1991"},{"key":"e_1_2_1_31_1","unstructured":"Laurent P.andVignollet L.(1994). \u2018Aspects statistique et dynamiques des g\u00e9n\u00e9rateurs de cas de test \u2019 In Proceedings of JVAL94 Troisieme Journ\u00e9e Validation des Syst\u00e9mes \u00e0 Base de Connaisances. Strasbourg France pp.B. 1\u2013B. 11."},{"key":"e_1_2_1_32_1","unstructured":"Leemans P. Treuer J.andWillems M.(1993). \u2018On the verification of knowledge\u2010based reasoning modules \u2019 Raport 1R\u2010346 Artificial Intelligence Group Department of Mathematics and Computer Science Vrije Universiteit Amsterdam De Boelelaan 1081a 1081 HV Amsterdam The Netherlands."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1468-0394.1986.tb00198.x"},{"key":"e_1_2_1_34_1","first-page":"273","article-title":"Formal verification of knowledge bases focused on consistency: two experiments based on ATMS techniques","volume":"4","author":"Loiseau S.","year":"1993","journal-title":"International Journal of Expert Systems: Research and Applications"},{"key":"e_1_2_1_35_1","first-page":"191","volume-title":"Validation, Verification and Test of Knowledge\u2010Based Systems","author":"Lopez B.","year":"1991"},{"key":"e_1_2_1_36_1","doi-asserted-by":"crossref","first-page":"58","DOI":"10.3233\/AIC-1990-3204","article-title":"Knowledge based systems validation: a state of the art","volume":"3","author":"Lopez B.","year":"1990","journal-title":"AICOM"},{"key":"e_1_2_1_37_1","unstructured":"Mengshoel O. J.(1991). \u2018KVAT: a tool for incremental knowledge validation in a knowledge engineering workbench\u2019 Report STF40 A91115 SINTEF DELAB 7034 Trondheim Norway."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/64.215224"},{"key":"e_1_2_1_39_1","unstructured":"Meseguer P. (1992). \u2018Incremental verification of rule\u2010based expert systems\u2019 InProceedings of the 10th European Conference on Artificial Intelligence ECAI\u201092 Vienna Austria Wiley Chichester U. K. pp.840\u2013844."},{"key":"e_1_2_1_40_1","unstructured":"Meseguer P. (1993). \u2018Expert system validation through knowledge base refinement\u2019 InProceedings of IJCAI\u201093 International Joint Conference on Artificial Intelligence Chambery France pp.477\u2013482."},{"key":"e_1_2_1_41_1","first-page":"163","article-title":"Verification of multi\u2010level rule\u2010based expert systems: theory and practice","volume":"4","author":"Meseguer P.","year":"1993","journal-title":"International Journal of Expert Systems: Research and Applications"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/0957-4174(90)90006-G"},{"key":"e_1_2_1_43_1","unstructured":"Miller L. A. Groundwater E.andMirsky S. M.(1993). \u2018Survey and assessment of conventional software verification and validation methods \u2019 SAIC\u201091\/6660 Science Applications International Corporation 11251 Roger Bacon Drive Reston VA 22090 USA."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(82)90012-1"},{"key":"e_1_2_1_45_1","first-page":"3","volume-title":"Validation, Verification and Test of Knowledge\u2010based Systems","author":"O'Leary E. D.","year":"1991"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1468-0394.1990.tb00234.x"},{"key":"e_1_2_1_47_1","unstructured":"Preston N. Desvergne J. M. Dorlet E.andEvrard J. M.(1994). \u2018Utilisation d'une representation abstraite KSL comme aide a la validation \u2019 In Proceedings of JVAL94 Troisieme Journee Validation des Syst\u00e9mes \u00e0 Base de Connaisances Strasbourg France pp.A.1\u2013A.14."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(87)90062-2"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.221138"},{"key":"e_1_2_1_50_1","unstructured":"Rousset M. C.(1988). \u2018On the consistency of knowledge bases: the COVADIS system\u2019 InProceedings of ECAI European Conference on Artificial Intelligence Munich Germany August pp.79\u201384."},{"key":"e_1_2_1_51_1","first-page":"739","volume-title":"Proceedings of ECA194","author":"Rousset M. C.","year":"1994"},{"key":"e_1_2_1_52_1","unstructured":"Rushby J.andWhitehurst R. A.(1989). \u2018Formal verification of AI software \u2019 NASA Contract 18226 SRI International Computer Science Laboratory. 333 Ravenswood Avenue Menlo Park CA 94025 U.S.A."},{"key":"e_1_2_1_53_1","first-page":"29","article-title":"Components of expertise","volume":"11","author":"Steels L.","year":"1990","journal-title":"Al Magazine"},{"key":"e_1_2_1_54_1","volume-title":"The Knowledge Level in Expert Systems","author":"Steels L.","year":"1994"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/0957-4174(90)90019-Q"},{"key":"e_1_2_1_56_1","unstructured":"Todd B. S.andLedger W. L.(1994). \u2018A computer\u2010based flowcharting system for clinical protocols \u2019 in preparation."},{"key":"e_1_2_1_57_1","unstructured":"Todd B. S.andStamper R.(1992). \u2018Formal specification of rule\u2010based expert systems\u2019 InProceedings of the 4th International Conference on Software Engineering and Knowledge Engineering SEKE 92 Capri Italy IEEE Computer Society Press Los Alamitos California U.S.A. pp.333\u2013340."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1016\/0020-7101(93)90030-A"},{"key":"e_1_2_1_59_1","volume-title":"Formal Specification of Complex Reasoning Systems","author":"Treur J.","year":"1993"},{"key":"e_1_2_1_60_1","unstructured":"Treur J.andWillems M.(1994). \u2018On verification in compositional knowledge\u2010based systems\u2019 InProceedings of the ECAI\u201094 Workshop on Validation of Knowledge\u2010Based Systems European Conference on Artificial Intelligence Amsterdam The Netherlands pp.4\u201320."},{"key":"e_1_2_1_61_1","unstructured":"Tsai W. T. Johnson P. E. Slagle J. R. Zaulkernan I. A. Heisler K. G. Jamal K. Han K. Volovik D. Gardiner D. A.andYang T. L. A.(1992). \u2018Requirements specifications for expert systems: a case study\u2019 Technical Report TR 88\u201344 Computer Science Department University of Minnesota Minneapolis MN 55455 U.S.A."},{"key":"e_1_2_1_62_1","first-page":"341","article-title":"Verification of knowledge base redundancy and subsumption using graph transformations","volume":"4","author":"Valiente G.","year":"1993","journal-title":"International Journal of Expert Systems: Research and Applications"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1016\/1042-8143(92)90017-U"},{"key":"e_1_2_1_64_1","unstructured":"Vermesan A. I.andWergeland T. H.(1994a ). \u2018A formal approach to knowledge base refinement and verification\u2019 inProceedings of ECAI\u201094 Workshop on Validation of Knowledge\u2010Based Systems European Conference on Artificial Intelligence Amsterdam The Netherlands pp.42\u201352."},{"key":"e_1_2_1_65_1","first-page":"22","volume-title":"Proceedings of the AAAI94 Workshop on the Validation of Knowledge\u2010Based Systems","author":"Vermesan A. I.","year":"1994"},{"key":"e_1_2_1_66_1","unstructured":"Vermesan A. I.andWergeland T. H.(1994c ). \u2018Expert system verification and validation: issues and approaches \u2019 Working paper No 82\/1994 Foundation for Research in Economics and Business Administration Breiviken 2 5035 Bergen Norway."},{"key":"e_1_2_1_67_1","unstructured":"Wells S. A.(1993). \u2018The VIVA method: a life\u2010cycle independent approach to KBS validation \u2019 In Proceedings of the AAAI93 Workshop on Validation and Verification of Knowledge\u2010Based Systems American Association for Artificial Intelligence Washington D. C. U.S.A. pp.109\u2013113."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1016\/1042-8143(92)90013-Q"},{"key":"e_1_2_1_69_1","unstructured":"Zlatareva N. (1992). \u2018Performance evaluation and refinement of expert system knowledge bases\u2019 InProceedings of ISMM International Conference on Information and Knowledge Management CIKM\u201092 The International Society for Mini and Microcomputers Baltimore Maryland U.S.A. pp.560\u2013567."}],"container-title":["Software Testing, Verification and Reliability"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fstvr.4370050404","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/stvr.4370050404","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,26]],"date-time":"2023-10-26T01:06:25Z","timestamp":1698282385000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/stvr.4370050404"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[1995,1]]},"references-count":68,"journal-issue":{"issue":"4","published-print":{"date-parts":[[1995,1]]}},"alternative-id":["10.1002\/stvr.4370050404"],"URL":"https:\/\/doi.org\/10.1002\/stvr.4370050404","archive":["Portico"],"relation":{},"ISSN":["0960-0833","1099-1689"],"issn-type":[{"value":"0960-0833","type":"print"},{"value":"1099-1689","type":"electronic"}],"subject":[],"published":{"date-parts":[[1995,1]]}}}