{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:30:03Z","timestamp":1760232603914,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2022,11,20]],"date-time":"2022-11-20T00:00:00Z","timestamp":1668902400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61906175","2016BSJJ037","212102210076","202102210351"],"award-info":[{"award-number":["61906175","2016BSJJ037","212102210076","202102210351"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Doctoral Research Fund of Zhengzhou University of Light Industry","award":["61906175","2016BSJJ037","212102210076","202102210351"],"award-info":[{"award-number":["61906175","2016BSJJ037","212102210076","202102210351"]}]},{"name":"Science and Technology Project of Henan Province","award":["61906175","2016BSJJ037","212102210076","202102210351"],"award-info":[{"award-number":["61906175","2016BSJJ037","212102210076","202102210351"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Software trustworthiness evaluation is regarded as a multi-criteria decision-making problem. However, most current software trustworthiness evaluation methods do not consider the relationships between criteria. In this paper, we present a software trustworthiness evaluation strategy via the relationships between criteria. Because the trustworthy attribute degree is evaluated by a criterion, a trustworthy attribute measurement method based on fuzzy theory is first proposed, and the relationships between criteria are described by cooperative and conflicting degrees between criteria. Then, a measure formula for the symmetric substitutivity between criteria is proposed, and the cooperative degree between criteria is taken as the approximation of the symmetric substitutivity between criteria. With the help of the symmetric substitutivity between criteria, the software trustworthiness measurement model obtained by axiomatic approaches is applied to aggregate the degree to which each optional software product meets each objective. Finally, the candidate software products are sorted according to the trustworthiness aggregation results, and the optimal product is obtained from the alternative software products on the basis of the sorting results. The theoretical rationality of the measurement model is validated by proving that it satisfies the desirable properties of software trustworthiness measures, and its effectiveness is demonstrated through a case study.<\/jats:p>","DOI":"10.3390\/sym14112458","type":"journal-article","created":{"date-parts":[[2022,11,21]],"date-time":"2022-11-21T03:09:30Z","timestamp":1669000170000},"page":"2458","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["A Novel Software Trustworthiness Evaluation Strategy via Relationships between Criteria"],"prefix":"10.3390","volume":"14","author":[{"given":"Hongwei","family":"Tao","sequence":"first","affiliation":[{"name":"College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, China"}]},{"given":"Qiaoling","family":"Cao","sequence":"additional","affiliation":[{"name":"College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, China"}]},{"given":"Haoran","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, China"}]},{"given":"Yang","family":"Xian","sequence":"additional","affiliation":[{"name":"College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, China"}]},{"given":"Songtao","family":"Shang","sequence":"additional","affiliation":[{"name":"College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, China"}]},{"given":"Xiaoxu","family":"Niu","sequence":"additional","affiliation":[{"name":"College of Computer and Communication Engineering, Zhengzhou University of Light Industry, Zhengzhou 450002, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/TDSC.2004.2","article-title":"Basic Concepts and Taxonomy of Dependable and Secure Computing","volume":"1","author":"Avizienis","year":"2004","journal-title":"IEEE Trans. Dependable Secur. Comput."},{"unstructured":"(2011). Systems and Software Engineering\u2014Systems and Software Quality Requirements and Evaluation (SQuaRE)\u2014System and Software Quality Models (Standard No. ISO\/IEC 25010:2011).","key":"ref_2"},{"doi-asserted-by":"crossref","unstructured":"McCall, J.A., Richards, P.K., and Walters, G.F. (1977). Factors in Software Quality: Concepts and Definitions of Software Quality, General Electric Co.","key":"ref_3","DOI":"10.21236\/ADA049014"},{"unstructured":"Boehm, B.W., Brown, J.R., Kaspar, H., Lipow, M., MacLeod, G.J., and Merritt, M.J. (1978). Characteristics of Software Quality, North-Holland Publishing Company.","key":"ref_4"},{"key":"ref_5","first-page":"291","article-title":"Review of the Achievements of Major Research Plan of Trustworthy Software","volume":"32","author":"He","year":"2018","journal-title":"Bull. Natl. Nat. Sci. Found. China"},{"doi-asserted-by":"crossref","unstructured":"Tao, H.W., Fu, L.Y., Chen, Y.X., Han, L., and Wang, X. (2022). Improved Allocation and Reallocation Approaches for Software Trustworthiness Based on Mathematical Programming. Symmetry, 14.","key":"ref_6","DOI":"10.3390\/sym14030628"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2364","DOI":"10.23940\/ijpe.19.09.p9.23642372","article-title":"A Survey of Software Trustworthiness Measurements","volume":"15","author":"Tao","year":"2019","journal-title":"Int. J. Perform. Eng."},{"doi-asserted-by":"crossref","unstructured":"Tao, H.W., Wu, H.Y., and Chen, Y.X. (2019). An Approach of Trustworthy Measurement Allocation Based on Sub-Attributes of Software. Mathematics, 7.","key":"ref_8","DOI":"10.3390\/math7030237"},{"doi-asserted-by":"crossref","unstructured":"Tao, H.W., Chen, Y.X., and Wu, H.Y. (2020). A Reallocation Approach for Software Trustworthiness Based on Trustworthy Attributes. Mathematics, 8.","key":"ref_9","DOI":"10.3390\/math8010014"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/0164-1212(94)90067-1","article-title":"Construction and Testing of Polynomials Predicting Software Maintainability","volume":"24","author":"Oman","year":"1994","journal-title":"J. Syst. Softw."},{"key":"ref_11","first-page":"817","article-title":"A Software Behavior Trustworthiness Measurement Method Based on Data Mining","volume":"4","author":"Yuan","year":"2012","journal-title":"Int. J. Comput. Intell. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"219174","DOI":"10.1109\/ACCESS.2020.3041181","article-title":"Vulnerable Code Detection Using Software Metrics and Machine Learning","volume":"8","author":"Medeiros","year":"2020","journal-title":"IEEE Acc."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1496","DOI":"10.1109\/TII.2020.2994747","article-title":"Trustworthiness in Industrial IoT Systems Based on Artificial Intelligence","volume":"17","author":"Lv","year":"2021","journal-title":"IEEE Trans. Industr. Inform."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"753","DOI":"10.1080\/09540091.2021.1889975","article-title":"NFMF: Neural Fusion Matrix Factorisation for QoS Prediction in Service Selection","volume":"33","author":"Xu","year":"2021","journal-title":"Connect. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"106233","DOI":"10.1016\/j.infsof.2019.106233","article-title":"Software Trustworthiness Evaluation Model Based on Behavior Trajectory Matrix","volume":"119","author":"Tian","year":"2020","journal-title":"Info. Softw. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1007\/s11235-011-9420-9","article-title":"A New Metric Model for Trustworthiness of Softwares","volume":"51","author":"Tao","year":"2012","journal-title":"Telecommun. Sys."},{"doi-asserted-by":"crossref","unstructured":"Tao, H.W., Chen, Y.X., and Wu, H.Y. (2020, January 11\u201314). Decomposition of Attributes Oriented Software Trustworthiness Measure Based on Axiomatic Approaches. Proceedings of the IEEE 20th International Conference on Software Quality, Reliability and Security Companion (QRS-C), Macau, China.","key":"ref_17","DOI":"10.1109\/QRS-C51114.2020.00060"},{"key":"ref_18","first-page":"32","article-title":"An Approach for Trustworthy Evidence of Source Code Oriented Aerospace Software Trustworthiness Measurement","volume":"47","author":"Liu","year":"2021","journal-title":"AER Control Appl."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1181","DOI":"10.1080\/09540091.2022.2061424","article-title":"Theoretical and Empirical Validation of Software Trustworthiness Measure Based on Decomposition of Attributes","volume":"34","author":"Tao","year":"2022","journal-title":"Connect. Sci."},{"key":"ref_20","first-page":"583","article-title":"Attributes Oriented Software Trustworthiness Measure Based on Axiomatic Approaches","volume":"23","author":"Tao","year":"2022","journal-title":"J. Int. Technol."},{"doi-asserted-by":"crossref","unstructured":"Shi, H.L., Ma, J., and Zou, F.Y. (2008, January 12\u201314). A Fuzzy Comprehensive Evaluation Model for Software Dependability Based on Entropy Weight. Proceedings of the 2008 International Conference on Computer Science and Software Engineering, Wuhan, China.","key":"ref_21","DOI":"10.1109\/CSSE.2008.351"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.4304\/jsw.4.10.1152-1159","article-title":"An Improved Comprehensive Evaluation Model of Software Dependability based on Rough Set Theory","volume":"4","author":"Li","year":"2009","journal-title":"J. Softw."},{"doi-asserted-by":"crossref","unstructured":"Gao, X.T., Ma, Y.F., and Zhou, W. (2020, January 11\u201314). The Trustworthiness Measurement Model of Component-based Software Based on the Subjective and Objective Weight Allocation Method. Proceedings of the IEEE 21st International Conference on Software Quality, Reliability and Security Companion (QRS-C), Macau, China.","key":"ref_23","DOI":"10.1109\/QRS-C55045.2021.00076"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"68618","DOI":"10.1109\/ACCESS.2022.3185684","article-title":"The Assessment of Cloud Service Trustworthiness State Based on D-S Theory and Markov Chain","volume":"10","author":"Yang","year":"2022","journal-title":"IEEE Acc."},{"key":"ref_25","first-page":"121","article-title":"Rating of Software Trustworthiness Via Scoring of System Testing Results","volume":"1","author":"Muhammad","year":"2018","journal-title":"Int. J. Dig. Enterp. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1007\/s11390-018-1843-2","article-title":"A Unified Measurement Solution of Software Trustworthiness Based on Social-to-Software Framework","volume":"33","author":"Yang","year":"2018","journal-title":"J. Comput. Sci. Technol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"60199","DOI":"10.1109\/ACCESS.2019.2892518","article-title":"Updating Model of Software Component Trustworthiness Based on Users Feedback","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Acc."},{"doi-asserted-by":"crossref","unstructured":"Alarcon, G.M., and Ryan, T.J. (2018, January 2\u20136). Trustworthiness Perceptions of Computer Code: A Heuristic-Systematic Processing Model. Proceedings of the 51st Hawaii International Conference on System Sciences, Waikoloa Village, HI, USA.","key":"ref_28","DOI":"10.24251\/HICSS.2018.671"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"241","DOI":"10.23940\/ijpe.21.02.p8.241252","article-title":"Framework for Trustworthiness in Software Development","volume":"17","author":"Maza","year":"2021","journal-title":"Int. J. Perform. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1145\/3178315.3178328","article-title":"Harnessing the Crowd Wisdom for Software Trustworthiness: Practices in China","volume":"43","author":"Wang","year":"2018","journal-title":"ACM SIGSOFT Softw. Eng. Notes"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/S0020-0255(97)10020-2","article-title":"Fuzzy Decision Making Through Trade-off Analysis Between Criteria","volume":"107","author":"Lee","year":"1998","journal-title":"Inf. Sci."},{"unstructured":"Lee, J., Kuo, J.Y., and Huang, W.T. (1996, January 11\u201314). Fuzzy Decision Making Through Relationships Analysis Between Criteria. Proceedings of the 1996 Asian Fuzzy Systems Symposium, Kenting, Taiwan.","key":"ref_32"},{"unstructured":"Xu, Z.S. (2020, January 22\u201323). Research on Software Trustworthiness Measurement Evaluation Model Based on Data Driven. Proceedings of the 2nd International Conference on Computer Science Communication And Network Security (CSCNS2020), Sanya, China.","key":"ref_33"},{"doi-asserted-by":"crossref","unstructured":"Deng, Y.X., Chen, Z.Z., Du, W.J., Mao, B.F., Liang, Z.Z., Lin, Q.S., and Li, J.H. (2020, January 11\u201314). Trustworthiness Derivation Tree: A Model of Evidence-Based Software Trustworthiness. Proceedings of the IEEE 21st International Conference on Software Quality, Reliability and Security Companion (QRS-C), Macau, China.","key":"ref_34","DOI":"10.1109\/QRS-C55045.2021.00077"},{"key":"ref_35","first-page":"2268","article-title":"Real-time Trust Measurement of Software: Behavior Trust Analysis Approach Based on Noninterference","volume":"30","author":"Zhang","year":"2019","journal-title":"J. Softw."},{"key":"ref_36","first-page":"1713","article-title":"Evidence-Based Software Process Trustworthiness Model and Evaluation Method","volume":"28","author":"Wang","year":"2017","journal-title":"J. Softw."},{"key":"ref_37","first-page":"3412","article-title":"Trustworthiness Evidence Supporting Evaluation of Software Process Trustworthiness","volume":"29","author":"Wang","year":"2018","journal-title":"J. Softw."},{"doi-asserted-by":"crossref","unstructured":"Tang, Y., Pan, Z., Pedrycz, W., Ren, F., and Song, X. (2022). Viewpoint-based Kernel Fuzzy Clustering with Weight Information Granules. IEEE Trans. Emerg. Top. Comput. Intell., in press.","key":"ref_38","DOI":"10.1109\/TETCI.2022.3201620"},{"key":"ref_39","first-page":"27","article-title":"Symmetric Implicational Algorithm Derived from Intuitionistic Fuzzy Entropy","volume":"19","author":"Tang","year":"2022","journal-title":"Iran. J. Fuzzy Syst."},{"key":"ref_40","first-page":"532","article-title":"Developing an Evaluation Approach for Software Trustworthiness Using Combination Weights and TOPSIS","volume":"7","author":"Li","year":"2012","journal-title":"J. Softw."},{"key":"ref_41","first-page":"217","article-title":"Representation, Ranking, Distance, and Similarity of L-R Type Fuzzy Number and Application","volume":"6","author":"Chen","year":"2000","journal-title":"Aust. J. Intell. Process. Syst."},{"unstructured":"Chen, Y.X., and Tao, H.W. (2019). Software Trustworthiness Measurement Evaluation and Enhancement Specification, Science Press.","key":"ref_42"},{"key":"ref_43","first-page":"114","article-title":"Revised TOPSIS Method Based on Vertical Projection Distance-Vertical Projection Method","volume":"24","author":"Hua","year":"2004","journal-title":"Syst. Eng. Theory Pract."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/14\/11\/2458\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:22:13Z","timestamp":1760145733000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/14\/11\/2458"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,20]]},"references-count":43,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["sym14112458"],"URL":"https:\/\/doi.org\/10.3390\/sym14112458","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2022,11,20]]}}}