{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T21:30:11Z","timestamp":1769635811940,"version":"3.49.0"},"reference-count":62,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2021,7,13]],"date-time":"2021-07-13T00:00:00Z","timestamp":1626134400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006021","name":"Direction G\u00e9n\u00e9rale de l\u2019Armement","doi-asserted-by":"publisher","award":["2014 93 0807"],"award-info":[{"award-number":["2014 93 0807"]}],"id":[{"id":"10.13039\/501100006021","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCP"],"abstract":"<jats:p>An effective network security requirement engineering is needed to help organizations in capturing cost-effective security solutions that protect networks against malicious attacks while meeting the business requirements. The diversity of currently available security requirement engineering methodologies leads security requirements engineers to an open question: How to choose one? We present a global evaluation methodology that we applied during the IREHDO2 project to find a requirement engineering method that could improve network security. Our evaluation methodology includes a process to determine pertinent evaluation criteria and a process to evaluate the requirement engineering methodologies. Our main contribution is to involve stakeholders (i.e., security requirements engineers) in the evaluation process by following a requirement engineering approach. We describe our experiments conducted during the project with security experts and the feedback we obtained. Although we applied it to evaluate three requirements engineering methods (KAOS, STS and SEPP) in the context of network security, our evaluation methodology can be instantiated in other contexts and other methods.<\/jats:p>","DOI":"10.3390\/jcp1030022","type":"journal-article","created":{"date-parts":[[2021,7,13]],"date-time":"2021-07-13T22:25:31Z","timestamp":1626215131000},"page":"422-452","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["A Methodological Approach to Evaluate Security Requirements Engineering Methodologies: Application to the IREHDO2 Project Context"],"prefix":"10.3390","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0943-6180","authenticated-orcid":false,"given":"Romain","family":"Laborde","sequence":"first","affiliation":[{"name":"IRIT\/Universit\u00e9 Paul Sabatier, 31062 Toulouse, France"}]},{"given":"Sravani Teja","family":"Bulusu","sequence":"additional","affiliation":[{"name":"IRIT\/Universit\u00e9 Paul Sabatier, 31062 Toulouse, France"},{"name":"Sopra Steria\u2014I2S, 31772 Colomiers, France"}]},{"given":"Ahmad Samer","family":"Wazan","sequence":"additional","affiliation":[{"name":"IRIT\/Universit\u00e9 Paul Sabatier, 31062 Toulouse, France"},{"name":"CAT Department, College of Technological Innovation, Zayed University, Abu Dabi 4783, United Arab Emirates"}]},{"given":"Arnaud","family":"Oglaza","sequence":"additional","affiliation":[{"name":"IRIT\/Universit\u00e9 Paul Sabatier, 31062 Toulouse, France"}]},{"given":"Abdelmalek","family":"Benzekri","sequence":"additional","affiliation":[{"name":"IRIT\/Universit\u00e9 Paul Sabatier, 31062 Toulouse, France"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,13]]},"reference":[{"key":"ref_1","unstructured":"SANS (2021, July 12). Securing Against the Most Common Vectors of Cyber Attacks 2017. Available online: https:\/\/www.sans.org\/white-papers\/37995\/."},{"key":"ref_2","unstructured":"(2021, July 11). ISO\/IEC 27033 IT Network Security Standard. Available online: http:\/\/www.iso27001security.com\/html\/27033.html."},{"key":"ref_3","unstructured":"SANS (2021, July 12). Infrastructure Security Architecture for Effective Security Monitoring 2015. Available online: https:\/\/www.sans.org\/white-papers\/36512\/."},{"key":"ref_4","first-page":"34","article-title":"Network Security Architecture","volume":"7","author":"Stawowski","year":"2009","journal-title":"ISSA J."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/s10922-007-9063-z","article-title":"Implementation of a Formal Security Policy Refinement Process in WBEM Architecture","volume":"15","author":"Laborde","year":"2007","journal-title":"J. Netw. Syst. Manag."},{"key":"ref_6","unstructured":"Laborde, R., Barr\u00e8re, F., and Benzekri, A. (2013, January 7\u201310). Toward authorization as a service: A study of the XACML standard. Proceedings of the 16th Communications & Networking Symposium, Society for Computer Simulation International, San Diego, CA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1007\/s12243-018-0673-0","article-title":"A situation-driven framework for dynamic security management","volume":"74","author":"Laborde","year":"2019","journal-title":"Ann. Telecommun."},{"key":"ref_8","unstructured":"Barrere, F., Benzekri, A., Grasset, F., and Laborde, R. (2002, January 5\u20137). A multi-domain security policy distribution architecture for dynamic IP based VPN management. Proceedings of the Policies for Distributed Systems and Networks, Monterey, CA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Sherwood, N.A. (2005). SABSA (Sherwood Applied Business Security Architecture)\u2014A Business-Driven Approach, CRC Press.","DOI":"10.1201\/b17776"},{"key":"ref_10","unstructured":"Hoo, K.S., Sudbury, A., and Jaquith, A. (2001). Tangible ROI through Secure Software Engineering. Security Business Q., 1."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Iqbal, J., Ahmad, R.B., Khan, M., Alyahya, S., Nasir, M.H.N., Akhunzada, A., and Shoaib, M. (2020). Requirements engineering issues causing software development outsourcing failure. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0229785"},{"key":"ref_12","unstructured":"Bulusu, S.T., Laborde, R., Wazan, A.S., Barr\u00e8re, F., and Benzekri, A. (2020, January 4\u20136). Towards the weaving of the characteristics of good security requirements. Proceedings of the International Conference on Risks and Security of Internet and Systems, Paris, France."},{"key":"ref_13","unstructured":"Bulusu, S.T., Laborde, R., Wazan, A.S., Barr\u00e8re, F., and Benzekri, A. (September, January 29). Which Security Requirements Engineering Methodology Should I Choose?: Towards a Requirements Engineering-based Evaluation Approach. Proceedings of the 12th International Conference on Availability, Reliability and Security, Reggio Calabria, Italy."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Bulusu, S.T., Laborde, R., Wazan, A.S., Barr\u00e8re, F., and Benzekri, A. (2018, January 9\u201313). Applying a Requirement Engineering Based Approach to Evaluate the Security Requirements Engineering Methodologies. Proceedings of the ACM SAC RE 2018, Pau, France.","DOI":"10.1145\/3167132.3167417"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Bulusu, S.T., Laborde, R., Wazan, A.S., Barr\u00e8re, F., and Benzekri, A. (2018). A Requirements Engineering-Based Approach for Evaluating Security Requirements Engineering Methodologies. Information Technology-New Generations, Springer.","DOI":"10.1145\/3167132.3167417"},{"key":"ref_16","unstructured":"Dalpiaz, F., Paja, E., and Giorgini, P. (2016). Security Requirements Engineering: Designing Secure Socio-Technical Systems, MIT Press."},{"key":"ref_17","unstructured":"Van Lamsweerde, A. (2009). Requirements Engineering: From System Goals to UML Models to Software Specifications, Wiley."},{"key":"ref_18","unstructured":"Hatebur, D., Heisel, M., and Schmidt, H. (2011, January 22\u201326). A pattern system for security requirements engineering. Proceedings of the 2011 Sixth International Conference on the Availability, Reliability and Security (ARES), Vienna, Austria."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Karpati, P., Sindre, G., and Opdahl, A.L. (2007, January 10\u201313). Characterising and analysing security requirements modelling initiatives. Proceedings of the 2011 Sixth International Conference on Availability, Reliability and Security (ARES), Vienna, Austria.","DOI":"10.1109\/ARES.2011.113"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1142\/S0218194002001062","article-title":"A synthesis of evaluation criteria for software specifications and specification techniques","volume":"12","author":"Khwaja","year":"2002","journal-title":"Int. J. Softw. Eng. Knowl. Eng."},{"key":"ref_21","unstructured":"Mayer, N. (2009). Model-Based Management of Information System Security Risk, University of Namur."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1007\/s00766-009-0092-x","article-title":"A comparison of security requirements engineering methods","volume":"15","author":"Fabian","year":"2010","journal-title":"Requir. Eng."},{"key":"ref_23","first-page":"21","article-title":"IT security and multilateral security","volume":"3","author":"Rannenberg","year":"1999","journal-title":"Multilater. Secur. Commun. Technol. Infrastruct. Econ."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Mu\u00f1ante, D., Chiprianov, V., Gallon, L., and Aniort\u00e9, P. (2014, January 8\u201312). A review of security requirements engineering methods with respect to risk analysis and model-driven engineering. Proceedings of the International Conference on Availability, Reliability, and Security, Fribourg, Switzerland.","DOI":"10.1007\/978-3-319-10975-6_6"},{"key":"ref_25","unstructured":"van Lamsweerde, A. (2004, January 23\u201328). Elaborating security requirements by construction of intentional anti-models. Proceedings of the ICSE 2004: 26th International Conference on Software Engineering, Washington, DC, USA."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Elahi, G., and Yu, E. (2007, January 5\u20139). A goal oriented approach for modeling and analyzing security trade-offs. Proceedings of the International Conference on Conceptual Modeling, Auckland, New Zealand.","DOI":"10.1007\/978-3-540-75563-0_26"},{"key":"ref_27","first-page":"1","article-title":"Reusable knowledge in security requirements engineering: A systematic mapping study","volume":"21","author":"Souag","year":"2015","journal-title":"Requir. Eng."},{"key":"ref_28","first-page":"2920","article-title":"Engineering Security into Distributed Systems: A Survey of Methodologies","volume":"18","author":"Uzunov","year":"2012","journal-title":"J. Ucs"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.csi.2010.01.006","article-title":"A systematic review of security requirements engineering","volume":"32","author":"Mellado","year":"2010","journal-title":"Comput. Stand. Interfaces"},{"key":"ref_30","unstructured":"(2016, May 27). IEEE 830 IEEE 830-1998\u2014IEEE Recommended Practice for Software Requirements Specifications. Available online: https:\/\/standards.ieee.org\/findstds\/standard\/830-1998.html."},{"key":"ref_31","unstructured":"Mead, N.R. (2021, July 12). How to Compare the Security Quality Requirements Engineering (SQUARE) Method with Other Methods, Available online: https:\/\/resources.sei.cmu.edu\/library\/asset-view.cfm?assetid=8257."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Nhlabatsi, A., Nuseibeh, B., and Yu, Y. (2012). Security requirements engineering for evolving software systems: A survey. Security-Aware Systems Applications and Software Development Methods, IGI Global.","DOI":"10.4018\/978-1-4666-1580-9.ch007"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"101852","DOI":"10.1016\/j.cose.2020.101852","article-title":"A maturity model for secure requirements engineering","volume":"95","author":"Niazi","year":"2020","journal-title":"Comput. Secur."},{"key":"ref_34","unstructured":"Sommerville, I., and Sawyer, P. (1997). Requirements Engineering: A Good Practice Guide, John Wiley & Sons, Inc."},{"key":"ref_35","unstructured":"(2021, July 12). ISO29148:2011 ISO\/IEC\/IEEE 29148:2011 Systems and Software Engineering\u2014Life Cycle Processes\u2014Requirements Engineering. Available online: https:\/\/www.iso.org\/standard\/45171.html."},{"key":"ref_36","unstructured":"ISO, I. (2009). ISO\/IEC 15408-1:2009 Information technology\u2014Security techniques\u2014Evaluation criteria for IT security\u2014Part 1: Introduction and general model. Int. Organ. Stand., Available online: https:\/\/standards.iso.org\/ittf\/PubliclyAvailableStandards\/c050341_ISO_IEC_15408-1_2009.zip."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1049\/sej.1996.0002","article-title":"Requirements engineering with viewpoints","volume":"11","author":"Kotonya","year":"1996","journal-title":"Softw. Eng. J."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"17","DOI":"10.5381\/jot.2007.6.1.c2","article-title":"Common Requirements Problems, Their Negative Consequences, and the Industry Best Practices to Help Solve Them","volume":"6","author":"Firesmith","year":"2007","journal-title":"J. Object Technol."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"77","DOI":"10.5381\/jot.2003.2.4.c7","article-title":"Specifying good requirements","volume":"2","author":"Firesmith","year":"2003","journal-title":"J. Object Technol."},{"key":"ref_40","unstructured":"Christian, T. (2010). Security Requirements Reusability and the SQUARE Methodology, Carnegie-Mellon Univ Pittsburgh Pa Software Engineering Inst."},{"key":"ref_41","first-page":"49","article-title":"From system goals to intruder anti-goals: Attack generation and resolution for security requirements engineering","volume":"3","author":"Brohez","year":"2003","journal-title":"Proc. RHAS"},{"key":"ref_42","unstructured":"Anderson, R.J. (2010). Security Engineering: A Guide to Building Dependable Distributed Systems, John Wiley & Sons."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1002\/j.2334-5837.1994.tb01679.x","article-title":"Requirements for development of software requirements","volume":"Volume 4","author":"Mar","year":"1994","journal-title":"Proceedings of the INCOSE International Symposium"},{"key":"ref_44","first-page":"44","article-title":"Writing quality requirements","volume":"7","author":"Wiegers","year":"1999","journal-title":"Softw. Dev."},{"key":"ref_45","unstructured":"Wieringa, R.J. (1996). Requirements Engineering: Frameworks for Understanding, John Wiley & Sons, Inc."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1109\/MS.1984.233702","article-title":"Verifying and validating software requirements and design specifications","volume":"1","author":"Boehm","year":"1984","journal-title":"IEEE Softw."},{"key":"ref_47","unstructured":"Pfleeger, S.L., and Atlee, J.M. (1998). Software Engineering: Theory and Practice, Pearson Education India. Available online: https:\/\/www.pearson.com\/us\/higher-education\/program\/Pfleeger-Software-Engineering-Theory-and-Practice-4th-Edition\/PGM58925.html."},{"key":"ref_48","unstructured":"Davis, A., Overmyer, S., Jordan, K., Caruso, J., Dandashi, F., Dinh, A., Kincaid, G., Ledeboer, G., Reynolds, P., and Sitaram, P. (1993, January 21\u201322). Identifying and measuring quality in a software requirements specification. Proceedings of the 1st International Software Metrics Symposium, Baltimore, MD, USA."},{"key":"ref_49","unstructured":"Young, R.R. (2004). The Requirements Engineering Handbook, Artech House."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Hull, E., Jackson, K., and Dick, J. (2010). Requirements Engineering, Springer Science & Business Media.","DOI":"10.1007\/978-1-84996-405-0"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1225","DOI":"10.1002\/j.2334-5837.1996.tb02142.x","article-title":"Requirements Management Working Group: Characteristics of Good Requirements","volume":"Volume 6","author":"Kar","year":"1996","journal-title":"Proceedings of the INCOSE International Symposium"},{"key":"ref_52","unstructured":"Zielczynski, P. (2008). Requirements Management Using IBM Rational RequisitePro, IBM Press\/Pearson plc."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1145\/224155.224157","article-title":"SMART requirements","volume":"20","author":"Mannion","year":"1995","journal-title":"ACM Sigsoft Softw. Eng. Notes"},{"key":"ref_54","unstructured":"(2021, July 12). IEEE 1233\u2014Guide for Developing System Requirements Specifications. Available online: https:\/\/ieeexplore.ieee.org\/document\/741940."},{"key":"ref_55","unstructured":"(2016, June 21). NetworkX developers NetworkX 2.1 Python Package. Available online: https:\/\/networkx.github.io\/documentation\/stable\/#."},{"key":"ref_56","unstructured":"(2016, June 21). Gephi.org Gephi 0.9.2\u2014The Open Graph Viz Platform. Available online: https:\/\/gephi.org\/."},{"key":"ref_57","unstructured":"Ahmad, S. (2012). Measuring the Effectiveness of Negotiation in Software Requirements Engineering, University of Western Australia."},{"key":"ref_58","unstructured":"(2018, September 07). David Lynas SABSA Foundation Courses Training\u2014David Lynas Consulting Limited. Available online: https:\/\/www.sabsacourses.com\/course-schedule\/."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1126\/science.103.2684.677","article-title":"On the Theory of Scales of Measurement","volume":"103","author":"Stevens","year":"1946","journal-title":"Science"},{"key":"ref_60","unstructured":"(2021, July 11). Respect-IT KAOS Tool\u2014Objectiver: HomePage. Available online: http:\/\/www.objectiver.com\/index.php?id=25."},{"key":"ref_61","unstructured":"Kletz, T.A. (1999). HAZOP and HAZAN: Identifying and Assessing Process Industry Hazards, IChemE."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Laborde, R., Bulusu, S.T., Wazan, A.S., Barr\u00e8re, F., and Benzekri, A. (2019, January 8\u201312). Logic-based methodology to help security architects in eliciting high-level network security requirements. Proceedings of the 34th ACM\/SIGAPP Symposium on Applied Computing, Limassol, Cyprus.","DOI":"10.1145\/3297280.3297437"}],"container-title":["Journal of Cybersecurity and Privacy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2624-800X\/1\/3\/22\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:30:05Z","timestamp":1760164205000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2624-800X\/1\/3\/22"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,13]]},"references-count":62,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["jcp1030022"],"URL":"https:\/\/doi.org\/10.3390\/jcp1030022","relation":{},"ISSN":["2624-800X"],"issn-type":[{"value":"2624-800X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,13]]}}}