{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,17]],"date-time":"2026-01-17T08:44:18Z","timestamp":1768639458689,"version":"3.49.0"},"reference-count":103,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T00:00:00Z","timestamp":1690329600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Software"],"abstract":"<jats:p>This study compares the reliability of estimation, productivity, and defect rate metrics for sprints driven by a specific instance of the agile approach (i.e., scrum) and an agile model-Bbased software engineering (MBSE) approach called the integrated Scrum Model-Based System Architecture Process (sMBSAP) when developing a software system. The quasi-experimental study conducted ten sprints using each approach. The approaches were then evaluated based on their effectiveness in helping the product development team estimate the backlog items that they could build during a time-boxed sprint and deliver more product backlog items (PBI) with fewer defects. The commitment reliability (CR) was calculated to compare the reliability of estimation with a measured average scrum-driven value of 0.81 versus a statistically different average sMBSAP-driven value of 0.94. Similarly, the average sprint velocity (SV) for the scrum-driven sprints was 26.8 versus 31.8 for the MBSAP-driven sprints. The average defect density (DD) for the scrum-driven sprints was 0.91, while that of the sMBSAP-driven sprints was 0.63. The average defect leakage (DL) for the scrum-driven sprints was 0.20, while that of the sMBSAP-driven sprints was 0.15. The t-test analysis concluded that the sMBSAP-driven sprints were associated with a statistically significant larger mean CR, SV, DD, and DL than that of the scrum-driven sprints. The overall results demonstrate formal quantitative benefits of an agile MBSE approach compared to an agile alone, thereby strengthening the case for considering agile MBSE methods within the software development community. Future work might include comparing agile and agile MBSE methods using alternative research designs and further software development objectives, techniques, and metrics.<\/jats:p>","DOI":"10.3390\/software2030015","type":"journal-article","created":{"date-parts":[[2023,7,27]],"date-time":"2023-07-27T02:14:48Z","timestamp":1690424088000},"page":"310-331","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Comparing Measured Agile Software Development Metrics Using an Agile Model-Based Software Engineering Approach versus Scrum Only"],"prefix":"10.3390","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5189-7725","authenticated-orcid":false,"given":"Moe","family":"Huss","sequence":"first","affiliation":[{"name":"Department of Systems Engineering, Walter Scott, Jr. College of Engineering, Colorado State University, Fort Collins, CO 80523, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4995-7375","authenticated-orcid":false,"given":"Daniel R.","family":"Herber","sequence":"additional","affiliation":[{"name":"Department of Systems Engineering, Walter Scott, Jr. College of Engineering, Colorado State University, Fort Collins, CO 80523, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0833-0130","authenticated-orcid":false,"given":"John M.","family":"Borky","sequence":"additional","affiliation":[{"name":"Department of Systems Engineering, Walter Scott, Jr. College of Engineering, Colorado State University, Fort Collins, CO 80523, USA"}]}],"member":"1968","published-online":{"date-parts":[[2023,7,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1016\/j.procs.2010.12.144","article-title":"The role of risk management in IT systems of organizations","volume":"3","author":"Tohidi","year":"2011","journal-title":"Procedia Comput. Sci."},{"key":"ref_2","unstructured":"Standish Group (2020). CHAOS Report 2020, Standish Group."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Altahtooh, U.A., and Emsley, M.W. (2014, January 6\u20139). Is a Challenged Project One of the Final Outcomes for an IT Project?. Proceedings of the 2014 47th Hawaii International Conference on System Sciences, Waikoloa, HI, USA.","DOI":"10.1109\/HICSS.2014.531"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1108\/ITP-09-2013-0167","article-title":"Influence processes of implementation effectiveness in challenged information technology projects in Africa","volume":"27","author":"Kandiri","year":"2014","journal-title":"Inf. Technol. People"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/S0263-7863(01)00075-8","article-title":"Critical failure factors in information system projects","volume":"20","author":"Yeo","year":"2022","journal-title":"Int. J. Proj. Manag."},{"key":"ref_6","first-page":"2","article-title":"Delivering large-scale IT projects on time, on budget, and on value","volume":"5","author":"Bloch","year":"2012","journal-title":"Harv. Bus. Rev."},{"key":"ref_7","first-page":"23","article-title":"Why Your IT Project May Be Riskier Than You Think","volume":"89","author":"Flyvbjerg","year":"2011","journal-title":"Harv. Bus. Rev."},{"key":"ref_8","unstructured":"Eadicicco, L., Peckham, M., Pullen, J.P., and Fitzpatrick, A. (2019, September 15). Time The 20 Most Successful Tech Failures of All Time. Time. Available online: https:\/\/time.com\/4704250\/most-successful-technology-tech-failures-gadgets-flops-bombs-fails\/."},{"key":"ref_9","unstructured":"Martin, J. (1991). Rapid Application Development, Macmillan Publishing Co., Inc."},{"key":"ref_10","unstructured":"Ambler, S. (2023, March 10). The Agile Unified Process (Aup). Available online: http:\/\/www.ambysoft.com\/unifiedprocess\/agileUP.html."},{"key":"ref_11","unstructured":"Turner, R. (2007). Toward Agile systems engineering processes. J. Def. Softw. Eng., 11\u201315. Available online: https:\/\/citeseerx.ist.psu.edu\/document?repid=rep1&type=pdf&doi=b807d8efe4acd265187a98357b19c063b5a9ad5e."},{"key":"ref_12","unstructured":"Schwaber, K. (1997). Business Object Design And Implementation, Springer. OOPSLA \u201995 Workshop Proceedings, Austin, TX, USA, 16 October 1995."},{"key":"ref_13","first-page":"1","article-title":"Issues and challenges in Scrum implementation","volume":"3","author":"Akif","year":"2012","journal-title":"Int. J. Sci. Eng. Res."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Cardozo, E.S.F., Ara\u00fajo Neto, J.B.F., Barza, A., Fran\u00e7a, A.C.C., and da Silva, F.Q.B. (2010, January 12\u201313). Scrum and productivity in software projects: A systematic literature review. Proceedings of the International Conference on Evaluation and Assessment in Software Engineering, Keele, UK.","DOI":"10.14236\/ewic\/EASE2010.16"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lehman, T., and Sharma, A. (April, January 29). Software development as a service: Agile experiences. Proceedings of the 2011 Annual SRII Global Conference, San Jose, CA, USA.","DOI":"10.1109\/SRII.2011.82"},{"key":"ref_16","unstructured":"(2023). Systems and Software Engineering\u2014System Life Cycle Processes (Standard No. ISO\/IEC\/IEEE 15288:2023)."},{"key":"ref_17","unstructured":"Dove, R., and Schindel, W. (2020, April 12). Agility in Systems Engineering\u2014Findings from Recent Studies; Working Paper; Paradigm Shift International and ICTT System Sciences: 2017. Available online: http:\/\/www.parshift.com\/s\/ASELCM-AgilityInSE-RecentFindings.pdf."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1002\/sys.21566","article-title":"Value and benefits of model-based systems engineering (MBSE): Evidence from the literature","volume":"24","author":"Henderson","year":"2021","journal-title":"Syst. Eng."},{"key":"ref_19","unstructured":"Douglass, B.P. (2021). Agile Model-Based Systems Engineering Cookbook, Packt Publishing. [1st ed.]."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3701","DOI":"10.1017\/dsi.2019.377","article-title":"Munich Agile MBSE concept (MAGIC)","volume":"1","author":"Salehi","year":"2019","journal-title":"Int. Conf. Eng. Des."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1016\/j.procir.2021.05.101","article-title":"Methodology for iterative system modeling in Agile product development","volume":"100","author":"Riesener","year":"2021","journal-title":"Procedia CIRP"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1080\/10429247.2019.1659701","article-title":"An analysis of theories supporting Agile Scrum and the use of Scrum in systems engineering","volume":"32","author":"Bott","year":"2020","journal-title":"Eng. Manag. J."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ciampa, P.D., and Nagel, B. (2021, January 2\u20136). Accelerating the development of complex systems in aeronautics via MBSE and MDAO: A roadmap to agility. Proceedings of the AIAA AVIATION Forum and Exposition, Virtual Event. Number AIAA 2021-3056.","DOI":"10.2514\/6.2021-3056"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"234","DOI":"10.3390\/software2020011","article-title":"An Agile model-based software engineering approach illustrated through the development of a health technology system","volume":"2","author":"Huss","year":"2023","journal-title":"Software"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Borky, J.M., and Bradley, T.H. (2019). Effective Model-Based Systems Engineering, Springer.","DOI":"10.1007\/978-3-319-95669-5"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Schwaber, K. (2004). Agile Project Management with Scrum, Microsoft Press.","DOI":"10.1007\/11499053_47"},{"key":"ref_27","first-page":"1","article-title":"Issues in Scrum Agile development principles and practices in software development","volume":"8","author":"Anand","year":"2015","journal-title":"Indian J. Sci. Technol."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Brambilla, M., Cabot, J., and Wimmer, M. (2017). Model-Driven Software Engineering in Practice, Springer.","DOI":"10.1007\/978-3-031-02549-5"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Bonnet, S., Voirin, J.L., Normand, V., and Exertier, D. (2015, January 13\u201316). Implementing the MBSE cultural change: Organization, coaching and lessons learned. Proceedings of the INCOSE International Symposium, Seattle, WA, USA.","DOI":"10.1002\/j.2334-5837.2015.00078.x"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Azhar, D., Mendes, E., and Riddle, P. (2012, January 21\u201322). A systematic review of web resource estimation. Proceedings of the International Conference on Predictive Models in Software Engineering, Lund, Sweden.","DOI":"10.1145\/2365324.2365332"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/TSE.2007.256943","article-title":"A systematic review of software development cost estimation studies","volume":"33","author":"Jorgensen","year":"2006","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_32","unstructured":"Molokken, K., and Jorgensen, M. (October, January 30). A review of software surveys on software effort estimation. Proceedings of the International Symposium on Empirical Software Engineering, Rome, Italy."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.infsof.2011.09.002","article-title":"Systematic literature review of machine learning based software development effort estimation models","volume":"54","author":"Wen","year":"2012","journal-title":"Inf. Softw. Technol."},{"key":"ref_34","unstructured":"Cohn, M. (2005). Agile Estimating and Planning, Pearson Education."},{"key":"ref_35","unstructured":"Grenning, J. (2002). Planning Poker or How to Avoid Analysis Paralysis While Release Planning, Renaissance Software Consulting. White Paper."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2086","DOI":"10.1016\/j.jss.2012.04.005","article-title":"On using planning poker for estimating user stories","volume":"85","author":"Hovelja","year":"2012","journal-title":"J. Syst. Softw."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Dalton, J. (2019). Great Big Agile, Springer.","DOI":"10.1007\/978-1-4842-4206-3"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Haugen, N.C. (2006, January 23\u201328). An empirical study of using planning poker for user story estimation. Proceedings of the AGILE, Minneapolis, MN, USA.","DOI":"10.1109\/AGILE.2006.16"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2106","DOI":"10.1016\/j.jss.2008.03.058","article-title":"Using planning poker for combining expert estimates in software projects","volume":"81","author":"Haugen","year":"2008","journal-title":"J. Syst. Softw."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Molokken-\u00d8stvold, K., and Haugen, N.C. (2007, January 10\u201313). Combining estimates with planning poker\u2013An empirical study. Proceedings of the Australian Software Engineering Conference, Melbourne, Australia.","DOI":"10.1109\/ASWEC.2007.15"},{"key":"ref_41","unstructured":"Mol\u00f8kken-\u00d8stvold, K., J\u00f8rgensen, M., Tanilkan, S.S., Gallis, H., Lien, A.C., and Hove, S.W. (2004, January 11\u201317). A survey on software estimation in the Norwegian industry. Proceedings of the International Symposium on Software Metrics, Chicago, IL, USA."},{"key":"ref_42","unstructured":"Tiwari, M. (2020, April 12). Understanding Scrum Metrics and KPIs. Online, 2019. Medium. Available online: https:\/\/medium.com\/@techiemanoj4\/understanding-scrum-metrics-and-kpis-fcf56833d488."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1109\/MS.2018.4321245","article-title":"To transform to have agility, dont do a capital A, capital T Agile transformation","volume":"35","author":"Smart","year":"2018","journal-title":"IEEE Softw."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1007\/s42107-018-0075-7","article-title":"Forecasting schedule reliability using the reliability of agents\u2019 promises","volume":"19","author":"Khanzadi","year":"2018","journal-title":"Asian J. Civ. Eng."},{"key":"ref_45","first-page":"34","article-title":"Introducing lean into the UK Highways Agency\u2019s supply chain","volume":"165","author":"Chen","year":"2012","journal-title":"Proc. Inst. Civ. Eng.-Civ. Eng."},{"key":"ref_46","unstructured":"Wagner, S., and Ruhe, M. (2018). A systematic review of productivity factors in software development. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1109\/52.820015","article-title":"Benchmarking software development productivity","volume":"17","author":"Maxwell","year":"2000","journal-title":"IEEE Softw."},{"key":"ref_48","unstructured":"Brodbeck, F.C. (1994). Produktivit\u00e4t und Qualit\u00e4t in Software-Projekten: Psychologische Analyse und Optimierung von Arbeitsprozessen in der Software-Entwicklung, Oldenbourg."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1023","DOI":"10.1109\/TSE.2004.104","article-title":"Software productivity measurement using multiple size measures","volume":"30","author":"Kitchenham","year":"2004","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Huijgens, H., and Solingen, R.v. (2014, January 3). A replicated study on correlating agile team velocity measured in function and story points. Proceedings of the International Workshop on Emerging Trends in Software Metrics, Hyderabad, India.","DOI":"10.1145\/2593868.2593874"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Alleman, G.B., Henderson, M., and Seggelke, R. (2003, January 25\u201328). Making Agile development work in a government contracting environment-measuring velocity with earned value. Proceedings of the Agile Development Conference, Salt Lake City, UT, USA.","DOI":"10.1109\/ADC.2003.1231460"},{"key":"ref_52","unstructured":"Javdani, T., Zulzalil, H., Ghani, A.A.A., Sultan, A.B.M., and Parizi, R.M. (2013). On the current measurement practices in Agile software development. arXiv."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/j.infsof.2015.02.005","article-title":"Using metrics in Agile and lean software development\u2013a systematic literature review of industrial studies","volume":"62","author":"Kupiainen","year":"2015","journal-title":"Inf. Softw. Technol."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1145\/3345626","article-title":"Velocity in software engineering","volume":"62","author":"Killalea","year":"2019","journal-title":"Commun. ACM"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Sharma, S., Kumar, D., and Fayad, M.E. (2021, January 3\u20134). An impact assessment of Agile ceremonies on sprint velocity under Agile software development. Proceedings of the International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions), Noida, India.","DOI":"10.1109\/ICRITO51393.2021.9596508"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"776","DOI":"10.1002\/smr.1661","article-title":"Understanding sprint velocity fluctuations for improved project plans with Scrum: A case study","volume":"26","author":"Caballero","year":"2014","journal-title":"J. Softw. Evol. Process."},{"key":"ref_57","unstructured":"(1990). IEEE Standard Glossary of Software Engineering Terminology (Standard No. IEEE Std 610 12)."},{"key":"ref_58","unstructured":"Tripathy, P., and Naik, K. (2011). Software Testing and Quality Assurance: Theory and Practice, John Wiley & Sons."},{"key":"ref_59","unstructured":"McDonald, M., Musson, R., and Smith, R. (2007). The Practical Guide to Defect Prevention, Microsoft Press."},{"key":"ref_60","unstructured":"Jones, C., and Bonsignour, O. (2011). The Economics of Software Quality, Addison-Wesley Professional."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1109\/2.976920","article-title":"Get ready for Agile methods, with care","volume":"35","author":"Boehm","year":"2002","journal-title":"Computer"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1109\/2.962984","article-title":"Top 10 list [software development]","volume":"34","author":"Boehm","year":"2001","journal-title":"Computer"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1109\/TSE.2010.90","article-title":"A general software defect-proneness prediction framework","volume":"37","author":"Song","year":"2010","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_64","unstructured":"Mohagheghi, P., Conradi, R., Killi, O.M., and Schwarz, H. (2004, January 30). An empirical study of software reuse vs. defect-density and stability. Proceedings of the International Conference on Software Engineering, Beijing, China."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Nagappan, N., and Ball, T. (2005, January 15\u201321). Use of relative code churn measures to predict system defect density. Proceedings of the International Conference on Software Engineering, St. Louis, MO, USA.","DOI":"10.1145\/1062455.1062514"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Shah, S.M.A., Morisio, M., and Torchiano, M. (2012, January 4\u20137). An overview of software defect density: A scoping study. Proceedings of the Asia-Pacific Software Engineering Conference, Hong Kong, China.","DOI":"10.1109\/APSEC.2012.93"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Mijumbi, R., Okumoto, K., Asthana, A., and Meekel, J. (2018, January 15\u201318). Recent advances in software reliability assurance. Proceedings of the IEEE International Symposium on Software Reliability Engineering Workshops, Memphis, TN, USA.","DOI":"10.1109\/ISSREW.2018.00-27"},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"Fehlmann, T.M., and Kranich, E. (2014, January 9\u201311). Measuring defects in a six sigma context. Proceedings of the International Conference on Lean Six Sigma, Durgapur, India.","DOI":"10.1007\/978-1-4471-5583-6_2"},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Li, J., Moe, N.B., and Dyb\u00e5, T. (2010, January 16\u201317). Transition from a plan-driven process to Scrum: A longitudinal case study on software quality. Proceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, Bolzano-Bozen, Italy.","DOI":"10.1145\/1852786.1852804"},{"key":"ref_70","unstructured":"Collofello, J.S., Yang, Z., Tvedt, J.D., Merrill, D., and Rus, I. (1996, January 27\u201329). Modeling software testing processes. Proceedings of the IEEE Fifteenth Annual International Phoenix Conference on Computers and Communications, Scottsdale, AZ, USA."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Jacob, P.M., and Prasanna, M. (2016, January 11\u201314). A comparative analysis on black box testing strategies. Proceedings of the International Conference on Information Science, Dublin, Ireland.","DOI":"10.1109\/INFOSCI.2016.7845290"},{"key":"ref_72","first-page":"288","article-title":"Software defect prediction models for quality improvement: A literature study","volume":"9","author":"Rawat","year":"2012","journal-title":"Int. J. Comput. Sci. Issues"},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"384","DOI":"10.4236\/jsea.2015.88038","article-title":"A framework for software defect prediction using neural networks","volume":"8","author":"Vashisht","year":"2015","journal-title":"J. Softw. Eng. Appl."},{"key":"ref_74","doi-asserted-by":"crossref","unstructured":"Carroll, E.R., and Malins, R.J. (2016). Systematic Literature Review: HOW Is Model-Based Systems Engineering Justified?, Sandia National Laboratories.","DOI":"10.2172\/1561164"},{"key":"ref_75","unstructured":"Friedenthal, S., Moore, A., and Steiner, R. (2014). A Practical Guide to SysML, Morgan Kaufmann."},{"key":"ref_76","unstructured":"Campbell, D.T., and Stanley, J.C. (1963). Experimental and Quasi-Experimental Designs for Research, Houghton Mifflin Company."},{"key":"ref_77","unstructured":"Reichardt, C.S. (2019). Quasi-Experimentation: A Guide to Design and Analysis, Guilford Publications."},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"15242","DOI":"10.1073\/pnas.1301687110","article-title":"The Hawthorne effect and energy awareness","volume":"110","author":"Schwartz","year":"2013","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_79","unstructured":"Satpathy, T. (2016). A Guide to the Scrum Body of Knowledge (SBOK\u2122 Guide), SCRUMstudy. [3rd ed.]."},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Buffardi, K., Robb, C., and Rahn, D. (2017, January 3\u20135). Learning agile with tech startup software engineering projects. Proceedings of the ACM Conference on Innovation and Technology in Computer Science Education, Bologna, Italy.","DOI":"10.1145\/3059009.3059063"},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1016\/j.jbusres.2018.06.013","article-title":"Agile business model innovation in digital entrepreneurship: Lean startup approaches","volume":"110","author":"Ghezzi","year":"2020","journal-title":"J. Bus. Res."},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Kuchta, D. (2018, January 17\u201318). Combination of the earned value method and the Agile approach\u2014A case study of a production system implementation. Proceedings of the Intelligent Systems in Production Engineering and Maintenance, Wroclaw, Poland.","DOI":"10.1007\/978-3-319-97490-3_9"},{"key":"ref_83","doi-asserted-by":"crossref","unstructured":"Tamrakar, R., and J\u00f8rgensen, M. (2012, January 14\u201315). Does the use of Fibonacci numbers in planning poker affect effort estimates?. Proceedings of the International Conference on Evaluation & Assessment in Software Engineering, Ciudad Real, Spain.","DOI":"10.1049\/ic.2012.0030"},{"key":"ref_84","unstructured":"(2023, February 28). ClickUp\u2122. Available online: https:\/\/www.clickup.com."},{"key":"ref_85","unstructured":"(2023, February 28). Powerbi\u2122. Available online: https:\/\/powerbi.microsoft.com\/en-us\/."},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"5","DOI":"10.11648\/j.ajtas.20160501.12","article-title":"A brief review of tests for normality","volume":"5","author":"Das","year":"2016","journal-title":"Am. J. Theor. Appl. Stat."},{"key":"ref_87","unstructured":"Rovai, A.P., Baker, J.D., and Ponton, M.K. (2013). Social Science Research Design and Statistics, Watertree Press."},{"key":"ref_88","doi-asserted-by":"crossref","first-page":"2706","DOI":"10.1016\/j.jspi.2006.04.011","article-title":"How robust are tests for two independent samples?","volume":"137","author":"Rasch","year":"2007","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_89","unstructured":"(2023, February 28). Prism\u2122. Available online: https:\/\/www.graphpad.com\/."},{"key":"ref_90","unstructured":"Saunders, M., Lewis, P., and Thornhill, A. (2019). Research Methods, Pearson Education. [8th ed.]."},{"key":"ref_91","unstructured":"Cook, T.D., and Reichardt, C.S. (1979). Qualitative and Quantitative Methods in Evaluation Research, Sage Publications."},{"key":"ref_92","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1097\/EDE.0000000000001161","article-title":"A graphical catalog of threats to validity: Linking social science with epidemiology","volume":"31","author":"Matthay","year":"2020","journal-title":"Epidemiology"},{"key":"ref_93","unstructured":"Howell, D.C. (2011). International Encyclopedia of Statistical Science, Springer."},{"key":"ref_94","unstructured":"Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics, Sage."},{"key":"ref_95","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.infsof.2008.04.006","article-title":"A systematic review of quasi-experiments in software engineering","volume":"51","author":"Kampenes","year":"2009","journal-title":"Inf. Softw. Technol."},{"key":"ref_96","unstructured":"Gall, M.D., Gall, J.P., and Borg, W.R. (2007). Educational Research: An Introduction, Pearson."},{"key":"ref_97","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1037\/1082-989X.10.4.444","article-title":"Is the meta-analysis of correlation coefficients accurate when population correlations vary?","volume":"10","author":"Field","year":"2005","journal-title":"Psychol. Methods"},{"key":"ref_98","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1214\/09-STS301","article-title":"The impact of Levene\u2019s test of equality of variances on statistical theory and practice","volume":"24","author":"Gastwirth","year":"2009","journal-title":"Stat. Sci."},{"key":"ref_99","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1027\/1614-2241\/a000016","article-title":"Is it really robust?","volume":"6","author":"Schmider","year":"2010","journal-title":"Methodology"},{"key":"ref_100","unstructured":"(2023, February 28). Microsoft 365. Available online: https:\/\/www.office.com."},{"key":"ref_101","unstructured":"(2023, February 28). Diagrams.net. Draw.io. Available online: https:\/\/draw.io."},{"key":"ref_102","unstructured":"Sparx Systems (2023, February 28). Enterprise Architect 15.2 User Guide. Available online: https:\/\/sparxsystems.com\/enterprise_architect_user_guide\/15.2."},{"key":"ref_103","doi-asserted-by":"crossref","unstructured":"Porru, S., Pinna, A., Marchesi, M., and Tonelli, R. (2017, January 20\u201328). Blockchain-oriented software engineering: Challenges and new directions. Proceedings of the 2017 IEEE\/ACM 39th International Conference On Software Engineering Companion (ICSE-C), Buenos Aires, Argentina.","DOI":"10.1109\/ICSE-C.2017.142"}],"container-title":["Software"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2674-113X\/2\/3\/15\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:19:41Z","timestamp":1760127581000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2674-113X\/2\/3\/15"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,26]]},"references-count":103,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["software2030015"],"URL":"https:\/\/doi.org\/10.3390\/software2030015","relation":{},"ISSN":["2674-113X"],"issn-type":[{"value":"2674-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,26]]}}}