{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T02:26:00Z","timestamp":1778034360598,"version":"3.51.4"},"reference-count":46,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,1,3]],"date-time":"2023-01-03T00:00:00Z","timestamp":1672704000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sustainability"],"abstract":"<jats:p>The implementation of an effective performance measurement system (PMS) is essential in organizations seeking continuous improvement and sustainability, especially in the context of Industry 4.0. However, a number of barriers\/obstacles hampers this effectiveness. For example, the inclusion of performance indicators without practical use, or presenting absurd results, are frequent in the industry, leading people to distrust and devalue such systems. The main research question in this study is: What are the main obstacles to effective performance measurement systems in organizations? These obstacles will be identified and categorized, and some conditions required to eliminate\/mitigate them will be explored. A systematic literature review is carried out, according to the PRISMA methodology, to identify the most frequent factors referred to as obstacles to PMS effectiveness, due to their negative influence on the implementation, use or maintenance of these systems. This review resulted in the identification of 175 obstacles that were then classified into 19 types of obstacles divided into 6 categories, whose definition was inspired by the 6M approach usually associated with Ishikawa diagrams. Then, the relationships between the obstacles are explored and investigated in order to identify interdependencies that may further influence the effectiveness of the PMS. Solutions and methodologies that can help mitigate\/eliminate their impact on PMS effectiveness are also identified.<\/jats:p>","DOI":"10.3390\/su15010867","type":"journal-article","created":{"date-parts":[[2023,1,4]],"date-time":"2023-01-04T04:28:17Z","timestamp":1672806497000},"page":"867","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Performance Measurement Systems in Continuous Improvement Environments: Obstacles to Their Effectiveness"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2313-2754","authenticated-orcid":false,"given":"Fl\u00e1vio","family":"Cunha","sequence":"first","affiliation":[{"name":"Department of Production and Systems, Algoritmi Centre, University of Minho, 4800-058 Guimar\u00e3es, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9724-0312","authenticated-orcid":false,"given":"Jos\u00e9","family":"Dinis-Carvalho","sequence":"additional","affiliation":[{"name":"Department of Production and Systems, Algoritmi Centre, University of Minho, 4800-058 Guimar\u00e3es, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2316-2793","authenticated-orcid":false,"given":"Rui M.","family":"Sousa","sequence":"additional","affiliation":[{"name":"Department of Production and Systems, Algoritmi Centre, University of Minho, 4800-058 Guimar\u00e3es, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,3]]},"reference":[{"key":"ref_1","unstructured":"Imai, M. (1997). Gemba Kaizen, McGraw-Hill. [2nd ed.]."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"959","DOI":"10.1080\/14783363.2018.1486536","article-title":"A Quality Scorecard for the era of Industry 4.0","volume":"29","author":"Shin","year":"2018","journal-title":"Total Qual. Manag. Bus. Excell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.ifacol.2018.08.227","article-title":"Developing a key performance indicators tree for lean and smart production systems","volume":"51","author":"Ante","year":"2018","journal-title":"IFAC-PapersOnLine"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Muhammad, U., Ferrer, B.R., Mohammed, W.M., and Lastra, J.L.M. (2018, January 15\u201318). An approach for implementing key performance indicators of a discrete manufacturing simulator based on the ISO 22400 standard. Proceedings of the 2018 IEEE Industrial Cyber-Physical Systems (ICPS), Saint Petersburg, Russia.","DOI":"10.1109\/ICPHYS.2018.8390779"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Womack, J., and Jones, D. (1996). Lean Thinking: Banish Waste and Create Wealth in Your Corporation, Simon & Schuster.","DOI":"10.1038\/sj.jors.2600967"},{"key":"ref_6","first-page":"235","article-title":"Toyota Inspired Excellence Models","volume":"610","author":"Macedo","year":"2021","journal-title":"IFIP Adv. Inf. Commun. Technol."},{"key":"ref_7","first-page":"241","article-title":"A literature review of lean manufacturing","volume":"8","author":"Gupta","year":"2013","journal-title":"Int. J. Manag. Sci. Eng. Manag."},{"key":"ref_8","unstructured":"Kaplan, R.S., and Norton, D.P. (1996). The Balanced Scorecard Translating Strategy into Action, Harvard Business School Press."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Rossi, A.H.G., Marcondes, G.B., Pontes, J., Leit\u00e3o, P., Treinta, F.T., De Resende, L.M.M., Mosconi, E., and Yoshino, R.T. (2022). Lean Tools in the Context of Industry 4.0: Literature Review, Implementation and Trends. Sustainability, 14.","DOI":"10.3390\/su141912295"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Nagy, J., Ol\u00e1h, J., Erdei, E., M\u00e1t\u00e9, D., and Popp, J. (2018). The Role and Impact of Industry 4.0 and the Internet of Things on the Business Strategy of the Value Chain\u2014The Case of Hungary. Sustainability, 10.","DOI":"10.3390\/su10103491"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1108\/01443579510083622","article-title":"Performance measurement system design: A literature review and research agenda","volume":"15","author":"Neely","year":"1995","journal-title":"Int. J. Oper. Prod. Manag."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bititci, U.S. (2015). Managing Business Performance, Wiley.","DOI":"10.1002\/9781119166542"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zairi, M. (1994). Measuring Performance for Business Results, Springer Science & Business Media.","DOI":"10.1007\/978-94-011-1302-1"},{"key":"ref_14","first-page":"34","article-title":"The Balanced Scorecard...the Eleventh Commandment","volume":"76","author":"McCunn","year":"1998","journal-title":"Manag. Account."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1371\/journal.pmed.1000097","article-title":"Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement","volume":"6","author":"Moher","year":"2009","journal-title":"PLoS Med."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liliana, L. (2016). A new model of Ishikawa diagram for quality assessment. IOP Conference Series: Materials Science and Engineering, IOP Publishing.","DOI":"10.1088\/1757-899X\/161\/1\/012099"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1288","DOI":"10.1108\/01443570210450329","article-title":"The success and failure of performance measurement initiatives: Perceptions of participating managers","volume":"22","author":"Bourne","year":"2002","journal-title":"Int. J. Oper. Prod. Manag."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1108\/09675421211281308","article-title":"New performance measurement and management control systems","volume":"13","author":"Watts","year":"2012","journal-title":"J. Appl. Account. Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"698","DOI":"10.1108\/00251740310496215","article-title":"Factors that play a role in \u2018managing through measures","volume":"41","author":"Franco","year":"2003","journal-title":"Manag. Decis."},{"key":"ref_20","unstructured":"Bourne, M. (2004). Handbook of Performance Measurement, GEE Publishing."},{"key":"ref_21","first-page":"614","article-title":"Modelling the barriers of supply chain performance measurement system implementation in the Indian automobile supply chain","volume":"5","author":"Charan","year":"2009","journal-title":"Int. J. Logist. Syst. Manag."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1108\/17410400910951026","article-title":"Lessons learned from performance management systems implementations","volume":"58","author":"Counet","year":"2009","journal-title":"Int. J. Product. Perform. Manag."},{"key":"ref_23","unstructured":"Franceschini, F., Galetto, M., and Maisano, D. (2007). Management by Measurement: Designing Key Indicators and Performance Measurement Systems, SpingerScience & Business Media."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1108\/01443579610125787","article-title":"The changing basis of performance measurement","volume":"16","author":"Ghalayini","year":"1996","journal-title":"Int. J. Oper. Prod. Manag."},{"key":"ref_25","unstructured":"Dixon, J.R., Nanni, A.J., and Vollmann, T.E. (1990). The New Performance Challenge, Dow Jones-Irwin."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1002\/npr.1108","article-title":"Why-and How-to Systematize Performance Measurement","volume":"20","author":"Hatten","year":"2001","journal-title":"J. Organ. Excell."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1504\/IJBPM.2003.003250","article-title":"Why some performance measurement initiatives fail: Lessons from the change management literature","volume":"5","author":"Bourne","year":"2003","journal-title":"Int. J. Bus. Perform. Manag."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1222","DOI":"10.1108\/01443570210450293","article-title":"A framework of the factors affecting the evolution of performance measurement systems","volume":"22","author":"Kennerley","year":"2002","journal-title":"Int. J. Oper. Prod. Manag."},{"key":"ref_29","first-page":"6","article-title":"Why balanced scorecards fail","volume":"2","author":"Schneiderman","year":"1999","journal-title":"J. Strateg. Perform. Meas."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1108\/13683040010362283","article-title":"Why Measurement Initiatives Fail","volume":"4","author":"Neely","year":"2000","journal-title":"Meas. Bus. Excell."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1080\/09540969509387888","article-title":"Unlocking the potential of performance measurement: A practical implementation guide","volume":"15","author":"Meekings","year":"1995","journal-title":"Public Money Manag."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"978","DOI":"10.1108\/AAAJ-04-2013-1312","article-title":"The challenges of integrated performance measurement systems: Integrating mechanisms for integrated measures","volume":"26","author":"Giovannoni","year":"2013","journal-title":"Account. Audit. Account. J."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1111\/j.1468-2370.2011.00318.x","article-title":"Performance Measurement: Challenges for Tomorrow","volume":"14","author":"Bititci","year":"2012","journal-title":"Int. J. Manag. Rev."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1177\/01708406030247003","article-title":"Performance Measures and the Rationalization of Organizations","volume":"24","author":"Townley","year":"2003","journal-title":"Organ. Stud."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"731","DOI":"10.1111\/ijmr.12184","article-title":"Performance measurement and management systems: A perspective from complexity theory","volume":"20","author":"Okwir","year":"2018","journal-title":"Int. J. Manag. Rev."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/S0377-2217(02)00918-9","article-title":"Designing a performance measurement system: A case study","volume":"156","author":"Lohman","year":"2004","journal-title":"Eur. J. Oper. Res."},{"key":"ref_37","unstructured":"Brown, M.G. (1996). Keeping Score: Using the Right Metrics to Drive World-Class Performance, CRC Press."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"231","DOI":"10.2478\/sbe-2019-0018","article-title":"Performance Management Systems\u2014Proposing and Testing a Conceptual Model","volume":"14","author":"Mihaela","year":"2019","journal-title":"Stud. Bus. Econ."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/j.cie.2010.11.010","article-title":"State of the art literature review on performance measurement","volume":"60","author":"Nudurupati","year":"2011","journal-title":"Comput. Ind. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"488","DOI":"10.1016\/j.aos.2007.05.002","article-title":"Developing performance-measurement systems as enabling formalization: A longitudinal field study of a logistics department","volume":"33","author":"Wouters","year":"2008","journal-title":"Account. Organ. Soc."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1108\/14635770610644628","article-title":"Performance measures in English small and medium enterprises: Survey results","volume":"13","author":"Sousa","year":"2006","journal-title":"Benchmarking Int. J."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"3","DOI":"10.2307\/3380448","article-title":"Recognizing Management Technique Dysfunctions: How Management Tools Often Create More Problems than They Solve","volume":"10","author":"Gabris","year":"1986","journal-title":"Public Product. Rev."},{"key":"ref_43","first-page":"131","article-title":"Vision of the balanced Scorecard in micro, small and medium enterprises","volume":"14","author":"Galdamez","year":"2019","journal-title":"Sist. Gest\u00e3o"},{"key":"ref_44","unstructured":"Simons, R. (1995). Control in an Age of Empowerment, Harvard Business Review Press."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1080\/09537280512331333011","article-title":"Researching performance measurement system implementation: The dynamics of success and failure","volume":"16","author":"Bourne","year":"2005","journal-title":"Prod. Plan. Control"},{"key":"ref_46","unstructured":"Amaro, P. (2022). The Integration of Lean Thinking in the Culture of Portuguese Organizations: Enablers and Inhibitors, University of Minho."}],"container-title":["Sustainability"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2071-1050\/15\/1\/867\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T17:57:40Z","timestamp":1760119060000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2071-1050\/15\/1\/867"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,3]]},"references-count":46,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["su15010867"],"URL":"https:\/\/doi.org\/10.3390\/su15010867","relation":{},"ISSN":["2071-1050"],"issn-type":[{"value":"2071-1050","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,3]]}}}