{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:25:59Z","timestamp":1760059559291,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2025,6,20]],"date-time":"2025-06-20T00:00:00Z","timestamp":1750377600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Governance, Competitiveness, and Public Policy","award":["UIDB\/04058\/2020"],"award-info":[{"award-number":["UIDB\/04058\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Applied Sciences"],"abstract":"<jats:p>Industry 4.0 technologies offer significant opportunities to enhance sustainable production and circular economy practices in the face of challenges arising from climate change. Considering the growing interest in this field, the literature review exposed that, particularly in the case of the Internet of Things (IoT), there is a need for empirical assessments of the impact of this technology on sustainability and circularity. This paper presents the validation process of an original assessment tool that evaluates IoT\u2019s alignment with the socioenvironmental and circular context of manufacturing organizations and their supply chains. Emphasis is placed on the construct titled \u201cIoT Technology Expectations\u201d. After systematically conducting a literature review, this study employed the Delphi method in conjunction with statistical analyses to refine or formulate new indicators or statements based on expert consensus, validating the proposed assessment tool. The findings of this research contribute to management practices by providing an instrument to assess the current stance of top management and other key managers (production, project and supply chain) on IoT use in manufacturing operations or supply chains from a socioenvironmental and circular perspective. The instrument serves as a starting point for exploring IoT\u2019s potential in circular economy practices. Academically, it provides a detailed explanation of the Delphi method and its application and outcomes.<\/jats:p>","DOI":"10.3390\/app15136982","type":"journal-article","created":{"date-parts":[[2025,6,20]],"date-time":"2025-06-20T09:04:49Z","timestamp":1750410289000},"page":"6982","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["IoT Socioenvironmental Assessment Instrument: Validation Process Applying Delphi Method"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9437-530X","authenticated-orcid":false,"given":"Adriane","family":"Cavalieri","sequence":"first","affiliation":[{"name":"Department of Assessments and Industrial Processes (DIAPI), National Institute of Technology (INT), Ministry of Science, Technology and Innovation (MCTI), Rio de Janeiro 20081-312, Brazil"},{"name":"GOVCOPP, Department of Economics, Management, Industrial Engineering and Tourism, Aveiro University, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-9683-8311","authenticated-orcid":false,"given":"Fabio","family":"Bottacci","sequence":"additional","affiliation":[{"name":"VINCI Digital, S\u00e3o Paulo 04558-030, Brazil"}]},{"given":"Jean-Christophe","family":"De Coster","sequence":"additional","affiliation":[{"name":"Managium Consulting, Rio de Janeiro 22210-030, Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4550-4654","authenticated-orcid":false,"given":"Amarildo","family":"Fernandes","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering, Federal University of Rio de Janeiro, Rio de Janeiro 21941-909, Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5303-9409","authenticated-orcid":false,"given":"Francisco","family":"Sabbadini","sequence":"additional","affiliation":[{"name":"Department of Production Engineering, Faculty of Technology, Rio de Janeiro State University, Resende 27537-000, Brazil"},{"name":"Department of Production Engineering, School of Metallurgical Industrial Engineering of Volta Redonda, Federal Fluminense University, Volta Redonda 27255-125, Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8504-0065","authenticated-orcid":false,"given":"Jo\u00e3o","family":"Reis","sequence":"additional","affiliation":[{"name":"GOVCOPP, Department of Economics, Management, Industrial Engineering and Tourism, Aveiro University, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0901-0614","authenticated-orcid":false,"given":"Marlene","family":"Amorim","sequence":"additional","affiliation":[{"name":"GOVCOPP, Department of Economics, Management, Industrial Engineering and Tourism, Aveiro University, 3810-193 Aveiro, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1080\/09537287.2011.591619","article-title":"Sustainable Manufacturing: Trends and Research Challenges","volume":"23","author":"Garetti","year":"2012","journal-title":"Prod. Plan. Control."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Malek, J., and Desai, T.N. (2020). A Systematic Literature Review to Map Literature Focus of Sustainable Manufacturing. J. Clean. Prod., 256.","DOI":"10.1016\/j.jclepro.2020.120345"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1299","DOI":"10.1002\/csr.2129","article-title":"Shortcomings in Reporting Contributions towards the Sustainable Development Goals","volume":"28","year":"2021","journal-title":"Corp. Soc. Responsib. Environ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1002\/csr.2202","article-title":"Organizations\u2019 Engagement with Sustainable Development Goals: From cherry-picking to SDG-washing?","volume":"29","author":"Urbieta","year":"2022","journal-title":"Corp. Soc. Responsib. Environ."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2315","DOI":"10.1002\/csr.2486","article-title":"The Efforts of Manufacturing Enterprises toward Sustainable Development under Voluntary Environmental Policy: The Roles of External Pressure, Attraction, and Internal Attributes","volume":"30","author":"Li","year":"2023","journal-title":"Corp. Soc. Responsib. Environ."},{"key":"ref_6","unstructured":"United Nations General Assembly (2015). Transforming Our World: The 2030 Agenda for Sustainable Development."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1108\/eb025539","article-title":"Accounting for the Triple Bottom Line","volume":"2","author":"Elkington","year":"1998","journal-title":"Meas. Bus. Excell."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Abubakr, M., Abbas, A., Tomaz, I., Soliman, M., Luqman, M., and Hegab, H. (2020). Sustainable and Smart Manufacturing: An Integrated Approach. Sustainability, 12.","DOI":"10.3390\/su12062280"},{"key":"ref_9","unstructured":"OECD (2011). The OECD Sustainable Manufacturing Toolkit."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1007\/s10479-023-05472-6","article-title":"Smart Manufacturing as a Strategic Tool to Mitigate Sustainable Manufacturing Challenges: A Case Approach","volume":"331","author":"Kannan","year":"2023","journal-title":"Ann. Oper. Res."},{"key":"ref_11","unstructured":"Ellen MacArthur Foundation (2015). Delivering the Circular Economy: A Toolkit for Policymakers, Ellen MacArthur Foundation."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1017","DOI":"10.1108\/IJPPM-04-2020-0196","article-title":"The Current Sustainability Scenario of Industry 4.0 Enabling Technologies in Indian Manufacturing Industries","volume":"70","author":"Pasi","year":"2021","journal-title":"Int. J. Product. Perform. Manag."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Laskurain-Iturbe, I., Arana-Land\u00edn, G., Landeta-Manzano, B., and Uriarte-Gallastegi, N. (2021). Exploring the Influence of Industry 4.0 Technologies on the Circular Economy. J. Clean. Prod., 321.","DOI":"10.1016\/j.jclepro.2021.128944"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Li, Y., Dai, J., and Cui, L. (2020). The Impact of Digital Technologies on Economic and Environmental Performance in the Context of Industry 4.0: A Moderated Mediation Model. Int. J. Prod. Econ., 229.","DOI":"10.1016\/j.ijpe.2020.107777"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1108\/TQM-07-2020-0160","article-title":"Digital Supplier Selection Reinforcing Supply Chain Quality Management Systems to Enhance Firm\u2019s Performance","volume":"35","author":"Sharma","year":"2020","journal-title":"TQM J."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1154","DOI":"10.1002\/bse.4031","article-title":"Digitalization for Circular Economy and Operational Excellence Practices: A Paradox Theory Perspective on Performance Improvement in Manufacturing Firms","volume":"34","author":"Prashar","year":"2025","journal-title":"Bus. Strat. Env."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Taddei, E., Sassanelli, C., Rosa, P., and Terzi, S. (2022). Circular Supply Chains in the Era of Industry 4.0: A Systematic Literature Review. Comput. Ind. Eng., 170.","DOI":"10.1016\/j.cie.2022.108268"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ni\u017eeti\u0107, S., \u0160oli\u0107, P., L\u00f3pez-de-Ipi\u00f1a Gonz\u00e1lez-de-Artaza, D., and Patrono, L. (2020). Internet of Things (IoT): Opportunities, Issues and Challenges towards a Smart and Sustainable Future. J. Clean. Prod., 274.","DOI":"10.1016\/j.jclepro.2020.122877"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Bavaresco, R., Arruda, H., Rocha, E., Barbosa, J., and Li, G.-P. (2021). Internet of Things and Occupational Well-Being in Industry 4.0: A Systematic Mapping Study and Taxonomy. Comput. Ind. Eng., 161.","DOI":"10.1016\/j.cie.2021.107670"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bag, S., Gupta, S., and Kumar, S. (2021). Industry 4.0 Adoption and 10R Advance Manufacturing Capabilities for Sustainable Development. Int. J. Prod. Econ., 231.","DOI":"10.1016\/j.ijpe.2020.107844"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1111\/jpim.12481","article-title":"Strategies for Integrating Stakeholders into Sustainability Innovation: A Configurational Perspective","volume":"36","author":"Juntunen","year":"2019","journal-title":"J. Prod. Innov. Manag."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4719","DOI":"10.1080\/00207543.2017.1402140","article-title":"Internet of Things and Supply Chain Management: A Literature Review","volume":"57","author":"Hassini","year":"2019","journal-title":"Int. J. Prod. Res."},{"key":"ref_23","unstructured":"De Schoenmakere, M., and Gillabel, J. (2017). Circular by Design: Products in the Circular Economy."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Piscicelli, L. (2023). The Sustainability Impact of a Digital Circular Economy. Curr. Opin. Environ. Sustain., 61.","DOI":"10.1016\/j.cosust.2022.101251"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Cavalieri, A., Reis, J., and Amorim, M. (2024). Socioenvironmental Assessment and Application Process for IOT: A Comprehensive Approach. J. Clean. Prod., 436.","DOI":"10.1016\/j.jclepro.2023.140348"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Cavalieri, A., Reis, J., and Amorim, M. (2022). A Conceptual Model Proposal to Assess the Effectiveness of IoT in Sustainability Orientation in Manufacturing Industry: An Environmental and Social Focus. Appl. Sci., 12.","DOI":"10.3390\/app12115661"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Moher, D., Liberati, A., Tetzlaff, J., and Altman, D.G. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. BMJ, 339.","DOI":"10.1136\/bmj.b2535"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1177\/160940690900800406","article-title":"Building a Conceptual Framework: Philosophy, Definitions, and Procedure","volume":"8","author":"Jabareen","year":"2009","journal-title":"Int. J. Qual. Methods"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1002\/nur.20147","article-title":"The Content Validity Index: Are You Sure You Know What\u2019s Being Reported? Critique and Recommendations","volume":"29","author":"Polit","year":"2006","journal-title":"Res. Nurs. Health"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Caldarelli, G. (2024). Expert Perspectives on Blockchain in the Circular Economy: A Delphi Study with Industry Specialists. J. Clean. Prod., 465.","DOI":"10.1016\/j.jclepro.2024.142781"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2687","DOI":"10.1080\/00207540500056102","article-title":"Delphi Study on Supply Chain Flexibility","volume":"43","author":"Lummus","year":"2005","journal-title":"Int. J. Prod. Res."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1097\/ACM.0000000000001092","article-title":"Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research","volume":"91","author":"Waggoner","year":"2016","journal-title":"Acad. Med."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.im.2003.11.002","article-title":"The Delphi Method as a Research Tool: An Example, Design Considerations and Applications","volume":"42","author":"Okoli","year":"2004","journal-title":"Inf. Manag."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1016\/j.im.2013.03.003","article-title":"A Systematic Assessment of Rigor in Information Systems Ranking-Type Delphi Studies","volume":"50","author":"Cameron","year":"2013","journal-title":"Inf. Manag."},{"key":"ref_35","unstructured":"Miller, L. (2006). Determining What Could\/Should Be: The Delphi Technique and Its Application, Western Educational Research Association."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Reis, J., Rosado, D., Cohen, Y., Pousa, C., and Cavalieri, A. (2022). Green Defense Industries in the European Union: The Case of the Battle Dress Uniform for Circular Economy. Sustainability, 14.","DOI":"10.3390\/su142013018"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1080\/01488376.2011.580697","article-title":"A Comparison of Psychometric Properties and Normality in 4-, 5-, 6-, and 11-Point Likert Scales","volume":"37","author":"Leung","year":"2011","journal-title":"J. Soc. Serv. Res."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Feroz, A.K., Zo, H., and Chiravuri, A. (2021). Digital Transformation and Environmental Sustainability: A Review and Research Agenda. Sustainability, 13.","DOI":"10.3390\/su13031530"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Chang, S.-I., Chang, L.-M., and Liao, J.-C. (2020). Risk Factors of Enterprise Internal Control under the Internet of Things Governance: A Qualitative Research Approach. Inf. Manag., 57.","DOI":"10.1016\/j.im.2020.103335"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1002\/leap.1417","article-title":"Questionnaires Mentioned in Academic Research 1996\u20132019: Rapid Increase but Declining Citation Impact","volume":"35","author":"Fairclough","year":"2021","journal-title":"Learn. Publ."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Ball, H.L. (2019). Conducting Online Surveys. J. Hum. Lact., 35.","DOI":"10.1177\/0890334419848734"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1016\/S0272-6963(98)00021-7","article-title":"An Assessment of Survey Research in POM: From Constructs to Theory","volume":"16","author":"Malhotra","year":"1998","journal-title":"J. Oper. Manag."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1494","DOI":"10.1016\/j.jclepro.2019.04.049","article-title":"Eco-Innovation Pathways to a Circular Economy: Envisioning Priorities through a Delphi Approach","volume":"228","author":"Antunes","year":"2019","journal-title":"J. Clean. Prod."}],"container-title":["Applied Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3417\/15\/13\/6982\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:55:52Z","timestamp":1760032552000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3417\/15\/13\/6982"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,20]]},"references-count":43,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["app15136982"],"URL":"https:\/\/doi.org\/10.3390\/app15136982","relation":{},"ISSN":["2076-3417"],"issn-type":[{"type":"electronic","value":"2076-3417"}],"subject":[],"published":{"date-parts":[[2025,6,20]]}}}