{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,15]],"date-time":"2025-08-15T01:28:45Z","timestamp":1755221325551,"version":"3.43.0"},"reference-count":56,"publisher":"FapUNIFESP (SciELO)","license":[{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"am","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003593","name":"National Council for Scientific and Technological Development","doi-asserted-by":"crossref","award":["373182\/2024-4"],"award-info":[{"award-number":["373182\/2024-4"]}],"id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Pesqui. Oper."],"abstract":"<jats:p>This paper proposes a complementary approach to assess the performance of Industrial Engineering undergraduate programs within the Brazilian Higher Education System. This approach presents an alternative Composite Indicator formulation derived from a Data Envelopment Analysis (DEA) model and the Benefit-of-Doubt (BoD) approach. This method enhances comparability among programs across diverse Higher Education Institutions (HEIs) by optimizing weights and incorporating weight restrictions. The application of the novel approach was conducted with the analysis of 73 undergraduate programs within the Brazilian Federal Public System in 2019, the study identified both programs with suboptimal performance levels and benchmark programs showcasing best practices. This feature of the BoD model can support the HEIs assessed as suboptimal in pursuing improvement opportunities in a benchmarking exercise. The incorporation of Weight Restrictions (WRs) enables a more comprehensive consideration of key performance indicators (KPIs) within the Preliminary Program Grade (CPC) framework, thereby revealing additional improvement opportunities. This research contributes to the field by integrating DEA and BoD into the official STEM performance criteria framework for the first time, offering insights into evaluating undergraduate Industrial Engineering programs within Brazilian HEIs.<\/jats:p>","DOI":"10.1590\/0101-7438.2025.045.00294388","type":"journal-article","created":{"date-parts":[[2025,8,8]],"date-time":"2025-08-08T13:21:36Z","timestamp":1754659296000},"source":"Crossref","is-referenced-by-count":0,"title":["A COMPLEMENTARY APPROACH BASED ON DATA ENVELOPMENT ANALYSIS TO ASSESS INDUSTRIAL ENGINEERING HIGHER EDUCATION UNDERGRADUATE PROGRAMS IN BRAZIL"],"prefix":"10.1590","volume":"45","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1904-7533","authenticated-orcid":false,"given":"Renata","family":"Oliveira","sequence":"first","affiliation":[{"name":"Universidade do Estado do Par\u00e1,  Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0651-0349","authenticated-orcid":false,"given":"Andreia","family":"Zanella","sequence":"additional","affiliation":[{"name":"Universidade Federal de Santa Catarina,  Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4891-8630","authenticated-orcid":false,"given":"Vitor","family":"Martins","sequence":"additional","affiliation":[{"name":"Universidade do Estado do Par\u00e1,  Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2619-2667","authenticated-orcid":false,"given":"Nathalia","family":"Monteiro","sequence":"additional","affiliation":[{"name":"Universidade do Estado do Par\u00e1,  Brazil"}]}],"member":"530","published-online":{"date-parts":[[2025]]},"reference":[{"issue":"1","key":"ref1","article-title":"University efficiency evaluation using data envelopment analysis: future research agenda","volume":"12","author":"ALMEIDA JPLD","year":"2024","journal-title":"Cogent Education"},{"issue":"4","key":"ref2","first-page":"33","article-title":"Avalia\u00e7\u00e3o institucional na Universidade Federal do Cear\u00e1 (UFC): Organiza\u00e7\u00e3o de sistema de dados e indicadores da qualidade institucional","volume":"9","author":"ANDRIOLA WB","year":"2004","journal-title":"Avalia\u00e7\u00e3o: Revista Da Avalia\u00e7\u00e3o Da Educa\u00e7\u00e3o Superior"},{"issue":"100","key":"ref3","first-page":"645","article-title":"Uso de indicadores para diagn\u00f3stico situacional de Institui\u00e7\u00f5es de Ensino Superior","volume":"26","author":"ANDRIOLA WB","year":"2018","journal-title":"Ensaio: Avalia\u00e7\u00e3o e Pol\u00edticas P\u00fablicas Em Educa\u00e7\u00e3o"},{"issue":"204","key":"ref4","doi-asserted-by":"crossref","first-page":"83","DOI":"10.15446\/dyna.v85n204.60207","article-title":"Evaluation of post-graduate programs using a network data envelopment analysis model","volume":"85","author":"ANGULO-MEZA L","year":"2018","journal-title":"DYNA"},{"issue":"1","key":"ref5","doi-asserted-by":"crossref","first-page":"44","DOI":"10.5902\/1983465942849","article-title":"The use of adaptive project management practices and methodologies in the development of a professional doctoral program","volume":"14","author":"AZEVEDO G","year":"2021","journal-title":"Revista De Administra\u00e7\u00e3o Da Ufsm"},{"issue":"1","key":"ref6","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1007\/s11294-015-9558-4","article-title":"Measuring efficiency in higher education: An empirical study using a bootstrapped data envelopment analysis","volume":"22","author":"BARRA C","year":"2016","journal-title":"International Advances in Economic Research"},{"issue":"1","key":"ref7","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1590\/S1517-97022014000100005","article-title":"Percurso da avalia\u00e7\u00e3o da educa\u00e7\u00e3o superior nos Governos Lula","volume":"40","author":"BARREYRO GB","year":"2014","journal-title":"Educa\u00e7\u00e3o e Pesquisa"},{"issue":"2","key":"ref8","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1016\/0305-0483(90)90064-G","article-title":"Comparing university departments","volume":"18","author":"BEASLEY JE","year":"1990","journal-title":"Omega"},{"issue":"3","key":"ref9","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1590\/S1414-40772010000300008","article-title":"Mudan\u00e7as nos pesos do CPC e seu impacto nos resultados de avalia\u00e7\u00e3o em universidades federais e privadas","volume":"15","author":"BITTENCOURT HR","year":"2010","journal-title":"Avalia\u00e7\u00e3o: Revista Da Avalia\u00e7\u00e3o Da Educa\u00e7\u00e3o Superior"},{"issue":"3","key":"ref10","first-page":"171","article-title":"Confrontando avalia\u00e7\u00f5es: os resultados do Exame Nacional de Cursos e os da An\u00e1lise Envolt\u00f3ria de Dados","volume":"11","author":"BORGES RS","year":"2006","journal-title":"Revista da Rede de Avalia\u00e7\u00e3o Institucional da Educa\u00e7\u00e3o Superior"},{"key":"ref11","series-title":"Lei n. 10.861, de 14 de abril de 2004. Institui o Sistema Nacional de Avalia\u00e7\u00e3o da Educa\u00e7\u00e3o Superior - Sinaes e d\u00e1 outras provid\u00eancias","year":"2004"},{"key":"ref12","series-title":"Portaria Normativa n. 4, de 5 de agosto de 2008. Regulamenta a aplica\u00e7\u00e3o do conceito preliminar de cursos superiores - CPC, para fins dos processos de renova\u00e7\u00e3o de reconhecimento respectivos, no \u00e2mbito do ciclo avaliativo do Sinaes","year":"2008"},{"key":"ref13","series-title":"Nota t\u00e9cnica n. 45\/2019. Metodologia de c\u00e1lculo do Indicador de Diferen\u00e7a entre os Desempenhos Observado e Esperado (IDD) referente ao ano de 2018","year":"2019"},{"issue":"2","key":"ref14","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.jhlste.2012.04.003","article-title":"Two-stage performance model for evaluating the managerial efficiency of higher education: Application by the Taiwanese tourism and leisure department","volume":"11","author":"CHANG TY","year":"2012","journal-title":"Journal of Hospitality, Leisure, Sport & Tourism Education"},{"issue":"6","key":"ref15","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1016\/0377-2217(78)90138-8","article-title":"Measuring the efficiency of decision making units","volume":"2","author":"CHARNES A","year":"1978","journal-title":"European Journal of Operational Research"},{"issue":"6","key":"ref16","doi-asserted-by":"crossref","first-page":"5263","DOI":"10.1007\/s11192-021-03982-3","article-title":"Measuring the efficiency of university departments: an empirical study using data envelopment analysis and cluster analysis","volume":"126","author":"CHEN SP","year":"2021","journal-title":"Scientometrics"},{"issue":"1","key":"ref17","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1007\/s11205-006-9029-7","article-title":"An introduction to \u2019benefit of the doubt\u2019 composite indicators","volume":"82","author":"CHERCHYE L","year":"2007","journal-title":"Social Indicators Research"},{"issue":"11","key":"ref18","doi-asserted-by":"crossref","first-page":"1302","DOI":"10.1287\/mnsc.36.11.1302","article-title":"A data envelopment model for aggregating prefence rankings","volume":"36","author":"COOK WD","year":"1990","journal-title":"Management Science"},{"key":"ref19","series-title":"Introduction to Data Envelopment Analysis and Its Uses","author":"COOPER WW","year":"2006","edition":"1"},{"issue":"6","key":"ref20","doi-asserted-by":"crossref","first-page":"1197","DOI":"10.1108\/IJSHE-07-2021-0278","article-title":"An analysis of the brazilian higher educational opportunity and challenge processes to achieve the 2030 agenda for the sustainable development","volume":"24","author":"DA\u00da G","year":"2023","journal-title":"International Journal of Sustainability in Higher Education"},{"key":"ref21","article-title":"Climbing university rankings under resources constraints: a combined approach integrating DEA and directed Louvain community detection","author":"DI LEO S","year":"2024","journal-title":"Annals of Operations Research"},{"issue":"4","key":"ref22","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1016\/j.jmse.2020.10.003","article-title":"Research performance evaluation of Chinese university: A non-homogeneous network DEA approach","volume":"6","author":"DING T","year":"2021","journal-title":"Journal of Management Science and Engineering"},{"issue":"6","key":"ref23","doi-asserted-by":"crossref","first-page":"563","DOI":"10.1057\/jors.1988.96","article-title":"Reducing Weight Flexibility in Data Envelopment Analysis","volume":"39","author":"DYSON RG","year":"1988","journal-title":"Journal of the Operational Research Society"},{"issue":"2","key":"ref24","doi-asserted-by":"crossref","first-page":"956","DOI":"10.1016\/j.ejor.2006.05.041","article-title":"Avoiding infeasibility in DEA models with weight restrictions","volume":"181","author":"ESTELLITA LINS M","year":"2007","journal-title":"European Journal of Operational Research"},{"issue":"2","key":"ref25","doi-asserted-by":"crossref","first-page":"319","DOI":"10.6018\/rie.513171","article-title":"Students\u2019 experiences with graduate education in brazil a confirmatory factor analysis approach","volume":"40","author":"FELICETTI V","year":"2022","journal-title":"Revista De Investigaci\u00f3n Educativa"},{"key":"ref26","article-title":"Evaluation of efficiency of brazilian federal universities: an approach through data envelopment analysis","volume":"28","author":"FILHO P","year":"2023","journal-title":"Avalia\u00e7\u00e3o: Revista Da Avalia\u00e7\u00e3o Da Educa\u00e7\u00e3o Superior"},{"issue":"2","key":"ref27","first-page":"236","article-title":"Determining the value of undergraduate business programs from market vs academic perspectives","volume":"31","author":"FISHER S","year":"2017","journal-title":"International Journal of Educational Management"},{"key":"ref28","series-title":"Assessing the performance of organizations with the hierarchical structure using data envelopment analysis: An efficiency analysis of Farhangian University","author":"GHASEMI N","year":"2020"},{"issue":"4","key":"ref29","doi-asserted-by":"crossref","first-page":"1108","DOI":"10.14807\/ijmp.v9i4.698","article-title":"Incorporating managed preferences in the evaluation of public organizations efficiency: a DEA approach","volume":"9","author":"GONTIJO T","year":"2018","journal-title":"Independent Journal of Management & Production"},{"key":"ref30","first-page":"1","article-title":"Avalia\u00e7\u00e3o da Educa\u00e7\u00e3o Superior: como avan\u00e7ar sem desqualificar","volume":"1","author":"GRIBOSKI CM","year":"2016","journal-title":"Observat\u00f3rio Universit\u00e1rio"},{"issue":"3","key":"ref31","doi-asserted-by":"crossref","first-page":"90","DOI":"10.21810\/sfuer.v12i3.1019","article-title":"Internationalization at home, coil and intercomprehension","volume":"12","author":"GUIMAR\u00c3ES F","year":"2019","journal-title":"Sfu Educational Review"},{"issue":"109","key":"ref32","first-page":"1076","article-title":"The efficiency of public expenditure on higher education: a study with brazilian federal universities","volume":"28","author":"HAMMES D","year":"2020","journal-title":"Ensaio: Avalia\u00e7\u00e3o e Pol\u00edticas P\u00fablicas Em Educa\u00e7\u00e3o"},{"issue":"66","key":"ref33","doi-asserted-by":"crossref","first-page":"938","DOI":"10.18222\/eae.v27i66.4039","article-title":"Sobre o Conceito Preliminar de Curso: concep\u00e7\u00e3o, aplica\u00e7\u00e3o e mudan\u00e7as metodol\u00f3gicas","volume":"27","author":"IKUTA CYS","year":"2016","journal-title":"Estudos Em Avalia\u00e7\u00e3o Educacional"},{"issue":"4","key":"ref34","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1016\/j.chieco.2008.08.004","article-title":"Measuring the research performance of Chinese higher education institutions using data envelopment analysis","volume":"19","author":"JOHNES J","year":"2008","journal-title":"China Economic Review"},{"issue":"5","key":"ref35","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1111\/j.1475-3995.2011.00813.x","article-title":"Departmental efficiency differences within a Greek university: An application of a DEA and Tobit analysis","volume":"18","author":"KOUNETAS K","year":"2011","journal-title":"International Transactions in Operational Research"},{"issue":"69","key":"ref36","first-page":"748","article-title":"Conceito Preliminar de Curso: conceito \u00fanico para uma realidade educacional m\u00faltipla","volume":"28","author":"LACERDA LLV","year":"2017","journal-title":"Estudos Em Avalia\u00e7\u00e3o Educacional"},{"key":"ref37","series-title":"Handbook on Constructing Composite Indicators","author":"NARDO M","year":"2008","edition":"1"},{"issue":"1","key":"ref38","doi-asserted-by":"crossref","first-page":"25","DOI":"10.3846\/20294913.2014.837116","article-title":"Application of DEA method in efficiency evaluation of public higher education institutions","volume":"20","author":"NAZARKO J","year":"2014","journal-title":"Technological and Economic Development of Economy"},{"issue":"4","key":"ref39","doi-asserted-by":"crossref","first-page":"563","DOI":"10.3390\/sym12040563","article-title":"Using data envelopment analysis and multi-criteria decision-making methods to evaluate teacher performance in higher education","volume":"12","author":"POPOVI\u0106 M","year":"2020","journal-title":"Symmetry"},{"issue":"1","key":"ref40","doi-asserted-by":"crossref","first-page":"113","DOI":"10.20397\/2177-6652\/2019.v19i1.1380","article-title":"Incorporando julgamentos de especialistas em educa\u00e7\u00e3o na avalia\u00e7\u00e3o da efici\u00eancia de cursos de gradua\u00e7\u00e3o: uma abordagem por data envelopment analysis","volume":"19","author":"RODRIGUES AC","year":"2019","journal-title":"Revista Gest\u00e3o & Tecnologia"},{"key":"ref41","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/S0377-2217(03)00402-8","article-title":"Restricting virtual weights in data envelopment analysis","volume":"159","author":"SARRICO CS","year":"2004","journal-title":"European Journal of Operational Research"},{"issue":"38","key":"ref42","first-page":"9","article-title":"O Conceito Preliminar e as boas pr\u00e1ticas de avalia\u00e7\u00e3o do ensino superior","volume":"26","author":"SCHWARTZMAN S","year":"2008","journal-title":"Estudos Revista da Associa\u00e7\u00e3o Brasileira de Mantenedoras de Ensino Superior"},{"issue":"2","key":"ref43","doi-asserted-by":"crossref","first-page":"188","DOI":"10.5902\/1983465910916","article-title":"Avalia\u00e7\u00e3o da efici\u00eancia t\u00e9cnica dos cursos de Administra\u00e7\u00e3o no Brasil","volume":"10","author":"SOLIMAN M","year":"2017","journal-title":"Revista De Administra\u00e7\u00e3o Da Ufsm"},{"issue":"2","key":"ref44","doi-asserted-by":"crossref","first-page":"1383","DOI":"10.1007\/s10479-021-04108-x","article-title":"Interdisciplinary graduate programs: application of the MACBETH multicriteria method for assessing their performances","volume":"316","author":"TAVARES R","year":"2022","journal-title":"Annals of Operations Research"},{"key":"ref45","doi-asserted-by":"crossref","first-page":"101984","DOI":"10.1016\/j.evalprogplan.2021.101984","article-title":"A proposed multistage evaluation approach for Higher Education Institutions based on network Data Envelopment Analysis: A Brazilian experience","volume":"89","author":"TAVARES RS","year":"2021","journal-title":"Evaluation and Program Planning"},{"issue":"1-2","key":"ref46","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/0304-4076(90)90049-Y","article-title":"The role of multiplier bounds in efficiency analysis with application to Kansas farming","volume":"46","author":"THOMPSON RG","year":"1990","journal-title":"Journal of Econometrics"},{"key":"ref47","series-title":"Transforming our world: the 2030 Agenda for Sustainable Development","year":"2015"},{"issue":"2","key":"ref48","doi-asserted-by":"crossref","first-page":"11","DOI":"10.32358\/rpd.2016.v2.156","article-title":"Produ\u00e7\u00e3o cient\u00edfica dos programas de p\u00f3s gradua\u00e7\u00e3o: avalia\u00e7\u00e3o da efici\u00eancia da \u00e1rea engenharias iii","volume":"2","author":"VASCONCELOS M","year":"2016","journal-title":"Revista Produ\u00e7\u00e3o E Desenvolvimento"},{"key":"ref49","series-title":"Avalia\u00e7\u00e3o Educacional: destando e reatando n\u00f3s","first-page":"173","volume-title":"A avalia\u00e7\u00e3o do desempenho de alunos de educa\u00e7\u00e3o superior: uma an\u00e1lise a partir da experi\u00eancia do ENADE","author":"VERHINE RE","year":"2009"},{"issue":"1","key":"ref50","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1111\/1468-0106.12310","article-title":"Survey on technical efficiency in higher education: a metafractional regression analysis","volume":"26","author":"VILLANO R","year":"2019","journal-title":"Pacific Economic Review"},{"issue":"9","key":"ref51","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1057\/jors.1990.120","article-title":"Restricting Weight Flexibility in Data Envelopment Analysis","volume":"41","author":"WONG YHB","year":"1990","journal-title":"The Journal of the Operational Research Society"},{"key":"ref52","doi-asserted-by":"crossref","first-page":"101317","DOI":"10.1016\/j.seps.2022.101317","article-title":"How to allocate multi-period research resources? Centralized resource allocation for public universities in China using a parallel DEA-based approach","volume":"82","author":"XIONG X","year":"2022","journal-title":"Socio-Economic Planning Sciences"},{"issue":"1","key":"ref53","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.joi.2017.11.002","article-title":"Measuring the inefficiency of Chinese research universities based on a two-stage network DEA model","volume":"12","author":"YANG GL","year":"2018","journal-title":"Journal of Informetrics"},{"issue":"2","key":"ref54","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1016\/j.ejor.2015.03.036","article-title":"Undesirable outputs and weighting schemes in composite indicators based on data envelopment analysis","volume":"245","author":"ZANELLA A","year":"2015","journal-title":"European Journal of Operational Research"},{"key":"ref55","doi-asserted-by":"crossref","DOI":"10.5902\/2179460X66024","article-title":"Avalia\u00e7\u00e3o de desempenho na educa\u00e7\u00e3o superior: uma abordagem utilizando a An\u00e1lise Envolt\u00f3ria de Dados","volume":"43","author":"ZANELLA A","year":"2021","journal-title":"Ci\u00eancia E Natura"},{"key":"ref56","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.cogsys.2018.11.004","article-title":"Research about the university teaching performance evaluation under the data envelopment method","volume":"56","author":"ZHANG X","year":"2019","journal-title":"Cognitive Systems Research"}],"container-title":["Pesquisa Operacional"],"original-title":[],"link":[{"URL":"http:\/\/www.scielo.br\/scielo.php?script=sci_pdf&pid=S0101-74382025000100224&tlng=en","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,8]],"date-time":"2025-08-08T13:21:43Z","timestamp":1754659303000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.scielo.br\/scielo.php?script=sci_arttext&pid=S0101-74382025000100224&tlng=en"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025]]},"references-count":56,"alternative-id":["S0101-74382025000100224"],"URL":"https:\/\/doi.org\/10.1590\/0101-7438.2025.045.00294388","relation":{},"ISSN":["1678-5142","0101-7438"],"issn-type":[{"value":"1678-5142","type":"electronic"},{"value":"0101-7438","type":"print"}],"subject":[],"published":{"date-parts":[[2025]]},"article-number":"e294388"}}