{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T12:53:59Z","timestamp":1782478439108,"version":"3.54.5"},"reference-count":60,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,6,2]],"date-time":"2020-06-02T00:00:00Z","timestamp":1591056000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>The development of information and communication technologies has revolutionized and changed the way we do business in various areas. The field of education did not remain immune to the mentioned changes; there was a gradual integration of the educational process and the mentioned technologies. As a result, platforms for distance learning, as well as the organization of e-learning courses of various types, have been developed. The rapid development of e-learning courses has led to the problem of e-learning course selection and evaluation. The problem of the e-learning course selection can be successfully solved by using multiple-criteria decision-making (MCDM) methods. Therefore, the aim of the paper is to propose an integrated approach based on the MCDM methods and symmetry principles for e-learning course selection. The pivot pairwise relative criteria importance assessment (PIPRECIA) method is used for determining the weights of criteria, and the interval-valued triangular fuzzy additive ratio assessment (ARAS) method is used for the ranking of alternatives i.e., e-learning courses. The suitability of the proposed integrated model is demonstrated through a numerical case study.<\/jats:p>","DOI":"10.3390\/sym12060928","type":"journal-article","created":{"date-parts":[[2020,6,3]],"date-time":"2020-06-03T04:12:09Z","timestamp":1591157529000},"page":"928","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":62,"title":["A Novel Integrated PIPRECIA\u2013Interval-Valued Triangular Fuzzy ARAS Model: E-Learning Course Selection"],"prefix":"10.3390","volume":"12","author":[{"given":"Kristina","family":"Jaukovic Jocic","sequence":"first","affiliation":[{"name":"Faculty of Applied Management, Economics and Finance, University Business Academy in Novi Sad, Jevrejska 24, 11000 Belgrade, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Goran","family":"Jocic","sequence":"additional","affiliation":[{"name":"Faculty of Applied Management, Economics and Finance, University Business Academy in Novi Sad, Jevrejska 24, 11000 Belgrade, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Darjan","family":"Karabasevic","sequence":"additional","affiliation":[{"name":"Faculty of Applied Management, Economics and Finance, University Business Academy in Novi Sad, Jevrejska 24, 11000 Belgrade, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gabrijela","family":"Popovic","sequence":"additional","affiliation":[{"name":"Faculty of Applied Management, Economics and Finance, University Business Academy in Novi Sad, Jevrejska 24, 11000 Belgrade, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6846-3074","authenticated-orcid":false,"given":"Dragisa","family":"Stanujkic","sequence":"additional","affiliation":[{"name":"Technical Faculty in Bor, University of Belgrade, Vojske Jugoslavije 12, 19210 Bor, Serbia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3201-949X","authenticated-orcid":false,"given":"Edmundas Kazimieras","family":"Zavadskas","sequence":"additional","affiliation":[{"name":"Institute of Sustainable Construction, Vilnius Gediminas Technical University, LT 10223 Vilnius, Lithuania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6180-2520","authenticated-orcid":false,"given":"Phong","family":"Thanh Nguyen","sequence":"additional","affiliation":[{"name":"Department of Project Management, Ho Chi Minh City Open University, Ho Chi Minh City 7000000, Vietnam"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,2]]},"reference":[{"key":"ref_1","unstructured":"Grimus, M. (2007). ICT and Creative Computing, Austrian Perspective in Teacher Education. Educational Technology: Opportunities and Challenges, University of Oulu."},{"key":"ref_2","first-page":"281","article-title":"ICT in Education Sector and its Impacts","volume":"7","author":"Trehan","year":"2019","journal-title":"Int. J. Manag. IT Eng."},{"key":"ref_3","unstructured":"Plomp, T., Anderson, R.E., Law, N., and Quale, A. (2009). CrossNational Information and Communication Technology Policies and Practices in Education, IAP. [2nd ed.]."},{"key":"ref_4","unstructured":"Pulkkinen, J. (2007). Cultural Globalization and Integration of ICT in Education. Educational Technology: Opportunities and Challenges, University of Oulu."},{"key":"ref_5","first-page":"148","article-title":"Systematic planning for ICT integration in topic learning","volume":"10","author":"Wang","year":"2007","journal-title":"J. Educ. Technol. Soc."},{"key":"ref_6","first-page":"230","article-title":"Integrating ICT at the Faculty Level: A Case Study","volume":"10","author":"Altun","year":"2011","journal-title":"Turkish Online J. Educ. Technol. -TOJET"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Garc\u00eda-Alcaraz, P., Mart\u00ednez-Loya, V., Garc\u00eda-Alcaraz, J.L., and S\u00e1nchez-Ram\u00edrez, C. (2019). The role of ICT in educational innovation. Managing Innovation in Highly Restrictive Environments, Springer.","DOI":"10.1007\/978-3-319-93716-8_7"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1080\/21532974.2011.10784681","article-title":"Creating Technology-Enhanced, Learner-Centered Classrooms","volume":"28","author":"An","year":"2011","journal-title":"J. Digit. Learn. Teach. Educ."},{"key":"ref_9","first-page":"2003","article-title":"E-learning continuance satisfaction in higher education: A unified perspective from instructors and students","volume":"43","author":"Teng","year":"2017","journal-title":"Stud. High. Educ."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1016\/j.compedu.2015.11.008","article-title":"The effects of integrating mobile devices with teaching and learning on students\u2019 learning performance: A meta-analysis and research synthesis","volume":"94","author":"Sung","year":"2016","journal-title":"Comput. Educ."},{"key":"ref_11","first-page":"1","article-title":"Problem-Based Learning in Online Environments","volume":"9","author":"An","year":"2008","journal-title":"Q. Rev. Distance Educ."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1111\/1467-8535.00326","article-title":"Interactivity and interactive functions in web-based learning systems: A technical framework for designers","volume":"34","author":"Chou","year":"2003","journal-title":"Br. J. Educ. Technol."},{"key":"ref_13","first-page":"318","article-title":"Twenty years of research on the academic performance differences between traditional and distance learning: Summative meta-analysis and trend examination","volume":"6","author":"Sachar","year":"2010","journal-title":"MERLOT J. Online Learn. Teach."},{"key":"ref_14","first-page":"106","article-title":"Using collaborative course development to achieve online course quality standards","volume":"11","author":"Chao","year":"2010","journal-title":"Int. Rev. Res. Open Distrib. Learn."},{"key":"ref_15","unstructured":"Stavredes, T., and Herder, T. (2014). A Guide to Online Course Design: Strategies for Student Success, John Wiley & Sons."},{"key":"ref_16","first-page":"7","article-title":"An approach for hotel type selection based on the Single-Valued Intuitionistic Fuzzy Numbers","volume":"1\u20132","author":"Sava","year":"2019","journal-title":"Int. Rev."},{"key":"ref_17","first-page":"41","article-title":"The selection of a location for potential roundabout construction\u2013a case study of Doboj","volume":"3","year":"2020","journal-title":"Oper. Res. Eng. Sci. Theory Appl."},{"key":"ref_18","unstructured":"MacCrimmon, K.R. (1968). Decision Marking Among Multiple-Attribute Alternatives: A Survey and Consolidated Approach. RAND Memorandum, RM-4823-ARPA, Rand Corporation."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Saaty, T.L. (1980). The Analytic Hierarchy Process for Decision in a Complex World, RWS Publications.","DOI":"10.21236\/ADA214804"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hwang, C.L., and Yoon, K. (1981). Multiple Attribute Decision Making: Methods and Applications, Springer.","DOI":"10.1007\/978-3-642-48318-9"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1287\/mnsc.31.6.647","article-title":"A preference ranking organization method: The PROMETHEE method for MCDM","volume":"31","author":"Brans","year":"1985","journal-title":"Manag. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1007\/BF00134132","article-title":"The outranking approach and the foundations of electre methods","volume":"31","author":"Roy","year":"1991","journal-title":"Theory Decis."},{"key":"ref_23","unstructured":"Opricovic, S. (1998). Multicriteria Optimization of Civil Engineering Systems, Faculty of Civil Engineering. (In Serbian)."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1016\/S0019-9958(65)90241-X","article-title":"Fuzzy sets","volume":"8","author":"Zadeh","year":"1965","journal-title":"Inf. Control."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1287\/mnsc.17.4.B141","article-title":"Decision-Making in a Fuzzy Environment","volume":"17","author":"Bellman","year":"1970","journal-title":"Manag. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/S0165-0114(86)80034-3","article-title":"Intuitionistic fuzzy sets","volume":"20","author":"Atanassov","year":"1986","journal-title":"Fuzzy Sets Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/S0165-0114(97)00303-5","article-title":"Correlation and information energy of interval-valued fuzzy numbers","volume":"103","author":"Wang","year":"1999","journal-title":"Fuzzy Sets Syst."},{"key":"ref_28","unstructured":"Lee, K.M. (2000, January 12\u201314). Bipolar-valued fuzzy sets and their basic operations. Proceedings of the International Conference on Intelligent Technologies, Bangkok, Thailand."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1007\/s00500-010-0563-5","article-title":"Extension of the ELECTRE method based on interval-valued fuzzy sets","volume":"15","author":"Vahdani","year":"2010","journal-title":"Soft Comput."},{"key":"ref_30","first-page":"175","article-title":"Decision-making in selecting reverse logistics alternative using interval-valued fuzzy sets combined with VIKOR approach","volume":"14","author":"Samantra","year":"2013","journal-title":"Int. J. Serv. Oper. Manag."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1007\/s00170-009-2241-2","article-title":"Extension of VIKOR method based on interval-valued fuzzy sets","volume":"47","author":"Vahdani","year":"2009","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1016\/j.eswa.2012.07.066","article-title":"Group multi-criteria decision making based upon interval-valued fuzzy numbers: An extension of the MULTIMOORA method","volume":"40","author":"Zeng","year":"2013","journal-title":"Expert Syst. Appl."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.asoc.2012.08.020","article-title":"Soft computing based on new interval-valued fuzzy modified multi-criteria decision-making method","volume":"13","author":"Vahdani","year":"2013","journal-title":"Appl. Soft Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"7050","DOI":"10.1016\/j.eswa.2010.03.013","article-title":"An extended TOPSIS method with interval-valued intuitionistic fuzzy numbers for virtual enterprise partner selection","volume":"37","author":"Ye","year":"2010","journal-title":"Expert Syst. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2544","DOI":"10.1016\/j.apm.2010.11.025","article-title":"Extension of the TOPSIS method for decision making problems under interval-valued intuitionistic fuzzy environment","volume":"35","author":"Park","year":"2011","journal-title":"Appl. Math. Model."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"159","DOI":"10.3846\/tede.2010.10","article-title":"A new additive ratio assessment (ARAS) method in multicriteria decision-making","volume":"16","author":"Zavadskas","year":"2010","journal-title":"Technol. Econ. Dev. Econ."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/S1644-9665(12)60141-1","article-title":"Multiple criteria analysis of foundation instalment alternatives by applying Additive Ratio Assessment (ARAS) method","volume":"10","author":"Zavadskas","year":"2010","journal-title":"Arch. Civ. Mech. Eng."},{"key":"ref_38","first-page":"43","article-title":"The MCDM Model for Personnel Selection Based on SWARA and ARAS Methods","volume":"20","author":"Stanujkic","year":"2015","journal-title":"Manag. Sustain. Bus. Manag. Solutions Emerg. Econ."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"43","DOI":"10.5937\/sjm11-7877","article-title":"Ranking of companies according to the indicators of corporate social responsibility based on SWARA and ARAS methods","volume":"11","author":"Paunkovic","year":"2016","journal-title":"Serbian J. Manag."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"724","DOI":"10.5937\/tehnika1805724K","article-title":"Selection of software testing method by using ARAS method","volume":"73","author":"Stanujkic","year":"2018","journal-title":"Tehnika"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"100382","DOI":"10.1016\/j.spasta.2019.100382","article-title":"BWM-ARAS: A new hybrid MCDM method for Cu prospectivity mapping in the Abhar area, NW Iran","volume":"33","author":"Bahrami","year":"2019","journal-title":"Spat. Stat."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"120685","DOI":"10.1016\/j.jclepro.2020.120685","article-title":"The optimization of engine operating parameters via SWARA and ARAS hybrid method in a small SI engine using alternative fuels","volume":"258","author":"Balki","year":"2020","journal-title":"J. Clean. Prod."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"423","DOI":"10.3846\/transport.2010.52","article-title":"A new fuzzy additive ratio assessment method (ARAS-F). Case study: The analysis of fuzzy multiple criteria in order to select the logistic centers location","volume":"25","author":"Turskis","year":"2010","journal-title":"Transport"},{"key":"ref_44","first-page":"597","article-title":"A Novel Method for Multiple Criteria Analysis: Grey Additive Ratio Assessment (ARAS-G) Method","volume":"21","author":"Turskis","year":"2010","journal-title":"Information"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"599","DOI":"10.3846\/16111699.2017.1327455","article-title":"New group decision-making ARCAS approach based on the integration of the SWARA and the ARAS methods adapted for negotiations","volume":"18","author":"Stanujkic","year":"2017","journal-title":"J. Bus. Econ. Manag."},{"key":"ref_46","first-page":"116","article-title":"The use of Pivot Pair-wise Relative Criteria Importance Assessment method for determining weights of criteria","volume":"20","author":"Stanujkic","year":"2017","journal-title":"Rom. J. Econ. Forecast."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Popovi\u0107, G. (2019). A framework for the quality control manager selection based on the PIPRECIA and WS PLP methods. EMAN 2019\u2013Economics & Management: How to Cope with Disrupted Times, Ljubljana, UdEkoM Balkan.","DOI":"10.31410\/EMAN.S.P.2019.33"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Toma\u0161evi\u0107, M., Lapuh, L., Stevi\u0107, \u017d., Stanujki\u0107, D., and Karaba\u0161evi\u0107, D. (2020). Evaluation of Criteria for the Implementation of High-Performance Computing (HPC) in Danube Region Countries Using Fuzzy PIPRECIA Method. Sustainability, 12.","DOI":"10.3390\/su12073017"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Stevi\u0107, \u017d., Stjepanovi\u0107, \u017d., Bo\u017ei\u010dkovi\u0107, Z., Das, D., and Stanujki\u0107, D. (2018). Assessment of Conditions for Implementing Information Technology in a Warehouse System: A Novel Fuzzy PIPRECIA Method. Symmetry, 10.","DOI":"10.3390\/sym10110586"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"80","DOI":"10.31181\/dmame2003114d","article-title":"A Novel Integrated Fuzzy PIPRECIA\u2013Interval Rough Saw Model: Green Supplier Selection","volume":"3","author":"Karamasa","year":"2020","journal-title":"Decis. Making Appl. Manag. Eng."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"47","DOI":"10.5937\/industrija47-24128","article-title":"Multiple criteria approach in the mining method selection","volume":"47","year":"2019","journal-title":"Industrija"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"335","DOI":"10.15388\/Informatica.2015.51","article-title":"Extension of the ARAS Method for Decision-Making Problems with Interval-Valued Triangular Fuzzy Numbers","volume":"26","author":"Stanujkic","year":"2015","journal-title":"Informatica"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"532","DOI":"10.2304\/elea.2014.11.6.532","article-title":"Evaluation Criterion for Quality Assessment of E-Learning Content","volume":"11","year":"2014","journal-title":"E-Learning Digit. Media"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"1517","DOI":"10.1016\/j.ins.2011.01.010","article-title":"Semantic Web Technologies for supporting learning assessment","volume":"181","year":"2011","journal-title":"Inf. Sci."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1028","DOI":"10.1016\/j.eswa.2006.02.004","article-title":"Evaluating intertwined effects in e-learning programs: A novel hybrid MCDM model based on factor analysis and DEMATEL","volume":"32","author":"Tzeng","year":"2007","journal-title":"Expert Syst. Appl."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"10657","DOI":"10.1016\/j.eswa.2009.02.047","article-title":"Evaluation of the criteria and effectiveness of distance e-learning with consistent fuzzy preference relations","volume":"36","author":"Chao","year":"2009","journal-title":"Expert Syst. Appl."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"351","DOI":"10.5267\/j.dsl.2017.2.003","article-title":"Fuzzy multi-attribute decision making evaluation of e-learning websites using FAHP, COPRAS, VIKOR, WDBA","volume":"6","author":"Garg","year":"2017","journal-title":"Decis. Sci. Lett."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/s40692-019-00135-7","article-title":"Selection of E-learning websites using a novel Proximity Indexed Value (PIV) MCDM method","volume":"6","author":"Khan","year":"2019","journal-title":"J. Comput. Educ."},{"key":"ref_59","first-page":"261","article-title":"Fuzzy analytical network process logic for performance measurement system of e-learning centers of Universities","volume":"11","author":"Ghannadpour","year":"2018","journal-title":"J. Ind. Syst. Eng."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"5927","DOI":"10.12973\/eurasia.2017.01041a","article-title":"The Use of a DANP with VIKOR Approach for Establishing the Model of E-Learning Service Quality","volume":"13","author":"Su","year":"2017","journal-title":"Eurasia J. Math. Sci. Technol. Educ."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/12\/6\/928\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:34:58Z","timestamp":1760175298000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/12\/6\/928"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,2]]},"references-count":60,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["sym12060928"],"URL":"https:\/\/doi.org\/10.3390\/sym12060928","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,2]]}}}