{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,21]],"date-time":"2025-12-21T06:24:23Z","timestamp":1766298263136,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T00:00:00Z","timestamp":1709769600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"RTA Program Universitas Gadjah Mada","award":["3550\/UN1.P.III\/Dit-Lit\/PT.01.05\/2022"],"award-info":[{"award-number":["3550\/UN1.P.III\/Dit-Lit\/PT.01.05\/2022"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Nielsen\u2019s heuristics are widely recognized for usability evaluation, but they are often considered insufficiently specific for assessing particular domains, such as e-learning. Currently, e-learning plays a pivotal role in higher education because of the shift in the educational paradigm from a teacher-centered approach to a student-centered approach. The criteria utilized in multiple sets of heuristics for evaluating e-learning are carefully examined based on the definitions of each criterion. If there are similarities in meaning among these criteria, they are consolidated into a single criterion, resulting in the creation of 20 new criteria (spanning three primary aspects) for the evaluation of e-learning. These 20 new criteria encompass key aspects related to the user interface, learning development, and motivation. Each aspect is assigned a weight to facilitate prioritization when implementing improvements to evaluate e-learning, which is especially beneficial for institutions with limited resources responsible for the relevant units. In terms of weighting, there is room for enhancement to attain more optimal weighting outcomes by employing a Fuzzy Preference Programming method known as Inverse Trigonometric Fuzzy Preference Programming (ITFPP). The higher the assigned weight, the greater the priority for implementing improvements.<\/jats:p>","DOI":"10.3390\/computers13030068","type":"journal-article","created":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T11:33:06Z","timestamp":1709811186000},"page":"68","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Inverse Trigonometric Fuzzy Preference Programming to Generate Weights with Optimal Solutions Implemented on Evaluation Criteria in E-Learning"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3476-2586","authenticated-orcid":false,"given":"Emi","family":"Iryanti","sequence":"first","affiliation":[{"name":"Department of Electrical and Information Engineering, Universitas Gadjah Mada, Yogyakarta 55281, Daerah Istimewa Yogyakarta, Indonesia"},{"name":"Department of Informatics, Institut Teknologi Telkom Purwokerto, Purwokerto 53147, Jawa Tengah, Indonesia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0581-2521","authenticated-orcid":false,"given":"Paulus Insap","family":"Santosa","sequence":"additional","affiliation":[{"name":"Department of Electrical and Information Engineering, Universitas Gadjah Mada, Yogyakarta 55281, Daerah Istimewa Yogyakarta, Indonesia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sri Suning","family":"Kusumawardani","sequence":"additional","affiliation":[{"name":"Department of Electrical and Information Engineering, Universitas Gadjah Mada, Yogyakarta 55281, Daerah Istimewa Yogyakarta, Indonesia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4044-4987","authenticated-orcid":false,"given":"Indriana","family":"Hidayah","sequence":"additional","affiliation":[{"name":"Department of Electrical and Information Engineering, Universitas Gadjah Mada, Yogyakarta 55281, Daerah Istimewa Yogyakarta, Indonesia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Nielsen, J., and Molich, R. (1990, January 1\u20135). Heuristic evaluation of user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Empowering People\u2014CHI \u201990, Seattle, WA, USA.","DOI":"10.1145\/97243.97281"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Nielsen, J. (1994, January 24\u201328). Usability Inspection. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems\u2014CHI \u201994, Boston, MA, USA.","DOI":"10.1145\/259963.260531"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"516","DOI":"10.1016\/j.ijinfomgt.2005.08.008","article-title":"The assessment of usability of electronic shopping: A heuristic evaluation","volume":"25","author":"Chen","year":"2005","journal-title":"Int. J. Inf. Manag."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Paz, F., Pow-Sang, J.A., and Collantes, L. (2014, January 7\u20139). Usability Heuristics for Transactional Web Sites. Proceedings of the 2014 11th International Conference on Information Technology: New Generations, Las Vegas, NV, USA.","DOI":"10.1109\/ITNG.2014.81"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1055\/s-0038-1666842","article-title":"The Development of Heuristics for Evaluation of Dashboard Visualizations","volume":"9","author":"Dowding","year":"2018","journal-title":"Appl. Clin. Inform."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.giq.2005.11.007","article-title":"Scenario evaluation of municipal Web sites: Development and use of an expert-focused evaluation tool","volume":"23","author":"Lentz","year":"2006","journal-title":"Gov. Inf. Q."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1207\/S15327590IJHC143&4_10","article-title":"Extending the Usability of Heuristics for Design and Evaluation: Lead, Follow, and Get Out of the Way","volume":"14","author":"Kamper","year":"2002","journal-title":"Int. J. Hum.-Comput. Interact."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Moraes, M.C., and Silveira, M.S. (2006, January 18\u201322). How am I ? Guidelines for Animated Interface Agents Evaluation. Proceedings of the IEEE\/WIC\/ACM International Conference on Intelligent Agent Technology (IAT\u201906), Hong Kong, China.","DOI":"10.1109\/IAT.2006.73"},{"key":"ref_9","unstructured":"Clarkson, E., and Arkin, R.C. (2024, February 29). Applying Heuristic Evaluation to Human-Robot Interaction Systems. Available online: https:\/\/sites.cc.gatech.edu\/ai\/robot-lab\/online-publications\/ClarksonArkinTechReport2006.pdf."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Mtebe, J., and Kissaka, M. (2015, January 6\u20138). Heuristics for Evaluating Usability of Learning Management Systems in Africa. Proceedings of the 2015 IST-Africa Conference, Lilongwe, Malawi.","DOI":"10.1109\/ISTAFRICA.2015.7190521"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.apergo.2015.11.016","article-title":"Establishing usability heuristics for heuristics evaluation in a specific domain: Is there a consensus?","volume":"56","author":"Hermawati","year":"2016","journal-title":"Appl. Ergon."},{"key":"ref_12","first-page":"248","article-title":"Using a new set of heuristics in evaluating Arabic interfaces","volume":"32","author":"Muhanna","year":"2018","journal-title":"J. King Saud Univ.-Comput. Inf. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/0377-2217(90)90057-I","article-title":"How to make a decision: The Analytic Hierarchy Process","volume":"48","author":"Saaty","year":"1990","journal-title":"Eur. J. Oper. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1007\/s11518-006-5021-7","article-title":"There is no mathematical validity for using fuzzy number crunching in the analytic hierarchy process","volume":"15","author":"Saaty","year":"2006","journal-title":"J. Syst. Sci. Syst. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"990","DOI":"10.1016\/j.asoc.2010.03.010","article-title":"Managing project risk using combined analytic hierarchy process and risk map","volume":"10","author":"Dey","year":"2010","journal-title":"Appl. Soft Comput."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.eswa.2017.08.003","article-title":"Sensitivity analysis method to address user disparities in the analytic hierarchy process","volume":"90","author":"Ivanco","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"108022","DOI":"10.1016\/j.asoc.2021.108022","article-title":"Classifiers selection based on analytic hierarchy process and similarity score for spam identification","volume":"113","author":"Mekouar","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"108248","DOI":"10.1016\/j.asoc.2021.108248","article-title":"Application of integrated factor evaluation\u2013analytic hierarchy process\u2013T-S fuzzy fault tree analysis in reliability allocation of industrial robot systems","volume":"115","author":"Bai","year":"2022","journal-title":"Appl. Soft Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1016\/0377-2217(95)00300-2","article-title":"Applications of the extent analysis method on fuzzy AHP","volume":"95","author":"Chang","year":"1996","journal-title":"Eur. J. Oper. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1016\/S0165-0114(83)80082-7","article-title":"A fuzzy extension of Saaty\u2019s priority theory","volume":"11","author":"Pedrycz","year":"1983","journal-title":"Fuzzy Sets Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/0165-0114(89)90187-5","article-title":"Multi-criteria decision analysis with fuzzy pairwise comparisons","volume":"29","author":"Boender","year":"1989","journal-title":"Fuzzy Sets Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.asoc.2004.04.001","article-title":"Evaluation of services using a fuzzy analytic hierarchy process","volume":"5","author":"Mikhailov","year":"2004","journal-title":"Appl. Soft Comput."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3055","DOI":"10.1016\/j.fss.2006.08.010","article-title":"A modified fuzzy logarithmic least squares method for fuzzy analytic hierarchy process","volume":"157","author":"Wang","year":"2006","journal-title":"Fuzzy Sets Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1016\/j.ejor.2007.01.050","article-title":"On the extent analysis method for fuzzy AHP and its applications","volume":"186","author":"Wang","year":"2008","journal-title":"Eur. J. Oper. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.fss.2016.03.006","article-title":"Fuzzy eigenvector method for obtaining normalized fuzzy weights from fuzzy pairwise comparison matrices","volume":"315","year":"2017","journal-title":"Fuzzy Sets Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1057\/palgrave.jors.2600899","article-title":"A fuzzy programming method for deriving priorities in the analytic hierarchy process","volume":"51","author":"Mikhailov","year":"2000","journal-title":"J. Oper. Res. Soc."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1016\/S0165-0114(02)00383-4","article-title":"Deriving priorities from fuzzy pairwise comparison judgements","volume":"134","author":"Mikhailov","year":"2003","journal-title":"Fuzzy Sets Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1016\/j.ijar.2010.12.004","article-title":"Fuzzy analytic hierarchy process: A logarithmic fuzzy preference programming methodology","volume":"52","author":"Wang","year":"2011","journal-title":"Int. J. Approx. Reason."},{"key":"ref_29","first-page":"238","article-title":"An integrated fuzzy AHP- fuzzy MULTIMOORA model for supply chain risk-benefit assessment and supplier selection","volume":"8","author":"Tavana","year":"2021","journal-title":"Int. J. Syst. Sci. Oper. Logist."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1016\/j.eswa.2017.11.001","article-title":"A fuzzy inference- fuzzy analytic hierarchy process-based clinical decision support system for diagnosis of heart diseases","volume":"95","author":"Nazari","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"115398","DOI":"10.1016\/j.eswa.2021.115398","article-title":"Budget optimized dynamic virtual machine provisioning in hybrid cloud using fuzzy analytic hierarchy process","volume":"183","author":"Radhika","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"105648","DOI":"10.1016\/j.asoc.2019.105648","article-title":"Fuzzy AHP based prioritization and taxonomy of software process improvement success factors in global software development","volume":"83","author":"Khan","year":"2019","journal-title":"Appl. Soft Comput."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1016\/j.cie.2018.01.015","article-title":"Fuzzy AHP-TOPSIS approaches to prioritizing solutions for reverse logistics barriers","volume":"117","author":"Sirisawat","year":"2018","journal-title":"Comput. Ind. Eng."},{"key":"ref_34","first-page":"110","article-title":"Double skin fa\u00e7ade assessment by fuzzy AHP and comparison with AHP","volume":"17","author":"Bostancioglu","year":"2021","journal-title":"Arch. Eng. Des. Manag."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.resconrec.2018.11.011","article-title":"Selecting sustainable energy conversion technologies for agricultural residues: A fuzzy AHP-VIKOR based prioritization from life cycle perspective","volume":"142","author":"Wang","year":"2019","journal-title":"Resour. Conserv. Recycl."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"800","DOI":"10.1016\/j.asoc.2017.05.026","article-title":"The utility based non-linear fuzzy AHP optimization model for network selection in heterogeneous wireless networks","volume":"67","author":"Goyal","year":"2018","journal-title":"Appl. Soft Comput."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1080\/13504509.2019.1605547","article-title":"A framework based on fuzzy AHP-TOPSIS for prioritizing solutions to overcome the barriers in the implementation of ecodesign practices in SMEs","volume":"26","author":"Singh","year":"2019","journal-title":"Int. J. Sustain. Dev. World Ecol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.eswa.2007.09.004","article-title":"Application of fuzzy extended AHP methodology on shipping registry selection: The case of Turkish maritime industry","volume":"36","author":"Celik","year":"2009","journal-title":"Expert Syst. Appl."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1016\/j.cie.2018.07.048","article-title":"Study of site selection of electric vehicle charging station based on extended GRP method under picture fuzzy environment","volume":"135","author":"Ju","year":"2019","journal-title":"Comput. Ind. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"100357","DOI":"10.1016\/j.smhl.2022.100357","article-title":"Heuristic evaluation of portable pulse oximeters for domiciliary use: Implications for its use in assessing medical device usability","volume":"27","author":"Black","year":"2023","journal-title":"Smart Health"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"103584","DOI":"10.1016\/j.apergo.2021.103584","article-title":"Usability of smart infusion pumps: A heuristic evaluation","volume":"98","author":"Klarich","year":"2022","journal-title":"Appl. Ergon."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"103216","DOI":"10.1016\/j.ergon.2021.103216","article-title":"Usability of state public health department websites for communication during a pandemic: A heuristic evaluation","volume":"86","author":"Momenipour","year":"2021","journal-title":"Int. J. Ind. Ergon."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/13\/3\/68\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:10:07Z","timestamp":1760105407000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/13\/3\/68"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,7]]},"references-count":42,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["computers13030068"],"URL":"https:\/\/doi.org\/10.3390\/computers13030068","relation":{},"ISSN":["2073-431X"],"issn-type":[{"type":"electronic","value":"2073-431X"}],"subject":[],"published":{"date-parts":[[2024,3,7]]}}}