{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,21]],"date-time":"2025-11-21T12:31:27Z","timestamp":1763728287683,"version":"3.41.2"},"reference-count":57,"publisher":"Emerald","issue":"4","license":[{"start":{"date-parts":[[2022,9,6]],"date-time":"2022-09-06T00:00:00Z","timestamp":1662422400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ITSE"],"published-print":{"date-parts":[[2022,10,27]]},"abstract":"<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title>\n<jats:p>The purpose of this study is, First, to leverage the limitation of annotated data and to identify the cognitive level of learning objectives efficiently, this study adopts transfer learning by using word2vec and a bidirectional gated recurrent units (GRU) that can fully take into account the context and improves the classification of the model. This study adds a layer based on attention mechanism (AM), which captures the context vector and gives keywords higher weight for text classification. Second, this study explains the authors\u2019 model\u2019s results with local interpretable model-agnostic explanations (LIME).<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title>\n<jats:p>Bloom's taxonomy levels of cognition are commonly used as a reference standard for identifying e-learning contents. Many action verbs in Bloom's taxonomy, however, overlap at different levels of the hierarchy, causing uncertainty regarding the cognitive level expected. Some studies have looked into the cognitive classification of e-learning content but none has looked into learning objectives. On the other hand, most of these research papers just adopt classical machine learning algorithms. The main constraint of this study is the availability of annotated learning objectives data sets. This study managed to build a data set of 2,400 learning objectives, but this size remains limited.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Findings<\/jats:title>\n<jats:p>This study\u2019s experiments show that the proposed model achieves highest scores of accuracy: 90.62%, F1-score and loss. The proposed model succeeds in classifying learning objectives, which contain ambiguous verb from the Bloom\u2019s taxonomy action verbs, while the same model without the attention layer fails. This study\u2019s LIME explainer aids in visualizing the most essential features of the text, which contributes to justifying the final classification.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title>\n<jats:p>In this study, the main objective is to propose a model that outperforms the baseline models for learning objectives classification based on the six cognitive levels of Bloom's taxonomy. In this sense, this study builds the bidirectional GRU (BiGRU)-attention model based on the combination of the BiGRU algorithm with the AM. This study feeds the architecture with word2vec embeddings. To prove the effectiveness of the proposed model, this study compares it with four classical machine learning algorithms that are widely used for the cognitive classification of text: Bayes naive, logistic regression, support vector machine and K-nearest neighbors and with GRU. The main constraint related to this study is the absence of annotated data; there is no annotated learning objective data set based on Bloom\u2019s taxonomy's cognitive levels. To overcome this problem, this study seemed to have no choice but to build the data set.<\/jats:p>\n<\/jats:sec>","DOI":"10.1108\/itse-10-2021-0188","type":"journal-article","created":{"date-parts":[[2022,9,6]],"date-time":"2022-09-06T01:02:02Z","timestamp":1662426122000},"page":"396-421","source":"Crossref","is-referenced-by-count":4,"title":["An explainable attention-based bidirectional GRU model for pedagogical classification of MOOCs"],"prefix":"10.1108","volume":"19","author":[{"given":"Hanane","family":"Sebbaq","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nour-eddine","family":"El Faddouli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","published-online":{"date-parts":[[2022,9,6]]},"reference":[{"key":"key2022102505491921900_ref001","doi-asserted-by":"publisher","DOI":"10.1016\/j.jksuci.2021.08.030","article-title":"Arabic aspect based sentiment analysis using bidirectional GRU based models","year":"2021","journal-title":"Journal of King Saud University \u2013 Computer and Information Sciences"},{"year":"2005","key":"key2022102505491921900_ref002","article-title":"Exam questions classification based on bloom\u2019s taxonomy cognitive level using classifiers combination"},{"first-page":"90","article-title":"A review of word embedding and document similarity algorithms applied to academic text","year":"2017","key":"key2022102505491921900_ref003"},{"key":"key2022102505491921900_ref005","unstructured":"Bahdanau, D., Cho, K. and Bengio, Y. (2016), \u201cNeural machine translation by jointly learning to align and translate\u201d, ArXiv14090473 Cs Stat, available at: http:\/\/arxiv.org\/abs\/1409.0473 (accessed 7 February 2022)."},{"key":"key2022102505491921900_ref006","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","year":"2020","journal-title":"Information Fusion"},{"key":"key2022102505491921900_ref007","unstructured":"Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D. and Rinzivillo, S. (2021), \u201cBenchmarking and survey of explanation methods for black box models\u201d, ArXiv210213076 Cs, available at: http:\/\/arxiv.org\/abs\/2102.13076 (accessed 7 February 2022)."},{"key":"key2022102505491921900_ref008","doi-asserted-by":"publisher","first-page":"661235","DOI":"10.3389\/fpsyg.2021.661235","article-title":"A cognitive level evaluation method based on a deep neural network for online learning: from a bloom\u2019s taxonomy of cognition objectives perspective","volume":"12","year":"2021","journal-title":"Frontiers in Psychology"},{"key":"key2022102505491921900_ref009","unstructured":"Chung, J., Gulcehre, C., Cho, K. and Bengio, Y. (2014), \u201cEmpirical evaluation of gated recurrent neural networks on sequence modeling\u201d, ArXiv14123555 Cs, available at: http:\/\/arxiv.org\/abs\/1412.3555 (accessed 24 March 2021)."},{"issue":"50","key":"key2022102505491921900_ref010","doi-asserted-by":"publisher","DOI":"10.6018\/red\/50\/2","article-title":"MOOCs as disruptive technologies: strategies for enhancing the learner experience and quality of MOOCs","year":"2016","journal-title":"Revista de Educaci\u00f3n a Distancia (RED)"},{"issue":"2","key":"key2022102505491921900_ref011","doi-asserted-by":"publisher","first-page":"ep275","DOI":"10.30935\/cedtech\/8341","article-title":"Identification of cognitive learning complexity of assessment questions using multi-class text classification","volume":"12","year":"2020","journal-title":"Contemporary Educational Technology"},{"key":"key2022102505491921900_ref012","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3231644.3231663","article-title":"Toward large-scale learning design: categorizing course designs in service of supporting learning outcomes","year":"2018"},{"key":"key2022102505491921900_ref013","doi-asserted-by":"publisher","first-page":"101182","DOI":"10.1016\/j.csl.2020.101182","article-title":"Attention-based BiLSTM fused CNN with gating mechanism model for chinese long text classification","volume":"68","year":"2021","journal-title":"Computer Speech and Language"},{"key":"key2022102505491921900_ref014","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1007\/978-3-642-53914-5_44","article-title":"Automatic labeling of forums using bloom\u2019s taxonomy","volume-title":"Adv. Data Min. Appl","year":"2013"},{"key":"key2022102505491921900_ref015","doi-asserted-by":"publisher","first-page":"1757","DOI":"10.1109\/ICICS.2003.1292768","article-title":"Question classification for E-learning by artificial neural network","year":"2003"},{"key":"key2022102505491921900_ref016","unstructured":"Grandini, M., Bagli, E. and Visani, G. (2020), \u201cMetrics for Multi-Class classification: an overview\u201d, ArXiv200805756 Cs Stat, available at: http:\/\/arxiv.org\/abs\/2008.05756 (accessed 24 March 2021)."},{"first-page":"410","article-title":"A rule-based approach in bloom\u2019s taxonomy question classification through natural language processing","year":"2012","key":"key2022102505491921900_ref017"},{"first-page":"7","article-title":"bloom\u2019s taxonomy question categorization using rules and N-gram approach","year":"2005","key":"key2022102505491921900_ref018"},{"issue":"3","key":"key2022102505491921900_ref019","doi-asserted-by":"publisher","DOI":"10.19030\/tlc.v2i3.1785","article-title":"How challenging? Using bloom\u2019s taxonomy to assess learning objectives in a degree completion program","volume":"2","year":"2005","journal-title":"Journal of College Teaching and Learning (TLC)"},{"key":"key2022102505491921900_ref020","unstructured":"Hu, Y., Huber, A., Anumula, J. and Liu, S.-C. (2019), \u201cOvercoming the vanishing gradient problem in plain recurrent networks\u201d, ArXiv180106105 Cs, available at: http:\/\/arxiv.org\/abs\/1801.06105 (accessed 24 March 2021)."},{"key":"key2022102505491921900_ref021","unstructured":"Kingma, D.P. and Ba, J. (2017), \u201cAdam: a method for stochastic optimization\u201d, ArXiv14126980 Cs, available at: http:\/\/arxiv.org\/abs\/1412.6980 (accessed 26 March 2021)."},{"first-page":"11","article-title":"do moocs need a special instructional design?","year":"2014","key":"key2022102505491921900_ref022"},{"issue":"4","key":"key2022102505491921900_ref023","doi-asserted-by":"publisher","first-page":"212","DOI":"10.1207\/s15430421tip4104_2","article-title":"A revision of bloom\u2019s taxonomy: an overview","volume":"41","year":"2002","journal-title":"Theory into Practice"},{"key":"key2022102505491921900_ref004","doi-asserted-by":"publisher","first-page":"20","DOI":"10.18178\/ijke.2019.5.1.111","article-title":"Bloom\u2019s taxonomy and rules based question analysis approach for measuring the quality of examination papers","year":"2019","journal-title":"International Journal of Knowledge Engineering"},{"issue":"3","key":"key2022102505491921900_ref024","doi-asserted-by":"publisher","first-page":"211","DOI":"10.14445\/22315381\/IJETT-V69I3P232","article-title":"Classifications of the summative assessment for revised bloom\u2019s taxonomy by using deep learning","volume":"69","year":"2021","journal-title":"International Journal of Engineering Trends and Technology"},{"key":"key2022102505491921900_ref025","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1016\/j.neucom.2019.01.078","article-title":"Bidirectional LSTM with attention mechanism and convolutional layer for text classification","volume":"337","year":"2019","journal-title":"Neurocomputing"},{"key":"key2022102505491921900_ref026","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1016\/j.compedu.2014.08.005","article-title":"Instructional quality of massive open online courses (MOOCs)","volume":"80","year":"2015","journal-title":"Computers and Education"},{"key":"key2022102505491921900_ref027","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1007\/978-3-030-66906-5_6","article-title":"Evaluation of approaches for automatic E-Assessment item annotation with levels of bloom\u2019s taxonomy","volume-title":"Learning Technology Systems","year":"2021"},{"first-page":"17","article-title":"First principles of instruction: a synthesis","year":"2007","key":"key2022102505491921900_ref028"},{"first-page":"17","article-title":"A combination method of syntactic and semantic approaches for classifying examination questions into bloom\u2019s taxonomy cognitive","year":"2019","key":"key2022102505491921900_ref029"},{"issue":"3","key":"key2022102505491921900_ref030","doi-asserted-by":"publisher","first-page":"e0230442","DOI":"10.1371\/journal.pone.0230442","article-title":"Question classification based on bloom\u2019s taxonomy cognitive domain using modified TF-IDF and word2vec","volume":"15","year":"2020","journal-title":"Plos One"},{"key":"key2022102505491921900_ref031","doi-asserted-by":"publisher","first-page":"391","DOI":"10.5220\/0005813303910396","article-title":"Automatic concepts classification based on bloom\u2019s taxonomy using text analysis and the na\u00efve bayes classifier method","year":"2016"},{"key":"key2022102505491921900_ref032","doi-asserted-by":"publisher","first-page":"19","DOI":"10.11114\/jets.v1i2.94","article-title":"Using action verbs as learning outcomes: applying bloom\u2019s taxonomy in measuring instructional objectives in introductory psychology","volume":"1","year":"2013","journal-title":"Journal of Education and Training Studies"},{"key":"key2022102505491921900_ref033","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1016\/j.sbspro.2012.09.278","article-title":"Automated analysis of exam questions according to bloom\u2019s taxonomy","volume":"59","year":"2012","journal-title":"Procedia \u2013 Social and Behavioral Sciences"},{"first-page":"8","article-title":"Classifications of exam questions using linguistically \u2013 motivated features: a case study based on Bloom\u2019s Taxonomy","year":"2016","key":"key2022102505491921900_ref034"},{"key":"key2022102505491921900_ref035","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/ICITEED.2016.7863293","article-title":"A comparison between shallow and deep architecture classifiers on small dataset","year":"2016"},{"key":"key2022102505491921900_ref036","unstructured":"PEDAGOGY (MAIN PHASE) (2014), available at: www.ide.iitkgp.ac.in\/Pedagogy_view\/index.jsp (accessed 6 April 2021)."},{"issue":"4","key":"key2022102505491921900_ref037","doi-asserted-by":"publisher","first-page":"516","DOI":"10.1177\/0165551518802522","article-title":"Text classification for cognitive domains: a case using lexical, syntactic and semantic features","volume":"45","year":"2019","journal-title":"Journal of Information Science"},{"issue":"4","key":"key2022102505491921900_ref038","doi-asserted-by":"publisher","DOI":"10.24059\/olj.v23i4.2084","article-title":"Characterizing MOOC pedagogies: exploring tools and methods for learning designers and researchers","volume":"23","year":"2019","journal-title":"Online Learning"},{"key":"key2022102505491921900_ref039","unstructured":"Ribeiro, M.T., Singh, S. and Guestrin, C. (2016), \u201cWhy should I trust you?\u201d Explaining the predictions of any classifier\u201d, ArXiv160204938 Cs Stat, available at: http:\/\/arxiv.org\/abs\/1602.04938 (accessed 8 February 2022)."},{"key":"key2022102505491921900_ref040","doi-asserted-by":"publisher","first-page":"621","DOI":"10.1007\/978-3-030-73882-2_56","article-title":"MOOCs semantic interoperability: towards unified and pedagogically enriched model for building a linked data repository","volume-title":"Digital Technologies and Applications","year":"2021"},{"key":"key2022102505491921900_ref041","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3419604.3419619","article-title":"Recommender system to support MOOCs teachers: Framework based on ontology and linked data","year":"2020"},{"key":"key2022102505491921900_ref042","doi-asserted-by":"publisher","first-page":"117887","DOI":"10.1109\/ACCESS.2021.3106443","article-title":"Bloom\u2019s learning outcomes\u2019 automatic classification using LSTM and pretrained word embeddings","volume":"9","year":"2021","journal-title":"IEEE Access"},{"key":"key2022102505491921900_ref043","doi-asserted-by":"publisher","first-page":"282","DOI":"10.1109\/ICOSEC49089.2020.9215319","article-title":"Review on Word2Vec word embedding neural net","year":"2020"},{"issue":"4","key":"key2022102505491921900_ref044","doi-asserted-by":"publisher","first-page":"37","DOI":"10.3390\/educsci6040037","article-title":"Reevaluating bloom\u2019s taxonomy: what measurable verbs can and cannot say about student learning","volume":"6","year":"2016","journal-title":"Education Sciences"},{"key":"key2022102505491921900_ref045","doi-asserted-by":"publisher","first-page":"100288","DOI":"10.1016\/j.cosrev.2020.100288","article-title":"Advancement from neural networks to deep learning in software effort estimation: perspective of two decades","volume":"38","year":"2020","journal-title":"Computer Science Review"},{"key":"key2022102505491921900_ref046","doi-asserted-by":"publisher","DOI":"10.15219\/em54.1098","article-title":"AMP: a tool for characterizing the pedagogical approaches of MOOCs","year":"2014","journal-title":"e-mentor"},{"first-page":"39","article-title":"Analyzing learning outcomes for electronic fundamentals using Bloom\u2019s taxonomy","year":"2019","key":"key2022102505491921900_ref047"},{"key":"key2022102505491921900_ref048","first-page":"9","article-title":"The impact of preprocessing on text classification","year":"2014","journal-title":"Information Processing & Management"},{"key":"key2022102505491921900_ref049","doi-asserted-by":"publisher","first-page":"107792","DOI":"10.1016\/j.asoc.2021.107792","article-title":"Sentiment classification using attention mechanism and bidirectional long short-term memory network","volume":"112","year":"2021","journal-title":"Applied Soft Computing"},{"first-page":"17","article-title":"Exploring the influences of MOOC design features on student performance and persistence","year":"2019","key":"key2022102505491921900_ref050"},{"key":"key2022102505491921900_ref051","doi-asserted-by":"publisher","first-page":"587","DOI":"10.1016\/j.sbspro.2013.10.277","article-title":"Analyzing the cognitive level of classroom questions using machine learning techniques","volume":"97","year":"2013","journal-title":"Procedia \u2013 Social and Behavioral Sciences"},{"key":"key2022102505491921900_ref052","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1109\/ISDA.2013.6920714","article-title":"Educational data mining: a case study of teacher\u2019s classroom questions","year":"2013"},{"key":"key2022102505491921900_ref053","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1007\/978-3-642-30732-4_17","article-title":"Bloom\u2019s taxonomy\u2013based classification for item bank questions using support vector machines","volume-title":"Modern Advanced Intelligent Systems Tools","year":"2012"},{"key":"key2022102505491921900_ref054","doi-asserted-by":"publisher","first-page":"1480","DOI":"10.18653\/v1\/N16-1174","article-title":"Hierarchical attention networks for document classification","year":"2016"},{"key":"key2022102505491921900_ref055","doi-asserted-by":"publisher","first-page":"866","DOI":"10.1109\/ISDA.2010.5687152","article-title":"Determination of bloom\u2019s cognitive level of question items using artificial neural network","year":"2010"},{"issue":"1\/4","key":"key2022102505491921900_ref056","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1007\/s13042-010-0001-0","article-title":"Understanding bag-of-words model: a statistical framework","volume":"1","year":"2010","journal-title":"International Journal of Machine Learning and Cybernetics"},{"key":"key2022102505491921900_ref057","unstructured":"Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H. and He, Q. (2020), \u201cA comprehensive survey on transfer learning\u201d, ArXiv191102685 Cs Stat, available at: http:\/\/arxiv.org\/abs\/1911.02685 (accessed 24 March 2021)."}],"container-title":["Interactive Technology and Smart Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/ITSE-10-2021-0188\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/ITSE-10-2021-0188\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T21:57:06Z","timestamp":1753394226000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/itse\/article\/19\/4\/396-421\/174696"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,6]]},"references-count":57,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,9,6]]},"published-print":{"date-parts":[[2022,10,27]]}},"alternative-id":["10.1108\/ITSE-10-2021-0188"],"URL":"https:\/\/doi.org\/10.1108\/itse-10-2021-0188","relation":{},"ISSN":["1741-5659","1741-5659"],"issn-type":[{"type":"print","value":"1741-5659"},{"type":"electronic","value":"1741-5659"}],"subject":[],"published":{"date-parts":[[2022,9,6]]}}}