{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:09:27Z","timestamp":1760238567999,"version":"build-2065373602"},"reference-count":58,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2020,8,18]],"date-time":"2020-08-18T00:00:00Z","timestamp":1597708800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["KK.01.1.1.01.0009"],"award-info":[{"award-number":["KK.01.1.1.01.0009"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003443","name":"Ministry of Education and Science of the Russian Federation","doi-asserted-by":"publisher","award":["074-U01"],"award-info":[{"award-number":["074-U01"]}],"id":[{"id":"10.13039\/501100003443","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Machines usually employ a guess-and-check strategy to analyze data: they take the data, make a guess, check the answer, adjust it with regard to the correct one if necessary, and try again on a new data set. An active learning environment guarantees better performance while training on less, but carefully chosen, data which reduces the costs of both annotating and analyzing large data sets. This issue becomes even more critical for deep learning applications. Human-like active learning integrates a variety of strategies and instructional models chosen by a teacher to contribute to learners\u2019 knowledge, while machine active learning strategies lack versatile tools for shifting the focus of instruction away from knowledge transmission to learners\u2019 knowledge construction. We approach this gap by considering an active learning environment in an educational setting. We propose a new strategy that measures the information capacity of data using the information function from the four-parameter logistic item response theory (4PL IRT). We compared the proposed strategy with the most common active learning strategies\u2014Least Confidence and Entropy Sampling. The results of computational experiments showed that the Information Capacity strategy shares similar behavior but provides a more flexible framework for building transparent knowledge models in deep learning.<\/jats:p>","DOI":"10.3390\/e22080906","type":"journal-article","created":{"date-parts":[[2020,8,18]],"date-time":"2020-08-18T11:15:27Z","timestamp":1597749327000},"page":"906","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["From Knowledge Transmission to Knowledge Construction: A Step towards Human-Like Active Learning"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6653-5978","authenticated-orcid":false,"given":"Ilona","family":"Kulikovskikh","sequence":"first","affiliation":[{"name":"Department of Information Systems and Technologies, Samara National Research University, Moskovskoe Shosse 34, 443086 Samara, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8037-8198","authenticated-orcid":false,"given":"Tomislav","family":"Lipic","sequence":"additional","affiliation":[{"name":"Division of Electronics, Ru\u0111er Bo\u0161kovi\u0107 Institute, Bijeni\u010dka cesta 54, 10000 Zagreb, Croatia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9185-9384","authenticated-orcid":false,"given":"Tomislav","family":"\u0160muc","sequence":"additional","affiliation":[{"name":"Division of Electronics, Ru\u0111er Bo\u0161kovi\u0107 Institute, Bijeni\u010dka cesta 54, 10000 Zagreb, Croatia"}]}],"member":"1968","published-online":{"date-parts":[[2020,8,18]]},"reference":[{"key":"ref_1","unstructured":"Bachman, P., Sordoni, A., and Trischler, A. (2017, January 6\u201311). Learning algorithms for active learning. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Fazakis, N., Kanas, V.G., Aridas, C.K., Karlos, S., and Kotsiantis, S. (2019). Combination of active learning and semi-supervised learning under a self-training scheme. Entropy, 21.","DOI":"10.3390\/e21100988"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hsu, W.-N., and Lin, H.-T. (2015, January 25\u201330). Active learning by learning. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA.","DOI":"10.1609\/aaai.v29i1.9597"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1109\/TPAMI.2014.2307881","article-title":"Active learning by querying informative and representative examples","volume":"36","author":"Huang","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_5","unstructured":"Konyushkova, K., Raphael, S., and Fua, P. (2017, January 4\u20139). Learning active learning from data. Proceedings of the Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1007\/s10618-016-0469-7","article-title":"Active learning: An empirical study of common baselines","volume":"31","author":"Sharma","year":"2017","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_7","unstructured":"Settles, B. (2009). Active Learning Literature Survey. Computer Sciences Technical Report, University of Wisconsin\u2013Madison."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sourati, J., Akcakaya, M., Dy, J.G., Leen, T.K., and Erdogmus, D. (2016). Classification Active Learning Based on Mutual Information. Entropy, 18.","DOI":"10.3390\/e18020051"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2023","DOI":"10.1109\/TPAMI.2017.2743707","article-title":"A probabilistic active learning algorithm based on Fisher information ratio","volume":"40","author":"Sourati","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3379504","article-title":"Multi-label active learning algorithms for image classification: Overview and future promise","volume":"53","author":"Wu","year":"2020","journal-title":"ACM Comput. Surv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Joshi, A.J., Porikli, F., and Papanikolopoulos, N. (2009, January 20\u201325). Multi-class active learning for image classification. Proceedings of the Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206627"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s11263-014-0781-x","article-title":"Multi-Class active learning by uncertainty sampling with diversity maximization","volume":"113","author":"Yang","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_13","first-page":"1469","article-title":"Activized Learning: Transforming passive to active with improved label complexity","volume":"13","author":"Hanneke","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_14","unstructured":"Bonwell, C., and Eison, J. (1991). Active Learning: Creating Excitement in the Classroom, Jossey-Bass."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1080\/03098265.2016.1185772","article-title":"Active learning through online quizzes: Better learning and less (busy) work","volume":"41","author":"Cook","year":"2017","journal-title":"J. Geogr. High. Educ."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1002\/j.2168-9830.2004.tb00809.x","article-title":"Does Active Learning Work? A review of the research","volume":"93","author":"Prince","year":"2004","journal-title":"J. Eng. Educ."},{"key":"ref_17","unstructured":"Aubrey, K., and Riley, A. (2015). Understanding and Using Educational Theories, Sage Publications Ltd."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mascharka, D., Tran, P., Soklaski, R., and Majumdar, A. (2018, January 18\u201323). Transparency by design: Closing the gap between performance and interpretability in visual reasoning. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00519"},{"key":"ref_19","unstructured":"Castro, R., Kalish, C., Nowak, R., Qian, R., Rogers, T., and Zhu, X. (2008, January 8\u201311). Human active learning. Proceedings of the Twenty-Second Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1332","DOI":"10.1126\/science.aab3050","article-title":"Human-level concept learning through probabilistic program induction","volume":"350","author":"Lake","year":"2015","journal-title":"Science"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e253","DOI":"10.1017\/S0140525X16001837","article-title":"Building machines that learn and think like people","volume":"40","author":"Lake","year":"2017","journal-title":"Behav. Brain Sci."},{"key":"ref_22","unstructured":"Mastorakis, G. (2018). Human-like machine learning: Limitations and suggestions. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-019-12552-4","article-title":"The Eighty Five Percent Rule for optimal learning","volume":"10","author":"Wilson","year":"2019","journal-title":"Nat. Commun."},{"key":"ref_24","unstructured":"De Ayala, R.J. (2009). The Theory and Practice of Item Response Theory (Methodology in the Social Sciences), The Guilford Press."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1082","DOI":"10.3102\/0034654317726529","article-title":"Developing, analyzing, and using distractors for multiple-choice tests in education: A comprehensive review","volume":"87","author":"Gierl","year":"2017","journal-title":"Rev. Educ. Res."},{"key":"ref_26","unstructured":"Hakel, M.D. (1998). Beyond Multiple Choice: Evaluating Alternatives to Traditional Testing for Selection, Lawrence Erlbaum Associates Publishers."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1002\/bdm.2101","article-title":"The test taker\u2019s fallacy: How students guess answers on multiple-choice tests","volume":"32","author":"Lee","year":"2019","journal-title":"Behav. Decis. Mak."},{"key":"ref_28","unstructured":"Lord, F.M. (1980). Applications of Item Response Theory to Practical Testing Problems, Erlbaum."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1111\/j.1745-3984.1989.tb00326.x","article-title":"Multiple-choice models: The distractors are also part of the item","volume":"26","author":"Thissen","year":"1989","journal-title":"J. Educ. Meas."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Mittelstadt, B., Russell, C., and Wachter, S. (2019, January 29\u201331). Explaining explanations in AI. Proceedings of the Fairness, Accountability, and Transparency (FAT*), Atlanta, GA, USA.","DOI":"10.1145\/3287560.3287574"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_32","unstructured":"Gal, Y., Islam, R., and Ghahramani, Z. (2017, January 6\u201311). Deep Bayesian active learning with image data. Proceedings of the 34th International Conference on MachineLearning, Sydney, Australia."},{"key":"ref_33","unstructured":"Geifman, Y., and El-Yaniv, R. (2019, January 8\u201314). Deep active learning with a neural architecture search. Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, BC, Canada."},{"key":"ref_34","unstructured":"Ouali, Y., Hudelot, C., and Tami, M. (2020). An overview of deep semi-supervised learning. arXiv."},{"key":"ref_35","unstructured":"Sener, O., and Savarese, S. (May, January 30). Active learning for convolutional neural networks: A core-set approach. Proceedings of the 6th International Conference on Learning Representations (ICLR 2018), Vancouver, BC, Canada."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Wang, D., and Shang, Y. (2014, January 6\u201311). A new active labeling method for deep learning. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Beijing, China.","DOI":"10.1109\/IJCNN.2014.6889457"},{"key":"ref_37","unstructured":"Budd, S., Robinson, E.C., and Kainz, B. (2019). A survey on active learning and human-in-the-loop deep learning for medical image analysis. arXiv."},{"key":"ref_38","unstructured":"Chen, Y., Filho, T.S., Prudencio, R.B.C., Diethe, T., and Flach, P. (2019). \u03b23-IRT: A new item response model and its applications. arXiv, Available online: https:\/\/arxiv.org\/abs\/1903.04016."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.artint.2018.09.004","article-title":"Item response theory in AI: Analysing machine learning classifiers at the instance Level","volume":"271","author":"Prudencio","year":"2019","journal-title":"Artif. Intell."},{"key":"ref_40","unstructured":"Whitehill, J., Ruvolo, P., Wu, T., Bergsma, J., and Movellan, J. (2009, January 7\u201310). Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. Proceedings of the Advances in Neural Information Processing Systems 22 (NIPS 2009), Vancouver, BC, Canada."},{"key":"ref_41","unstructured":"Yeung, C.K. (2019). Deep-IRT: Make deep learning based knowledge tracing explainable using item response theory. arXiv."},{"key":"ref_42","unstructured":"Lalor, J.P., Wu, H., and Yu, H. (2017). CIFT: Crowd-informed fine-tuning to improve machine learning ability. arXiv."},{"key":"ref_43","unstructured":"Ravi, S., and Larochelle, H. (2018, June 04). Meta-Learning for Batch Mode Active Learning. Available online: https:\/\/openreview.net\/forum?id=r1PsGFJPz."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"80","DOI":"10.2307\/3001968","article-title":"Individual comparisons by ranking methods","volume":"1","author":"Wilcoxon","year":"1945","journal-title":"Biom. Bull."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1080\/01621459.1961.10482090","article-title":"Multiple comparisons among means","volume":"56","author":"Dunn","year":"1961","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1136\/bmj.316.7139.1236","article-title":"What\u2019s wrong with Bonferroni adjustments","volume":"316","author":"Perneger","year":"1998","journal-title":"BMJ"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.chb.2017.05.001","article-title":"Promoting collaborative learning through regulation of guessing in clickers","volume":"75","author":"Kulikovskikh","year":"2017","journal-title":"Comput. Hum. Behav."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1080\/0305764X.2016.1259389","article-title":"Collaborative learning practices: Teacher and student perceived obstacles to effective student collaboration","volume":"48","author":"Le","year":"2018","journal-title":"Camb. J. Educ."},{"key":"ref_49","unstructured":"Obeid, R., Schwartz, A., Shane-Simpson, C., and Brooks, P.J. (2018). Cooperative and collaborative learning: Getting the best of both words. How We Teach Now: The GSTA Guide to Student-Centered Teaching, Society for the Teaching of Psychology."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1177\/0146621616664047","article-title":"Unfolding IRT models for Likert-type items with a don\u2019t know option","volume":"49","author":"Liu","year":"2016","journal-title":"Appl. Psychol. Meas."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1177\/0146621618762743","article-title":"A general unfolding IRT model for multiple response styles","volume":"43","author":"Liu","year":"2019","journal-title":"Appl. Psychol. Meas."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1177\/0013164416637107","article-title":"Improving measures via examining the behavior of distractors in multiple-choice tests: Assessment and remediation","volume":"77","author":"Sideridis","year":"2016","journal-title":"Educ. Psychol. Meas."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Bonifay, W. (2020). Multidimensional Item Response Theory (Quantitative Applications in the Social Sciences), SAGE Publications.","DOI":"10.4135\/9781506384276"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1177\/0013164406294778","article-title":"\u201cGuessing\u201d parameter estimates for multidimensional Item Response Theory models","volume":"67","author":"DeMars","year":"2007","journal-title":"Educ. Psychol. Meas."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Gin, B., Sim, N., Skrondal, A., and Rabe-Hesketh, S. (2019). A dyadic IRT model. arXiv.","DOI":"10.1007\/s11336-020-09718-1"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Reckase, M.D. (2009). Multidimensional Item Response Theory, Springer.","DOI":"10.1007\/978-0-387-89976-3"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Frieden, B.R. (2004). Science from Fisher Information: A Unification, Cambridge University Press.","DOI":"10.1017\/CBO9780511616907"},{"key":"ref_58","unstructured":"Lehmann, E.L., and Casella, G. (1998). Theory of Point Estimation, Springer. [2nd ed.]."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/8\/906\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:02:38Z","timestamp":1760176958000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/8\/906"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,18]]},"references-count":58,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,8]]}},"alternative-id":["e22080906"],"URL":"https:\/\/doi.org\/10.3390\/e22080906","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2020,8,18]]}}}