{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:04:46Z","timestamp":1760058286795,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,3,29]],"date-time":"2025-03-29T00:00:00Z","timestamp":1743206400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62176107","62076111","62076215"],"award-info":[{"award-number":["62176107","62076111","62076215"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>With the rapid growth of digital computing, communication, and storage devices applied in various real-world scenarios, more and more data have been collected and stored to drive the development of machine learning techniques. It is also noted that the data that emerge in real-world applications tend to become more complex. In this study, we regard a complex data type, i.e., multi-label data, acquired with a time constraint in a dynamic online scenario. Under such conditions, constructing a learning model has to face two challenges: it requires dynamically adapting the variances in label correlations and imbalanced data distributions and it requires more labeling consumptions. To solve these two issues, we propose a novel online multi-label active learning (OMAL) algorithm that considers simultaneously adopting uncertainty (using the average entropy of prediction probabilities) and diversity (using the average cosine distance between feature vectors) as an active query strategy. Specifically, to focus on label correlations, we use a classifier chain (CC) as the multi-label learning model and design a label co-occurrence ranking strategy to arrange label sequence in CC. To adapt the naturally imbalanced distribution of the multi-label data, we select weight extreme learning machine (WELM) as the basic binary-class classifier in CC. The experimental results on ten benchmark multi-label datasets that were transformed into streams show that our proposed method is superior to several popular static multi-label active learning algorithms in terms of both the Macro-F1 and Micro-F1 metrics, indicating its specifical adaptions in the dynamic data stream environment.<\/jats:p>","DOI":"10.3390\/e27040363","type":"journal-article","created":{"date-parts":[[2025,3,31]],"date-time":"2025-03-31T05:10:07Z","timestamp":1743397807000},"page":"363","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["OMAL: A Multi-Label Active Learning Approach from Data Streams"],"prefix":"10.3390","volume":"27","author":[{"given":"Qiao","family":"Fang","sequence":"first","affiliation":[{"name":"School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chen","family":"Xiang","sequence":"additional","affiliation":[{"name":"School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5656-4559","authenticated-orcid":false,"given":"Jicong","family":"Duan","sequence":"additional","affiliation":[{"name":"School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benallal","family":"Soufiyan","sequence":"additional","affiliation":[{"name":"School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3417-2727","authenticated-orcid":false,"given":"Changbin","family":"Shao","sequence":"additional","affiliation":[{"name":"School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xibei","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sen","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Information Technology, Yancheng Institute of Technology, Yancheng 224051, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9621-4158","authenticated-orcid":false,"given":"Hualong","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"120391","DOI":"10.1016\/j.eswa.2023.120391","article-title":"Deep learning based active learning technique for data annotation and improve the overall performance of classification models","volume":"228","author":"Amin","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2026","DOI":"10.1109\/TPAMI.2011.20","article-title":"Active learning based on locally linear reconstruction","volume":"33","author":"Zhang","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1577","DOI":"10.1016\/j.patcog.2014.12.009","article-title":"An active learning-based SVM multi-class classification model","volume":"48","author":"Guo","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1007\/s11633-015-0912-z","article-title":"A novel active learning method using SVM for text classification","volume":"15","author":"Goudjil","year":"2018","journal-title":"Int. J. Autom. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1007\/s10618-008-0115-0","article-title":"Active learning for object classification: From exploration to exploitation","volume":"18","author":"Cebron","year":"2009","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sourati, J., Akcakaya, M., Dy, J.G., Leen, T.K., and Erdogmus, D. (2016). Classification active learning based on mutual information. Entropy, 18.","DOI":"10.3390\/e18020051"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Karaca, A.C., and Bilgin, G. (2025). An end-to-end active learning framework for limited labelled hyperspectral image classification. Int. J. Remote Sens., 1\u201328.","DOI":"10.1080\/01431161.2025.2467294"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1116","DOI":"10.1109\/TMM.2021.3119868","article-title":"Deep co-image-label hashing for multi-label image retrieval","volume":"24","author":"Shen","year":"2021","journal-title":"IEEE Trans. Multimed."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1016\/j.ejor.2019.05.037","article-title":"Evaluating multi-label classifiers and recommender systems in the financial service sector","volume":"279","author":"Bogaert","year":"2019","journal-title":"Eur. J. Oper. Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1007\/s13748-017-0133-5","article-title":"An adapted incremental graded multi-label classification model for recommendation systems","volume":"7","author":"Laghmari","year":"2018","journal-title":"Prog. Artif. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"102441","DOI":"10.1016\/j.ipm.2020.102441","article-title":"A novel reasoning mechanism for multi-label text classification","volume":"58","author":"Wang","year":"2021","journal-title":"Inf. Process. Manag."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhou, L., Zheng, X., Yang, D., Wang, Y., Bai, X., and Ye, X. (2021). Application of multi-label classification models for the diagnosis of diabetic complications. BMC Med. Inform. Decis. Mak., 21.","DOI":"10.1186\/s12911-021-01525-7"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"115565","DOI":"10.1016\/j.eswa.2021.115565","article-title":"Single and multi-label fault classification in rotors from unprocessed multi-sensor data through deep and parallel CNN architectures","volume":"185","author":"Sonkul","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"100324","DOI":"10.1016\/j.iot.2020.100324","article-title":"Multi-label classification based ensemble learning for human activity recognition in smart home","volume":"12","author":"Jethanandani","year":"2020","journal-title":"Internet Things"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3379504","article-title":"Multi-label active learning algorithms for image classification: Overview and future promise","volume":"53","author":"Wu","year":"2020","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/s10994-021-06003-9","article-title":"How to measure uncertainty in uncertainty sampling for active learning","volume":"111","author":"Nguyen","year":"2022","journal-title":"Mach. Learn."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1460","DOI":"10.1109\/TFUZZ.2017.2717803","article-title":"Incorporating diversity and informativeness in multiple-instance active learning","volume":"25","author":"Wang","year":"2017","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wu, J., Sheng, V.S., Zhang, J., Zhao, P., and Cui, Z. (2014, January 27\u201330). Multi-label active learning for image classification. Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), Paris, France.","DOI":"10.1109\/ICIP.2014.7026058"},{"key":"ref_19","first-page":"1479","article-title":"Active Learning with Multi-Label SVM Classification","volume":"13","author":"Li","year":"2013","journal-title":"Int. Jt. Conf. Artif. Intell. (IJCAI)"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1016\/j.neucom.2017.08.001","article-title":"Effective active learning strategy for multi-label learning","volume":"273","author":"Reyes","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3952","DOI":"10.1109\/TNNLS.2021.3122531","article-title":"Learning data streams with changing distributions and temporal dependency","volume":"34","author":"Song","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3657286","article-title":"A systematic literature review of novelty detection in data streams: Challenges and opportunities","volume":"56","author":"Gaudreault","year":"2024","journal-title":"ACM Comput. Surv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2005","DOI":"10.1109\/TPDS.2021.3055265","article-title":"Pebbles: Leveraging sketches for processing voluminous, high velocity data streams","volume":"32","author":"Buddhika","year":"2021","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e1405","DOI":"10.1002\/widm.1405","article-title":"Data stream analysis: Foundations, major tasks and tools","volume":"11","author":"Bahri","year":"2021","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_25","first-page":"2346","article-title":"Learning under concept drift: A review","volume":"31","author":"Lu","year":"2018","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2523813","article-title":"A survey on concept drift adaptation","volume":"46","author":"Gama","year":"2014","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"964","DOI":"10.1007\/s10618-015-0448-4","article-title":"Characterizing concept drift","volume":"30","author":"Webb","year":"2016","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, P., Zhang, P., and Guo, L. (2012, January 26\u201328). Mining multi-label data streams using ensemble-based active learning. Proceedings of the 2012 SIAM International Conference on Data Mining, Anaheim, CA, USA.","DOI":"10.1137\/1.9781611972825.97"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1081","DOI":"10.1109\/TKDE.2017.2785795","article-title":"Multi-label learning with global and local label correlation","volume":"30","author":"Zhu","year":"2017","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"107506","DOI":"10.1016\/j.engappai.2023.107506","article-title":"A partition-based problem transformation algorithm for classifying imbalanced multi-label data","volume":"128","author":"Duan","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"121366","DOI":"10.1016\/j.eswa.2023.121366","article-title":"ECC++: An algorithm family based on ensemble of classifier chains for classifying imbalanced multi-label data","volume":"236","author":"Duan","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"4459","DOI":"10.1109\/TCYB.2020.3027509","article-title":"Towards class-imbalance aware multi-label learning","volume":"52","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Cybern."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1007\/s10994-011-5256-5","article-title":"Classifier chains for multi-label classification","volume":"85","author":"Read","year":"2011","journal-title":"Mach. Learn."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"683","DOI":"10.1613\/jair.1.12376","article-title":"Classifier chains: A review and perspectives","volume":"70","author":"Read","year":"2021","journal-title":"J. Artif. Intell. Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1016\/j.neucom.2012.08.010","article-title":"Weighted extreme learning machine for imbalance learning","volume":"101","author":"Zong","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1016\/j.neucom.2015.04.019","article-title":"AL-ELM: One uncertainty-based active learning algorithm using extreme learning machine","volume":"166","author":"Yu","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Gon\u00e7alves, E.C., Plastino, A., and Freitas, A.A. (2013, January 4\u20136). A genetic algorithm for optimizing the label ordering in multi-label classifier chains. Proceedings of the 2013 IEEE 25th International Conference on Tools with Artificial Intelligence, Herndon, VA, USA.","DOI":"10.1109\/ICTAI.2013.76"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"108395","DOI":"10.1016\/j.asoc.2021.108395","article-title":"Linear ordering problem based classifier chain using genetic algorithm for multi-label classification","volume":"117","author":"Mishra","year":"2022","journal-title":"Appl. Soft Comput."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2401","DOI":"10.1007\/s10462-022-10232-2","article-title":"Unsupervised concept drift detection for multi-label data streams","volume":"56","author":"Gulcan","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.ins.2020.07.014","article-title":"RCSMOTE: Range-Controlled synthetic minority over-sampling technique for handling the class imbalance problem","volume":"542","author":"Soltanzadeh","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.knosys.2015.10.012","article-title":"ODOC-ELM: Optimal decision outputs compensation-based extreme learning machine for classifying imbalanced data","volume":"92","author":"Yu","year":"2016","journal-title":"Knowl.-Based Syst."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"122778","DOI":"10.1016\/j.eswa.2023.122778","article-title":"A review of ensemble learning and data augmentation models for class imbalanced problems: Combination, implementation and evaluation","volume":"244","author":"Khan","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"3559","DOI":"10.1007\/s40747-021-00614-4","article-title":"Combining weighted SMOTE with ensemble learning for the class-imbalanced prediction of small business credit risk","volume":"9","author":"Abedin","year":"2023","journal-title":"Complex Intell. Syst."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.neucom.2005.12.126","article-title":"Extreme learning machine: Theory and applications","volume":"70","author":"Huang","year":"2006","journal-title":"Neurocomputing"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1016\/j.neunet.2014.10.001","article-title":"Trends in extreme learning machines: A review","volume":"61","author":"Huang","year":"2015","journal-title":"Neural Netw."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"120578","DOI":"10.1109\/ACCESS.2022.3222178","article-title":"DME: An adaptive and just-in-time weighted ensemble learning method for classifying block-based concept drift steam","volume":"10","author":"Feng","year":"2022","journal-title":"IEEE Access"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"28488","DOI":"10.1109\/ACCESS.2018.2839340","article-title":"LW-ELM: A fast and flexible cost-sensitive learning framework for classifying imbalanced data","volume":"6","author":"Yu","year":"2018","journal-title":"IEEE Access"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1016\/j.neucom.2020.04.098","article-title":"Boosting label weighted extreme learning machine for classifying multi-label imbalanced data","volume":"403","author":"Cheng","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"2044","DOI":"10.1016\/j.ins.2009.12.010","article-title":"Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power","volume":"180","author":"Luengo","year":"2010","journal-title":"Inf. Sci."},{"key":"ref_50","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","year":"2006","journal-title":"J. Mach. Learn. Res."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/4\/363\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:05:31Z","timestamp":1760029531000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/4\/363"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,29]]},"references-count":50,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["e27040363"],"URL":"https:\/\/doi.org\/10.3390\/e27040363","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2025,3,29]]}}}