{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T18:24:41Z","timestamp":1775240681431,"version":"3.50.1"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T00:00:00Z","timestamp":1615507200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T00:00:00Z","timestamp":1615507200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100010909","name":"Young Scientists Fund","doi-asserted-by":"publisher","award":["61806005"],"award-info":[{"award-number":["61806005"]}],"id":[{"id":"10.13039\/501100010909","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001695","name":"Japan Science and Technology Corporation","doi-asserted-by":"publisher","award":["191400000190"],"award-info":[{"award-number":["191400000190"]}],"id":[{"id":"10.13039\/501100001695","id-type":"DOI","asserted-by":"publisher"}]},{"name":"JST-AIP","award":["JPMJCR19U4"],"award-info":[{"award-number":["JPMJCR19U4"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Data Min Knowl Disc"],"published-print":{"date-parts":[[2021,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Multi-label learning deals with data examples which are associated with multiple class labels simultaneously. Despite the success of existing approaches to multi-label learning, there is still a problem neglected by researchers, i.e., not only are some of the values of observed labels missing, but also some of the labels are completely unobserved for the training data. We refer to the problem as<jats:italic>multi-label learning with missing and completely unobserved labels<\/jats:italic>, and argue that it is necessary to discover these completely unobserved labels in order to mine useful knowledge and make a deeper understanding of what is behind the data. In this paper, we propose a new approach named MCUL to solve multi-label learning with Missing and Completely Unobserved Labels. We try to discover the unobserved labels of a multi-label data set with a clustering based regularization term and describe the semantic meanings of them based on the label-specific features learned by MCUL, and overcome the problem of missing labels by exploiting label correlations. The proposed method MCUL can predict both the observed and newly discovered labels simultaneously for unseen data examples. Experimental results validated over ten benchmark datasets demonstrate that the proposed method can outperform other state-of-the-art approaches on observed labels and obtain an acceptable performance on the new discovered labels as well.<\/jats:p>","DOI":"10.1007\/s10618-021-00743-x","type":"journal-article","created":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T17:03:08Z","timestamp":1615568588000},"page":"1061-1086","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["Multi-label learning with missing and completely unobserved labels"],"prefix":"10.1007","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2022-5747","authenticated-orcid":false,"given":"Jun","family":"Huang","sequence":"first","affiliation":[]},{"given":"Linchuan","family":"Xu","sequence":"additional","affiliation":[]},{"given":"Kun","family":"Qian","sequence":"additional","affiliation":[]},{"given":"Jing","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Kenji","family":"Yamanishi","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,3,12]]},"reference":[{"issue":"1","key":"743_CR1","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1137\/080716542","volume":"2","author":"A Beck","year":"2009","unstructured":"Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci 2(1):183\u2013202","journal-title":"SIAM J Imaging Sci"},{"key":"743_CR2","unstructured":"Bhatia K, Jain H, Kar P, Varma M, Jain P (2016) Sparse local embeddings for extreme multi-label classification. In: Neural information processing systems (NIPS), pp 730\u2013738"},{"issue":"9","key":"743_CR3","doi-asserted-by":"publisher","first-page":"1757","DOI":"10.1016\/j.patcog.2004.03.009","volume":"37","author":"MR Boutell","year":"2004","unstructured":"Boutell MR, Luo JB, Shen XP, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757\u20131771","journal-title":"Pattern Recognit"},{"issue":"1","key":"743_CR4","doi-asserted-by":"publisher","first-page":"1193","DOI":"10.1007\/s10994-018-5773-6","volume":"108","author":"HM Chu","year":"2019","unstructured":"Chu HM, Huang KH, Lin HT (2019) Dynamic principal projection for cost-sensitive online multi-label classification. Mach Learn 108(1):1193\u20131230","journal-title":"Mach Learn"},{"key":"743_CR5","doi-asserted-by":"crossref","unstructured":"Da Q, Yu Y, Zhou ZH (2014) Learning with augmented class by exploiting unlabeled data. In: AAAI conference on artificial intelligence (AAAI), pp 2373\u20132379","DOI":"10.1609\/aaai.v28i1.8997"},{"key":"743_CR6","doi-asserted-by":"crossref","unstructured":"Decubber S, Mortier T, Dembczy\u0144ski K, Waegeman W (2019) Deep f-measure maximization in multi-label classification: A comparative study. In: European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD), pp 290\u2013305","DOI":"10.1007\/978-3-030-10925-7_18"},{"key":"743_CR7","unstructured":"Dembczy\u0144ski K, Cheng W, H\u00fcllermeier E (2010) Bayes optimal multilabel classification via probabilistic classifier chains. In: International conference on machine learning (ICML), pp 1609\u20131614"},{"key":"743_CR8","first-page":"1","volume":"7","author":"J Dem\u0161ar","year":"2006","unstructured":"Dem\u0161ar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1\u201330","journal-title":"J Mach Learn Res"},{"key":"743_CR9","doi-asserted-by":"crossref","unstructured":"Ding C, He X, Simon HD (2005) On the equivalence of nonnegative matrix factorization and spectral clustering. In: SIAM international conference on data mining (SDM), pp 606\u2013610","DOI":"10.1137\/1.9781611972757.70"},{"key":"743_CR10","unstructured":"Elisseeff A, Jason W (2001) A kernel method for multi-labelled classification. In: Neural information processing systems (NIPS), pp 681\u2013687"},{"issue":"2","key":"743_CR11","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1007\/s10994-008-5064-8","volume":"73","author":"J F\u00fcrnkranz","year":"2008","unstructured":"F\u00fcrnkranz J, H\u00fcllermeier E, Loza Menc\u00eda E, Brinker K (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133\u2013153","journal-title":"Mach Learn"},{"issue":"3","key":"743_CR12","doi-asserted-by":"publisher","first-page":"52:1","DOI":"10.1145\/2716262","volume":"47","author":"E Gibaja","year":"2015","unstructured":"Gibaja E, Ventura S (2015) A tutorial on multilabel learning. ACM Comput Surv 47(3):52:1\u201352:38","journal-title":"ACM Comput Surv"},{"key":"743_CR13","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-41111-8","volume-title":"Multilabel classification: problem analysis, metrics and techniques","author":"F Herrera","year":"2016","unstructured":"Herrera F, Charte F, Rivera AJ, del Jesus MJ (2016) Multilabel classification: problem analysis, metrics and techniques. Springer, Berlin"},{"issue":"12","key":"743_CR14","doi-asserted-by":"publisher","first-page":"3309","DOI":"10.1109\/TKDE.2016.2608339","volume":"28","author":"J Huang","year":"2016","unstructured":"Huang J, Li G, Huang Q, Wu X (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309\u20133323","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"3","key":"743_CR15","doi-asserted-by":"publisher","first-page":"876","DOI":"10.1109\/TCYB.2017.2663838","volume":"48","author":"J Huang","year":"2018","unstructured":"Huang J, Li G, Huang Q, Wu X (2018) Joint feature selection and classification for multilabel learning. IEEE Trans Cybern 48(3):876\u2013889","journal-title":"IEEE Trans Cybern"},{"key":"743_CR16","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1016\/j.ins.2019.04.021","volume":"492","author":"J Huang","year":"2019","unstructured":"Huang J, Qin F, Zheng X, Cheng Z, Yuan Z, Zhang W, Huang Q (2019) Improving multi-label classification with missing labels by learning label-specific features. Inf Sci 492:124\u2013146","journal-title":"Inf Sci"},{"key":"743_CR17","unstructured":"Liu W (2019) Copula multi-label learning. In: Neural information processing systems (NIPS), pp 6334\u20136343"},{"key":"743_CR18","unstructured":"Liu W, Shen X (2019) Sparse extreme multi-label learning with oracle property. In: International conference on machine learning (ICML), pp 4032\u20134041"},{"key":"743_CR19","doi-asserted-by":"crossref","unstructured":"Masera L, Blanzieri E (2019) Awx: an integrated approach to hierarchical-multilabel classification. In: European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD), pp 322\u2013336","DOI":"10.1007\/978-3-030-10925-7_20"},{"key":"743_CR20","doi-asserted-by":"crossref","unstructured":"Mu X, Zhu F, Du J, Lim EP, Zhou ZH (2017) Streaming classification with emerging new class by class matrix sketching. In: AAAI conference on artificial intelligence (AAAI), pp 2373\u20132379","DOI":"10.1609\/aaai.v31i1.10842"},{"key":"743_CR21","unstructured":"Nguyen V, H\u00fcllermeier E (2019) Reliable multi-label classification: prediction with partial abstention. CoRR arXiv:1904.09235"},{"key":"743_CR22","unstructured":"Park LAF, Read J (2019) A blended metric for multi-label optimisation and evaluation. In: European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD), pp 719\u2013734"},{"key":"743_CR23","unstructured":"Pham A, Raich R, Fern X, Arriaga JP (2015) Multi-instance multi-label learning in the presence of novel class instances. In: International conference on machine learning (ICML), pp 2427\u20132435"},{"key":"743_CR24","doi-asserted-by":"crossref","unstructured":"Qu W, Zhang Y, Zhu J, Qiu Q (2009) Mining multi-label concept-drifting data streams using dynamic classifier ensemble. In: Asian conference on machine learning (ACML), pp 308\u2013321","DOI":"10.1007\/978-3-642-05224-8_24"},{"key":"743_CR25","doi-asserted-by":"crossref","unstructured":"Read J, Pfahringer B, Holmes G (2008) Multi-label classification using ensembles of pruned sets. In: IEEE international conference on data mining (ICDM), pp 995\u20131000","DOI":"10.1109\/ICDM.2008.74"},{"key":"743_CR26","doi-asserted-by":"crossref","unstructured":"Read J, Pfahringer B, Holmes G, Frank E (2009) Classifier chains for multi-label classification. In: European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD), pp 254\u2013269","DOI":"10.1007\/978-3-642-04174-7_17"},{"key":"743_CR27","unstructured":"Read J, Bifet A, Holmes G, Pfahringer B (2011) Streaming multi-label classification. In: Proceedings of 2nd workshop applications of pattern analysis, vol 17, pp 19\u201325"},{"key":"743_CR28","doi-asserted-by":"crossref","unstructured":"Shi Z, Xue Y, Wen Y, Cai G (2014) Efficient class incremental learning for multi-label classification of evolving data streams. In: International joint conference on neural network (IJCNN), pp 2093\u20132099","DOI":"10.1109\/IJCNN.2014.6889926"},{"key":"743_CR29","doi-asserted-by":"crossref","unstructured":"Sun Y, Zhang Y, Zhou Z (2010) Multi-label learning with weak label. In: AAAI conference on artificial intelligence (AAAI), pp 593\u2013598","DOI":"10.1609\/aaai.v24i1.7699"},{"key":"743_CR30","doi-asserted-by":"crossref","unstructured":"Tan Q, Yu G, Domeniconi C, Wang J, Zhang Z (2018) Incomplete multi-view weak-label learning. In: International joint conference on artificial intelligence (IJCAI), pp 2703\u20132709","DOI":"10.24963\/ijcai.2018\/375"},{"key":"743_CR31","unstructured":"Tsoumakas G, Katakis I, Vlahavas I (2010) Mining multi-label data. In: Data mining and knowledge discovery handbook, pp 667\u2013685"},{"issue":"7","key":"743_CR32","doi-asserted-by":"publisher","first-page":"1079","DOI":"10.1109\/TKDE.2010.164","volume":"23","author":"G Tsoumakas","year":"2011","unstructured":"Tsoumakas G, Katakis I, Vlahavas L (2011) Random k-labelsets for multilabel classification. IEEE Trans Knowl Data Eng 23(7):1079\u20131089","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"743_CR33","doi-asserted-by":"crossref","unstructured":"Wei T, Tu W, Li Y (2019) Learning for tail label data: a label-specific feature approach. In: International joint conference on artificial intelligence (IJCAI), pp 3842\u20133848","DOI":"10.24963\/ijcai.2019\/533"},{"key":"743_CR34","doi-asserted-by":"crossref","unstructured":"Wu X, Chen Q, Hu Y, Wang D, Chang X, Wang X, Zhang ML (2019) Multi-view multi-label learning with view-specific information extraction. In: International joint conference on artificial intelligence (IJCAI), pp 3884\u20133890","DOI":"10.24963\/ijcai.2019\/539"},{"key":"743_CR35","unstructured":"Wydmuch M, Jasinska K, Kuznetsov M, Busa-Fekete R, Dembczynski K (2018) A no-regret generalization of hierarchical softmax to extreme multi-label classification. In: Neural information processing systems (NIPS), pp 6355\u20136366"},{"key":"743_CR36","unstructured":"Xu M, Jin R, Zhou Z (2013) Speedup matrix completion with side information: application to multi-label learning. In: Neural information processing systems (NIPS), pp 2301\u20132309"},{"key":"743_CR37","unstructured":"Yu H, Jain P, Kar P, Dhillon IS (2014) Large-scale multi-label learning with missing labels. In: International conference on machine learning (ICML), pp 593\u2013601"},{"key":"743_CR38","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2951668","author":"J Zhang","year":"2019","unstructured":"Zhang J, Wu X (2019) Multi-label truth inference for crowdsourcing using mixture models. IEEE Trans Knowl Data Eng. https:\/\/doi.org\/10.1109\/TKDE.2019.2951668","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"1","key":"743_CR39","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1109\/TPAMI.2014.2339815","volume":"37","author":"ML Zhang","year":"2015","unstructured":"Zhang ML, Wu L (2015) Lift: Multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107\u2013120","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"10","key":"743_CR40","doi-asserted-by":"publisher","first-page":"1338","DOI":"10.1109\/TKDE.2006.162","volume":"18","author":"ML Zhang","year":"2006","unstructured":"Zhang ML, Zhou ZH (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338\u20131351","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"7","key":"743_CR41","doi-asserted-by":"publisher","first-page":"2038","DOI":"10.1016\/j.patcog.2006.12.019","volume":"40","author":"ML Zhang","year":"2007","unstructured":"Zhang ML, Zhou ZH (2007) Ml-knn: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038\u20132048","journal-title":"Pattern Recognit"},{"issue":"8","key":"743_CR42","doi-asserted-by":"publisher","first-page":"1819","DOI":"10.1109\/TKDE.2013.39","volume":"26","author":"ML Zhang","year":"2014","unstructured":"Zhang ML, Zhou ZH (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819\u20131837","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"743_CR43","unstructured":"Zhang Y, Henao R, Gan Z, Li Y, Carin L (2018) Multi-label learning from medical plain text with convolutional residual models. In: Proceedings of the 3rd machine learning for healthcare conference, pp 280\u2013294"},{"key":"743_CR44","unstructured":"Zhang YJ, Zhao P, Zhou ZH (2020) Exploratory machine learning with unknown unknowns. CoRR arXiv:2002.01605"},{"key":"743_CR45","doi-asserted-by":"crossref","unstructured":"Zhu Y, Ting KM, Zhou ZH (2017) Discover multiple novel labels in multi-instance multi-label learning. In: AAAI conference on artificial intelligence (AAAI), pp 2977\u20132983","DOI":"10.1609\/aaai.v31i1.10838"},{"issue":"6","key":"743_CR46","doi-asserted-by":"publisher","first-page":"1081","DOI":"10.1109\/TKDE.2017.2785795","volume":"30","author":"Y Zhu","year":"2018","unstructured":"Zhu Y, Kwok JT, Zhou ZH (2018) Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng 30(6):1081\u20131094","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"10","key":"743_CR47","doi-asserted-by":"publisher","first-page":"1901","DOI":"10.1109\/TKDE.2018.2810872","volume":"30","author":"Y Zhu","year":"2018","unstructured":"Zhu Y, Ting KM, Zhou ZH (2018) Multi-label learning with emerging new labels. IEEE Trans Knowl Data Eng 30(10):1901\u20131914","journal-title":"IEEE Trans Knowl Data Eng"}],"container-title":["Data Mining and Knowledge Discovery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-021-00743-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10618-021-00743-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-021-00743-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T09:31:24Z","timestamp":1671615084000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10618-021-00743-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,12]]},"references-count":47,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,5]]}},"alternative-id":["743"],"URL":"https:\/\/doi.org\/10.1007\/s10618-021-00743-x","relation":{},"ISSN":["1384-5810","1573-756X"],"issn-type":[{"value":"1384-5810","type":"print"},{"value":"1573-756X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,12]]},"assertion":[{"value":"30 April 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 February 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 March 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}