{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T05:05:51Z","timestamp":1782709551078,"version":"3.54.5"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2009,5,1]],"date-time":"2009-05-01T00:00:00Z","timestamp":1241136000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["1R01-GM079688-01"],"award-info":[{"award-number":["1R01-GM079688-01"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000145","name":"Division of Information and Intelligent Systems","doi-asserted-by":"publisher","award":["IIS-0643494"],"award-info":[{"award-number":["IIS-0643494"]}],"id":[{"id":"10.13039\/100000145","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001459","name":"Ministry of Education - Singapore","doi-asserted-by":"publisher","award":["RG67\/07"],"award-info":[{"award-number":["RG67\/07"]}],"id":[{"id":"10.13039\/501100001459","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2009,5]]},"abstract":"<jats:p>Support vector machine (SVM) active learning is one popular and successful technique for relevance feedback in content-based image retrieval (CBIR). Despite the success, conventional SVM active learning has two main drawbacks. First, the performance of SVM is usually limited by the number of labeled examples. It often suffers a poor performance for the small-sized labeled examples, which is the case in relevance feedback. Second, conventional approaches do not take into account the redundancy among examples, and could select multiple examples that are similar (or even identical). In this work, we propose a novel scheme for explicitly addressing the drawbacks. It first learns a kernel function from a mixture of labeled and unlabeled data, and therefore alleviates the problem of small-sized training data. The kernel will then be used for a batch mode active learning method to identify the most informative and diverse examples via a min-max framework. Two novel algorithms are proposed to solve the related combinatorial optimization: the first approach approximates the problem into a quadratic program, and the second solves the combinatorial optimization approximately by a greedy algorithm that exploits the merits of submodular functions. Extensive experiments with image retrieval using both natural photo images and medical images show that the proposed algorithms are significantly more effective than the state-of-the-art approaches. A demo is available at http:\/\/msm.cais.ntu.edu.sg\/LSCBIR\/.<\/jats:p>","DOI":"10.1145\/1508850.1508854","type":"journal-article","created":{"date-parts":[[2009,5,19]],"date-time":"2009-05-19T16:47:42Z","timestamp":1242751662000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":103,"title":["Semisupervised SVM batch mode active learning with applications to image retrieval"],"prefix":"10.1145","volume":"27","author":[{"given":"Steven C. H.","family":"Hoi","sequence":"first","affiliation":[{"name":"Nanyang Technological University, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rong","family":"Jin","sequence":"additional","affiliation":[{"name":"Michigan State University, East Lansing, MI"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianke","family":"Zhu","sequence":"additional","affiliation":[{"name":"Chinese University of Hong Kong, Hong Kong, S.A.R"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael R.","family":"Lyu","sequence":"additional","affiliation":[{"name":"Chinese University of Hong Kong, Hong Kong, S.A.R"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2009,5,19]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Boyd S. and Vandenberghe L. 2004. Convex Optimization. Cambridge University Press Cambridge U.K.   Boyd S. and Vandenberghe L. 2004. Convex Optimization. Cambridge University Press Cambridge U.K.","DOI":"10.1017\/CBO9780511804441"},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the International Conference on Machine Learning (ICML).","author":"Brinker K.","year":"2003","unstructured":"Brinker , K. 2003 . Incorporating diversity in active learning with support vector machines . In Proceedings of the International Conference on Machine Learning (ICML). Brinker, K. 2003. Incorporating diversity in active learning with support vector machines. In Proceedings of the International Conference on Machine Learning (ICML)."},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Chapelle O. Sch\u00f6lkopf B. and Zien A. 2006. Semisupervised Learning. MIT Press Cambridge MA.   Chapelle O. Sch\u00f6lkopf B. and Zien A. 2006. Semisupervised Learning. MIT Press Cambridge MA.","DOI":"10.7551\/mitpress\/9780262033589.001.0001"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the NIPS.","author":"Cohn D. A.","unstructured":"Cohn , D. A. , Ghahramani , Z. , and Jordan , M. I . 1995. Active learning with statistical models . In Proceedings of the NIPS. Cohn, D. A., Ghahramani, Z., and Jordan, M. I. 1995. Active learning with statistical models. In Proceedings of the NIPS."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/11788034_13"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1027527.1027664"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1143844.1143897"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1027527.1027533"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1135777.1135870"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.44"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1150402.1150426"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2006.53"},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the IEEE International Conference on Image Processing (ICIP).","volume":"3","author":"Hong P.","unstructured":"Hong , P. , Tian , Q. , and Huang , T. S . 2000. Incorporate support vector machines to content-based image retrieval with relevant feedback . In Proceedings of the IEEE International Conference on Image Processing (ICIP). Vol. 3 . 750--753. Hong, P., Tian, Q., and Huang, T. S. 2000. Incorporate support vector machines to content-based image retrieval with relevant feedback. In Proceedings of the IEEE International Conference on Image Processing (ICIP). Vol. 3. 750--753."},{"key":"e_1_2_1_14_1","volume-title":"Proceedings of the IEEE International Conference on Image Processing (ICIP).","volume":"3","author":"Huang T. S.","unstructured":"Huang , T. S. and Zhou , X. S . 2001. Image retrieval by relevance feedback: From heuristic weight adjustment to optimal learning methods . In Proceedings of the IEEE International Conference on Image Processing (ICIP). Vol. 3 . 2--5. Huang, T. S. and Zhou, X. S. 2001. Image retrieval by relevance feedback: From heuristic weight adjustment to optimal learning methods. In Proceedings of the IEEE International Conference on Image Processing (ICIP). Vol. 3. 2--5."},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the 24th International Conference on Very Large Data Bases (VLDB). 218--227","author":"Ishikawa Y.","unstructured":"Ishikawa , Y. , Subramanya , R. , and Faloutsos , C . 1998. MindReader: Querying databases through multiple examples . In Proceedings of the 24th International Conference on Very Large Data Bases (VLDB). 218--227 . Ishikawa, Y., Subramanya, R., and Faloutsos, C. 1998. MindReader: Querying databases through multiple examples. In Proceedings of the 24th International Conference on Very Large Data Bases (VLDB). 218--227."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0031-3203(03)00043-8"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the International Joint Conference on Neural Networks (IJCNN","author":"Laaksonen J.","unstructured":"Laaksonen , J. , Koskela , M. , and Oja , E . 1999. Picsom: Self-organizing maps for content-based image retrieval . In Proceedings of the International Joint Conference on Neural Networks (IJCNN , Washington, DC). Laaksonen, J., Koskela, M., and Oja, E. 1999. Picsom: Self-organizing maps for content-based image retrieval. In Proceedings of the International Joint Conference on Neural Networks (IJCNN, Washington, DC)."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1126004.1126005"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2006.881938"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of AAAI.","author":"Liere R.","unstructured":"Liere , R. and Tadepalli , P . 1997. Active learning with committees for text categorization . In Proceedings of AAAI. Liere, R. and Tadepalli, P. 1997. Active learning with committees for text categorization. In Proceedings of AAAI."},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the IEEE Workshop on Content-Based Access of lmage and Video Libraries. 68--72","author":"MacArthur S.","unstructured":"MacArthur , S. , Brodley , C. , and Shyu , C . 2000. Relevance feedback decision trees in content-based image retrieval . In Proceedings of the IEEE Workshop on Content-Based Access of lmage and Video Libraries. 68--72 . MacArthur, S., Brodley, C., and Shyu, C. 2000. Relevance feedback decision trees in content-based image retrieval. In Proceedings of the IEEE Workshop on Content-Based Access of lmage and Video Libraries. 68--72."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.531803"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the International Conference on Machine Learning (ICML).","author":"McCallum A. K.","unstructured":"McCallum , A. K. and Nigam , K . 1998. Employing EM and pool-based active learning for text classification . In Proceedings of the International Conference on Machine Learning (ICML). McCallum, A. K. and Nigam, K. 1998. Employing EM and pool-based active learning for text classification. In Proceedings of the International Conference on Machine Learning (ICML)."},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the 7th Workshop of Cross-Language Evaluation Forum (CLEF).","author":"Muller H.","unstructured":"Muller , H. , Deselaers , T. , Lehmann , T. , Clough , P. , and Hersh , W . 2007. Overview of the imageclefmed 2006 medical retrieval annotation tasks . In Proceedings of the 7th Workshop of Cross-Language Evaluation Forum (CLEF). Muller, H., Deselaers, T., Lehmann, T., Clough, P., and Hersh, W. 2007. Overview of the imageclefmed 2006 medical retrieval annotation tasks. In Proceedings of the 7th Workshop of Cross-Language Evaluation Forum (CLEF)."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01588971"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-006-0043-1"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the International Conference on Multimedia and Expo (ICME).","volume":"1","author":"Qian F.","unstructured":"Qian , F. , Li , M. , Zhang , L. , Zhang , H.-J. , and Zhang , B . 2002. Gaussian mixture model for relevance feedback in image retrieval . In Proceedings of the International Conference on Multimedia and Expo (ICME). Vol. 1 . 229--232. Qian, F., Li, M., Zhang, L., Zhang, H.-J., and Zhang, B. 2002. Gaussian mixture model for relevance feedback in image retrieval. In Proceedings of the International Conference on Multimedia and Expo (ICME). Vol. 1. 229--232."},{"key":"e_1_2_1_28_1","volume-title":"II. In Proceedings of the IEEE International Conference on Image Processing. 815--818","author":"Rui Y.","unstructured":"Rui , Y. , Huang , T. , and Mehrotra , S . 1997. Content-based image retrieval with relevance feedback in mars . II. In Proceedings of the IEEE International Conference on Image Processing. 815--818 . Rui, Y., Huang, T., and Mehrotra, S. 1997. Content-based image retrieval with relevance feedback in mars. II. In Proceedings of the IEEE International Conference on Image Processing. 815--818."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/76.718510"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 17th International Conference on Machine Learning (ICML).","author":"Schohn G.","unstructured":"Schohn , G. and Cohn , D . 2000. Less is more: Active learning with support vector machines . In Proceedings of the 17th International Conference on Machine Learning (ICML). Schohn, G. and Cohn, D. 2000. Less is more: Active learning with support vector machines. In Proceedings of the 17th International Conference on Machine Learning (ICML)."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102455"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.895972"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2007.1003"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.5555\/1018428.1020883"},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Tao D.","unstructured":"Tao , D. and Tang , X . 2004b. Random sampling based svm for relevance feedback image retrieval . In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). Tao, D. and Tang, X. 2004b. Random sampling based svm for relevance feedback image retrieval. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2005.861375"},{"key":"e_1_2_1_37_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","volume":"1","author":"Tieu K.","unstructured":"Tieu , K. and Viola , P . 2000. Boosting image retrieval . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 1 . 228--235. Tieu, K. and Viola, P. 2000. Boosting image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 1. 228--235."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/500141.500159"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the 17th International Conference on Machine Learning (ICML).","author":"Tong S.","unstructured":"Tong , S. and Koller , D . 2000. Support vector machine active learning with applications to text classification . In Proceedings of the 17th International Conference on Machine Learning (ICML). Tong, S. and Koller, D. 2000. Support vector machine active learning with applications to text classification. In Proceedings of the 17th International Conference on Machine Learning (ICML)."},{"key":"e_1_2_1_40_1","volume-title":"Statistical Learning Theory","author":"Vapnik V. N.","unstructured":"Vapnik , V. N. 1998. Statistical Learning Theory . Wiley , New York, NY . Vapnik, V. N. 1998. Statistical Learning Theory. Wiley, New York, NY."},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of NIPS.","author":"Vasconcelos N.","unstructured":"Vasconcelos , N. and Lippman , A . 1999. Learning from user feedback in image retrieval systems . In Proceedings of NIPS. Vasconcelos, N. and Lippman, A. 1999. Learning from user feedback in image retrieval systems. In Proceedings of NIPS."},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of the CVPR.","author":"Wang L.","unstructured":"Wang , L. , Chan , K. L. , and Zhang , Z . 2003. Bootstrapping SVM active learning by incorporating unlabelled images for image retrieval . In Proceedings of the CVPR. Wang, L., Chan, K. L., and Zhang, Z. 2003. Bootstrapping SVM active learning by incorporating unlabelled images for image retrieval. In Proceedings of the CVPR."},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Wu Y.","unstructured":"Wu , Y. , Tian , Q. , and Huang , T. S . 2000. Discriminant-EM algorithm with application to image retrieval . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Wu, Y., Tian, Q., and Huang, T. S. 2000. Discriminant-EM algorithm with application to image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/957013.957087"},{"key":"e_1_2_1_45_1","volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS).","author":"Yuhong Guo D. S.","year":"2007","unstructured":"Yuhong Guo , D. S. 2007 . Discriminative batch mode active learning . In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS). Yuhong Guo, D. S. 2007. Discriminative batch mode active learning. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS)."},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the International Conference on Image Processing (ICIP).","volume":"2","author":"Zhang L.","unstructured":"Zhang , L. , Lin , F. , and Zhang , B . 2001. Support vector machine learning for image retrieval . In Proceedings of the International Conference on Image Processing (ICIP). Vol. 2 . 721--724. Zhang, L., Lin, F., and Zhang, B. 2001. Support vector machine learning for image retrieval. In Proceedings of the International Conference on Image Processing (ICIP). Vol. 2. 721--724."},{"key":"e_1_2_1_47_1","volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS).","author":"Zhang T.","unstructured":"Zhang , T. and Ando , R. K . 2005. Analysis of spectral kernel design based semisupervised learning . In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS). Zhang, T. and Ando, R. K. 2005. Analysis of spectral kernel design based semisupervised learning. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS)."},{"key":"e_1_2_1_48_1","volume-title":"Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Zhou X. S.","unstructured":"Zhou , X. S. and Huang , T. S . 2001. Small sample learning during multimedia retrieval using biasmap . In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). Zhou, X. S. and Huang, T. S. 2001. Small sample learning during multimedia retrieval using biasmap. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00530-002-0070-3"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1508850.1508854","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1508850.1508854","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T13:29:42Z","timestamp":1750253382000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1508850.1508854"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,5]]},"references-count":49,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2009,5]]}},"alternative-id":["10.1145\/1508850.1508854"],"URL":"https:\/\/doi.org\/10.1145\/1508850.1508854","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2009,5]]},"assertion":[{"value":"2008-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2008-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2009-05-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}