{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:09:18Z","timestamp":1750306158790,"version":"3.41.0"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2017,1,12]],"date-time":"2017-01-12T00:00:00Z","timestamp":1484179200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Chinese National Program on Key Basic Research Project","award":["2013CB329304, 2014CB744604"],"award-info":[{"award-number":["2013CB329304, 2014CB744604"]}]},{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61402324, 61272265, 61373122"],"award-info":[{"award-number":["61402324, 61272265, 61373122"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2017,5,31]]},"abstract":"<jats:p>In many research and application areas, such as information retrieval and machine learning, we often encounter dealing with a probability distribution that is mixed by one distribution that is relevant to our task in hand and the other that is irrelevant and that we want to get rid of. Thus, it is an essential problem to separate the irrelevant distribution from the mixture distribution. This article is focused on the application in Information Retrieval, where relevance feedback is a widely used technique to build a refined query model based on a set of feedback documents. However, in practice, the relevance feedback set, even provided by users explicitly or implicitly, is often a mixture of relevant and irrelevant documents. Consequently, the resultant query model (typically a term distribution) is often a mixture rather than a true relevance term distribution, leading to a negative impact on the retrieval performance. To tackle this problem, we recently proposed a Distribution Separation Method (DSM), which aims to approximate the true relevance distribution by separating a seed irrelevance distribution from the mixture one. While it achieved a promising performance in an empirical evaluation with simulated explicit irrelevance feedback data, it has not been deployed in the scenario where one should automatically obtain the irrelevance feedback data. In this article, we propose a substantial extension of the basic DSM from two perspectives: developing a further regularization framework and deploying DSM in the automatic irrelevance feedback scenario. Specifically, in order to avoid the output distribution of DSM drifting away from the true relevance distribution when the quality of seed irrelevant distribution (as the input to DSM) is not guaranteed, we propose a DSM regularization framework to constrain the estimation for the relevance distribution. This regularization framework includes three algorithms, each corresponding to a regularization strategy incorporated in the objective function of DSM. In addition, we exploit DSM in automatic (i.e., pseudo) irrelevance feedback, by automatically detecting the seed irrelevant documents via three different document reranking methods. We have carried out extensive experiments based on various TREC datasets, in order to systematically evaluate the proposed methods. The experimental results demonstrate the effectiveness of our proposed approaches in comparison with various strong baselines.<\/jats:p>","DOI":"10.1145\/2994608","type":"journal-article","created":{"date-parts":[[2017,1,17]],"date-time":"2017-01-17T13:42:08Z","timestamp":1484660528000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["A Distribution Separation Method Using Irrelevance Feedback Data for Information Retrieval"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0228-9330","authenticated-orcid":false,"given":"Peng","family":"Zhang","sequence":"first","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qian","family":"Yu","sequence":"additional","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuexian","family":"Hou","sequence":"additional","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dawei","family":"Song","sequence":"additional","affiliation":[{"name":"Tianjin University and The Open University, Milton Keynes, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingfei","family":"Li","sequence":"additional","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bin","family":"Hu","sequence":"additional","affiliation":[{"name":"Lanzhou University, Lanzhou, Gansu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,1,12]]},"reference":[{"doi-asserted-by":"crossref","unstructured":"Nasreen Abdul-Jaleel James Allan W. Bruce Croft Fernando Diaz Leah Larkey Xiaoyan Li Donald Metzler Mark D. Smucker Trevor Strohman Howard Turtle and Courtney Wade. 2004. Umass at trec 2004: Novelty and hard. In TREC\u201904.  Nasreen Abdul-Jaleel James Allan W. Bruce Croft Fernando Diaz Leah Larkey Xiaoyan Li Donald Metzler Mark D. Smucker Trevor Strohman Howard Turtle and Courtney Wade. 2004. Umass at trec 2004: Novelty and hard. In TREC\u201904.","key":"e_1_2_2_1_1","DOI":"10.21236\/ADA460118"},{"doi-asserted-by":"publisher","key":"e_1_2_2_2_1","DOI":"10.1007\/3-540-45681-3_2"},{"doi-asserted-by":"publisher","key":"e_1_2_2_3_1","DOI":"10.1145\/2348283.2348312"},{"doi-asserted-by":"publisher","key":"e_1_2_2_4_1","DOI":"10.1145\/215206.215383"},{"key":"e_1_2_2_5_1","first-page":"23","article-title":"From ranknet to lambdarank to lambdamart: An overview","volume":"11","author":"Burges Christopher J. C.","year":"2010","journal-title":"Learning"},{"doi-asserted-by":"publisher","key":"e_1_2_2_6_1","DOI":"10.1145\/1526709.1526711"},{"doi-asserted-by":"publisher","key":"e_1_2_2_7_1","DOI":"10.1145\/1007730.1007733"},{"doi-asserted-by":"publisher","key":"e_1_2_2_8_1","DOI":"10.1007\/978-3-319-30671-1_38"},{"doi-asserted-by":"publisher","key":"e_1_2_2_9_1","DOI":"10.1145\/959258.959266"},{"doi-asserted-by":"publisher","key":"e_1_2_2_10_1","DOI":"10.1145\/248625.248650"},{"doi-asserted-by":"publisher","key":"e_1_2_2_11_1","DOI":"10.1145\/1059981.1059982"},{"doi-asserted-by":"publisher","key":"e_1_2_2_12_1","DOI":"10.1145\/1571941.1572003"},{"doi-asserted-by":"publisher","key":"e_1_2_2_13_1","DOI":"10.1016\/j.ins.2011.03.007"},{"doi-asserted-by":"publisher","key":"e_1_2_2_14_1","DOI":"10.1016\/S0306-4573(99)00056-4"},{"doi-asserted-by":"publisher","key":"e_1_2_2_15_1","DOI":"10.1145\/2063576.2063586"},{"doi-asserted-by":"publisher","key":"e_1_2_2_16_1","DOI":"10.1145\/383952.383970"},{"doi-asserted-by":"publisher","key":"e_1_2_2_17_1","DOI":"10.1145\/383952.383972"},{"doi-asserted-by":"publisher","key":"e_1_2_2_18_1","DOI":"10.1145\/1857947.1857951"},{"doi-asserted-by":"publisher","key":"e_1_2_2_19_1","DOI":"10.1145\/1835449.1835546"},{"doi-asserted-by":"publisher","key":"e_1_2_2_20_1","DOI":"10.1561\/1500000023"},{"volume-title":"Proceedings of the ACM 11th Text Retrieval Conference (TREC\u201902)","year":"2002","author":"Ogilvie Paul","key":"e_1_2_2_21_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_22_1","DOI":"10.1145\/335191.335437"},{"unstructured":"Cornelis Joost van Rijsbergen. 1979. Information Retrieval. Butterworth-Heinemann.  Cornelis Joost van Rijsbergen. 1979. Information Retrieval. Butterworth-Heinemann.","key":"e_1_2_2_23_1"},{"volume-title":"The SMART Retrieval System\u2014Experiments in Automatic Document Processing","author":"Rocchio Joseph John","key":"e_1_2_2_24_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_25_1","DOI":"10.2307\/2685263"},{"doi-asserted-by":"publisher","key":"e_1_2_2_26_1","DOI":"10.1145\/258525.258530"},{"doi-asserted-by":"publisher","key":"e_1_2_2_27_1","DOI":"10.1007\/978-3-540-78646-7_31"},{"doi-asserted-by":"publisher","key":"e_1_2_2_28_1","DOI":"10.1016\/j.eswa.2013.03.001"},{"doi-asserted-by":"publisher","key":"e_1_2_2_29_1","DOI":"10.1145\/1148170.1148201"},{"doi-asserted-by":"publisher","key":"e_1_2_2_30_1","DOI":"10.1007\/s10115-003-0115-8"},{"doi-asserted-by":"publisher","key":"e_1_2_2_31_1","DOI":"10.1145\/1571941.1571963"},{"doi-asserted-by":"publisher","key":"e_1_2_2_32_1","DOI":"10.1145\/1390334.1390374"},{"doi-asserted-by":"publisher","key":"e_1_2_2_33_1","DOI":"10.1145\/1076034.1076044"},{"doi-asserted-by":"publisher","key":"e_1_2_2_34_1","DOI":"10.1145\/502585.502654"},{"doi-asserted-by":"publisher","key":"e_1_2_2_35_1","DOI":"10.1145\/383952.384019"},{"doi-asserted-by":"publisher","key":"e_1_2_2_36_1","DOI":"10.1145\/1014052.1014150"},{"doi-asserted-by":"publisher","key":"e_1_2_2_37_1","DOI":"10.1145\/1571941.1571962"},{"doi-asserted-by":"publisher","key":"e_1_2_2_38_1","DOI":"10.3390\/e18040105"},{"doi-asserted-by":"publisher","key":"e_1_2_2_39_1","DOI":"10.1016\/j.ipm.2007.12.003"},{"volume-title":"Jimmy Xiangji Huang, and Ben He","year":"2011","author":"Zhao Jiashu","key":"e_1_2_2_40_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_41_1","DOI":"10.1007\/978-3-642-12275-0_32"},{"doi-asserted-by":"crossref","unstructured":"Guido Zuccon Leif Azzopardi and C. J. van Rijsbergen. 2010. Has portfolio theory got any principles?. In SIGIR\u201910. 755--756.  Guido Zuccon Leif Azzopardi and C. J. van Rijsbergen. 2010. Has portfolio theory got any principles?. In SIGIR\u201910. 755--756.","key":"e_1_2_2_42_1","DOI":"10.1145\/1835449.1835600"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2994608","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2994608","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:39:48Z","timestamp":1750217988000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2994608"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,1,12]]},"references-count":42,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2017,5,31]]}},"alternative-id":["10.1145\/2994608"],"URL":"https:\/\/doi.org\/10.1145\/2994608","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"type":"print","value":"2157-6904"},{"type":"electronic","value":"2157-6912"}],"subject":[],"published":{"date-parts":[[2017,1,12]]},"assertion":[{"value":"2015-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-01-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}