{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T17:35:03Z","timestamp":1754156103784,"version":"3.41.2"},"reference-count":45,"publisher":"Emerald","issue":"4","license":[{"start":{"date-parts":[[2016,11,7]],"date-time":"2016-11-07T00:00:00Z","timestamp":1478476800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJWIS"],"published-print":{"date-parts":[[2016,11,7]]},"abstract":"<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title>\n<jats:p>Learning to rank algorithms inherently faces many challenges. The most important challenges could be listed as high-dimensionality of the training data, the dynamic nature of Web information resources and lack of click-through data. High dimensionality of the training data affects effectiveness and efficiency of learning algorithms. Besides, most of learning to rank benchmark datasets do not include click-through data as a very rich source of information about the search behavior of users while dealing with the ranked lists of search results. To deal with these limitations, this paper aims to introduce a novel learning to rank algorithm by using a set of complex click-through features in a reinforcement learning (RL) model. These features are calculated from the existing click-through information in the data set or even from data sets without any explicit click-through information.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title>\n<jats:p>The proposed ranking algorithm (QRC-Rank) applies RL techniques on a set of calculated click-through features. QRC-Rank is as a two-steps process. In the first step, Transformation phase, a compact benchmark data set is created which contains a set of click-through features. These feature are calculated from the original click-through information available in the data set and constitute a compact representation of click-through information. To find most effective click-through feature, a number of scenarios are investigated. The second phase is Model-Generation, in which a RL model is built to rank the documents. This model is created by applying temporal difference learning methods such as Q-Learning and SARSA.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Findings<\/jats:title>\n<jats:p>The proposed learning to rank method, QRC-rank, is evaluated on WCL2R and LETOR4.0 data sets. Experimental results demonstrate that QRC-Rank outperforms the state-of-the-art learning to rank methods such as SVMRank, RankBoost, ListNet and AdaRank based on the precision and normalized discount cumulative gain evaluation criteria. The use of the click-through features calculated from the training data set is a major contributor to the performance of the system.<\/jats:p>\n<\/jats:sec>\n<jats:sec>\n<jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title>\n<jats:p>In this paper, we have demonstrated the viability of the proposed features that provide a compact representation for the click through data in a learning to rank application. These compact click-through features are calculated from the original features of the learning to rank benchmark data set. In addition, a Markov Decision Process model is proposed for the learning to rank problem using RL, including the sets of states, actions, rewarding strategy and the transition function.<\/jats:p>\n<\/jats:sec>","DOI":"10.1108\/ijwis-12-2015-0046","type":"journal-article","created":{"date-parts":[[2016,10,28]],"date-time":"2016-10-28T03:28:25Z","timestamp":1477625305000},"page":"448-476","source":"Crossref","is-referenced-by-count":4,"title":["Learning to rank with click-through features in a reinforcement learning framework"],"prefix":"10.1108","volume":"12","author":[{"given":"Amir Hosein","family":"Keyhanipour","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Behzad","family":"Moshiri","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maryam","family":"Piroozmand","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Farhad","family":"Oroumchian","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ali","family":"Moeini","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","reference":[{"first-page":"19","article-title":"Improving web search ranking by incorporating user behavior information","year":"2006","key":"key2020121102031139100_ref001"},{"first-page":"3","article-title":"Learning user interaction models for predicting web search result preferences","year":"2006","key":"key2020121102031139100_ref002"},{"issue":"3","key":"key2020121102031139100_ref003","first-page":"551","article-title":"WCL2R: a benchmark collection for learning to rank research with click through data","volume":"1","year":"2010","journal-title":"Journal of Information and Data Management"},{"first-page":"399","article-title":"A combined component approach for finding collection-adapted ranking functions based on genetic programming","year":"2007","key":"key2020121102031139100_ref004"},{"year":"2013","key":"key2020121102031139100_ref006","article-title":"Learning high-level image representation for image retrieval via multi-task dnn using clickthrough data"},{"first-page":"129","article-title":"Learning to rank: from pairwise approach to list wise approach","year":"2007","key":"key2020121102031139100_ref007"},{"issue":"4","key":"key2020121102031139100_ref008","doi-asserted-by":"crossref","first-page":"1686","DOI":"10.1016\/j.asoc.2012.12.023","article-title":"Applying reinforcement learning for web pages ranking algorithms","volume":"13","year":"2013","journal-title":"Applied Soft Computing"},{"first-page":"73","article-title":"Are click-through data adequate for learning web search rankings?","year":"2008","key":"key2020121102031139100_ref009"},{"first-page":"181","article-title":"A model to estimate intrinsic document relevance from the click-through logs of a web search engine","year":"2010","key":"key2020121102031139100_ref010"},{"issue":"1","key":"key2020121102031139100_ref011","first-page":"933","article-title":"An efficient boosting algorithm for combining preferences","volume":"4","year":"2003","journal-title":"Journal of Machine Learning Research"},{"first-page":"355","article-title":"Smoothing clickthrough data for web search ranking","year":"2009","key":"key2020121102031139100_ref012"},{"key":"key2020121102031139100_ref013","unstructured":"Google (1998), \u201cGoogle search engine\u201d, available at: www.google.com (accessed 20 December 2015)."},{"issue":"3","key":"key2020121102031139100_ref014","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/0165-0114(94)00174-6","article-title":"Fuzzy integral in multicriteria decision making","volume":"69","year":"1995","journal-title":"Fuzzy Sets and Systems"},{"first-page":"478","article-title":"Eye-tracking analysis of user behavior in WWW search","year":"2004","key":"key2020121102031139100_ref015"},{"article-title":"Large margin rank boundaries for ordinal regression","volume-title":"Advances in Large Margin Classifiers","year":"2000","key":"key2020121102031139100_ref016"},{"first-page":"183","article-title":"Reusing historical interaction data for faster online learning to rank for IR","year":"2013","key":"key2020121102031139100_ref017"},{"first-page":"2333","article-title":"Learning deep structured semantic models for web search using click-through data","year":"2013","key":"key2020121102031139100_ref019"},{"first-page":"277","article-title":"Learning to re-rank: query-dependent image re-ranking using click data","year":"2011","key":"key2020121102031139100_ref020"},{"first-page":"35","article-title":"Global ranking by exploiting user clicks","year":"2009","key":"key2020121102031139100_ref021"},{"first-page":"133","article-title":"Optimizing search engines using clickthrough data","year":"2002","key":"key2020121102031139100_ref022"},{"first-page":"217","article-title":"Training linear SVMs in linear time","year":"2006","key":"key2020121102031139100_ref023"},{"volume-title":"Rank Correlation Methods","year":"1948","key":"key2020121102031139100_ref024"},{"issue":"4","key":"key2020121102031139100_ref025","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/j.knosys.2006.08.001","article-title":"Aggregation of multiple search engines based on users\u2019 preferences in WebFusion","volume":"20","year":"2007","journal-title":"Knowledge-Based Systems"},{"key":"key2020121102031139100_ref026","unstructured":"LETOR4.0 Datasets (2009), \u201cLETOR4.0 datasets\u201d, available at: http:\/\/research.microsoft.com\/en-us\/um\/beijing\/projects\/letor\/letor4dataset.aspx (accessed 20 December 2015)."},{"key":"key2020121102031139100_ref027","unstructured":"LETOR4.0\u2019s Features List (2009), \u201cLETOR4.0\u2019s features list\u201d, available at: http:\/\/research.microsoft.com\/en-us\/um\/beijing\/projects\/letor\/LETOR4.0\/Data\/Features_in_LETOR4.pdf (accessed 20 December 2015)."},{"first-page":"75","article-title":"Usefulness of quality click-through data for training","year":"2009","key":"key2020121102031139100_ref029"},{"issue":"5","key":"key2020121102031139100_ref030","doi-asserted-by":"crossref","first-page":"584","DOI":"10.1007\/s10791-012-9209-9","article-title":"The whens and hows of learning to rank for web search","volume":"16","year":"2013","journal-title":"Information Retrieval"},{"first-page":"709","article-title":"Learning latent semantic relations from click-through data for query suggestion","year":"2008","key":"key2020121102031139100_ref028"},{"volume-title":"An Introduction to Information Retrieval","year":"2008","key":"key2020121102031139100_ref031"},{"key":"key2020121102031139100_ref032","unstructured":"Miller, M. (2012), \u201c53% of Organic search clicks go to first link\u201d, available at: http:\/\/searchenginewatch.com\/article\/2215868\/53-of-Organic-Search-Clicks-Go-to-First-Link-Study (accessed 20 December 2015)."},{"key":"key2020121102031139100_ref033","first-page":"3","article-title":"LETOR: benchmark dataset for research on learning to rank for information retrieval","year":"2007","journal-title":"SIGIR 2007 Workshop on Learning to Rank for Information Retrieval"},{"first-page":"239","article-title":"Query chains: learning to rank from implicit feedback","year":"2005","key":"key2020121102031139100_ref034"},{"first-page":"570","article-title":"Active exploration for learning rankings from click-through data","year":"2007","key":"key2020121102031139100_ref035"},{"volume-title":"A Mathematical Theory of Evidence","year":"1976","key":"key2020121102031139100_ref036"},{"volume-title":"Reinforcement Learning: An Introduction","year":"1998","key":"key2020121102031139100_ref037"},{"volume-title":"Algorithms for Reinforcement Learning","year":"2010","key":"key2020121102031139100_ref038"},{"key":"key2020121102031139100_ref039","unstructured":"TodoCL (2002), \u201cTodoCL search engine website\u201d, available at: www.todocl.cl (accessed 20 December 2015)."},{"first-page":"267","article-title":"Learning to rank at query-time using association rules","year":"2008","key":"key2020121102031139100_ref040"},{"first-page":"203","article-title":"User modeling in search logs via a nonparametric bayesian approach","year":"2014","key":"key2020121102031139100_ref042"},{"key":"key2020121102031139100_ref043","unstructured":"WCL2R (2010), \u201cWCL2R website\u201d, available at: www.latin.dcc.ufmg.br\/collections\/wcl2r (accessed 20 December 2015)."},{"first-page":"171","article-title":"Improving quality of training data for learning to rank using click-through data","year":"2010","key":"key2020121102031139100_ref045"},{"first-page":"391","article-title":"Adarank: a boosting algorithm for information retrieval","year":"2007","key":"key2020121102031139100_ref044"},{"key":"key2020121102031139100_ref046","unstructured":"Xu, Q. (2010), \u201cLearning to rank from clickthrough data\u201d, available at: http:\/\/isabel-drost.de\/projects\/tuberlin\/imsem2010\/learningtorank_paper_2010.pdf (accessed 20 December 2015)."},{"issue":"2","key":"key2020121102031139100_ref047","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.ipm.2009.12.005","article-title":"A3CRank: an adaptive ranking method based on connectivity, content and click-through data","volume":"46","year":"2010","journal-title":"Information Processing and Management"},{"first-page":"334","article-title":"A study of smoothing methods for language models applied to Ad Hoc information retrieval","year":"2001","key":"key2020121102031139100_ref048"}],"container-title":["International Journal of Web Information Systems"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/IJWIS-12-2015-0046","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJWIS-12-2015-0046\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJWIS-12-2015-0046\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T22:24:28Z","timestamp":1753395868000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/ijwis\/article\/12\/4\/448-476\/164487"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,11,7]]},"references-count":45,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2016,11,7]]}},"alternative-id":["10.1108\/IJWIS-12-2015-0046"],"URL":"https:\/\/doi.org\/10.1108\/ijwis-12-2015-0046","relation":{},"ISSN":["1744-0084"],"issn-type":[{"type":"print","value":"1744-0084"}],"subject":[],"published":{"date-parts":[[2016,11,7]]}}}