{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:52:29Z","timestamp":1760241149582,"version":"build-2065373602"},"reference-count":30,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2019,12,9]],"date-time":"2019-12-09T00:00:00Z","timestamp":1575849600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Deanship of Scientific Research at Northern Border University, Arar, 304 Saudi Arabia","award":["SCI-2018-3-9-F-0000."],"award-info":[{"award-number":["SCI-2018-3-9-F-0000."]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Peer-to-peer (P2P) systems have offered users an efficient way to share various resources and access diverse services over the Internet. In unstructured P2P systems, resource storage and indexation are fully distributed among participating peers. Therefore, locating peers sharing pertinent resources for a specific user query is a challenging issue. In fact, effective query routing requires smart decisions to select a certain number of peers with respect to their relevance for the query instead of choosing them at random. In this respect, we introduce here a new query-oriented approach, called the reinforcement learning-based query routing approach (RLQR). The main goal of RLQR is to reach high retrieval effectiveness as well as a lower search cost by reducing the number of exchanged messages and contacted peers. To achieve this, the RLQR relies on information gathered from previously sent queries to identify relevant peers for forthcoming queries. Indeed, we formulate the query routing issue as the reinforcement learning problem and introduce a fully distributed approach for addressing it. In addition, RLQR addresses the well-known cold-start issue during the training stage, which allows it to improve its retrieval effectiveness and search cost continuously, and, therefore, goes quickly through the cold-start phase. Performed simulations demonstrate that RLQR outperforms pioneering query routing approaches in terms of retrieval effectiveness and communications cost.<\/jats:p>","DOI":"10.3390\/fi11120253","type":"journal-article","created":{"date-parts":[[2019,12,9]],"date-time":"2019-12-09T05:54:51Z","timestamp":1575870891000},"page":"253","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Reinforcement Learning Based Query Routing Approach for P2P Systems"],"prefix":"10.3390","volume":"11","author":[{"given":"Fawaz","family":"Alanazi","sequence":"first","affiliation":[{"name":"College of Science, Northern Border University, Arar 73222, Saudi Arabia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4600-8131","authenticated-orcid":false,"given":"Taoufik","family":"Yeferny","sequence":"additional","affiliation":[{"name":"College of Science, Northern Border University, Arar 73222, Saudi Arabia"}]}],"member":"1968","published-online":{"date-parts":[[2019,12,9]]},"reference":[{"key":"ref_1","unstructured":"Chernov, S., Serdyukov, P., Bender, M., Michel, S., Weikum, G., and Zimmer, C. (2005, January 28\u201329). Database selection and result merging in P2P web search. Proceedings of the 3rd International Workshop on Databases, Information Systems, and Peer-to-Peer Computing (DBISP2P 2005), Trondheim, Norway. Lecture Notes in Computer Science."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chawathe, Y., Ratnasamy, S., and Breslau, L. (2003, January 25\u201329). Making gnutella-like P2P systems scalable. Proceedings of the 2003 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, Karlsruhe, Germany.","DOI":"10.1145\/863955.864000"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1109\/TNET.2002.808407","article-title":"Chord: A Scalable Peer-to-Peer Lookup Protocol for Internet Applications","volume":"11","author":"Stoica","year":"2003","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Rowstron, A., and Druschel, P. (2001, January 12\u201316). Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. Proceedings of the IFIP\/ACM International Conference on Distributed Systems Platforms, Heidelberg, Germany.","DOI":"10.1007\/3-540-45518-3_18"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"49059","DOI":"10.1109\/ACCESS.2019.2910117","article-title":"Query Learning-Based Scheme for Pertinent Resource Lookup in Mobile P2P Networks","volume":"7","author":"Taoufik","year":"2019","journal-title":"IEEE Access"},{"key":"ref_6","unstructured":"Deshpande, M., and Venkatasubramanian, N. (2004, January 25\u201327). The Different Dimensions of Dynamicity. Proceedings of the 4th International Conference on Peer-to-Peer Computing (P2P\u201904), Zurich, Switzerland."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Yeferny, T., and Arour, K. (2012, January 26\u201329). Efficient routing method in p2p systems based upon training knowledge. Proceedings of the 26th International Conference on Advanced Information Networking and Applications Workshops, Fukuoka, Japan.","DOI":"10.1109\/WAINA.2012.211"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yeferny, T., Arour, K., and Bouzeghoub, A. (2013, January 25\u201328). An efficient peer-to-peer semantic overlay network for learning query routing. Proceedings of the 27th IEEE International Conference on Advanced Information Networking and Applications (AINA), Barcelona, Spain.","DOI":"10.1109\/AINA.2013.129"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1007\/s12083-014-0282-2","article-title":"Learning model for efficient query routing in P2P information retrieval systems","volume":"8","author":"Arour","year":"2015","journal-title":"Peer-to-Peer Netw. Appl."},{"key":"ref_10","first-page":"28","article-title":"CDP: A Content Discovery Protocol for Mobile P2P Systems","volume":"18","author":"Yeferny","year":"2018","journal-title":"Int. J. Comput. Sci. Netw. Secur."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lv, Q., Cao, P., Cohen, E., Li, K., and Shenker, S. (2002, January 22\u201326). Search and replication in unstructured peer-to-peer networks. Proceedings of the 16th International Conference on Supercomputing, New York, NY, USA.","DOI":"10.1145\/514191.514206"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"648","DOI":"10.1016\/S1004-4132(06)60111-4","article-title":"Random walk search in unstructured P2P","volume":"17","author":"Jia","year":"2006","journal-title":"J. Syst. Eng. Electron."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kalogeraki, V., Gunopulos, D., and Zeinalipour-Yazti, D. (2002, January 4\u20139). A local search mechanism for peer-to-peer networks. Proceedings of the Eleventh International Conference on Information and Knowledge Management, McLean, VA, USA.","DOI":"10.1145\/584792.584842"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/S0166-218X(03)00257-9","article-title":"Gossiping and broadcasting versus computing functions in networks","volume":"137","author":"Dietzfelbinger","year":"2004","journal-title":"Discret. Appl. Math."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1445","DOI":"10.1016\/j.comcom.2009.04.005","article-title":"Enhancing peer-to-peer content discovery techniques over mobile ad hoc networks","volume":"32","author":"Macedo","year":"2009","journal-title":"Comput. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1016\/j.jnca.2008.09.001","article-title":"Reducing query overhead through route learning in unstructured peer-to-peer network","volume":"32","author":"Ciraci","year":"2009","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, L., Chu, W., Langford, J., and Schapire, R.E. (2010, January 26\u201330). A Contextual-bandit Approach to Personalized News Article Recommendation. Proceedings of the 19th International Conference on World Wide Web WWW \u201910, Raleigh, NC, USA.","DOI":"10.1145\/1772690.1772758"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wang, L., Wang, C., Wang, K., and He, X. (2017, January 9\u201310). BiUCB: A Contextual Bandit Algorithm for Cold-Start and Diversified Recommendation. Proceedings of the 2017 IEEE International Conference on Big Knowledge (ICBK), Hefei, China.","DOI":"10.1109\/ICBK.2017.49"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Qiao, R., Yan, S., and Shen, B. (2018, January 14\u201316). A Reinforcement Learning Solution to Cold-Start Problem in Software Crowdsourcing Recommendations. Proceedings of the 2018 IEEE International Conference on Progress in Informatics and Computing (PIC), Suzhou, China.","DOI":"10.1109\/PIC.2018.8706279"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lu, J., and Callan, J. (2003, January 3\u20138). Content-based retrieval in hybrid peer-to-peer networks. Proceedings of the Twelfth International Conference on Information and Knowledge Management, CIKM \u201903, New Orleans, LA, USA.","DOI":"10.1145\/956900.956903"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.procs.2013.09.010","article-title":"OBAME: Optimized Bio-inspired Algorithm to Maximize Search Efficiency in P2P Databases","volume":"21","author":"Kurid","year":"2013","journal-title":"Procedia Comput. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/S1005-8885(09)60509-4","article-title":"SKIP: An efficient search mechanism in unstructured P2P networks","volume":"17","author":"Shen","year":"2011","journal-title":"J. China Univ. Posts Telecommun."},{"key":"ref_23","unstructured":"Christoph, T., Steffen, S., and Adrian, W. (2004, January 17\u201320). Semantic Query Routing in Peer-to-Peer Networks based on Social Metaphors. Proceedings of the 13th International World Wide Web Conference, New York, NY, USA."},{"key":"ref_24","unstructured":"Ganter, B., and Wille, R. (1997). Formal Concept Analysis: Mathematical Foundations, Springer."},{"key":"ref_25","first-page":"20","article-title":"Bandit Problems. Sequential Allocation of Experiments. Monographs on Statistics and Applied Probability","volume":"29","author":"Berry","year":"1987","journal-title":"Biom. J."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.2307\/1427934","article-title":"Sample Mean Based Index Policies with O(log n) Regret for the Multi-Armed Bandit Problem","volume":"27","author":"Agrawal","year":"1995","journal-title":"Adv. Appl. Probab."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1093\/biomet\/25.3-4.285","article-title":"On the likelihood that one unknown probability exceeds another in view of the evidence of two samples","volume":"25","author":"Thompson","year":"1933","journal-title":"Biometrika"},{"key":"ref_28","unstructured":"Jelasity, M., Montresor, A., Jesi, G.P., and Voulgaris, S. (2019, December 07). The Peersim Simulator. Available online: http:\/\/peersim.sf.net."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zammali, S., and Arour, K. (2010, January 1\u20132). P2PIRB: Benchmarking framework for P2PIR. Proceedings of the Third International Conference on Data Management in Grid and Peer-to-Peer Systems (Globe), Bilbao, Spain.","DOI":"10.1007\/978-3-642-15108-8_9"},{"key":"ref_30","unstructured":"Makhoul, J., Kubala, F., Schwartz, R., and Weischedel, R. (March, January 28). Performance Measures For Information Extraction. Proceedings of the DARPA Broadcast News Workshop, Herndon, VA, USA."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/11\/12\/253\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:40:38Z","timestamp":1760190038000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/11\/12\/253"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,9]]},"references-count":30,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["fi11120253"],"URL":"https:\/\/doi.org\/10.3390\/fi11120253","relation":{},"ISSN":["1999-5903"],"issn-type":[{"type":"electronic","value":"1999-5903"}],"subject":[],"published":{"date-parts":[[2019,12,9]]}}}