{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T15:02:09Z","timestamp":1782313329653,"version":"3.54.5"},"reference-count":32,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T00:00:00Z","timestamp":1730246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["61070015"],"award-info":[{"award-number":["61070015"]}]},{"name":"National Natural Science Foundation of China","award":["2022GCZX013"],"award-info":[{"award-number":["2022GCZX013"]}]},{"name":"National Natural Science Foundation of China","award":["ZNJW2024KFQN012"],"award-info":[{"award-number":["ZNJW2024KFQN012"]}]},{"name":"National Natural Science Foundation of China","award":["SDNY-2024001"],"award-info":[{"award-number":["SDNY-2024001"]}]},{"name":"National Natural Science Foundation of China","award":["D202405310814162131"],"award-info":[{"award-number":["D202405310814162131"]}]},{"name":"Guangdong Provincial Key Technology Research Center for Object Digitization and Epidemic Intelligence Prevention and Control, Guangdong Provincial Engineering Technology Research Center for General Colleges and Universities, Guangdong Provincial Department of Education","award":["61070015"],"award-info":[{"award-number":["61070015"]}]},{"name":"Guangdong Provincial Key Technology Research Center for Object Digitization and Epidemic Intelligence Prevention and Control, Guangdong Provincial Engineering Technology Research Center for General Colleges and Universities, Guangdong Provincial Department of Education","award":["2022GCZX013"],"award-info":[{"award-number":["2022GCZX013"]}]},{"name":"Guangdong Provincial Key Technology Research Center for Object Digitization and Epidemic Intelligence Prevention and Control, Guangdong Provincial Engineering Technology Research Center for General Colleges and Universities, Guangdong Provincial Department of Education","award":["ZNJW2024KFQN012"],"award-info":[{"award-number":["ZNJW2024KFQN012"]}]},{"name":"Guangdong Provincial Key Technology Research Center for Object Digitization and Epidemic Intelligence Prevention and Control, Guangdong Provincial Engineering Technology Research Center for General Colleges and Universities, Guangdong Provincial Department of Education","award":["SDNY-2024001"],"award-info":[{"award-number":["SDNY-2024001"]}]},{"name":"Guangdong Provincial Key Technology Research Center for Object Digitization and Epidemic Intelligence Prevention and Control, Guangdong Provincial Engineering Technology Research Center for General Colleges and Universities, Guangdong Provincial Department of Education","award":["D202405310814162131"],"award-info":[{"award-number":["D202405310814162131"]}]},{"name":"Intelligent Policing Key Laboratory of Sichuan Province","award":["61070015"],"award-info":[{"award-number":["61070015"]}]},{"name":"Intelligent Policing Key Laboratory of Sichuan Province","award":["2022GCZX013"],"award-info":[{"award-number":["2022GCZX013"]}]},{"name":"Intelligent Policing Key Laboratory of Sichuan Province","award":["ZNJW2024KFQN012"],"award-info":[{"award-number":["ZNJW2024KFQN012"]}]},{"name":"Intelligent Policing Key Laboratory of Sichuan Province","award":["SDNY-2024001"],"award-info":[{"award-number":["SDNY-2024001"]}]},{"name":"Intelligent Policing Key Laboratory of Sichuan Province","award":["D202405310814162131"],"award-info":[{"award-number":["D202405310814162131"]}]},{"name":"Open Research Subject of Sichuan Provincial Engineering Research Center of Hydroelectric Energy Power Equipment Technology","award":["61070015"],"award-info":[{"award-number":["61070015"]}]},{"name":"Open Research Subject of Sichuan Provincial Engineering Research Center of Hydroelectric Energy Power Equipment Technology","award":["2022GCZX013"],"award-info":[{"award-number":["2022GCZX013"]}]},{"name":"Open Research Subject of Sichuan Provincial Engineering Research Center of Hydroelectric Energy Power Equipment Technology","award":["ZNJW2024KFQN012"],"award-info":[{"award-number":["ZNJW2024KFQN012"]}]},{"name":"Open Research Subject of Sichuan Provincial Engineering Research Center of Hydroelectric Energy Power Equipment Technology","award":["SDNY-2024001"],"award-info":[{"award-number":["SDNY-2024001"]}]},{"name":"Open Research Subject of Sichuan Provincial Engineering Research Center of Hydroelectric Energy Power Equipment Technology","award":["D202405310814162131"],"award-info":[{"award-number":["D202405310814162131"]}]},{"name":"College Student Innovation and Entrepreneurship Training Project of Sichuan Province","award":["61070015"],"award-info":[{"award-number":["61070015"]}]},{"name":"College Student Innovation and Entrepreneurship Training Project of Sichuan Province","award":["2022GCZX013"],"award-info":[{"award-number":["2022GCZX013"]}]},{"name":"College Student Innovation and Entrepreneurship Training Project of Sichuan Province","award":["ZNJW2024KFQN012"],"award-info":[{"award-number":["ZNJW2024KFQN012"]}]},{"name":"College Student Innovation and Entrepreneurship Training Project of Sichuan Province","award":["SDNY-2024001"],"award-info":[{"award-number":["SDNY-2024001"]}]},{"name":"College Student Innovation and Entrepreneurship Training Project of Sichuan Province","award":["D202405310814162131"],"award-info":[{"award-number":["D202405310814162131"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>A focused crawler automatically retrieves, organizes, and extracts specific topic-related information from the internet for analysis and application. Currently, most focused crawlers assess the relevance of web pages to a given topic through methods such as keyword matching, semantic analysis, and link structures. However, these existing focused crawlers suffer from issues such as misleading directions and reduced accuracy due to the lack of semantic analysis of topic terms, as well as biased computation of topic relevance caused by the absence of effective weighting factors. To solve the above-mentioned problems, this study proposes a semantic and optimized focused crawler based on Semantic Graph and Genetic Algorithm. The proposed crawler eliminates ambiguous terms by constructing a semantic graph, optimizes the weighting factors of topic relevance with asymmetry by using a genetic algorithm, and combines both above two points to predict the priority of each unvisited hyperlink. The experiment results indicate that the proposed SG-GA Crawler improves the evaluation indicators compared with the other three focused crawlers, including VSM Crawler, SSRM Crawler, and SG Crawler. More specifically, the percentage improvement achieved by the proposed method exceeds 19%, 19%, and 13% in terms of three evaluation indicators, including the number of relevant web pages, acquisition rate, and average relevance, respectively. In conclusion, the proposed focused crawler can grab more quantity and higher quality topic-related web pages from the Internet.<\/jats:p>","DOI":"10.3390\/sym16111439","type":"journal-article","created":{"date-parts":[[2024,10,31]],"date-time":"2024-10-31T06:55:37Z","timestamp":1730357737000},"page":"1439","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A Semantic and Optimized Focused Crawler Based on Semantic Graph and Genetic Algorithm"],"prefix":"10.3390","volume":"16","author":[{"given":"Wenhao","family":"Huang","sequence":"first","affiliation":[{"name":"School of Software Engineering, South China University of Technology, Guangzhou 510006, China"},{"name":"School of Computer Science, South China Business College Guangdong University of Foreign Studies, Guangzhou 510545, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoyan","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer and Software Engineering, Xihua University, Chengdu 610039, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiao","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Computer and Software Engineering, Xihua University, Chengdu 610039, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deyu","family":"Qi","sequence":"additional","affiliation":[{"name":"School of Software Engineering, South China University of Technology, Guangzhou 510006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianqing","family":"Xi","sequence":"additional","affiliation":[{"name":"School of Software Engineering, South China University of Technology, Guangzhou 510006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9240-5484","authenticated-orcid":false,"given":"Wenjun","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer and Software Engineering, Xihua University, Chengdu 610039, China"},{"name":"Sichuan Provincial Engineering Research Center of Hydroelectric Energy Power Equipment Technology, Xihua University, Chengdu 610039, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feiyu","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Computer and Software Engineering, Xihua University, Chengdu 610039, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Liu, C.H., You, S.D., and Chiu, Y.C. (2024). A Reinforcement Learning Approach to Guide Web Crawler to Explore Web Applications for Improving Code Coverage. Electronics, 13.","DOI":"10.3390\/electronics13020427"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"17400","DOI":"10.1007\/s11227-024-06133-6","article-title":"TS-Finder: Privacy enhanced web crawler detection model using temporal-spatial access behaviors","volume":"80","author":"Zhao","year":"2024","journal-title":"J. Supercomput."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"7589727","DOI":"10.1155\/2022\/7589727","article-title":"Construction of Online English Corpus Based on Web Crawler Technology","volume":"2022","author":"Qi","year":"2022","journal-title":"Wirel. Commun. Mob. Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"60343","DOI":"10.1007\/s11042-023-17925-y","article-title":"A novel combining method of dynamic and static web crawler with parallel computing","volume":"83","author":"Liu","year":"2024","journal-title":"Multimed. Tools Appl."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/j.neucom.2022.11.059","article-title":"Reinforcement learning based web crawler detection for diversity and dynamics","volume":"520","author":"Gao","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_6","first-page":"310","article-title":"A Topic-Specific Web Crawler using Deep Convolutional Networks","volume":"20","author":"ALqaraleh","year":"2023","journal-title":"Int. Arab. J. Inf. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2037","DOI":"10.1080\/03772063.2021.1885312","article-title":"LEARNING-based Focused WEB Crawler","volume":"69","author":"Kumar","year":"2021","journal-title":"IETE J. Res."},{"key":"ref_8","first-page":"105","article-title":"An Enhanced Semantic Focused Web Crawler Based on Hybrid String Matching Algorithm","volume":"21","author":"Prabha","year":"2021","journal-title":"Cybern. Inf. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"107944","DOI":"10.1016\/j.engappai.2024.107944","article-title":"Weakly supervised learning for an effective focused web crawler","volume":"132","author":"Dhanith","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1007\/s13198-022-01808-w","article-title":"Kaushik. An efficient focused crawler using LSTM-CNN based deep learning","volume":"14","author":"Shrivastava","year":"2023","journal-title":"Int. J. Syst. Assur. Eng. Manag."},{"key":"ref_11","first-page":"819","article-title":"Focused Crawler Based on Reinforcement Learning and Decaying Epsilon-Greedy Exploration Policy","volume":"20","author":"Kaleel","year":"2023","journal-title":"Int. Arab. J. Inf. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Mary, J.D.P.N.R., Balasubramanian, S., and Raj, R.S.P. (2021). A Critique Empirical Evaluation of Relevance Computation for Focused Web Crawlers. J. Braz. Arch. Biol. Technol., 64.","DOI":"10.1590\/1678-4324-2021210223"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1007\/s40747-023-01121-4","article-title":"Applying particle swarm optimization-based dynamic adaptive hyperlink evaluation to focused crawler for meteorological disasters","volume":"10","author":"Liu","year":"2024","journal-title":"Complex Intell. Syst."},{"key":"ref_14","first-page":"8888","article-title":"Crawl-shing: A focused crawler for fetching phishing contents based on graph isomorphism","volume":"34","author":"Tchakounte","year":"2022","journal-title":"J. King Saud Univ. Comput. Inf. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.patrec.2020.12.003","article-title":"Keyword weight optimization using gradient strategies in event focused web crawling","volume":"142","author":"Rajiv","year":"2021","journal-title":"J. Pattern Recognit. Lett."},{"key":"ref_16","first-page":"122","article-title":"A Word Embedding Based Approach for Focused Web Crawling Using the Recurrent Neural Network","volume":"6","author":"Dhanith","year":"2021","journal-title":"Int. J. Interact. Multimed. Artif. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e12993","DOI":"10.1111\/exsy.12993","article-title":"A supervised learning-based approach for focused web crawling for IoMT using global co-occurrence matrix","volume":"40","author":"Rajiv","year":"2023","journal-title":"Expert Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"7577","DOI":"10.1007\/s11042-019-08252-2","article-title":"An ontology-driven multimedia focused crawler based on linked open data and deep learning techniques","volume":"79","author":"Capuano","year":"2020","journal-title":"Multimed. Tools Appl."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"116741","DOI":"10.1016\/j.eswa.2022.116741","article-title":"Applying ontology learning and multi-objective ant colony optimization method for focused crawling to meteorological disasters domain knowledge","volume":"198","author":"Liu","year":"2022","journal-title":"J. Expert Syst. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1007\/s13278-022-00860-5","article-title":"A local updating algorithm for personalized PageRank via Chebyshev polynomials","volume":"12","author":"Bautista","year":"2022","journal-title":"J. Soc. Netw. Anal. Min."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ni, J.J., Cai, Y., Tang, G.Y., and Xie, Y.J. (2021). Collaborative Filtering Recommendation Algorithm Based on TF-IDF and User Characteristics. J. Appl. Sci., 11.","DOI":"10.3390\/app11209554"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1007\/s00354-017-0029-8","article-title":"Efficient Topical Focused Crawling Through Neighborhood Feature","volume":"36","author":"Suebchua","year":"2018","journal-title":"New Gener. Comput."},{"key":"ref_23","first-page":"36","article-title":"Efficacious Hyperlink Based Similarity Measure Using Heterogeneous Propagation of PageRank Scores","volume":"9","author":"Thangasamy","year":"2019","journal-title":"Int. J. Inf. Retr. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"108495","DOI":"10.1016\/j.knosys.2022.108495","article-title":"A novel focused crawler combining Web space evolution and domain ontology","volume":"243","author":"Liu","year":"2022","journal-title":"J. Knowl. Based Syst."},{"key":"ref_25","first-page":"461","article-title":"UCrawler: A learning-based web crawler using a URL knowledge base","volume":"21","author":"Wang","year":"2021","journal-title":"J. Comput. Methods Sci. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"695","DOI":"10.3233\/IDA-205107","article-title":"Online learning agents for cost-sensitive topical data acquisition from the web","volume":"26","author":"Naghibi","year":"2022","journal-title":"J. Intell. Data Anal."},{"key":"ref_27","first-page":"1043","article-title":"Improved algorithm of Context Graph based on feature selection","volume":"20","author":"Liu","year":"2020","journal-title":"J. Comput. Methods Sci. Eng."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"6406901","DOI":"10.1155\/2016\/6406901","article-title":"An Improved Focused Crawler: Using Web Page Classification and Link Priority Evaluation","volume":"2016","author":"Lu","year":"2016","journal-title":"Math. Probl. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.asoc.2016.12.028","article-title":"A web page distillation strategy for efficient focused crawling based on optimized Na\u00efve bayes (ONB) classifier","volume":"53","author":"Saleh","year":"2017","journal-title":"Appl. Soft Comput."},{"key":"ref_30","first-page":"613","article-title":"A vector space model for automatic indexing","volume":"18","author":"Salton","year":"1975","journal-title":"Commun. Assoc. Comput. Mach."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Varelas, G., Voutsakis, E., Raftopoulou, P., Petrakis, E.G.M., and Milios, E.E. (2005, January 5). Semantic similarity methods in WordNet and their application to information retrieval on the web. Proceedings of the 7th Annual ACM International Workshop on Web Information and Data Management, Bremen, Germany.","DOI":"10.1145\/1097047.1097051"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"7390","DOI":"10.1007\/s10489-022-03180-5","article-title":"A semantic and intelligent focused crawler based on semantic vector space model and membrane computing optimization algorithm","volume":"53","author":"Liu","year":"2022","journal-title":"Appl. Intell."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/11\/1439\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:24:29Z","timestamp":1760113469000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/11\/1439"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,30]]},"references-count":32,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["sym16111439"],"URL":"https:\/\/doi.org\/10.3390\/sym16111439","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,30]]}}}