{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:16:48Z","timestamp":1760242608478,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2017,12,22]],"date-time":"2017-12-22T00:00:00Z","timestamp":1513900800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this work we present the design and implementation of WARCProcessor, a novel multiplatform integrative tool aimed to build scientific datasets to facilitate experimentation in web spam research. The developed application allows the user to specify multiple criteria that change the way in which new corpora are generated whilst reducing the number of repetitive and error prone tasks related with existing corpus maintenance. For this goal, WARCProcessor supports up to six commonly used data sources for web spam research, being able to store output corpus in standard WARC format together with complementary metadata files. Additionally, the application facilitates the automatic and concurrent download of web sites from Internet, giving the possibility of configuring the deep of the links to be followed as well as the behaviour when redirected URLs appear. WARCProcessor supports both an interactive GUI interface and a command line utility for being executed in background.<\/jats:p>","DOI":"10.3390\/s18010016","type":"journal-article","created":{"date-parts":[[2017,12,22]],"date-time":"2017-12-22T05:50:19Z","timestamp":1513921819000},"page":"16","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["WARCProcessor: An Integrative Tool for Building and Management of Web Spam Corpora"],"prefix":"10.3390","volume":"18","author":[{"given":"Miguel","family":"Call\u00f3n","sequence":"first","affiliation":[{"name":"ESEI: Higher Technical School of Computer Engineering, University of Vigo, 32004 Ourense, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jorge","family":"Fdez-Glez","sequence":"additional","affiliation":[{"name":"CITI: Centre for Research Transference and Innovation, Avda. Galicia 2, Parque Tecnol\u00f3xico, San Cibrao das Vi\u00f1as, 32900 Ourense, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Ruano-Ord\u00e1s","sequence":"additional","affiliation":[{"name":"ESEI: Higher Technical School of Computer Engineering, University of Vigo, 32004 Ourense, Spain"},{"name":"CINBIO: Biomedical Research Centre, Campus Universitario Lagoas-Marcosende, 36310 Vigo, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1889-1337","authenticated-orcid":false,"given":"Rosal\u00eda","family":"Laza","sequence":"additional","affiliation":[{"name":"ESEI: Higher Technical School of Computer Engineering, University of Vigo, 32004 Ourense, Spain"},{"name":"CINBIO: Biomedical Research Centre, Campus Universitario Lagoas-Marcosende, 36310 Vigo, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Reyes","family":"Pav\u00f3n","sequence":"additional","affiliation":[{"name":"ESEI: Higher Technical School of Computer Engineering, University of Vigo, 32004 Ourense, Spain"},{"name":"CINBIO: Biomedical Research Centre, Campus Universitario Lagoas-Marcosende, 36310 Vigo, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3943-8013","authenticated-orcid":false,"given":"Florentino","family":"Fdez-Riverola","sequence":"additional","affiliation":[{"name":"ESEI: Higher Technical School of Computer Engineering, University of Vigo, 32004 Ourense, Spain"},{"name":"CINBIO: Biomedical Research Centre, Campus Universitario Lagoas-Marcosende, 36310 Vigo, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jose","family":"M\u00e9ndez","sequence":"additional","affiliation":[{"name":"ESEI: Higher Technical School of Computer Engineering, University of Vigo, 32004 Ourense, Spain"},{"name":"CINBIO: Biomedical Research Centre, Campus Universitario Lagoas-Marcosende, 36310 Vigo, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,12,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.ipm.2009.12.005","article-title":"A3CRank: An adaptive ranking method based on connectivity, content and click-through data","volume":"46","author":"Bidoki","year":"2010","journal-title":"Inf. Process. Manag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1016\/j.proeng.2011.11.2507","article-title":"Countering web spam of link based ranking based on link analysis","volume":"23","author":"Wang","year":"2011","journal-title":"Procedia Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1686","DOI":"10.1016\/j.asoc.2012.12.023","article-title":"Applying reinforce-ment learning for web pages ranking algorithms","volume":"13","author":"Derhami","year":"2013","journal-title":"Appl. Soft Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1561\/1500000021","article-title":"Adversarial web search","volume":"4","author":"Castillo","year":"2011","journal-title":"Found. Trends Inf. Retr."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ghiam, S., and Pour, A.N. (arXiv, 2012). A Survey on web spam detection methods: Taxonomy, arXiv.","DOI":"10.5121\/ijnsa.2012.4510"},{"key":"ref_6","unstructured":"John, J.P., Yu, F., Xie, Y., Krishnamurthy, A., and Abadi, M. (2011, January 8\u201312). deSEO: Combating search-result poisoning. Proceedings of the USENIX Security Symposium, San Francisco, CA, USA."},{"key":"ref_7","first-page":"9","article-title":"Awareness, Knowledge and perception of online spam","volume":"4","author":"Ridzuan","year":"2013","journal-title":"J. Next Gener. Inf. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3634","DOI":"10.1016\/j.eswa.2014.12.029","article-title":"Detection of review spam: A survey","volume":"42","author":"Heydari","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1016\/j.ipm.2014.11.001","article-title":"Detecting positive and negative deceptive opinions using PU-learning","volume":"51","author":"Rosso","year":"2015","journal-title":"Inf. Process. Manag."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hayati, P.M., and Potdar, V. (2009, January 23\u201326). Toward spam 2.0: An evaluation of Web 2.0 anti-spam methods. Proceedings of the 7th IEEE International Conference on Industrial Informatics, Cardiff, Wales, UK.","DOI":"10.1109\/INDIN.2009.5195918"},{"key":"ref_11","first-page":"634","article-title":"A Survey on Web Spam and Spam 2.0","volume":"4","author":"Chandra","year":"2014","journal-title":"Int. J. Adv. Res. Comput. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3086637","article-title":"Robust spammer detection in microblogs: Leveraging user carefulness","volume":"8","author":"Fu","year":"2017","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"976","DOI":"10.1109\/TIFS.2014.2316975","article-title":"SybilBelief: A semi-supervised learning approach for structure-based sybil detection","volume":"9","author":"Gong","year":"2014","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Wang, B., Zhang, L., and Gong, N.Z. (2017, January 1\u20134). SybilSCAR: Sybil detection in online social networks via local rule based propagation. Proceedings of the IEEE International Conference on Computer Communications (INFOCOM 2017), Atlanta, GA, USA.","DOI":"10.1109\/INFOCOM.2017.8057066"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Jia, J., Wang, B., and Gong, N.Z. (2017, January 26\u201329). Random walk based fake account detection in online social networks. Proceedings of the 47th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN 2017), Denver, CO, USA.","DOI":"10.1109\/DSN.2017.55"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1145\/2207243.2207252","article-title":"Survey on web spam detection: Principles and algorithms","volume":"13","author":"Spirin","year":"2011","journal-title":"SIGKDD Explor."},{"key":"ref_17","unstructured":"Geng, G.G., Jin, X.-B., Zhang, X.-C., and Zhang, D.X. (arXiv, 2013). Evaluating web content quality via multi-scale features, arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1080\/15427951.2013.850456","article-title":"The classification power of Web features","volume":"10","author":"Kiss","year":"2014","journal-title":"Internet Math."},{"key":"ref_19","first-page":"86","article-title":"Machine learning methods for spamdexing detection","volume":"2","author":"Silva","year":"2013","journal-title":"Int. J. Inf. Secur. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Prieto, V.M., \u00c1lvarez, M., L\u00f3pez-Garc\u00eda, R., and Cacheda, R. (2012, January 2\u20133). Analysis and detection of web spam by means of web content. Proceedings of the 5th Information Retrieval Facility Conference (IRFC 2012), Vienna, Austria.","DOI":"10.1007\/978-3-642-31274-8_4"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1016\/j.procs.2015.10.069","article-title":"Comprehensive literature review on machine learning structures for web spam classification","volume":"70","author":"Goh","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"17","DOI":"10.4018\/jdcf.2012010102","article-title":"Spam 2.0 state of the art","volume":"4","author":"Hayati","year":"2012","journal-title":"Int. J. Digit. Crime Forensics"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Choo, E., Yu, T., and Chi, M. (2015, January 13\u201315). Detecting opinion spammer groups through community discovery and sentiment analysis. Proceedings of the 29th Annual IFIP WG 11.3 Working Conference on Data and Applications Security and Privacy, Fairfax, VA, USA.","DOI":"10.1007\/978-3-319-20810-7_11"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Rosenthal, S., Nakov, P., Kiritchenko, S., Mohammad, S.M., Ritter, A., and Stoyanov, V. (2015, January 4\u20135). SemEval-2015 Task 10: Sentiment analysis in twitter. Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Denver, CO, USA.","DOI":"10.18653\/v1\/S15-2078"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1016\/j.ipm.2015.11.004","article-title":"Unsupervised adaptive microblog filtering for broad dynamic topics","volume":"52","author":"Magdy","year":"2016","journal-title":"Inf. Process. Manag."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.ipm.2015.03.002","article-title":"Multi-lingual opinion mining on YouTube","volume":"52","author":"Severyn","year":"2016","journal-title":"Inf. Process. Manag."},{"key":"ref_27","unstructured":"Ott, M., Cardie, C., and Hancock, J.T. (2013, January 9\u201314). Negative deceptive opinion spam. Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Atlanta, GA, USA."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liang, D., Liu, X., and Shen, H. (2014, January 9\u201310). Detecting spam reviewers by combing reviewer feature and relationship. Proceedings of the International Conference on Informative and Cybernetics for Computational Social Systems (ICCSS), Qingdao, China.","DOI":"10.1109\/ICCSS.2014.6961824"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/s13278-014-0189-1","article-title":"SPADE: A social-spam analytics and detection framework","volume":"4","author":"Wang","year":"2014","journal-title":"Soc. Netw. Anal. Min."},{"key":"ref_30","first-page":"14","article-title":"Using machine Learning algorithms to detect content-based arabic web Spam","volume":"7","author":"Wahsheh","year":"2012","journal-title":"J. Inf. Assur. Secur."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Garz\u00f3, A., Dar\u00f3czy, B., Kiss, T., Sikl\u00f3si, D., and Bencz\u00far, A.A. (2013, January 13\u201317). Cross-lingual web spam classification. Proceedings of the 3rd Joint WICOW\/AIRWeb Workshop on Web Quality, Rio de Janeiro, Brazil.","DOI":"10.1145\/2487788.2488139"},{"key":"ref_32","first-page":"1","article-title":"The strange case of reproducibility versus representativeness in contextual suggestion test collections","volume":"19","author":"Samar","year":"2015","journal-title":"Inf. Retr. J."},{"key":"ref_33","first-page":"63","article-title":"Experimental reproducibility and the experimenters\u2019 regress","volume":"1992","author":"Radder","year":"1992","journal-title":"Philos. Sci."},{"key":"ref_34","unstructured":"Thomas, S., and Kai, W. (2014). EXMARaLDA. Handbook on Corpus Phonology, Oxford University Press."},{"key":"ref_35","unstructured":"Smith, S. (2015, January 6\u20138). Construction and use of thematic corpora by academic English learners. Proceedings of the 17th International CALL Conference, Tarragona, Spain."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1007\/s10791-014-9248-5","article-title":"A genetic programming framework to schedule webpage updates","volume":"18","author":"Cristiano","year":"2015","journal-title":"Inf. Retr. J."},{"key":"ref_37","first-page":"235","article-title":"Scalable online incremental learning for web spam detection","volume":"124","author":"Han","year":"2012","journal-title":"Recent Adv. Comput. Sci. Inf. Eng."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1075\/ijcl.17.3.04har","article-title":"CQPweb\u2014Combining power, flexibility and usability in a corpus analysis tool","volume":"17","author":"Hardie","year":"2012","journal-title":"Int. J. Corpus Linguist."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/1\/16\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:55:06Z","timestamp":1760208906000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/1\/16"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,12,22]]},"references-count":38,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2018,1]]}},"alternative-id":["s18010016"],"URL":"https:\/\/doi.org\/10.3390\/s18010016","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2017,12,22]]}}}