{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,3]],"date-time":"2025-12-03T18:14:05Z","timestamp":1764785645179,"version":"3.41.0"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,6,5]],"date-time":"2025-06-05T00:00:00Z","timestamp":1749081600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,6,5]],"date-time":"2025-06-05T00:00:00Z","timestamp":1749081600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100014440","name":"Ministerio de Ciencia, Innovaci\u00f3n y Universidades","doi-asserted-by":"publisher","award":["MICIN\/AEI\/10.13039 \/501100011033"],"award-info":[{"award-number":["MICIN\/AEI\/10.13039 \/501100011033"]}],"id":[{"id":"10.13039\/100014440","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Cybersecurity"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>This paper introduces Biblio-US17, a labeled dataset collected over 6 months from the log files of a popular public website at the University of Seville. It contains 47 million records, each including the method, uniform resource identifier (URI) and associated response code and size of every request received by the web server. Records have been classified as either normal or attack using a comprehensive semi-automated process, which involved signature-based detection, assisted inspection of URIs vocabulary, and substantial expert manual supervision. Unlike comparable datasets, this one offers a genuine real-world perspective on the normal operation of an active website, along with an unbiased proportion of actual attacks (i.e., non-synthetic). This makes it ideal for evaluating and comparing anomaly-based approaches in a realistic environment. Its extensive size and duration also make it valuable for addressing challenges like data shift and insufficient training. This paper describes the collection and labeling processes, dataset structure, and most relevant properties. We also include an example of an application for assessing the performance of a simple anomaly detector. Biblio-US17, now available to the scientific community, can also be used to model the URIs used by current web servers.<\/jats:p>","DOI":"10.1186\/s42400-024-00336-3","type":"journal-article","created":{"date-parts":[[2025,6,5]],"date-time":"2025-06-05T01:01:36Z","timestamp":1749085296000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Building a large, realistic and labeled HTTP URI dataset for anomaly-based intrusion detection systems: Biblio-US17"],"prefix":"10.1186","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8424-9932","authenticated-orcid":false,"given":"Jes\u00fas","family":"D\u00edaz-Verdejo","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8505-1920","authenticated-orcid":false,"given":"Rafael","family":"Estepa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1841-3973","authenticated-orcid":false,"given":"Antonio","family":"Estepa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8146-8438","authenticated-orcid":false,"given":"Javier","family":"Mu\u00f1oz-Calle","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6376-4620","authenticated-orcid":false,"given":"Germ\u00e1n","family":"Madinabeitia","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,6,5]]},"reference":[{"key":"336_CR1","unstructured":"(2023) Cyber Security Statistics The Ultimate List Of Stats Data, & Trends For 2023 . https:\/\/purplesec.us\/resources\/cyber-security-statistics\/#ZeroDay. Accessed 19-May-2023"},{"issue":"18","key":"336_CR6","doi-asserted-by":"publisher","first-page":"7760","DOI":"10.3390\/s23187760","volume":"23","author":"S Abad","year":"2023","unstructured":"Abad S, Gholamy H, Aslani M (2023) Classification of malicious urls using machine learning. Sensors 23(18):7760","journal-title":"Sensors"},{"key":"336_CR7","doi-asserted-by":"crossref","unstructured":"Agarwal N, Hussain SZ (2018) A closer look at intrusion detection system for web applications. Secur Commun Netw 2018","DOI":"10.1155\/2018\/9601357"},{"issue":"2","key":"336_CR8","doi-asserted-by":"publisher","first-page":"91","DOI":"10.48161\/qaj.v1n2a52","volume":"1","author":"ZS Ageed","year":"2021","unstructured":"Ageed ZS, Zeebaree SR, Sadeeq MM et al (2021) A survey of data mining implementation in smart city applications. Qubahan Acad J 1(2):91\u201399","journal-title":"Qubahan Acad J"},{"key":"336_CR9","unstructured":"Apps SC (2021) Cyberattacks 2021: phishing, ransomware & data breach statistics from the last year. https:\/\/spanning.com\/blog\/cyberattacks-2021-phishing-ransomware-data-breach-statistics\/"},{"key":"336_CR10","unstructured":"Ben\u00a0Martin CA, Denis\u00a0Sinegubko RE, Pellegrini T (2022) 2022 website threat research report. https:\/\/sucuri.net\/reports\/. Accessed 28-Aug-2024"},{"key":"336_CR11","doi-asserted-by":"crossref","unstructured":"Betarte G, Gim\u00e9nez E, Mart\u00ednez R et\u00a0al (2018) Improving web application firewalls through anomaly detection. In: 2018 17th IEEE international conference on machine learning and applications (ICMLA). IEEE, pp 779\u2013784","DOI":"10.1109\/ICMLA.2018.00124"},{"key":"336_CR12","first-page":"22","volume":"1","author":"ST Brugger","year":"2007","unstructured":"Brugger ST, Chow J (2007) An assessment of the darpa ids evaluation dataset using snort. UCDAVIS Dept Comput Sci 1:22","journal-title":"UCDAVIS Dept Comput Sci"},{"key":"336_CR13","doi-asserted-by":"crossref","unstructured":"Chodak G, Suchacka G, Chawla Y (2020) EClog: HTTP-level e-commerce data based on server access logs for an online store","DOI":"10.1016\/j.comnet.2020.107589"},{"key":"336_CR14","doi-asserted-by":"crossref","unstructured":"Cretu GF, Stavrou A, Locasto ME et\u00a0al (2008) Casting out demons: sanitizing training data for anomaly sensors. In: 2008 IEEE symposium on security and privacy (sp 2008). IEEE, pp 81\u201395","DOI":"10.1109\/SP.2008.11"},{"key":"336_CR15","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1016\/j.future.2020.03.033","volume":"109","author":"JE D\u00edaz-Verdejo","year":"2020","unstructured":"D\u00edaz-Verdejo JE, Estepa A, Estepa R et al (2020) A methodology for conducting efficient sanitization of http training datasets. Future Gener Comput Syst 109:67\u201382","journal-title":"Future Gener Comput Syst"},{"issue":"2","key":"336_CR16","doi-asserted-by":"publisher","first-page":"852","DOI":"10.3390\/app12020852","volume":"12","author":"J D\u00edaz-Verdejo","year":"2022","unstructured":"D\u00edaz-Verdejo J, Mu\u00f1oz-Calle J, Estepa Alonso A et al (2022) On the detection capabilities of signature-based intrusion detection systems in the context of web attacks. Appl Sci 12(2):852","journal-title":"Appl Sci"},{"key":"336_CR17","doi-asserted-by":"crossref","unstructured":"D\u00edaz Verdejo J, Estepa Alonso RM, Estepa Alonso AJ, Mu\u00f1oz Calle FJ et\u00a0al (2023) Labeled HTTP requests dataset: dataset Biblio-US17. https:\/\/doi.org\/10.12795\/11441\/148254. Accessed 29-Jul-2023","DOI":"10.12795\/11441\/148254"},{"key":"336_CR18","doi-asserted-by":"crossref","unstructured":"Diaz-Verdejo J, Estepa R, Alonso AE et\u00a0al (2024a) Insights into anomaly-based intrusion detection systems usability. a case study using real http requests. In: European interdisciplinary cybersecurity conference, pp 82\u201389","DOI":"10.1145\/3655693.3655745"},{"key":"336_CR19","doi-asserted-by":"crossref","unstructured":"Diaz-Verdejo J, Estepa\u00a0Alonso R, Estepa\u00a0Alonso A et\u00a0al (2024b) Biblio-us17: a labeled real url dataset for anomaly-based intrusion detection systems development. In: European interdisciplinary cybersecurity conference, pp 217\u2013218","DOI":"10.1145\/3655693.3661319"},{"key":"336_CR20","doi-asserted-by":"crossref","unstructured":"D\u00edaz-Verdejo J, Mu\u00f1oz-Calle J, Estepa Alonso R et\u00a0al (2024c) Inspectorlog: a new tool for offline attack detection over web log trace files. In: Proceedings of the 21st international conference on security and cryptography\u2014SECRYPT, INSTICC. SciTePress, pp 692\u2013697. https:\/\/doi.org\/10.5220\/0012764000003767","DOI":"10.5220\/0012764000003767"},{"key":"336_CR21","doi-asserted-by":"publisher","first-page":"44410","DOI":"10.1109\/ACCESS.2020.2977591","volume":"8","author":"R Estepa","year":"2020","unstructured":"Estepa R, D\u00edaz-Verdejo JE, Estepa A et al (2020) How much training data is enough? A case study for http anomaly-based intrusion detection. IEEE Access 8:44410\u201344425","journal-title":"IEEE Access"},{"key":"336_CR2","unstructured":"Etopen (emerging threats open) rule sets (2023) https:\/\/community.emergingthreats.net\/. Accessed 17-Nov-2023"},{"key":"336_CR22","unstructured":"Forbes (2023) Top Website Statistics For 2023. https:\/\/www.forbes.com\/advisor\/business\/software\/website-statistics\/. Accessed 19-May-2023"},{"issue":"4","key":"336_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1749603.1749605","volume":"42","author":"BC Fung","year":"2010","unstructured":"Fung BC, Wang K, Chen R et al (2010) Privacy-preserving data publishing: a survey of recent developments. ACM Comput Surv 42(4):1\u201353","journal-title":"ACM Comput Surv"},{"key":"336_CR24","doi-asserted-by":"crossref","unstructured":"Gao Y, Ma Y, Li D (2017) Anomaly detection of malicious users\u2019 behaviors for web applications based on web logs. In: 2017 IEEE 17th international conference on communication technology (ICCT). IEEE, pp 1352\u20131355","DOI":"10.1109\/ICCT.2017.8359854"},{"issue":"1\u20132","key":"336_CR25","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.cose.2008.08.003","volume":"28","author":"P Garcia-Teodoro","year":"2009","unstructured":"Garcia-Teodoro P, Diaz-Verdejo J, Maci\u00e1-Fern\u00e1ndez G et al (2009) Anomaly-based network intrusion detection: techniques, systems and challenges. Comput Secur 28(1\u20132):18\u201328","journal-title":"Comput Secur"},{"issue":"4","key":"336_CR26","doi-asserted-by":"publisher","DOI":"10.1002\/ett.4240","volume":"32","author":"S Hajj","year":"2021","unstructured":"Hajj S, El Sibai R, Bou Abdo J et al (2021) Anomaly-based intrusion detection systems: the requirements, methods, measurements, and datasets. Trans Emerg Telecommun Technol 32(4):e4240","journal-title":"Trans Emerg Telecommun Technol"},{"key":"336_CR27","doi-asserted-by":"crossref","unstructured":"Hjerppe K, Ruohonen J, Lepp\u00e4nen V (2019) The general data protection regulation: requirements, architectures, and constraints. In: 2019 IEEE 27th international requirements engineering conference (RE). IEEE, pp 265\u2013275","DOI":"10.1109\/RE.2019.00036"},{"key":"336_CR28","unstructured":"Hus\u00e1k M, Apruzzese G, Yang SJ et\u00a0al (2021) Towards an efficient detection of pivoting activity. In: 2021 IFIP\/IEEE international symposium on integrated network management (IM). IEEE, pp 980\u2013985"},{"key":"336_CR29","first-page":"21","volume":"19","author":"IM Ibrahim","year":"2021","unstructured":"Ibrahim IM, Ameen SY, Yasin HM et al (2021) Web server performance improvement using dynamic load balancing techniques: a review. System 19:21","journal-title":"System"},{"issue":"1","key":"336_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s42400-019-0038-7","volume":"2","author":"A Khraisat","year":"2019","unstructured":"Khraisat A, Gondal I, Vamplew P et al (2019) Survey of intrusion detection systems: techniques, datasets and challenges. Cybersecurity 2(1):1\u201322","journal-title":"Cybersecurity"},{"issue":"1","key":"336_CR31","doi-asserted-by":"publisher","first-page":"402","DOI":"10.1109\/TR.2020.3031317","volume":"70","author":"M Landauer","year":"2020","unstructured":"Landauer M, Skopik F, Wurzenberger M et al (2020) Have it your way: generating customized log datasets with a model-driven simulation testbed. IEEE Trans Reliab 70(1):402\u2013415","journal-title":"IEEE Trans Reliab"},{"key":"336_CR32","doi-asserted-by":"crossref","unstructured":"Landauer M, Skopik F, H\u00f6ld G et\u00a0al (2022) A user and entity behavior analytics log data set for anomaly detection in cloud computing. In: 2022 IEEE international conference on big data (big data). IEEE, pp 4285\u20134294","DOI":"10.1109\/BigData55660.2022.10020672"},{"key":"336_CR33","doi-asserted-by":"crossref","unstructured":"Lara-Romero A, Ternero-Muniz J, Estepa R et\u00a0al (2023) Http cyberattacks detection through automatic signature generation in multi-site iot deployments. In: Proceedings of the 2023 European interdisciplinary cybersecurity conference. ACM, pp 65\u201370","DOI":"10.1145\/3590777.3590788"},{"key":"336_CR34","doi-asserted-by":"crossref","unstructured":"Mahoney MV, Chan PK (2003) An analysis of the 1999 darpa\/lincoln laboratory evaluation data for network anomaly detection. In: Recent advances in intrusion detection: 6th international symposium, RAID 2003, Pittsburgh, PA, USA, September 8\u201310, 2003. Proceedings 6, Springer, pp 220\u2013237","DOI":"10.1007\/978-3-540-45248-5_13"},{"key":"336_CR3","unstructured":"Nemesida rules (2023) https:\/\/rlinfo.nemesida-security.com\/. Accessed 17-Nov-2023"},{"key":"336_CR4","unstructured":"Owasp modsecurity core rule set (crs) (2023), https:\/\/github.com\/coreruleset\/coreruleset\/releases\/. Accessed 17-Nov-2023"},{"key":"336_CR35","doi-asserted-by":"crossref","unstructured":"Pa\u0142ka D, Zachara M (2011) Learning web application firewall-benefits and caveats. In: Availability, reliability and security for business, enterprise and health information systems: IFIP WG 8.4\/8.9 international cross domain conference and workshop, ARES 2011, Vienna, Austria, August 22-26, 2011. Proceedings 6, Springer, pp 295\u2013308","DOI":"10.1007\/978-3-642-23300-5_23"},{"key":"336_CR36","doi-asserted-by":"crossref","unstructured":"Paxson V (2004) Strategies for sound internet measurement. In: Proceedings of the 4th ACM SIGCOMM conference on internet measurement, pp 263\u2013271","DOI":"10.1145\/1028788.1028824"},{"issue":"1","key":"336_CR37","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1145\/290163.290168","volume":"1","author":"MK Reiter","year":"1998","unstructured":"Reiter MK, Rubin AD (1998) Crowds: anonymity for web transactions. ACM Trans Inf Syst Secur 1(1):66\u201392","journal-title":"ACM Trans Inf Syst Secur"},{"key":"336_CR38","unstructured":"Salazar-Hern\u00e1ndez R, D\u00edaz-Verdejo JE (2010) Anonimizaci\u00f3n de payloads para el desarrollo de aids basados en protocolos. Actas de las IX Jornadas de Ingenier\u00eda Telem\u00e1tica (JITEL 2010) pp 260\u2013267"},{"key":"336_CR39","doi-asserted-by":"crossref","unstructured":"Saxena A, Arora A, Saxena S et\u00a0al (2022) Detection of web attacks using machine learning based url classification techniques. In: 2022 2nd international conference on intelligent technologies (CONIT). IEEE, pp 1\u201313","DOI":"10.1109\/CONIT55038.2022.9847838"},{"key":"336_CR40","doi-asserted-by":"crossref","unstructured":"Schulz H, Okanovi\u0107 D, van Hoorn A et\u00a0al (2021) Context-tailored workload model generation for continuous representative load testing. In: Proceedings of the ACM\/SPEC international conference on performance engineering, pp 21\u201332","DOI":"10.1145\/3427921.3450240"},{"key":"336_CR41","first-page":"108","volume":"1","author":"I Sharafaldin","year":"2018","unstructured":"Sharafaldin I, Lashkari AH, Ghorbani AA (2018) Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp 1:108\u2013116","journal-title":"ICISSp"},{"key":"336_CR5","unstructured":"Talos (vrt group) community rules, (2023). https:\/\/www.snort.org\/talos. Accessed 17-Nov-2023"},{"key":"336_CR42","doi-asserted-by":"crossref","unstructured":"Uetz R, Hemminghaus C, Hackl\u00e4nder L et\u00a0al (2021) Reproducible and adaptable log data generation for sound cybersecurity experiments. In: Annual computer security applications conference, pp 690\u2013705","DOI":"10.1145\/3485832.3488020"},{"key":"336_CR43","doi-asserted-by":"publisher","first-page":"200","DOI":"10.1016\/j.comnet.2017.08.013","volume":"127","author":"EK Viegas","year":"2017","unstructured":"Viegas EK, Santin AO, Oliveira LS (2017) Toward a reliable anomaly-based intrusion detection in real-world environments. Comput Netw 127:200\u2013216","journal-title":"Comput Netw"}],"container-title":["Cybersecurity"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s42400-024-00336-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s42400-024-00336-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s42400-024-00336-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,5]],"date-time":"2025-06-05T01:01:50Z","timestamp":1749085310000},"score":1,"resource":{"primary":{"URL":"https:\/\/cybersecurity.springeropen.com\/articles\/10.1186\/s42400-024-00336-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,5]]},"references-count":43,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["336"],"URL":"https:\/\/doi.org\/10.1186\/s42400-024-00336-3","relation":{},"ISSN":["2523-3246"],"issn-type":[{"value":"2523-3246","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,5]]},"assertion":[{"value":"7 March 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 November 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 June 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"38"}}