{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T05:49:09Z","timestamp":1774590549890,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,12,16]],"date-time":"2024-12-16T00:00:00Z","timestamp":1734307200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>The accurate and timely detection of cyber threats is critical to keeping our online economy and data safe. A key technique in early detection is the classification of unusual patterns of network behaviour, often hidden as low-frequency events within complex time-series packet flows. One of the ways in which such anomalies can be detected is to analyse the information entropy of the payload within individual packets, since changes in entropy can often indicate suspicious activity\u2014such as whether session encryption has been compromised, or whether a plaintext channel has been co-opted as a covert channel. To decide whether activity is anomalous, we need to compare real-time entropy values with baseline values, and while the analysis of entropy in packet data is not particularly new, to the best of our knowledge, there are no published baselines for payload entropy across commonly used network services. We offer two contributions: (1) we analyse several large packet datasets to establish baseline payload information entropy values for standard network services, and (2) we present an efficient method for engineering entropy metrics from packet flows from real-time and offline packet data. Such entropy metrics can be included within feature subsets, thus making the feature set richer for subsequent analysis and machine learning applications.<\/jats:p>","DOI":"10.3390\/fi16120470","type":"journal-article","created":{"date-parts":[[2024,12,16]],"date-time":"2024-12-16T10:08:53Z","timestamp":1734343733000},"page":"470","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Characterising Payload Entropy in Packet Flows\u2014Baseline Entropy Analysis for Network Anomaly Detection"],"prefix":"10.3390","volume":"16","author":[{"given":"Anthony","family":"Kenyon","sequence":"first","affiliation":[{"name":"Hyperscalar Ltd., High Wycombe HP22 4LW, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8986-884X","authenticated-orcid":false,"given":"Lipika","family":"Deka","sequence":"additional","affiliation":[{"name":"School of Computer Science and Informatics, De Montfort University, Leicester LE1 9BH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7398-5870","authenticated-orcid":false,"given":"David","family":"Elizondo","sequence":"additional","affiliation":[{"name":"School of Computer Science and Informatics, De Montfort University, Leicester LE1 9BH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"8309","DOI":"10.1007\/s10586-018-1755-5","article-title":"Anomaly network traffic detection algorithm based on information entropy measurement under the cloud computing environment","volume":"22","author":"Yang","year":"2019","journal-title":"Clust. Comput."},{"key":"ref_2","unstructured":"Tellenbach, B., Burkhart, M., Sornette, D., and Maillart, T. (2009). Beyond shannon: Characterizing internet traffic with generalized entropy metrics. Passive and Active Network Measurement: 10th International Conference, PAM 2009, Seoul, Republic of Korea, 1\u20133 April 2009, Springer. Proceedings 10."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_4","unstructured":"Shannon, C.E., and Weaver, W. (2015). The Mathematical Theory of Communication, University of Illinois Press."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"102022","DOI":"10.1016\/j.cose.2020.102022","article-title":"Are public intrusion datasets fit for purpose characterising the state of the art in intrusion event datasets","volume":"99","author":"Kenyon","year":"2020","journal-title":"Comput. Secur."},{"key":"ref_6","unstructured":"Goubault-Larrecq, J., and Olivain, J. (2006). Detecting Subverted Cryptographic Protocols by Entropy Checking. [Ph.D. Thesis, LSV]."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1109\/MSP.2007.48","article-title":"Using entropy analysis to find encrypted and packed malware","volume":"5","author":"Lyda","year":"2007","journal-title":"IEEE Secur. Priv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Gibert, D., Mateu, C., Planes, J., and Vicens, R. (2018, January 2\u20137). Classification of malware by using structural entropy on convolutional neural networks. Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 32, No. 1), New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11409"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10207-014-0242-0","article-title":"Malware analysis using visualized images and entropy graphs","volume":"14","author":"Han","year":"2015","journal-title":"Int. J. Inf. Secur."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wang, Y., Zhang, Z., Guo, L., and Li, S. (2011, January 28\u201330). Using entropy to classify traffic more deeply. Proceedings of the 2011 IEEE Sixth International Conference on Networking, Architecture, and Storage, Dalian, China.","DOI":"10.1109\/NAS.2011.18"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zi, L., Yearwood, J., and Wu, X.W. (2010, January 1\u20133). Adaptive clustering with feature ranking for DDoS attacks detection. Proceedings of the 2010 Fourth International Conference on Network and System Security, Melbourne, Australia.","DOI":"10.1109\/NSS.2010.70"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2004","DOI":"10.1109\/TPDS.2012.316","article-title":"Identification of peer-to-peer voip sessions using entropy and codec properties","volume":"24","author":"Gomes","year":"2012","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1016\/j.protcy.2013.04.045","article-title":"Entropy and flow-based approach for anomalous traffic filtering","volume":"7","year":"2013","journal-title":"Procedia Technol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Roma\u00f1a, D.A.L., Kubota, S., Sugitani, K., and Musashi, Y. (2008, January 20\u201323). DNS based spam bots detection in a university. Proceedings of the 2008 First International Conference on Intelligent Networks and Intelligent Systems, Toronto, ON, Canada.","DOI":"10.1109\/ICINIS.2008.54"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Altaher, A., Ramadass, S., and Almomani, A. (2011, January 19\u201321). Real time network anomaly detection using relative entropy. Proceedings of the 8th International Conference on High-Capacity Optical Networks and Emerging Technologies, Riyadh, Saudi Arabia.","DOI":"10.1109\/HONET.2011.6149829"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gu, Y., McCallum, A., and Towsley, D. (2005, January 19\u201321). Detecting anomalies in network traffic using maximum entropy estimation. Proceedings of the 5th ACM SIGCOMM conference on Internet Measurement, Berkeley, CA, USA.","DOI":"10.1145\/1330107.1330148"},{"key":"ref_17","unstructured":"Mamun, M.S.I., Ghorbani, A.A., and Stakhanova, N. (2016). An entropy based encrypted traffic classifier. Information and Communications Security: 17th International Conference, ICICS 2015, Beijing, China, 9\u201311 December 2015, Springer International Publishing. Revised Selected Papers 17."},{"key":"ref_18","unstructured":"Croll, G.J. (2013). Bientropy-the approximate entropy of a finite binary string. arXiv."},{"key":"ref_19","unstructured":"Zhiyong, C., and Yong, Z. (2009, January 19\u201320). Entropy based taxonomy of network convert channels. Proceedings of the 2009 2nd International Conference on Power Electronics and Intelligent Transportation System (PEITS), Shenzhen, China."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chow, J.K., Li, X., and Mountrouidou, X. (2017, January 8\u201311). Raising flags: Detecting covert storage channels using relative entropy. Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education, Seattle, WA, USA.","DOI":"10.1145\/3017680.3022454"},{"key":"ref_21","unstructured":"Homem, I., Papapetrou, P., and Dosis, S. (2017). Entropy-based prediction of network protocols in the forensic analysis of dns tunnels. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Kenyon, T. (2018). Transportation cyber-physical systems security and privacy. Transportation Cyber-Physical Systems, Elsevier.","DOI":"10.1016\/B978-0-12-814295-0.00005-8"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Li, H., and Chasaki, D. (2022, January 25\u201328). Network-Based Machine Learning Detection of Covert Channel Attacks on Cyber-Physical Systems. Proceedings of the 2022 IEEE 20th International Conference on Industrial Informatics (INDIN), Perth, Australia.","DOI":"10.1109\/INDIN51773.2022.9976152"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"\u00d6zdel, S., Ate\u015f, \u00c7., Ate\u015f, P.D., Koca, M., and Anar\u0131m, E. (September, January 29). Payload-Based Network Traffic Analysis for Application Classification and Intrusion Detection. Proceedings of the 2022 30th European Signal Processing Conference (EUSIPCO), Belgrade, Serbia.","DOI":"10.23919\/EUSIPCO55093.2022.9909683"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1126\/science.267.5199.843","article-title":"Gauging similarity with n-grams: Language-independent categorization of text","volume":"267","author":"Damashek","year":"1995","journal-title":"Science"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Arackaparambil, C., Bratus, S., Brody, J., and Shubina, A. (2010, January 19\u201323). Distributed monitoring of conditional entropy for anomaly detection in streams. Proceedings of the 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW), Atlanta, GA, USA.","DOI":"10.1109\/IPDPSW.2010.5470852"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1109\/TIT.1981.1056331","article-title":"The performance of universal encoding","volume":"27","author":"Krichevsky","year":"1981","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_28","unstructured":"Kenyon, A., Elizondo, D., and Deka, L. (2023). Improved Flow Recovery from Packet Data. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2037","DOI":"10.1109\/COMST.2014.2321898","article-title":"Flow Monitoring Explained: From packet capture to data analysis with NetFlow and IPFIX","volume":"16","author":"Hofstede","year":"2014","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_30","unstructured":"Patterson, M.A. (2013). Unleasing the Power of NetFlow and IPFIX, Plixer International, Inc."},{"key":"ref_31","unstructured":"Internet Engineering Task Force (2014, August 07). RFC-7011 Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information. IETF [online]. Available online: https:\/\/tools.ietf.org\/html\/rfc7011."},{"key":"ref_32","unstructured":"Chen, J., and Zhang, Q. (2014). AMS-Sampling in Distributed Monitoring, with Applications to Entropy. arXiv."},{"key":"ref_33","unstructured":"Kerr, D.R., Bruins, B.L., and Cisco Systems, Inc. (2001). Network Flow Switching and Flow Data Export. (6,243,667), U.S. Patent."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Claise, B. (2004). RFC 3954: Cisco Systems NetFlow Services Export Version 9, RFC Editor.","DOI":"10.17487\/rfc3954"},{"key":"ref_35","unstructured":"Internet Engineering Task Force (2014, August 07). RFC-6313 Export of Structure Data in IP Flow Information Export (IPFIX). IETF [online]. Available online: https:\/\/tools.ietf.org\/html\/rfc6313."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Claise, B., and Trammell, B. (2013). RFC 7012: Information Model fo IP Flow Information Export (IPFIX), RFC Editor.","DOI":"10.17487\/rfc7012"},{"key":"ref_37","unstructured":"(2024, December 08). Wilipedia Page, List of TCP and UDP Port Numbers. Available online: https:\/\/en.wikipedia.org\/wiki\/List_of_TCP_and_UDP_port_numbers."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1145\/362375.362389","article-title":"A note on the confinement problem","volume":"16","author":"Lampson","year":"1973","journal-title":"Commun. ACM"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1109\/COMST.2007.4317620","article-title":"A survey of covert channels and countermeasures in computer network protocols","volume":"9","author":"Zander","year":"2007","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_40","unstructured":"(2024, December 08). CVE-2014-0160 Record for Heartbleed. Available online: https:\/\/cve.mitre.org\/cgi-bin\/cvename.cgi?name=CVE-2014-0160."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Dorfinger, P., Panholzer, G., and John, W. (2011). Entropy estimation for real-time encrypted traffic identification (short paper). Traffic Monitoring and Analysis: Third International Workshop, TMA 2011, Vienna, Austria, 27 April 2011, Springer. Proceedings 3.","DOI":"10.1007\/978-3-642-20305-3_14"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/12\/470\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:52:57Z","timestamp":1760115177000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/12\/470"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,16]]},"references-count":41,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["fi16120470"],"URL":"https:\/\/doi.org\/10.3390\/fi16120470","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,16]]}}}