{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T00:27:37Z","timestamp":1773448057778,"version":"3.50.1"},"reference-count":65,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,6,23]],"date-time":"2020-06-23T00:00:00Z","timestamp":1592870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>The cyber security landscape is fundamentally changing over the past years. While technology is evolving and new sophisticated applications are being developed, a new threat scenario is emerging in alarming proportions. Sophisticated threats with multi-vectored, multi-staged and polymorphic characteristics are performing complex attacks, making the processes of detection and mitigation far more complicated. Thus, organizations were encouraged to change their traditional defense models and to use and to develop new systems with a proactive approach. Such changes are necessary because the old approaches are not effective anymore to detect advanced attacks. Also, the organizations are encouraged to develop the ability to respond to incidents in real-time using complex threat intelligence platforms. However, since the field is growing rapidly, today Cyber Threat Intelligence concept lacks a consistent definition and a heterogeneous market has emerged, including diverse systems and tools, with different capabilities and goals. This work aims to provide a comprehensive evaluation methodology of threat intelligence standards and cyber threat intelligence platforms. The proposed methodology is based on the selection of the most relevant candidates to establish the evaluation criteria. In addition, this work studies the Cyber Threat Intelligence ecosystem and Threat Intelligence standards and platforms existing in state-of-the-art.<\/jats:p>","DOI":"10.3390\/fi12060108","type":"journal-article","created":{"date-parts":[[2020,6,24]],"date-time":"2020-06-24T08:54:50Z","timestamp":1592988890000},"page":"108","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":52,"title":["A Methodology to Evaluate Standards and Platforms within Cyber Threat Intelligence"],"prefix":"10.3390","volume":"12","author":[{"given":"Alessandra","family":"de Melo e Silva","sequence":"first","affiliation":[{"name":"Post Graduation in Electrical Engineering (PPEE), Department of Electrical Engineering, University of Bras\u00edlia, Bras\u00edlia 70910-900, Brazil"}]},{"given":"Jo\u00e3o Jos\u00e9","family":"Costa Gondim","sequence":"additional","affiliation":[{"name":"Post Graduation in Electrical Engineering (PPEE), Department of Electrical Engineering, University of Bras\u00edlia, Bras\u00edlia 70910-900, Brazil"},{"name":"Department of Computer Science (CIC), University of Brasilia (UnB), Brasilia-DF 70910-900, Brazil"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6717-3374","authenticated-orcid":false,"given":"Robson","family":"de Oliveira Albuquerque","sequence":"additional","affiliation":[{"name":"Post Graduation in Electrical Engineering (PPEE), Department of Electrical Engineering, University of Bras\u00edlia, Bras\u00edlia 70910-900, Brazil"},{"name":"Group of Analysis, Security and Systems (GASS), Department of Software Engineering and Artificial Intelligence (DISIA), Faculty of Computer Science and Engineering, Office 431, Universidad Complutense de Madrid (UCM), Calle Profesor Jos\u00e9 Garc\u00eda Santesmases, 9, Ciudad Universitaria, 28040 Madrid, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7573-6272","authenticated-orcid":false,"given":"Luis Javier","family":"Garc\u00eda Villalba","sequence":"additional","affiliation":[{"name":"Group of Analysis, Security and Systems (GASS), Department of Software Engineering and Artificial Intelligence (DISIA), Faculty of Computer Science and Engineering, Office 431, Universidad Complutense de Madrid (UCM), Calle Profesor Jos\u00e9 Garc\u00eda Santesmases, 9, Ciudad Universitaria, 28040 Madrid, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,23]]},"reference":[{"key":"ref_1","unstructured":"Pokorny, Z. (2019). The Threat Intelligence Handbook: Moving toward a Security Intelligence Program, CyberEdge Group."},{"key":"ref_2","unstructured":"Bissell, K., LaSalle, R., and Dal Cin, P. (2020, May 04). TThe Cost of Cybercrime\u2014Ninth Annual Cost of Cybercrime Study. Available online: https:\/\/www.accenture.com\/_acnmedia\/pdf-96\/accenture-2019-cost-of-cybercrime-study-final.pdf."},{"key":"ref_3","unstructured":"Bissell, K., LaSalle, R., and Dal Cin, P. (2020, May 04). The 2020 Cyber Security Report. Available online: https:\/\/pages.checkpoint.com\/cyber-security-report-2020."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Tounsi, W. (2019). What is Cyber Threat Intelligence and How is it Evolving?. Cyber-Vigilance and Digital Trust, John Wiley & Sons, Inc.","DOI":"10.1002\/9781119618393"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1851","DOI":"10.1109\/COMST.2019.2891891","article-title":"A survey on advanced persistent threats: Techniques, solutions, challenges, and research opportunities","volume":"21","author":"Alshamrani","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wu, J. (2020). New Approaches to Cyber Defense. Cyberspace Mimic Defense, Springer.","DOI":"10.1007\/978-3-030-29844-9"},{"key":"ref_7","first-page":"371","article-title":"Cyber Threat Intelligence\u2014Issue and Challenges","volume":"10","author":"Abu","year":"2018","journal-title":"Indones. J. Electr. Eng. Comput. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1016\/j.future.2019.06.026","article-title":"A cloud-edge based data security architecture for sharing and analysing cyber threat information","volume":"102","author":"Chadwick","year":"2020","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"101867","DOI":"10.1016\/j.cose.2020.101867","article-title":"TIMiner: Automatically extracting and analyzing categorized cyber threat intelligence from social data","volume":"95","author":"Zhao","year":"2020","journal-title":"Comput. Secur."},{"key":"ref_10","unstructured":"Gao, Y., LI, X., PENG, H., Fang, B., and Yu, P. (2020). HinCTI: A Cyber Threat Intelligence Modeling and Identification System Based on Heterogeneous Information Network. IEEE Trans. Knowl. Data Eng., 1."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1007\/s11235-019-00613-4","article-title":"Cybersecurity threat intelligence knowledge exchange based on blockchain","volume":"73","author":"Riesco","year":"2019","journal-title":"Telecommun. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Rantos, K., Spyros, A., Papanikolaou, A., Kritsas, A., Ilioudis, C., and Katos, V. (2020). Interoperability Challenges in the Cybersecurity Information Sharing Ecosystem. Computers, 9.","DOI":"10.3390\/computers9010018"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ramsdale, A., Shiaeles, S., and Kolokotronis, N. (2020). A Comparative Analysis of Cyber-Threat Intelligence Sources, Formats and Languages. Electronics, 9.","DOI":"10.3390\/electronics9050824"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Bauer, S., Fischer, D., Sauerwein, C., Latzel, S., Stelzer, D., and Breu, R. (2020, January 7\u201310). Towards an Evaluation Framework for Threat Intelligence Sharing Platforms. Proceedings of the 53rd Hawaii International Conference on System Sciences, Maui, HI, USA.","DOI":"10.24251\/HICSS.2020.239"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"101761","DOI":"10.1016\/j.cose.2020.101761","article-title":"A review and theoretical explanation of the \u2018Cyberthreat-Intelligence (CTI) capability\u2019 that needs to be fostered in information security practitioners and how this can be accomplished","volume":"92","author":"Shin","year":"2020","journal-title":"Comput. Secur."},{"key":"ref_16","unstructured":"Sauerwein, C., Sillaber, C., Mussmann, A., and Breu, R. (2017, January 12\u201315). Threat intelligence sharing platforms: An exploratory study of software vendors and research perspectives. Proceedings of the 13th International Conference on Wirtschaftsinformatik, St.Gallen, Switzerland."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.cose.2016.04.003","article-title":"A problem shared is a problem halved: A survey on the dimensions of collective cyber defense through security information sharing","volume":"60","author":"Skopik","year":"2016","journal-title":"Comput. Secur."},{"key":"ref_18","unstructured":"ENISA (2020, March 16). Exploring the Opportunities and Limitations of Current Threat Intelligence Platforms. Available online: https:\/\/www.enisa.europa.eu\/publications\/exploring-the-opportunities-and-limitations-of-current-threat-intelligence-platforms."},{"key":"ref_19","unstructured":"Poputa-Clean, P., and Stingley, M. (2020, March 23). Automated Defense-Using Threat Intelligence to Augment Security. Available online: https:\/\/www.sans.org\/reading-room\/whitepapers\/threats\/paper\/35692."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"101589","DOI":"10.1016\/j.cose.2019.101589","article-title":"Cyber threat intelligence sharing: Survey and research directions","volume":"87","author":"Wagner","year":"2019","journal-title":"Comput. Secur."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Sarker, I.H., Abushark, Y.B., and Khan, A.I. (2020). ContextPCA: Predicting Context-Aware Smartphone Apps Usage Based On Machine Learning Techniques. Symmetry, 12.","DOI":"10.3390\/sym12040499"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sarker, I.H., Kayes, A.S.M., and Watters, P. (2019). Effectiveness analysis of machine learning classification models for predicting personalized context-aware smartphone usage. J. Big Data, 6.","DOI":"10.1186\/s40537-019-0219-y"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Sarker, I.H., Abushark, Y.B., Alsolami, F., and Khan, A.I. (2020). IntruDTree: A Machine Learning-Based Cyber Security Intrusion Detection Model. Symmetry, 12.","DOI":"10.20944\/preprints202004.0481.v1"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Truong, T.C., Zelinka, I., Plucar, J., \u010cand\u00edk, M., and \u0160ulc, V. (2020). Artificial Intelligence and Cybersecurity: Past, Presence, and Future. Advances in Intelligent Systems and Computing, Springer.","DOI":"10.1007\/978-981-15-0199-9_30"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.future.2019.02.013","article-title":"A machine learning-based FinTech cyber threat attribution framework using high-level indicators of compromise","volume":"96","author":"Noor","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_26","unstructured":"Dalton, A., Aghaei, E., Al-Shaer, E., Bhatia, A., Castillo, E., Cheng, Z., Dhaduvai, S., Duan, Q., Islam, M.M., and Karimi, Y. (2020). The Panacea Threat Intelligence and Active Defense Platform. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Kazato, Y., Nakagawa, Y., and Nakatani, Y. (2020, January 10\u201313). Improving Maliciousness Estimation of Indicator of Compromise Using Graph Convolutional Networks. Proceedings of the 2020 IEEE 17th Annual Consumer Communications & Networking Conference (CCNC), Las Vegas, NV, USA.","DOI":"10.1109\/CCNC46108.2020.9045113"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Albakri, A., Boiten, E., and Lemos, R.D. (2019). Sharing Cyber Threat Intelligence Under the General Data Protection Regulation. Privacy Technologies and Policy, Springer.","DOI":"10.1007\/978-3-030-21752-5_3"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wu, Y., Qiao, Y., Ye, Y., and Lee, B. (2019, January 22\u201325). Towards Improved Trust in Threat Intelligence Sharing using Blockchain and Trusted Computing. Proceedings of the 2019 Sixth International Conference on Internet of Things: Systems, Management and Security (IOTSMS), Granada, Spain.","DOI":"10.1109\/IOTSMS48152.2019.8939192"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Tlelo-Cuautle, E., D\u00edaz-Mu\u00f1oz, J.D., Gonz\u00e1lez-Zapata, A.M., Li, R., Le\u00f3n-Salas, W.D., Fern\u00e1ndez, F.V., Guill\u00e9n-Fern\u00e1ndez, O., and Cruz-Vega, I. (2020). Chaotic Image Encryption Using Hopfield and Hindmarsh\u2013Rose Neurons Implemented on FPGA. Sensors, 20.","DOI":"10.3390\/s20051326"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Khan, M., Masood, F., and Alghafis, A. (2019). Secure image encryption scheme based on fractals key with Fibonacci series and discrete dynamical system. Neural Comput. Appl.","DOI":"10.1007\/s00521-019-04667-y"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Burger, E.W., Goodman, M.D., Kampanakis, P., and Zhu, K.A. (2014, January 3\u20137). Taxonomy Model for Cyber Threat Intelligence Information Exchange Technologies. Proceedings of the 2014 ACM Workshop on Information Sharing & Collaborative Security\u2014WISCS-14, Scottsdale, AZ, USA.","DOI":"10.1145\/2663876.2663883"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Mavroeidis, V., and Bromander, S. (2017, January 11\u201313). Cyber Threat Intelligence Model: An Evaluation of Taxonomies, Sharing Standards, and Ontologies within Cyber Threat Intelligence. Proceedings of the 2017 European Intelligence and Security Informatics Conference (EISIC), Athens, Greece.","DOI":"10.1109\/EISIC.2017.20"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Asgarli, E., and Burger, E. (2016, January 10\u201311). Semantic ontologies for cyber threat sharing standards. Proceedings of the 2016 IEEE Symposium on Technologies for Homeland Security (HST), Waltham, MA, USA.","DOI":"10.1109\/THS.2016.7568896"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Steinberger, J., Sperotto, A., Golling, M., and Baier, H. (2015, January 11\u201315). How to exchange security events? Overview and evaluation of formats and protocols. Proceedings of the 2015 IFIP\/IEEE International Symposium on Integrated Network Management (IM), Ottawa, ON, Canada.","DOI":"10.1109\/INM.2015.7140300"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1016\/j.cose.2017.09.001","article-title":"A survey on technical threat intelligence in the age of sophisticated cyber attacks","volume":"72","author":"Tounsi","year":"2018","journal-title":"Comput. Secur."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/j.cose.2017.10.009","article-title":"A comparative analysis of incident reporting formats","volume":"73","author":"Menges","year":"2018","journal-title":"Comput. Secur."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1007\/s10586-017-0729-3","article-title":"Clust. Comput. Security analysis of a proposed internet of things middleware","volume":"20","author":"Ferreira","year":"2017","journal-title":"Clust. Comput."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1711","DOI":"10.1007\/s11277-015-3168-6","article-title":"Design and Evaluation of a Services Interface for the Internet of Things","volume":"91","author":"Ferreira","year":"2016","journal-title":"Wirel. Pers. Commun."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Sillaber, C., Sauerwein, C., Mussmann, A., and Breu, R. (2016, January 24\u201328). Data Quality Challenges and Future Research Directions in Threat Intelligence Sharing Practice. Proceedings of the 2016 ACM on Workshop on Information Sharing and Collaborative Security\u2014WISCS16, Vienna, Austria.","DOI":"10.1145\/2994539.2994546"},{"key":"ref_41","unstructured":"Barnum, S. (2020, March 17). Standardizing Cyber Threat Intelligence Information with the Structured Threat Information eXpression (STIX). Available online: https:\/\/www.mitre.org\/publications\/technical-papers\/standardizing-cyber-threat-intelligence-information-with-the."},{"key":"ref_42","unstructured":"Chismon, D., and Ruks, M. (2015). Threat Intelligence: Collecting, Analysing, Evaluating, MWR InfoSecurity Ltd."},{"key":"ref_43","unstructured":"Friedman, J., and Bouchard, M. (2015). Definitive Guide to Cyber Threat Intelligence: Using Knowledge about Adversaries to Win the War against Targeted Attacks, CyberEdge Group."},{"key":"ref_44","unstructured":"CERT-UK (2020, May 04). An Introduction to Threat Intelligence, Available online: http:\/\/dl.icdst.org\/pdfs\/files\/85d0b11467a3e30bf12a5bbc6c3e543c.pdf."},{"key":"ref_45","unstructured":"Shackleford, D. (2020, May 12). Cyber Threat Intelligence Uses, Successes and Failures: The Sans 2017 Cti Survey. Available online: https:\/\/www.sans.org\/reading-room\/whitepapers\/threats\/paper\/37677."},{"key":"ref_46","unstructured":"OASIS (2020, May 18). STIX Version 2.0. Available online: http:\/\/docs.oasis-open.org\/cti\/stix\/v2.0\/stix-v2.0-part1-stix-core.html."},{"key":"ref_47","unstructured":"OASIS (2020, May 18). STIX Version 2.0. Available online: http:\/\/docs.oasis-open.org\/cti\/stix\/v2.0\/stix-v2.0-part2-stix-objects.html."},{"key":"ref_48","unstructured":"Corporation, M. (2020, May 21). Cyber Observable eXpression (CybOX\u2122) Archive Website. Available online: https:\/\/cyboxproject.github.io\/."},{"key":"ref_49","unstructured":"OASIS (2020, May 18). STIX\u2122 Version 2.0. Available online: http:\/\/docs.oasis-open.org\/cti\/stix\/v2.0\/stix-v2.0-part3-cyber-observable-core.pdf."},{"key":"ref_50","unstructured":"OASIS (2020, May 21). TAXII Version 2.0. Available online: http:\/\/docs.oasis-open.org\/cti\/taxii\/v2.0\/taxii-v2.0.html."},{"key":"ref_51","unstructured":"Danyliw, R., Meijer, J., and Demchenko, Y. (2020, May 25). The Incident Object Description Exchange Format. Available online: https:\/\/tools.ietf.org\/html\/rfc5070."},{"key":"ref_52","unstructured":"Danyliw, R. (2020, May 25). The Incident Object Description Exchange Format Version 2. Available online: https:\/\/tools.ietf.org\/html\/rfc7970."},{"key":"ref_53","unstructured":"Moriarty, K. (2020, May 27). Real-Time Inter-Network Defense (RID). Available online: https:\/\/tools.ietf.org\/html\/rfc6545."},{"key":"ref_54","unstructured":"Inc., M (2020, May 27). An Introduction to Open IOC. Available online: https:\/\/www.academia.edu\/31820654\/An_Introduction_to_Open_IOC."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Wagner, T.D., Palomar, E., Mahbub, K., and Abdallah, A.E. (2017). Relevance Filtering for Shared Cyber Threat Intelligence (Short Paper). Information Security Practice and Experience, Springer.","DOI":"10.1007\/978-3-319-72359-4_35"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Liu, R., Zhao, Z., Sun, C., Yang, X., Gong, X., and Zhang, J. (2017). A Research and Analysis Method of Open Source Threat Intelligence Data. Communications in Computer and Information Science, Springer.","DOI":"10.1007\/978-981-10-6385-5_30"},{"key":"ref_57","unstructured":"ANSSI (2020, May 29). OpenCTI\u2014The Open Source Solution for Processing and Sharing Threat Intelligence Knowledge, Available online: https:\/\/www.ssi.gouv.fr\/en\/actualite\/opencti-the-open-source-solution-for-processing-and-sharing-threat-intelligence-knowledge\/."},{"key":"ref_58","unstructured":"Garnier, F. (2020, May 30). CTI & Information Fusion Benefits and Challenges. Available online: https:\/\/www.enisa.europa.eu\/events\/2019-cti-eu\/presentations\/200130-cti-info-fusion-tlp-white."},{"key":"ref_59","unstructured":"Project M. (2020, May 20). MISP\u2014Open Source Threat Intelligence Platform & Open Standards For Threat Information Sharing. Available online: https:\/\/www.misp-project.org\/features.html."},{"key":"ref_60","unstructured":"Corporation, M. (2020, May 26). Welcome to CRITs. Available online: https:\/\/github.com\/crits\/crits#readme."},{"key":"ref_61","unstructured":"Corporation, M. (2020, May 26). Collaborative Research into Threats. Available online: https:\/\/crits.github.io\/#nav."},{"key":"ref_62","unstructured":"OpenCTI (2020, May 29). OpenCTI Documentation 3.0.2. Available online: https:\/\/opencti-platform.github.io\/docs\/getting-started\/introduction."},{"key":"ref_63","unstructured":"GADGETS, C. (2020, May 20). The FASTEST Way to Consume Threat Intelligence. Available online: https:\/\/csirtgadgets.com\/collective-intelligence-framework."},{"key":"ref_64","unstructured":"Iovino, G. (2020, May 20). What Is the Collective Intelligence Framework?. Available online: https:\/\/github.com\/csirtgadgets\/massive-octo-spice\/wiki\/What-is-the-Collective-Intelligence-Framework%3F."},{"key":"ref_65","unstructured":"Anomali (2020, May 19). Anomali STAXX\u2014Installation and Administration Guide. Available online: https:\/\/update.anomali.com\/staxx\/docs\/Anomali_STAXX_Installation_&_Administration_Guide.pdf."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/12\/6\/108\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:41:57Z","timestamp":1760175717000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/12\/6\/108"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,23]]},"references-count":65,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["fi12060108"],"URL":"https:\/\/doi.org\/10.3390\/fi12060108","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,23]]}}}