{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T15:15:01Z","timestamp":1777734901506,"version":"3.51.4"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T00:00:00Z","timestamp":1689206400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T00:00:00Z","timestamp":1689206400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["PCI2019-103762"],"award-info":[{"award-number":["PCI2019-103762"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Inf Syst Front"],"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>IoT platforms are in charge of extracting and processing the data that come from IoT networks, generating additional value, and providing access to the user through usable interfaces. However, the ever growing number of devices, networks, services and applications within the IoT ecosystem, and the recently adopted edge\/cloud architecture, increase the complexity. Therefore, IoT platforms should integrate monitoring and visualization tools to facilitate deployment, management and maintenance tasks. In this work, we present the implementation and performance evaluation of an IoT modular platform for distributed architectures that combines the use of Elastic Stack tools\u00a0(Elasticsearch, Kibana and Beats) and Apache Kafka. We have developed a monitoring framework based on Beats agents that supervise the platform performance attending to different metrics; and adapted the Kibana visualization tools to provide friendly and accessible information to platform administrators and users. Finally, we have deployed and evaluated the IoT platform in four real use cases, identifying the factors that affect the performance of the different modules: Edge Node, Data Streaming, Cloud Server and Search Engine.<\/jats:p>","DOI":"10.1007\/s10796-023-10409-2","type":"journal-article","created":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T13:01:51Z","timestamp":1689253311000},"page":"2373-2389","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Monitoring Framework for the Performance Evaluation of an IoT Platform with Elasticsearch and Apache Kafka"],"prefix":"10.1007","volume":"26","author":[{"given":"Gonzalo","family":"Calderon","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guillermo","family":"del\u00a0Campo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Edgar","family":"Saavedra","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Asunci\u00f3n","family":"Santamar\u00eda","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,7,13]]},"reference":[{"key":"10409_CR1","doi-asserted-by":"publisher","unstructured":"Adhikaree, A., Kim, T., Vagdoda, J., Ochoa, A., Hernandez, P.\u00a0J., & Lee, Y. (2017). Cloud-based battery condition monitoring platform for large-scale lithium-ion battery energy storage systems using internet-of-things (IoT). In IEEE Energy Conversion Congress and Exposition (ECCE) (pp. 1004\u20131009). https:\/\/doi.org\/10.1109\/ECCE.2017.8095896","DOI":"10.1109\/ECCE.2017.8095896"},{"key":"10409_CR2","doi-asserted-by":"crossref","unstructured":"Ahmed, F., Jahangir, U., Rahim, H., Ali, K., & et\u00a0al. (2020). Centralized log management using elasticsearch, logstash and kibana. In IEEE International Conference on Information Science and Communication Technology (ICISCT). IEEE, Karachi, Pakistan, (pp. 1\u20137)","DOI":"10.1109\/ICISCT49550.2020.9080053"},{"key":"10409_CR3","doi-asserted-by":"publisher","unstructured":"Anusas-Amornkul, T., & Sangrat, S. (2017). Linux server monitoring and self-healing system using nagios. In International Conference on Mobile Web and Information Systems (pp. 290\u2013302). https:\/\/doi.org\/10.1007\/978-3-319-65515-4_24","DOI":"10.1007\/978-3-319-65515-4_24"},{"key":"10409_CR4","doi-asserted-by":"crossref","unstructured":"Babun, L., Denney, K., Celik, Z. B., McDaniel, P., & Uluagac, A. S. (2021). A survey on IoT platforms: Communication, security, and privacy perspectives. Computer Networks, 192: 108,040","DOI":"10.1016\/j.comnet.2021.108040"},{"key":"10409_CR5","doi-asserted-by":"publisher","unstructured":"Bajer, M. (2017). Building an IoT Data Hub with Elasticsearch, Logstash and Kibana. In IEEE 5th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW) (pp. 63\u201368). https:\/\/doi.org\/10.1109\/FiCloudW.2017.101","DOI":"10.1109\/FiCloudW.2017.101"},{"key":"10409_CR6","unstructured":"Barth, W. (2008). Nagios: System and network monitoring. No Starch Press"},{"key":"10409_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.comcom.2014.09.008","volume":"54","author":"E Borgia","year":"2014","unstructured":"Borgia, E. (2014). The internet of things vision: Key features, applications and open issues. Computer Communications, 54, 1\u201331. https:\/\/doi.org\/10.1016\/j.comcom.2014.09.008","journal-title":"Computer Communications"},{"key":"10409_CR8","doi-asserted-by":"publisher","unstructured":"Calderon, G., Del\u00a0Campo, G., Saavedra, E., & Santamaria, A. (2021). Management and monitoring IoT networks through an elastic stack-based platform. In IEEE 2021 8th International Conference on Future Internet of Things and Cloud (FiCloud) (pp. 184\u2013191). https:\/\/doi.org\/10.1109\/FiCloud49777.2021.00034","DOI":"10.1109\/FiCloud49777.2021.00034"},{"key":"10409_CR9","doi-asserted-by":"crossref","unstructured":"Chakraborty, M., & Kundan, A.\u00a0P. (2021). Grafana. In Monitoring Cloud-Native Applications (pp. 187\u2013240). Springer","DOI":"10.1007\/978-1-4842-6888-9_6"},{"issue":"4","key":"10409_CR10","doi-asserted-by":"publisher","first-page":"1219","DOI":"10.1109\/TLA.2018.8362160","volume":"16","author":"A Cravero","year":"2018","unstructured":"Cravero, A. (2018). Big data architectures and the internet of things: A systematic mapping study. IEEE Latin America Transactions, 16(4), 1219\u20131226.","journal-title":"IEEE Latin America Transactions"},{"issue":"10","key":"10409_CR11","doi-asserted-by":"publisher","first-page":"4151","DOI":"10.1007\/s12652-017-0659-1","volume":"10","author":"A Darwish","year":"2019","unstructured":"Darwish, A., Hassanien, A. E., Elhoseny, M., Sangaiah, A. K., & Muhammad, K. (2019). The impact of the hybrid platform of internet of things and cloud computing on healthcare systems: opportunities, challenges, and open problems. Journal of Ambient Intelligence and Humanized Computing, 10(10), 4151\u20134166. https:\/\/doi.org\/10.1007\/s12652-017-0659-1","journal-title":"Journal of Ambient Intelligence and Humanized Computing"},{"key":"10409_CR12","doi-asserted-by":"publisher","unstructured":"Del\u00a0Campo, G., Montoya, E., Mart\u00edn, J., G\u00f3mez, I., & Santamar\u00eda, A. (2012). Batnet: a 6lowpan-based sensors and actuators network. In International Conference on Ubiquitous Computing and Ambient Intelligence (pp. 58\u201365). Springer. https:\/\/doi.org\/10.1007\/978-3-642-35377-2_8","DOI":"10.1007\/978-3-642-35377-2_8"},{"key":"10409_CR13","doi-asserted-by":"publisher","unstructured":"Dharur, S., & Swaminathan, K. (2018). Efficient Surveillance and Monitoring using the ELK Stack for IoT powered Smart Buildings. In IEEE 2nd International Conference on Inventive Systems and Control (ICISC) (pp. 700\u2013705). https:\/\/doi.org\/10.1109\/ICISC.2018.8398888","DOI":"10.1109\/ICISC.2018.8398888"},{"key":"10409_CR14","doi-asserted-by":"publisher","unstructured":"Espinoza, H., Kling, G., McGroarty, F., O\u2019Mahony, M., & Ziouvelou, X. (2020). Estimating the Impact of the Internet of Things on Productivity in Europe. Heliyon, 6(5). https:\/\/doi.org\/10.1016\/j.heliyon.2020.e03935","DOI":"10.1016\/j.heliyon.2020.e03935"},{"key":"10409_CR15","unstructured":"Gormley, C., & Tong, Z. (2015). Elasticsearch: the definitive guide: a distributed real-time search and analytics engine. O\u2019Reilly Media, Inc"},{"key":"10409_CR16","doi-asserted-by":"crossref","unstructured":"Gos, K., & Zabierowski, W. (2020). The comparison of microservice and monolithic architecture. In 2020 IEEE XVIth International Conference on the Perspective Technologies and Methods in MEMS Design (MEMSTECH). IEEE, Lviv, Ukraine, pp. 150\u2013153","DOI":"10.1109\/MEMSTECH49584.2020.9109514"},{"key":"10409_CR17","unstructured":"G\u00fclc\u00fc, C. (2003). The complete log4j manual. QOS.ch"},{"key":"10409_CR18","doi-asserted-by":"publisher","unstructured":"Guth, J., Breitenb\u00fccher, U., Falkenthal, M., Fremantle, P., Kopp, O., Leymann, F., & Reinfurt, L. (2018). A detailed analysis of IoT platform architectures: concepts, similarities, and differences. In Internet of Everything. IEEE, Paris, France, (pp. 81\u2013101). Springer. https:\/\/doi.org\/10.1007\/978-981-10-5861-5_4","DOI":"10.1007\/978-981-10-5861-5_4"},{"key":"10409_CR19","doi-asserted-by":"publisher","unstructured":"Guth, J., Breitenb\u00fccher, U., Falkenthal, M., Leymann, F., & Reinfurt, L. (2016). Comparison of IoT platform architectures: A field study based on a reference architecture. In IEEE 2016 Cloudification of the Internet of Things (CIoT) (pp. 1\u20136). https:\/\/doi.org\/10.1109\/CIOT.2016.7872918","DOI":"10.1109\/CIOT.2016.7872918"},{"key":"10409_CR20","unstructured":"Hunt, P., Konar, M., Junqueira, F.\u00a0P., & Reed, B. (2010). Zookeeper: Wait-free coordination for internet-scale systems. In USENIX annual technical conference (vol.\u00a08)"},{"key":"10409_CR21","doi-asserted-by":"publisher","unstructured":"Jaya, N.\u00a0I., & Hossain, M.\u00a0F. (2018). A prototype air flow control system for home automation using mqtt over websocket in aws IoT core. In International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC) (pp. 111\u20131116). https:\/\/doi.org\/10.1109\/CyberC.2018.00032","DOI":"10.1109\/CyberC.2018.00032"},{"key":"10409_CR22","doi-asserted-by":"publisher","unstructured":"Kang, D., Park, M., Kim, H., Kim, D., Kim, S., Son, H., & Lee, S. (2017). Room temperature control and fire alarm\/suppression IoT service using mqtt on aws. In International Conference on Platform Technology and Service (PlatCon) (pp. 1\u20135). https:\/\/doi.org\/10.1109\/PlatCon.2017.7883724","DOI":"10.1109\/PlatCon.2017.7883724"},{"key":"10409_CR23","doi-asserted-by":"publisher","unstructured":"Kazmi, A., Jan, Z., Zappa, A., & Serrano, M. (2017). Overcoming the Heterogeneity in the Internet of Things for Smart Cities. In International workshop on interoperability and open-source solutions (pp. 20\u201335). https:\/\/doi.org\/10.1007\/978-3-319-56877-5_2","DOI":"10.1007\/978-3-319-56877-5_2"},{"key":"10409_CR24","doi-asserted-by":"publisher","unstructured":"Kotenko, I., Kuleshov, A., & Ushakov, I. (2017). Aggregation of Elastic Stack Instruments for Collecting, Storing and Processing of Security Information and Events. In IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld\/SCALCOM\/UIC\/ATC\/CBDCom\/IOP\/SCI) (pp. 1\u20138). https:\/\/doi.org\/10.1109\/UIC-ATC.2017.8397627","DOI":"10.1109\/UIC-ATC.2017.8397627"},{"key":"10409_CR25","unstructured":"Kreps, J., Narkhede, N., Rao, J., & et\u00a0al. (2011). Kafka: A distributed messaging system for log processing. In: The 6th International Workshop on Networking Meets Databases. NetDB 2011, Athens, Greece, (vol. 11, pp. 1\u20137)"},{"key":"10409_CR26","unstructured":"Langels, H.-J. (2008). Knx ip using ip networks as knx medium. In Proceedings of the KNX Scientific Conference 2008. Siemens AG, Regensburg"},{"key":"10409_CR27","doi-asserted-by":"publisher","unstructured":"Lee, H., Mun, H., & Lee, Y. (2020). Comparing response time of home IoT devices with or without cloud. In 2020 IEEE International Conference on Consumer Electronics (ICCE) (pp. 1\u20136). https:\/\/doi.org\/10.1109\/ICCE46568.2020.9043102","DOI":"10.1109\/ICCE46568.2020.9043102"},{"issue":"10","key":"10409_CR28","doi-asserted-by":"publisher","first-page":"10695","DOI":"10.1109\/JIOT.2020.3012763","volume":"7","author":"MA L\u00f3pez-Pe\u00f1a","year":"2020","unstructured":"L\u00f3pez-Pe\u00f1a, M. A., D\u00edaz, J., P\u00e9rez, J. E., & Humanes, H. (2020). Devops for IoT systems: Fast and continuous monitoring feedback of system availability. IEEE Internet of Things Journal, 7(10), 10695\u201310707. https:\/\/doi.org\/10.1109\/JIOT.2020.3012763","journal-title":"IEEE Internet of Things Journal"},{"key":"10409_CR29","doi-asserted-by":"publisher","unstructured":"Nasajpour, M., Pouriyeh, S., Parizi, R.\u00a0M., Dorodchi, M., Valero, M., & Arabnia, H.\u00a0R. (2020). Internet of Things for Current COVID-19 and Future Pandemics: An Exploratory Study. Journal of healthcare informatics research (pp. 1\u201340). https:\/\/doi.org\/10.1016\/j.dsx.2020.04.041","DOI":"10.1016\/j.dsx.2020.04.041"},{"key":"10409_CR30","unstructured":"Pasha, S. (2016). Thingspeak based sensing and monitoring system for IoT with matlab analysis. International Journal of New Technology and Research, 2(6): 19\u201323"},{"key":"10409_CR31","doi-asserted-by":"crossref","unstructured":"Patti, E. & Acquaviva, A. (2016). IoT platform for smart cities: Requirements and implementation case studies. In IEEE 2nd International Forum on Research and Technologies for Society and Industry Leveraging a better tomorrow (RTSI). IEEE, Bologna, Italy, (pp. 1\u20136)","DOI":"10.1109\/RTSI.2016.7740618"},{"key":"10409_CR32","doi-asserted-by":"crossref","unstructured":"Peddoju, S.\u00a0K. & Upadhyay, H. (2020). Evaluation of IoT data visualization tools and techniques. Data visualization: Trends and challenges toward multidisciplinary perception. Springer, Singapore, (pp. 115\u2013139)","DOI":"10.1007\/978-981-15-2282-6_7"},{"key":"10409_CR33","doi-asserted-by":"publisher","unstructured":"Pelle, I., Czentye, J., D\u00f3ka, J., & Sonkoly, B. (2019). Towards Latency Sensitive Cloud Native Applications: A Performance Study on AWS. In IEEE 12th International Conference on Cloud Computing (CLOUD) (pp. 272\u2013280). https:\/\/doi.org\/10.1109\/CLOUD.2019.00054","DOI":"10.1109\/CLOUD.2019.00054"},{"key":"10409_CR34","doi-asserted-by":"crossref","unstructured":"Renita, J. & Elizabeth, N.\u00a0E. (2017). Network\u2019s server monitoring and analysis using nagios. In 2017 IEEE International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET). IEEE, Chennai, India, (pp. 1904\u20131909)","DOI":"10.1109\/WiSPNET.2017.8300092"},{"key":"10409_CR35","unstructured":"Sahay, M. R., Sukumaran, M. K., Amarnath, S., & Palani, T. N. D. (2019). Environmental monitoring system using IoT and cloud service at real-time. EasyChair Preprint, 968, pp. 1\u20138"},{"key":"10409_CR36","doi-asserted-by":"crossref","unstructured":"Shelby, Z., Hartke, K., & Bormann, C. (2014). The constrained application protocol (coap). Internet Engineering Task Force (IETF), https:\/\/datatracker.ietf.org\/doc\/html\/rfc7252","DOI":"10.17487\/rfc7252"},{"issue":"4","key":"10409_CR37","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1016\/j.dsx.2020.04.041","volume":"14","author":"RP Singh","year":"2020","unstructured":"Singh, R. P., Javaid, M., Haleem, A., & Suman, R. (2020). Internet of Things (IoT) Applications to Fight Against COVID-19 Pandemic. Diabetes & Metabolic Syndrome: Clinical Research & Reviews, 14(4), 521\u2013524. https:\/\/doi.org\/10.1016\/j.dsx.2020.04.041","journal-title":"Diabetes & Metabolic Syndrome: Clinical Research & Reviews"},{"key":"10409_CR38","unstructured":"Tader, P. (2010). Server monitoring with zabbix. Linux Journal, 2010(195), 7."},{"key":"10409_CR39","doi-asserted-by":"publisher","unstructured":"Tala\u015f, A., Pop, F., & Neagu, G. (2017). Elastic Stack in Action for Smart Cities: Making Sense of Big Data. In 13th IEEE International Conference on Intelligent Computer Communication and Processing (ICCP) (pp. 469\u2013476). https:\/\/doi.org\/10.1109\/ICCP.2017.8117049","DOI":"10.1109\/ICCP.2017.8117049"},{"key":"10409_CR40","doi-asserted-by":"crossref","unstructured":"Vaarandi, R. & Pihelgas, M. (2014). Using security logs for collecting and reporting technical security metrics. In IEEE Military communications conference. IEEE, Baltimore, MD, USA, (pp. 294\u2013299)","DOI":"10.1109\/MILCOM.2014.53"},{"key":"10409_CR41","doi-asserted-by":"publisher","unstructured":"Venkatramulu, S., Phridviraj, M., Srinivas, C., & Rao, V. C.\u00a0S. (2021). Implementation of grafana as open source visualization and query processing platform for data scientists and researchers. Materials Today: Proceedings. https:\/\/doi.org\/10.1016\/j.matpr.2021.03.364","DOI":"10.1016\/j.matpr.2021.03.364"},{"key":"10409_CR42","unstructured":"Z\u00fa\u00f1iga-Prieto, M., Rodr\u00edguez, D., Rodr\u00edguez, J., Solano, L., Insfran, E., & Abrah\u00e3o, S. (2018). IoT-adl: An adl for describing cloud IoT applications. Journal of Computers, 29(6), 264\u2013273"}],"container-title":["Information Systems Frontiers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10796-023-10409-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10796-023-10409-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10796-023-10409-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,26]],"date-time":"2025-02-26T02:04:06Z","timestamp":1740535446000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10796-023-10409-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,13]]},"references-count":42,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["10409"],"URL":"https:\/\/doi.org\/10.1007\/s10796-023-10409-2","relation":{},"ISSN":["1387-3326","1572-9419"],"issn-type":[{"value":"1387-3326","type":"print"},{"value":"1572-9419","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,13]]},"assertion":[{"value":"18 May 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 July 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}]}}