{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T18:11:08Z","timestamp":1771265468626,"version":"3.50.1"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T00:00:00Z","timestamp":1758844800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T00:00:00Z","timestamp":1758844800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>This article introduces ThingsBoard Message Queue (TBMQ), a scalable and fault-tolerant Message Queuing Telemetry Transport (MQTT) broker designed to address the challenges of data handling in large-scale Internet of Things (IoT) ecosystems. TBMQ\u2019s distributed architecture enables horizontal scalability, fault tolerance, and reliable data delivery, ensuring high availability and data durability under heavy load conditions. The broker efficiently handles two prevalent communication patterns in big data IoT applications: the fan-in pattern, where a large number of devices generate high volumes of messages for processing, and the fan-out pattern, where a few requests trigger significant outgoing data to multiple devices. Leveraging Apache Kafka (Kafka), TBMQ supports over 100 million concurrent client connections and a throughput exceeding 3 million messages per second. This paper explores the architectural decisions, technical innovations, and strategic use of Kafka, providing insights into building reliable and scalable messaging systems suitable for the demanding requirements of big data in IoT.<\/jats:p>","DOI":"10.1186\/s40537-025-01271-x","type":"journal-article","created":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T10:32:10Z","timestamp":1758882730000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["A distributed architecture for MQTT messaging: the case of TBMQ"],"prefix":"10.1186","volume":"12","author":[{"given":"Andrii","family":"Shvaika","sequence":"first","affiliation":[]},{"given":"Dmytro","family":"Shvaika","sequence":"additional","affiliation":[]},{"given":"Dmytro","family":"Landiak","sequence":"additional","affiliation":[]},{"given":"Volodymyr","family":"Artemchuk","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,9,26]]},"reference":[{"key":"1271_CR1","doi-asserted-by":"publisher","first-page":"57","DOI":"10.3934\/ElectrEng.2020.1.57","volume":"4","author":"LO Aghenta","year":"2020","unstructured":"Aghenta LO, Iqbal MT. Design and implementation of a low-cost, open source IoT-based SCADA system using ESP32 with OLED, ThingsBoard and MQTT protocol. AIMS Electr Eng. 2020;4:57\u201386.","journal-title":"AIMS Electr Eng"},{"key":"1271_CR2","doi-asserted-by":"publisher","unstructured":"Casillo M, Colace F, De Santo M, Lorusso A, Mosca R, Santaniello D. $$\\text{VIOT}_{{\\text{ Lab }}}$$: A Virtual Remote Laboratory for Internet of Things Based on ThingsBoard Platform. In: Proceedings of the 2021 IEEE Frontiers in Education Conference (FIE). IEEE. Lincoln. 2021;pp 1-6. https:\/\/doi.org\/10.1109\/FIE49875.2021.9637317.","DOI":"10.1109\/FIE49875.2021.9637317"},{"key":"1271_CR3","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1109\/SWC50871.2021.00064","volume":"2021","author":"E Okhovat","year":"2021","unstructured":"Okhovat E, Bauer M. Monitoring the Smart City Sensor Data Using ThingsBoard and Node-Red. In: 2021 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Internet of People and Smart City Innovation (SmartWorld\/SCALCOM\/UIC\/ATC\/IOP\/SCI). IEEE Atlanta. 2021;2021:425\u201332. https:\/\/doi.org\/10.1109\/SWC50871.2021.00064.","journal-title":"IEEE Atlanta"},{"issue":"06","key":"1271_CR4","doi-asserted-by":"publisher","first-page":"4","DOI":"10.3991\/ijim.v17i06.34129","volume":"17","author":"DN Bestari","year":"2023","unstructured":"Bestari DN, Wibowo A. An IoT-based real-time weather monitoring system using telegram bot and ThingsBoard platform. Int J Interact Mob Technol. 2023;17(06):4\u201319. https:\/\/doi.org\/10.3991\/ijim.v17i06.34129.","journal-title":"Int J Interact Mob Technol"},{"issue":"5","key":"1271_CR5","doi-asserted-by":"publisher","first-page":"574","DOI":"10.3844\/jcssp.2024.574.584","volume":"20","author":"P Di Felice","year":"2024","unstructured":"Di Felice P, Paolone G. Papers Mentioning ThingsBoard: A Systematic Mapping Study. J Comput Sci. 2024;20(5):574\u201384. https:\/\/doi.org\/10.3844\/jcssp.2024.574.584.","journal-title":"J Comput Sci"},{"key":"1271_CR6","doi-asserted-by":"publisher","unstructured":"Zhang Y, Li H, Wu F, Wang X. Survey and Analysis of Industrial MQTT Broker Designs: Performance, Persistence, and Deployment Considerations, IEEE Transactions on Industrial Informatics, early access, 2025. https:\/\/doi.org\/10.1109\/TII.2025.3359142.","DOI":"10.1109\/TII.2025.3359142"},{"key":"1271_CR7","unstructured":"MQTT: The Standard for IoT Messaging (2022) https:\/\/mqtt.org\/. Accessed 16 Oct 2024."},{"issue":"3","key":"1271_CR8","doi-asserted-by":"publisher","DOI":"10.1002\/nem.2115","volume":"31","author":"E Bertrand-Martinez","year":"2021","unstructured":"Bertrand-Martinez E, Feio PD, de Brito NV, Kon F, Abel\u00e9m A. Classification and evaluation of IoT brokers: A methodology. Int J Netw Manag. 2021;31(3): e2115. https:\/\/doi.org\/10.1002\/nem.2115.","journal-title":"Int J Netw Manag"},{"key":"1271_CR9","doi-asserted-by":"publisher","first-page":"201071","DOI":"10.1109\/ACCESS.2020.3035849","volume":"8","author":"B Mishra","year":"2020","unstructured":"Mishra B, Kertesz A. The use of MQTT in M2M and IoT systems: A survey. IEEE Access. 2020;8:201071\u201386. https:\/\/doi.org\/10.1109\/ACCESS.2020.3035849.","journal-title":"IEEE Access"},{"key":"1271_CR10","unstructured":"An open source MQTT broker. https:\/\/mosquitto.org\/. Accessed 17 Oct 2024."},{"key":"1271_CR11","unstructured":"HiveMQ MQTT Broker. https:\/\/www.hivemq.com\/. Accessed 17 Oct 2024."},{"key":"1271_CR12","unstructured":"Highly reliable MQTT data persistence based on RocksDB. https:\/\/www.emqx.com\/en\/blog\/mqtt-persistence-based-on-rocksdb. Accessed 17 Oct 2024."},{"key":"1271_CR13","unstructured":"The most scalable MQTT broker for IoT. https:\/\/www.emqx.io\/. Accessed 17 Oct 2024."},{"key":"1271_CR14","unstructured":"Clustering MQTT for high availability and scalability. https:\/\/vernemq.com\/. Accessed 17 Oct 2024."},{"key":"1271_CR15","doi-asserted-by":"publisher","unstructured":"Bender M, Kirdan E, Pahl M-O, Carle G. Open-source MQTT evaluation. IEEE 18th Annual Consumer Communications & Networking Conference (CCNC). IEEE, Las Vegas. 2021;pp 1-4. https:\/\/doi.org\/10.1109\/CCNC49032.2021.9369499.","DOI":"10.1109\/CCNC49032.2021.9369499"},{"key":"1271_CR16","doi-asserted-by":"publisher","first-page":"13704","DOI":"10.1109\/ACCESS.2024.3355670","volume":"12","author":"AM Dey","year":"2024","unstructured":"Dey AM, Rana BT, Bhattacharjee A. An Overview of MQTT Broker Architectures for Large-Scale IoT Deployments. IEEE Access. 2024;12:13704\u201326. https:\/\/doi.org\/10.1109\/ACCESS.2024.3355670.","journal-title":"IEEE Access"},{"key":"1271_CR17","unstructured":"TMBQ architecture. https:\/\/thingsboard.io\/docs\/mqtt-broker\/architecture\/. Accessed 17 Oct 2024."},{"key":"1271_CR18","unstructured":"Protocol Buffers. https:\/\/developers.google.com\/. Accessed 17 Oct 2024."},{"key":"1271_CR19","doi-asserted-by":"publisher","unstructured":"Popi\u0107 S, Pezer D, Mrazovac B, Tesli\u0107 N. Performance evaluation of using Protocol Buffers in the Internet of Things communication. 2016 International Conference on Smart Systems and Technologies (SST). IEEE, Osijek, Croatia. 2016;pp 261-265. https:\/\/doi.org\/10.1109\/SST.2016.7765670.","DOI":"10.1109\/SST.2016.7765670"},{"key":"1271_CR20","unstructured":"Shvaika D, Shvaika A, Artemchuk V. Data serialization protocols in IoT: Problems and solutions using the ThingsBoard platform as an example. Proceedings of the 4th Edge Computing Workshop (doors 2024). Zhytomyr, Ukraine(3666). 2024;pp 70-75. https:\/\/ceur-ws.org\/Vol-3666\/paper11.pdf."},{"key":"1271_CR21","doi-asserted-by":"publisher","DOI":"10.55056\/jec.745","author":"DI Shvaika","year":"2024","unstructured":"Shvaika DI, Shvaika AI, Artemchuk VO. Advancing IoT interoperability: dynamic data serialization using ThingsBoard. J Edge Comput. 2024. https:\/\/doi.org\/10.55056\/jec.745.","journal-title":"J Edge Comput"},{"issue":"3","key":"1271_CR22","doi-asserted-by":"publisher","first-page":"34","DOI":"10.3390\/iot6030034","volume":"6","author":"DI Shvaika","year":"2025","unstructured":"Shvaika DI, Shvaika AI, Artemchuk VO. MQTT Broker Architectural Enhancements for High-Performance P2P Messaging: TBMQ Scalability and Reliability in Distributed IoT Systems. IoT. 2025;6(3):34. https:\/\/doi.org\/10.3390\/iot6030034.","journal-title":"IoT"},{"key":"1271_CR23","unstructured":"Build, Operate, and Secure Distributed Applications https:\/\/akka.io\/. Accessed 17 Oct 2024"},{"key":"1271_CR24","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/1086.001.0001","volume-title":"Actors: a model of concurrent computation in distributed systems","author":"G Agha","year":"1986","unstructured":"Agha G. Actors: a model of concurrent computation in distributed systems. Cambridge: MIT Press; 1986."},{"key":"1271_CR25","doi-asserted-by":"publisher","unstructured":"Camilleri, C., Vella, J.G., and Nezval, V. (2023). Actor Model Frameworks: An Empirical Performance Analysis. In: Garg, L., et al. (eds) Key Digital Trends Shaping the Future of Information and Management Science. ISMS 2022. Lecture Notes in Networks and Systems, vol 671. Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-031-31153-6_37","DOI":"10.1007\/978-3-031-31153-6_37"},{"key":"1271_CR26","unstructured":"Poggi A. Developing scalable applications with actors. WSEAS Trans Comput. 2014;13:660\u20139. https:\/\/wseas.com\/journals\/computers\/2014\/a425705-661.pdf"},{"key":"1271_CR27","unstructured":"The Netty project. https:\/\/netty.io\/. Accessed 17 Oct 2024."},{"key":"1271_CR28","unstructured":"Maurer N, Wolfthal M. Netty in action. Manning; 2015."},{"key":"1271_CR29","doi-asserted-by":"publisher","unstructured":"Wang Y, Huang L, Liu X, Sun T, Lei K. Performance comparison and evaluation of WebSocket frameworks: Netty, Undertow, Vert.x, Grizzly and Jetty. 2018 1st IEEE International Conference on Hot Information-Centric Networking (HotICN). IEEE, Shenzhen. 2018;pp 13-17. https:\/\/doi.org\/10.1109\/HOTICN.2018.8605989.","DOI":"10.1109\/HOTICN.2018.8605989"},{"key":"1271_CR30","unstructured":"Apache Kafka. https:\/\/kafka.apache.org\/. Accessed 17 Oct 2024."},{"key":"1271_CR31","doi-asserted-by":"publisher","unstructured":"Vyas S, Tyagi RK, Jain C, Sahu S.Performance evaluation of Apache Kafka \u2013 A modern platform for real time data streaming. 2022 2nd International Conference on Innovative Practices in Technology and Management (ICIPTM). IEEE, Gautam Buddha Nagar. 2022;pp 465-470. https:\/\/doi.org\/10.1109\/ICIPTM54933.2022.9754154.","DOI":"10.1109\/ICIPTM54933.2022.9754154"},{"key":"1271_CR32","doi-asserted-by":"publisher","first-page":"96131","DOI":"10.1109\/ACCESS.2023.3305586","volume":"11","author":"S Park","year":"2023","unstructured":"Park S, Huh J-H. A study on big data collecting and utilizing smart factory based grid networking big data using Apache Kafka. IEEE Access. 2023;11:96131\u201342. https:\/\/doi.org\/10.1109\/ACCESS.2023.3305586.","journal-title":"IEEE Access"},{"issue":"47","key":"1271_CR33","first-page":"9478","volume":"3","author":"KMM Thein","year":"2014","unstructured":"Thein KMM. Apache Kafka: Next generation distributed messaging system. Int J Sci Eng Technol Res. 2014;3(47):9478\u201383.","journal-title":"Int J Sci Eng Technol Res"},{"key":"1271_CR34","unstructured":"PostgreSQL: The world\u2019s most advanced open source relational database. https:\/\/www.postgresql.org\/. Accessed 17 Oct 2024."},{"key":"1271_CR35","unstructured":"Redis. Available online: https:\/\/redis.io\/ Accessed on 21 Jul 2025."},{"key":"1271_CR36","doi-asserted-by":"publisher","unstructured":"Gruener S, Koziolek H, R\u00fcckert J. Towards resilient IoT messaging: An experience report analyzing MQTT brokers. 2021 IEEE 18th International Conference on Software Architecture (ICSA). IEEE, Stuttgar. 2021;pp 69-79. https:\/\/doi.org\/10.1109\/ICSA51549.2021.00015.","DOI":"10.1109\/ICSA51549.2021.00015"},{"key":"1271_CR37","unstructured":"AWS. https:\/\/aws.amazon.com\/. Accessed 17 Oct 2024."},{"key":"1271_CR38","unstructured":"Wittig A, Wittig M. Amazon Web Services in action, third edition: An in-depth guide to AWS. Manning; 2021. pp 231-252."},{"key":"1271_CR39","unstructured":"Kubernetes documentation. https:\/\/kubernetes.io\/. Accessed 17 Oct 2024."},{"key":"1271_CR40","unstructured":"Orr M, Cao Y.Mastering Elastic Kubernetes Service on AWS: Deploy and manage EKS clusters to support cloud-native applications in AWS. Packt Publishing; 2023."},{"key":"1271_CR41","unstructured":"Open MQTT Benchmarking Comparison: MQTT Brokers in 2023. https:\/\/www.emqx.com\/en\/blog\/open-mqtt-benchmarking-comparison-mqtt-brokers-in-2023. Accessed 22 June 2025."},{"key":"1271_CR42","unstructured":"Reaching 100M MQTT connections with EMQX 5.0. https:\/\/www.emqx.com\/en\/blog\/reaching-100m-mqtt-connections-with-emqx-5-0. Accessed 22 June 2025."},{"issue":"2","key":"1271_CR43","doi-asserted-by":"publisher","DOI":"10.1002\/spy2.318","volume":"7","author":"D Chatzoglou","year":"2024","unstructured":"Chatzoglou D, Tsoutsos M, Panagopoulos A. Performance Benchmarking and Privacy Evaluation of Open-Source MQTT Brokers. Secur Priv. 2024;7(2): e318. https:\/\/doi.org\/10.1002\/spy2.318.","journal-title":"Secur Priv"},{"key":"1271_CR44","unstructured":"NATS. Available online: https:\/\/nats.io\/. Accessed 23 July 2025."}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-025-01271-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40537-025-01271-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-025-01271-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T10:32:14Z","timestamp":1758882734000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-025-01271-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,26]]},"references-count":44,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["1271"],"URL":"https:\/\/doi.org\/10.1186\/s40537-025-01271-x","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,26]]},"assertion":[{"value":"23 October 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 August 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 September 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Financial interests: Authors A.S owns stock in ThingsBoard Inc., Authors D.S. and D.L. has received research support from ThingsBoard, Inc. Author V.A has no financial interest. Non-financial interests: none.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"224"}}