{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T09:32:29Z","timestamp":1772789549170,"version":"3.50.1"},"reference-count":26,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2019,11,20]],"date-time":"2019-11-20T00:00:00Z","timestamp":1574208000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>One of the main challenges in modern Internet of Things (IoT) systems is the efficient collection, routing and management of data streams from heterogeneous sources, including sources with high ingestion rates. Despite the existence of various IoT data streaming frameworks, there is still no easy way for collecting and routing IoT streams in efficient and configurable ways that are easy to be implemented and deployed in realistic environments. In this paper, we introduce a programmable engine for Distributed Data Analytics (DDA), which eases the task of collecting IoT streams from different sources and accordingly, routing them to appropriate consumers. The engine provides also the means for preprocessing and analysis of data streams, which are two of the most important tasks in Big Data analytics applications. At the heart of the engine lies a Domain Specific Language (DSL) that enables the zero-programming definition of data routing and preprocessing tasks. This DSL is outlined in the paper, along with the middleware that supports its runtime execution. As part of the paper, we present the architecture of the engine, as well as the digital models that it uses for modelling data streams in the digital world. We also discuss the validation of the DDA in several data intensive IoT use cases in industrial environments, including use cases in pilot productions lines and in several real-life manufacturing environments. The latter manifest the configurability, programmability and flexibility of the DDA engine, as well as its ability to support practical applications.<\/jats:p>","DOI":"10.3390\/info10120360","type":"journal-article","created":{"date-parts":[[2019,11,20]],"date-time":"2019-11-20T11:06:03Z","timestamp":1574247963000},"page":"360","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Configurable Distributed Data Management for the Internet of the Things"],"prefix":"10.3390","volume":"10","author":[{"given":"Nikos","family":"Kefalakis","sequence":"first","affiliation":[{"name":"Athens Information Technology, 15125 Athens, Greece"}]},{"given":"Aikaterini","family":"Roukounaki","sequence":"additional","affiliation":[{"name":"Athens Information Technology, 15125 Athens, Greece"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6668-3911","authenticated-orcid":false,"given":"John","family":"Soldatos","sequence":"additional","affiliation":[{"name":"Athens Information Technology, 15125 Athens, Greece"}]}],"member":"1968","published-online":{"date-parts":[[2019,11,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1007\/s00778-003-0095-z","article-title":"Aurora: A new model and architecture for data stream management","volume":"12","author":"Abadi","year":"2003","journal-title":"VLDB J."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chandrasekaran, S., Cooper, O., Deshpande, A., Franklin, M., Hellerstein, J., Hong, W., Krishnamurthy, S., Reiss, F., and Shah, M. (2003, January 9\u201312). TelegraphCQ: Continuous dataflow processing. Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data (SIGMOD \u203203), New York, NY, USA.","DOI":"10.1145\/872853.872857"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1007\/s00778-004-0147-z","article-title":"The CQL continuous query language: Semantic foundations and query execution","volume":"15","author":"Arasu","year":"2006","journal-title":"VLDB J."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Ahmad, Y., Berg, B., Cetintemrel, U., Humphrey, M., Hwang, J., Jhingran, A., Maskey, A., Papaemmanouil, O., Rasin, A., and Tatbul, N. (2005, January 14\u201316). Distributed operation in the Borealis stream processing engine. Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD \u203205), New York, NY, USA.","DOI":"10.1145\/1066157.1066274"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Biem, A., Bouillet, E., Feng, H., Ranganathan, A., Riabov, A., Verscheure, O., Koutsopoulos, H., and Moran, C. (2010, January 6\u201310). IBM infosphere streams for scalable, real-time, intelligent transportation services. Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD \u203210), New York, NY, USA.","DOI":"10.1145\/1807167.1807291"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2351","DOI":"10.1109\/TPDS.2012.24","article-title":"StreamCloud: An Elastic and Scalable Data Streaming System","volume":"23","author":"Gulisano","year":"2012","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1145\/2934664","article-title":"Apache Spark: A Unified Engine for Big Data Processing","volume":"59","author":"Zaharia","year":"2016","journal-title":"Commun. ACM"},{"key":"ref_8","first-page":"28","article-title":"Apache Flink\u2122: Stream and Batch Processing in a Single Engine","volume":"38","author":"Carbone","year":"2015","journal-title":"IEEE Data Eng. Bull."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1634","DOI":"10.14778\/3137765.3137770","article-title":"Samza: Stateful scalable stream processing at Linked","volume":"10","author":"Noghabi","year":"2017","journal-title":"Proc. VLDB Endow."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Murray, D., McSherry, F., Isaacs, R., Isard, M., Barham, P., and Abadi, M. (2013, January 3\u20136). Naiad: A timely dataflow system. Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (SOSP \u203213), New York, NY, USA.","DOI":"10.1145\/2517349.2522738"},{"key":"ref_11","unstructured":"Kreps, J., Narkhede, N., and Rao, J. (2011, January 12\u201316). Kafka: A Distributed Messaging System for Log Processing. Proceedings of the 6th International Workshop on Networking Meets Databases (NetDB), Athens, Greece."},{"key":"ref_12","unstructured":"Isaja, M., Soldatos, J., and Gezer, V. (2017, January 12\u201316). Combining Edge Computing and Blockchains for Flexibility and Performance in Industrial Automation. Proceedings of the Eleventh International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies (UBICOMM), Barcelona, Spain."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.infsof.2015.11.001","article-title":"Domain-Specific Languages: A Systematic Mapping Study","volume":"71","author":"Kosar","year":"2016","journal-title":"Inf. Softw. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1145\/1118890.1118892","article-title":"When and how to develop domain-specific languages","volume":"37","author":"Mernik","year":"2005","journal-title":"ACM Comput. Surv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.pmcj.2008.08.004","article-title":"REFiLL: A lightweight programmable middleware platform for cost effective RFID application development","volume":"5","author":"Anagnostopoulos","year":"2009","journal-title":"Pervasive Mob. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1244","DOI":"10.1016\/j.jss.2011.02.036","article-title":"APDL: A reference XML schema for process-centered definition of RFID solutions","volume":"84","author":"Kefalakis","year":"2011","journal-title":"J. Syst. Softw."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"390","DOI":"10.1016\/j.infsof.2007.04.002","article-title":"A preliminary study on various implementation approaches of domain-specific language","volume":"50","author":"Kosar","year":"2008","journal-title":"Inf. Softw. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2206","DOI":"10.1007\/s10664-016-9483-z","article-title":"Effectiveness and efficiency of a domain-specific language for high-performance marine ecosystem simulation: A controlled experiment","volume":"22","author":"Johanson","year":"2017","journal-title":"Empir. Softw. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2734","DOI":"10.1007\/s10664-017-9593-2","article-title":"Program comprehension of domain-specific and general-purpose languages: Replication of a family of experiments using integrated development environments","volume":"23","author":"Kosar","year":"2018","journal-title":"Empir. Softw. Eng."},{"key":"ref_20","first-page":"139","article-title":"Model-driven engineering: A survey supported by a unified conceptual model","volume":"43","author":"Silva","year":"2015","journal-title":"Comput. Lang. Syst. Struct."},{"key":"ref_21","first-page":"1","article-title":"Applying Model-Driven Engineering to High-Performance Computing: Experience Report, Lessons Learned, and Remaining Challenges","volume":"55","author":"Lelandais","year":"2019","journal-title":"J. Comput. Lang."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Petrali, P., Isaja, M., and Soldatos, J. (2018). Edge Computing and Distributed Ledger Technologies for Flexible Production Lines: A White-Appliances Industry Case, In IFAC.","DOI":"10.1016\/j.ifacol.2018.08.324"},{"key":"ref_23","unstructured":"Soldatos, J., Lazaro, O., and Cavadini, F. (2019, November 18). The Digital Shopfloor: Industrial Automation in the Industry 4.0 Era Forthcoming Performance Analysis and Applications. River Publishers Series in Automation, Control and Robotics, ISBN 9788770220415, e-ISBN 9788770220408. Available online: https:\/\/www.riverpublishers.com\/book_details.php?book_id=676."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kefalakis, N., Roukounaki, A., and Soldatos, J. (2019, January 29\u201331). A Configurable Distributed Data Analytics Infrastructure for the Industrial Internet of things. Proceedings of the DCOSS, Santorini Island, Greece.","DOI":"10.1109\/DCOSS.2019.00048"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Mathew, J., Kennedy, J., Ma, L., Tan, A., and Anderson, D. (2006). A Review of the MIMOSA OSA-EAI Database for Condition Monitoring Systems. Engineering Asset Management, Springer.","DOI":"10.1007\/978-1-84628-814-2_88"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Soldatos, J. (2019). Introduction to Industry 4.0 and the Digital Shopfloor Vision. The Digital Shopfloor: Industrial Automation in the Industry 4.0 Era, River Publishers.","DOI":"10.13052\/rp-9788770220408"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/10\/12\/360\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:35:55Z","timestamp":1760189755000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/10\/12\/360"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,20]]},"references-count":26,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["info10120360"],"URL":"https:\/\/doi.org\/10.3390\/info10120360","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,20]]}}}