{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,8]],"date-time":"2025-11-08T22:57:25Z","timestamp":1762642645183,"version":"build-2065373602"},"reference-count":49,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2021,12,17]],"date-time":"2021-12-17T00:00:00Z","timestamp":1639699200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000780","name":"European Commission","doi-asserted-by":"publisher","award":["HERIT DATA"],"award-info":[{"award-number":["HERIT DATA"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The Internet of things has produced several heterogeneous devices and data models for sensors\/actuators, physical and virtual. Corresponding data must be aggregated and their models have to be put in relationships with the general knowledge to make them immediately usable by visual analytics tools, APIs, and other devices. In this paper, models and tools for data ingestion and regularization are presented to simplify and enable the automated visual representation of corresponding data. The addressed problems are related to the (i) regularization of the high heterogeneity of data that are available in the IoT devices (physical or virtual) and KPIs (key performance indicators), thus allowing such data in elements of hypercubes to be reported, and (ii) the possibility of providing final users with an index on views and data structures that can be directly exploited by graphical widgets of visual analytics tools, according to different operators. The solution analyzes the loaded data to extract and generate the IoT device model, as well as to create the instances of the device and generate eventual time series. The whole process allows data for visual analytics and dashboarding to be prepared in a few clicks. The proposed IoT device model is compliant with FIWARE NGSI and is supported by a formal definition of data characterization in terms of value type, value unit, and data type. The resulting data model has been enforced into the Snap4City dashboard wizard and tool, which is a GDPR-compliant multitenant architecture. The solution has been developed and validated by considering six different pilots in Europe for collecting big data to monitor and reason people flows and tourism with the aim of improving quality of service; it has been developed in the context of the HERIT-DATA Interreg project and on top of Snap4City infrastructure and tools. The model turned out to be capable of meeting all the requirements of HERIT-DATA, while some of the visual representation tools still need to be updated and furtherly developed to add a few features.<\/jats:p>","DOI":"10.3390\/s21248429","type":"journal-article","created":{"date-parts":[[2021,12,20]],"date-time":"2021-12-20T02:40:32Z","timestamp":1639968032000},"page":"8429","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Automating IoT Data Ingestion Enabling Visual Representation"],"prefix":"10.3390","volume":"21","author":[{"given":"Ala","family":"Arman","sequence":"first","affiliation":[{"name":"Distributed Systems and Internet Technology Lab DISIT, University of Florence, 50139 Firenze, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8167-1003","authenticated-orcid":false,"given":"Pierfrancesco","family":"Bellini","sequence":"additional","affiliation":[{"name":"Distributed Systems and Internet Technology Lab DISIT, University of Florence, 50139 Firenze, Italy"}]},{"given":"Daniele","family":"Bologna","sequence":"additional","affiliation":[{"name":"Distributed Systems and Internet Technology Lab DISIT, University of Florence, 50139 Firenze, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1044-3107","authenticated-orcid":false,"given":"Paolo","family":"Nesi","sequence":"additional","affiliation":[{"name":"Distributed Systems and Internet Technology Lab DISIT, University of Florence, 50139 Firenze, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9235-437X","authenticated-orcid":false,"given":"Gianni","family":"Pantaleo","sequence":"additional","affiliation":[{"name":"Distributed Systems and Internet Technology Lab DISIT, University of Florence, 50139 Firenze, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0643-171X","authenticated-orcid":false,"given":"Michela","family":"Paolucci","sequence":"additional","affiliation":[{"name":"Distributed Systems and Internet Technology Lab DISIT, University of Florence, 50139 Firenze, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Bellini, P., Bologna, D., Han, Q., Nesi, P., Pantaleo, G., and Paolucci, M. (2020, January 14\u201317). Data Ingestion and Inspection for Smart City Applications. Proceedings of the 2020 IEEE International Conference on Smart Computing (SMARTCOMP), Bologna, Italy.","DOI":"10.1109\/SMARTCOMP50058.2020.00052"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1145\/2858789","article-title":"Smart Cities: Concepts, Architectures, Research Opportunities","volume":"59","author":"Khatoun","year":"2016","journal-title":"Commun. ACM"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Badii, C., Bilotta, S., Cenni, D., Difino, A., Nesi, P., Paoli, I., and Paolucci, M. (2020). High Density Real-Time Air Quality Derived Services from IoT Networks. Sensors, 20.","DOI":"10.3390\/s20185435"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Ranjan, R., Mitra, K., Prakash Jayaraman, P., Wang, L., and Zomaya, A.Y. (2020). Internet of Things (IoT) and Cloud Computing Enabled Disaster Management. Handbook of Integration of Cloud Computing, Cyber Physical Systems and Internet of Things, Springer International Publishing.","DOI":"10.1007\/978-3-030-43795-4"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Samant, S.S., Chhetri, M.B., Vo, Q.B., Kowalczyk, R., and Nepal, S. (2017, January 15\u201317). Towards Quality-Assured Data Delivery in Cloud-Based IoT Platforms for Smart Cities. Proceedings of the 2017 IEEE 3rd International Conference on Collaboration and Internet Computing (CIC), San Jose, CA, USA.","DOI":"10.1109\/CIC.2017.00046"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Singh, P.K., Singh, Y., Kolekar, M.H., Kar, A.K., Chhabra, J.K., and Sen, A. (2021). Data Ingestion and Analysis Framework for Geoscience Data. Proceedings of the International Conference on Recent Innovations in Computing, Jammu, India, 8\u20139 May 2021, Springer.","DOI":"10.1007\/978-981-15-8297-4"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/j.future.2019.02.011","article-title":"BIGSEA: A Big Data Analytics Platform for Public Transportation Information","volume":"96","author":"Alic","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Nanthaamornphong, A., Holmes, J., and Asawateera, P. (2020, January 24\u201327). A Case Study: Phuket City Data Platform. Proceedings of the 2020 17th International Conference on Electrical Engineering\/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), Phuket, Thailand.","DOI":"10.1109\/ECTI-CON49241.2020.9158101"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"54","DOI":"10.26555\/jiteki.v16i1.17105","article-title":"Analyzing challenging aspects of IPv6 over IPv4","volume":"6","author":"Ashraf","year":"2020","journal-title":"J. Ilm. Tek. Elektro Komput. Dan Inform."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Farmanbar, M., and Rong, C. (2020). Triangulum City Dashboard: An Interactive Data Analytic Platform for Visualizing Smart City Performance. Processes, 8.","DOI":"10.3390\/pr8020250"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Komamizu, T., Amagasa, T., Shaikh, S.A., Shiokawa, H., and Kitagawa, H. (2016, January 12\u201314). Towards Real-Time Analysis of Smart City Data: A Case Study on City Facility Utilizations. Proceedings of the 2016 IEEE 18th International Conference on High Performance Computing and Communications; IEEE 14th International Conference on Smart City; IEEE 2nd International Conference on Data Science and Systems (HPCC\/SmartCity\/DSS), Sydney, NSW, Australia.","DOI":"10.1109\/HPCC-SmartCity-DSS.2016.0192"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Beheshti, A., Hashmi, M., Dong, H., and Zhang, W.E. (2018). Scalable Architecture for Personalized Healthcare Service Recommendation Using Big Data Lake. Service Research and Innovation, Springer International Publishing.","DOI":"10.1007\/978-3-319-76587-7"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.future.2017.05.001","article-title":"Analysis and Assessment of a Knowledge Based Smart City Architecture Providing Service APIs","volume":"75","author":"Badii","year":"2017","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_14","unstructured":"(2021, November 25). HERIT-DATA Interreg Project. Available online: https:\/\/herit-data.interreg-med.eu\/."},{"key":"ref_15","unstructured":"(2021, November 25). Snap4City: Smart aNalytic APp Builder for Sentient Cities and IOT. Available online: https:\/\/www.snap4city.org."},{"key":"ref_16","unstructured":"(2021, October 09). General Data Protection Regulation. Available online: https:\/\/eur-lex.europa.eu\/eli\/reg\/2016\/679\/oj."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"5267","DOI":"10.1109\/JIOT.2020.2978770","article-title":"Smart City IoT Services Creation Through Large-Scale Collaboration","volume":"7","author":"Cirillo","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1109\/MC.2018.2880019","article-title":"CPaaS.Io: An EU-Japan Collaboration on Open Smart-City Platforms","volume":"51","author":"Koshizuka","year":"2018","journal-title":"Computer"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1443","DOI":"10.1109\/TII.2014.2306384","article-title":"An IoT-Oriented Data Storage Framework in Cloud Computing Platform","volume":"10","author":"Jiang","year":"2014","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Harris, A., Stovall, J., and Sartipi, M. (2019, January 9\u201312). MLK Smart Corridor: An Urban Testbed for Smart City Applications. Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA.","DOI":"10.1109\/BigData47090.2019.9006382"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1109\/JIOT.2017.2722378","article-title":"An Ingestion and Analytics Architecture for IoT Applied to Smart City Use Cases","volume":"5","author":"Akbar","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1186\/s13174-015-0039-z","article-title":"CitySDK Tourism API\u2014Building Value around Open Data","volume":"6","author":"Pereira","year":"2015","journal-title":"J. Internet Serv. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hefnawy, A., Bouras, A., and Cherifi, C. (2016). IoT for Smart City Services: Lifecycle Approach. Proceedings of the International Conference on Internet of Things and Cloud Computing, Cambridge, UK, 22\u201323 March 2016, Association for Computing Machinery.","DOI":"10.1145\/2896387.2896440"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1016\/j.future.2016.10.030","article-title":"Knowledge-Infused and Consistent Complex Event Processing over Real-Time and Persistent Streams","volume":"76","author":"Zhou","year":"2017","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ordonez, C., Tahsin Al-Amin, S., and Bellatreche, L. (2020, January 10\u201313). An ER-Flow Diagram for Big Data. Proceedings of the 2020 IEEE International Conference on Big Data (Big Data), Atlanta, GA, USA.","DOI":"10.1109\/BigData50022.2020.9378088"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Erraissi, A., Banane, M., Belangour, A., and Azzouazi, M. (2020, January 26\u201327). Big Data Storage Using Model Driven Engineering: From Big Data Meta-Model to Cloudera PSM Meta-Model. Proceedings of the 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI), Sakheer, Bahrain.","DOI":"10.1109\/ICDABI51230.2020.9325674"},{"key":"ref_27","unstructured":"White, T. (2012). Hadoop: The Definitive Guide, O\u2019Reilly Media, Inc."},{"key":"ref_28","unstructured":"Menon, R. (2014). Cloudera Administration Handbook, Packt Publishing Ltd."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"8943","DOI":"10.1109\/JIOT.2020.2999446","article-title":"Graph-Deep-Learning-Based Inference of Fine-Grained Air Quality from Mobile IoT Sensors","volume":"7","author":"Do","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Datta, S.K., and Bonnet, C. (2018, January 24\u201326). Next-Generation, Data Centric and End-to-End IoT Architecture Based on Microservices. Proceedings of the 2018 IEEE International Conference on Consumer Electronics\u2014Asia (ICCE-Asia), Jeju, Korea.","DOI":"10.1109\/ICCE-ASIA.2018.8552135"},{"key":"ref_31","unstructured":"(2021, September 10). CitySDK. Available online: https:\/\/www.citysdk.eu\/."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Bellini, E., Bellini, P., Cenni, D., Nesi, P., Pantaleo, G., Paoli, I., and Paolucci, M. (2021). An IoE and Big Multimedia Data Approach for Urban Transport System Resilience Management in Smart Cities. Sensors, 21.","DOI":"10.3390\/s21020435"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Arman, A., Bellini, P., Nesi, P., and Paolucci, M. (2019, January 10). Analyzing Public Transportation Offer Wrt Mobility Demand. Proceedings of the 1st ACM International Workshop on Technology Enablers and Innovative Applications for Smart Cities and Communities, New York, NY, USA.","DOI":"10.1145\/3364544.3364828"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1016\/j.compind.2018.12.010","article-title":"A Big Data Platform for Smart Meter Data Analytics","volume":"105","author":"Wilcox","year":"2019","journal-title":"Comput. Ind."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"748","DOI":"10.1016\/j.ijinfomgt.2016.05.002","article-title":"The Role of Big Data in Smart City","volume":"36","author":"Hashem","year":"2016","journal-title":"Int. J. Inf. Manag."},{"key":"ref_36","unstructured":"FIWARE Internet of Things (IoT) (2021, November 09). Services Enablement Architecture. Available online: https:\/\/www.fiware.org\/developers\/."},{"key":"ref_37","unstructured":"(2021, November 09). FIWARE NGSI API. Available online: http:\/\/fiware.github.io\/specifications\/ngsiv2\/stable\/."},{"key":"ref_38","unstructured":"(2021, September 10). FIWARE-NGSI Specification. Available online: https:\/\/knowage.readthedocs.io\/en\/6.1.1\/user\/NGSI\/README\/index.html."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"03025","DOI":"10.1051\/itmconf\/20181703025","article-title":"Data Lake: A New Ideology in Big Data Era","volume":"17","author":"Khine","year":"2018","journal-title":"ITM Web Conf."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1145\/248603.248616","article-title":"An Overview of Data Warehousing and OLAP Technology","volume":"26","author":"Chaudhuri","year":"1997","journal-title":"SIGMOD Rec."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"112","DOI":"10.4018\/IJDWM.2020100107","article-title":"A Temporal Multidimensional Model and OLAP Operators","volume":"16","author":"Ahmed","year":"2020","journal-title":"Int. J. Data Warehous. Min."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Badii, C., Difino, A., Nesi, P., Paoli, I., and Paolucci, M. (2021). Classification of Users\u2019 Transportation Modalities from Mobiles in Real Operating Conditions. Multimed. Tools Appl., 1\u201326.","DOI":"10.1007\/s11042-021-10993-y"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1016\/j.future.2017.08.047","article-title":"Government Affairs Service Platform for Smart City","volume":"81","author":"Lv","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Badii, C., Belay, E., Bellini, P., Cenni, D., Marazzini, M., Mesiti, M., Nesi, P., Pantaleo, G., Paolucci, M., and Valtolina, S. (2018, January 8\u201312). Snap4City: A Scalable IOT\/IOE Platform for Developing Smart City Applications. Proceedings of the 2018 IEEE SmartWorld, Ubiquitous Intelligence Computing, Advanced Trusted Computing, Scalable Computing Communications, Cloud Big Data Computing, Internet of People and Smart City Innovation (SmartWorld\/SCALCOM\/UIC\/ATC\/CBDCom\/IOP\/SCI), Guangzhou, China.","DOI":"10.1109\/SmartWorld.2018.00353"},{"key":"ref_45","unstructured":"Vassiliadis, P., Simitsis, A., and Skiadopoulos, S. Conceptual Modeling for ETL Processes. Proceedings of the 5th ACM International Workshop on Data Warehousing and OLAP."},{"key":"ref_46","first-page":"100276","article-title":"High Level Control of Chemical Plant by Industry 4.0 Solutions","volume":"2021","author":"Bellini","year":"2021","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.engappai.2016.01.011","article-title":"Geographical Localization of Web Domains and Organization Addresses Recognition by Employing Natural Language Processing, Pattern Matching and Clustering","volume":"51","author":"Nesi","year":"2016","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Badii, C., Bellini, P., Difino, A., Nesi, P., Pantaleo, G., and Paolucci, M. (2019). MicroServices Suite for Smart City Applications. Sensors, 19.","DOI":"10.3390\/s19214798"},{"key":"ref_49","unstructured":"(2021, August 03). OpenStreetMap. Available online: https:\/\/www.disit.org\/smosm\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/24\/8429\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:50:41Z","timestamp":1760169041000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/24\/8429"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,17]]},"references-count":49,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["s21248429"],"URL":"https:\/\/doi.org\/10.3390\/s21248429","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,12,17]]}}}