{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T21:10:02Z","timestamp":1781039402838,"version":"3.54.1"},"reference-count":54,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2017,6,24]],"date-time":"2017-06-24T00:00:00Z","timestamp":1498262400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>The digitization of our lives cause a shift in the data production as well as in the required data management. Numerous nodes are capable of producing huge volumes of data in our everyday activities. Sensors, personal smart devices as well as the Internet of Things (IoT) paradigm lead to a vast infrastructure that covers all the aspects of activities in modern societies. In the most of the cases, the critical issue for public authorities (usually, local, like municipalities) is the efficient management of data towards the support of novel services. The reason is that analytics provided on top of the collected data could help in the delivery of new applications that will facilitate citizens\u2019 lives. However, the provision of analytics demands intelligent techniques for the underlying data management. The most known technique is the separation of huge volumes of data into a number of parts and their parallel management to limit the required time for the delivery of analytics. Afterwards, analytics requests in the form of queries could be realized and derive the necessary knowledge for supporting intelligent applications. In this paper, we define the concept of a Query Controller (    Q C    ) that receives queries for analytics and assigns each of them to a processor placed in front of each data partition. We discuss an intelligent process for query assignments that adopts Machine Learning (ML). We adopt two learning schemes, i.e., Reinforcement Learning (RL) and clustering. We report on the comparison of the two schemes and elaborate on their combination. Our aim is to provide an efficient framework to support the decision making of the QC that should swiftly select the appropriate processor for each query. We provide mathematical formulations for the discussed problem and present simulation results. Through a comprehensive experimental evaluation, we reveal the advantages of the proposed models and describe the outcomes results while comparing them with a deterministic framework.<\/jats:p>","DOI":"10.3390\/informatics4030016","type":"journal-article","created":{"date-parts":[[2017,6,27]],"date-time":"2017-06-27T02:58:05Z","timestamp":1498532285000},"page":"16","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Reinforcement Learning for Predictive Analytics in Smart Cities"],"prefix":"10.3390","volume":"4","author":[{"given":"Kostas","family":"Kolomvatsos","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Thessaly, Papasiopoulou 2-4, Lamia 35100, Greece;"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christos","family":"Anagnostopoulos","sequence":"additional","affiliation":[{"name":"School of Computing Science, University of Glasgow, 17 Lilybank Gardens, Glasgow G12 8QQ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2017,6,24]]},"reference":[{"key":"ref_1","unstructured":"Scholl, H., and Scholl, M. (2014, January 4\u20137). Smart Governance: A Roadmap for Research and Practice. Proceedings of the iConference Summary, Berlin, Germany."},{"key":"ref_2","unstructured":"Meijer, A., and Bolivar, M. (2013, January 11\u201313). Governing the Smart City: Scaling-Up the Search for Socio-Techno Synergy. Proceedings of the 2013 EGPA Conference, Scotland, UK."},{"key":"ref_3","unstructured":"Bolivar, M.P.R. (2015). Smart Cities: Big Cities, Complex Governance?. Transforming City Governments for Successful Smart Cities, Springer International Publishing. Public Administration and Information Technology 8."},{"key":"ref_4","unstructured":"Hemmet, D., and Townsend, A. (2013). Design Rules for Smarter Cities. \u2018Smart Citizens\u2019 Future Everything, Future Everything Publications."},{"key":"ref_5","first-page":"9429676","article-title":"Big Data Analytics Embedded Smart City Architecture for Performance Enhancement through Real-Time Data Processing and Decision-Making","volume":"2017","author":"Silva","year":"2017","journal-title":"Wirel. Commun. Mobile Comput."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1007\/s00778-003-0095-z","article-title":"Aurora: A New Model and Architecture for Data Stream Management","volume":"12","author":"Abadi","year":"2003","journal-title":"VLDB J."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Arasu, A., Babcock, B., Babu, S., Cieslewicz, J., Datar, M., Ito, K., Motwani, R., Srivastava, U., and Widom, J. (2004). STREAM: The Stanford Data Stream Management System, Springer.","DOI":"10.1145\/872757.872854"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1007\/s00778-003-0096-y","article-title":"Soup: A system for streaming queries over streaming data","volume":"12","author":"Chandrasekaran","year":"2003","journal-title":"VLDB J."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Cranor, C., Johnson, T., Spataschek, O., and Shkapenyuk, V. (2003, January 9\u201312). Gigascope: A Stream Database for Network Applications. Proceedings of the International Conference on Management of Data and Symposium on Principles Database and Systems (SIGMOD), San Diego, CA, USA.","DOI":"10.1145\/872757.872838"},{"key":"ref_10","unstructured":"Hammad, M.A., Ghanem, T.M., Aref, W.G., Elmagarmid, A.K., and Mokbel, M.F. (2003). Efficient Pipelined Execution of Sliding-Window Queries Over Data Streams, Department of Computer Science, Purdue University. Technical Report."},{"key":"ref_11","unstructured":"Motwani, R., Widom, J., Arasu, A., Babcock, B., Babu, S., Datar, M., Manku, G.S., Olston, C., Rosenstein, J., and Varma, R. (2003, January 5\u20138). Query Processing, Approximation, and Resource Management in a Data Stream Management System. Proceedings of the First Biennial Conference on Innovative Data Systems Research (CIDR), Asilomar, CA, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1145\/601858.601861","article-title":"The Cougar Approach to In-Network Query Processing in Sensor Networks","volume":"31","author":"Yao","year":"2002","journal-title":"SIGMOD Rec."},{"key":"ref_13","first-page":"4","article-title":"Continuous Query Processing of Spatio-Temporal Data Streams in PLACE","volume":"9","author":"Mokbel","year":"2005","journal-title":"Geoinformatics"},{"key":"ref_14","unstructured":"(2017, March 20). Funellblack, Supporting Multiple Query Processors. Available online: https:\/\/docs.funnelback.com\/more\/extra\/supporting_multiple_query_processors.html."},{"key":"ref_15","unstructured":"(2017, March 20). Oracle Parallel Server Concepts and Administration. Available online: https:\/\/docs.oracle.com\/cd\/A58617_01\/server.804\/a58238\/ch1_unde.htm."},{"key":"ref_16","unstructured":"Ciaccia, P., and Patella, M. (2001). Approximate Similarity Queries: A Survey, University of Bolognia."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ramakrishnan, G., and Uma Maheswari, S. (2015). Novel Approach for Predicting Difficult Keyword Queries over Databases using Effective Ranking. Int. J. Eng. Res. Technol., 4.","DOI":"10.17577\/IJERTV4IS030251"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhang, W., and Li, J. (2006). Processing Probabilistic Range Query over Imprecise Data Based on Quality of Result. APWeb Workshops, Springer. LNCS.","DOI":"10.1007\/11610496_57"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_20","unstructured":"Bishop, C. (2009). Pattern Recognition and Machine Learning, Springer."},{"key":"ref_21","unstructured":"Alpaydin, E. (2010). Introduction to Machine Learning, The MIT Press. [2nd ed.]."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Kolomvatsos, K., and Hadjiefthymiades, S. (2016). Learning the Engagement of Query Processors for Intelligent Analytics. Springer Appl. Intell. J., 1\u201317.","DOI":"10.1007\/s10489-016-0821-z"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Singh, S., and Singh, N. (2012, January 19\u201320). Big Data Analytics. Proceedings of the International Conference on Communication, Information, Computing Technology, Mumbai, India.","DOI":"10.1109\/ICCICT.2012.6398180"},{"key":"ref_24","unstructured":"Herodotou, H., Lim, H., Luo, G., Borisov, N., Dong, L., Cetin, F.B., and Babu, S. (2011, January 9\u201312). Starfish: A Self-tuning System for Big Data Analytics. Proceedings of the Biennial Conference on Innovative Data Systems Research (CIDR), Asilomar, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Agarwal, S., Milner, H., Kleiner, A., Talwalkar, A., Jordan, M., Madden, S., Mozafari, B., and Stoica, I. (2014, January 22\u201327). Knowing When You\u2019re Wrong: Building Fast and Reliable Approximate Query Processing Systems. Proceedings of the ACM SIGMOD Conference, Snowbird, UT, USA.","DOI":"10.1145\/2588555.2593667"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chaudhuri, S., Das, G., and Srivastava, U. (2004, January 13\u201318). Effective use of block-level sampling in statistics estimation. Proceedings of the 2004 ACM SIGMOD international conference on Management of data, Paris, France.","DOI":"10.1145\/1007568.1007602"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1023\/A:1009835310546","article-title":"Informix under control: Online query Processing","volume":"4","author":"Hellerstein","year":"2000","journal-title":"Data Min. Knowl. Discov. J."},{"key":"ref_28","unstructured":"Pansare, N., Borkar, V.R., Jermaine, C., and Condie, T. (September, January 29). Online aggregation for large MapReduce jobs. Proceedings of the 37th International Conference on Very Large Data Bases (VLDB), Seattle, WA, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Doucet, A., Briers, M., and Senecal, S. (2006). Efficient block sampling strategies for sequential Monte Carlo methods. J. Comput. Graphical Stat.","DOI":"10.1198\/106186006X142744"},{"key":"ref_30","unstructured":"Raman, V., Raman, B., and Hellerstein, J.M. (1999, January 7\u201310). Online dynamic reordering for interactive data processing. Proceedings of the 25th International Conference on Very Large Data Bases (VLDB), Scotland, UK."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1726","DOI":"10.14778\/2556549.2556557","article-title":"Scalable Progressive Analytics on Big Data in the Cloud","volume":"6","author":"Chandramouli","year":"2013","journal-title":"Proc. VLDB Endow."},{"key":"ref_32","unstructured":"Condie, T., Conway, N., Alvaro, P., Hellerstein, J.M., Elmeleegy, K., and Sears, R. (2010, January 28\u201330). MapReduce online. Proceedings of the 7th Conference on Networked Systems Design and Implementation, San Jose, CA, USA."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1472","DOI":"10.14778\/1454159.1454204","article-title":"Ad-hoc Data Processing in the Cloud","volume":"1","author":"Logothetis","year":"2008","journal-title":"Proc. VLDB Endow."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Jermaine, C., Arumugam, S., Pol, A., and Dobra, A. (2007, January 11\u201314). Scalable approximate query processing with the DBO engine. Proceedings of the \u201907 International Conference on Management of Data (SIGMOD), Beijing, China.","DOI":"10.1145\/1247480.1247560"},{"key":"ref_35","unstructured":"Bakhtiar, A., and Joan, L. (2015, January 21\u201326). Sketch of Big Data Real-Time Analytics Model. Proceedings of the The Fifth International Conference on Advances in Information Mining and Management, Brussels, Belgium."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1016\/j.compeleceng.2016.05.018","article-title":"Symmetric Matrix-based Predictive Classifier for Big Data computation and information sharing in Cloud","volume":"56","author":"Vennila","year":"2016","journal-title":"Comput. Electr. Eng."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hassani, M., Spaus, P., Cuzzocrea, A., and Seidi, T. (2016, January 16\u201319). I-HASTREAM: Density-Based Hierarchical Clustering of Big Data Streams and Its Application to Big Graph Analytics Tools. Proceedings of the 16th IEEE\/ACM International Symposium on Cluster, Cloud and Grid Computing, Cartagena, Colombia.","DOI":"10.1109\/CCGrid.2016.102"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Ma, S., and Liang, Z. (2015, January 24\u201327). Design and Implementation of Smart City Big Data Processing Platform Based on Distributed Architecture. Proceedings of the 10th International Conference on Intelligent Systems and Knowledge Engineering (ISKE), Taipei, Taiwan.","DOI":"10.1109\/ISKE.2015.7"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Arasteh, H., Hosseinnezhad, V., Loia, V., Tommasetti, A., Troisi, O., Shafie-khah, M., and Siano, P. (2016, January 6\u20138). IoT-based Smart Cities: A Survey. Proceedings of the IEEE 16th International Conference on Environment and Electrical Engineering (EEEIC), Florence, Italy.","DOI":"10.1109\/EEEIC.2016.7555867"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/s13174-015-0041-5","article-title":"Applications of big data to smart cities","volume":"6","author":"Eiman","year":"2015","journal-title":"J. Internet Ser. Appl."},{"key":"ref_41","first-page":"12","article-title":"Role of Big Data and Analytics in Smart Cities","volume":"5","author":"Kumar","year":"2016","journal-title":"Int. J. Sci. Res."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"766","DOI":"10.1109\/ACCESS.2016.2529723","article-title":"Internet of Things and Big data Analytics for Smart Cities and Connected Communities","volume":"4","author":"Sun","year":"2016","journal-title":"IEEE Access"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Spatti, D.H., and Bartocci Lobini, L.H. (2016). Computational Tools for Data Processing in Smart Cities. Smart Cities Technologies, InTech.","DOI":"10.5772\/65208"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Bonino, D., Rizzo, F., Pastrone, C., Soto, J.A.C., Ahlsen, M., and Axling, M. (2016, January 12\u201315). Block-based realtime big-data processing for smart cities. Proceedings of the IEEE International Smart Cities Conference (ISC2), Trento, Italy.","DOI":"10.1109\/ISC2.2016.7580768"},{"key":"ref_45","unstructured":"Lei, C., Zhuang, Z., Rundensteiner, E.A., and Eltabakh, M. (2014, January 1\u20135). Redoop Infrastructure for Recurring Big Data Queries. Proceedings of the 40th International Conference on very Large Data Bases (VLDB), Hangzhou, China."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1007\/s00778-012-0269-7","article-title":"The HaLoop approach to large-scale iterative data analysis","volume":"21","author":"Bu","year":"2012","journal-title":"VLDB J."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Ekanayake, J., Li, H., Zhang, B., Gunarathne, T., Bae, S.-H., Qiu, J., and Fox, G. (2010, January 21\u201325). Twister: A runtime for iterative mapreduce. Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, Chicago, IL, USA.","DOI":"10.1145\/1851476.1851593"},{"key":"ref_48","unstructured":"Olston, C., Chiou, G., Chitnis, L., Liu, F., Han, Y., Larsson, M., Neumann, A., Rao, V.B.N., Karasubramanian, V., and Seth, S. (2011, January 12\u201316). Nova: continuous pig\/hadoop workflows. Proceedings of the SIGMOD Conference, Athens, Greece."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/j.bdr.2015.04.001","article-title":"Efficient Machine Learning for Big Data: A Review","volume":"2","author":"Yoo","year":"2015","journal-title":"Big Data Res."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"350","DOI":"10.1016\/j.neucom.2017.01.026","article-title":"Machine learning on big data: Opportunities and challenges","volume":"237","author":"Zhou","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Jang, J.-S.R., Sun, C.T., and Mizutani, E. (1997). Neuro-Fuzzy and Soft Computing\u2014A Computational Approach to Learning and Machine Intelligence, Prentice Hall.","DOI":"10.1109\/TAC.1997.633847"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Boulougaris, G., Kolomvatsos, K., and Hadjiefthymiades, S. (2010, January 18\u201323). Building the Knowledge Base of a Buyer Agent Using Reinforcement Learning Techniques. Proceedings of the 2010 IEEE World Congress on Computational Intelligence (WCCI 2010), IJCNN, Barcelona, Spain.","DOI":"10.1109\/IJCNN.2010.5596601"},{"key":"ref_53","unstructured":"Manyika, J., and Durrant-Whyte, H. (1994). Data Fusion and Sensor Management: A Decentralized Information-Theoretic Approach, Ellis Horwood."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Charikar, M., Chekuri, C., Feder, T., and Motwani, R. (1997, January 4\u20136). Incremental clustering and dynamic infomation retrieval. Proceedings of the ACM 29th Annual Symposium on Theory of Computing, El Paso, TX, USA.","DOI":"10.1145\/258533.258657"}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/4\/3\/16\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:40:17Z","timestamp":1760208017000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/4\/3\/16"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,6,24]]},"references-count":54,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2017,9]]}},"alternative-id":["informatics4030016"],"URL":"https:\/\/doi.org\/10.3390\/informatics4030016","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,6,24]]}}}