{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T03:47:40Z","timestamp":1781754460492,"version":"3.54.5"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,11,27]],"date-time":"2019-11-27T00:00:00Z","timestamp":1574812800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,11,27]],"date-time":"2019-11-27T00:00:00Z","timestamp":1574812800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>A number of technologies enabled by Internet of Thing (IoT) have been used for the prevention of various chronic diseases, continuous and real-time tracking system is a particularly important one. Wearable medical devices with sensor, health cloud and mobile applications have continuously generating a huge amount of data which is often called as streaming big data. Due to the higher speed of the data generation, it is difficult to collect, process and analyze such massive data in real-time in order to perform real-time actions in case of emergencies and extracting hidden value. using traditional methods which are limited and time-consuming. Therefore, there is a significant need to real-time big data stream processing to ensure an effective and scalable solution. In order to overcome this issue, this work proposes a new architecture for real-time health status prediction and analytics system using big data technologies. The system focus on applying distributed machine learning model on streaming health data events ingested to Spark streaming through Kafka topics. Firstly, we transform the standard decision tree (DT) (C4.5) algorithm into a parallel, distributed, scalable and fast DT using Spark instead of Hadoop MapReduce which becomes limited for real-time computing. Secondly, this model is applied to streaming data coming from distributed sources of various diseases to predict health status. Based on several input attributes, the system predicts health status, send an alert message to care providers and store the details in a distributed database to perform health data analytics and stream reporting. We measure the performance of Spark DT against traditional machine learning tools including Weka. Finally, performance evaluation parameters such as throughput and execution time are calculated to show the effectiveness of the proposed architecture. The experimental results show that the proposed system is able to effectively process and predict real-time and massive amount of medical data enabled by IoT from distributed and various diseases.<\/jats:p>","DOI":"10.1186\/s40537-019-0271-7","type":"journal-article","created":{"date-parts":[[2019,11,27]],"date-time":"2019-11-27T09:04:51Z","timestamp":1574845491000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":72,"title":["A new Internet of Things architecture for real-time prediction of various diseases using machine learning on big data environment"],"prefix":"10.1186","volume":"6","author":[{"given":"Abderrahmane","family":"Ed-daoudy","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Khalil","family":"Maalmi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2019,11,27]]},"reference":[{"issue":"1\u20132","key":"271_CR1","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1504\/IJAIP.2018.089494","volume":"10","author":"G Manogaran","year":"2018","unstructured":"Manogaran G, Lopez D. Health data analytics using scalable logistic regression with stochastic gradient descent. Int J Adv Intell Paradigms. 2018;10(1\u20132):118\u201332.","journal-title":"Int J Adv Intell Paradigms"},{"key":"271_CR2","doi-asserted-by":"publisher","first-page":"652","DOI":"10.1109\/ACCESS.2014.2332453","volume":"2","author":"H Hu","year":"2014","unstructured":"Hu H, Wen Y, Chua T-S, Li X. Toward scalable systems for big data analytics: a technology tutorial. IEEE Access. 2014;2:652\u201387.","journal-title":"IEEE Access"},{"issue":"4","key":"271_CR3","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1145\/1978915.1978919","volume":"39","author":"R Cattell","year":"2011","unstructured":"Cattell R. Scalable sql and NoSQL data stores. ACM Sigmod Record. 2011;39(4):12\u201327.","journal-title":"ACM Sigmod Record"},{"key":"271_CR4","unstructured":"Moniruzzaman A, Hossain SA. NoSQL database: New era of databases for big data analytics-classification, characteristics and comparison. 2013. arXiv preprint arXiv:1307.0191."},{"issue":"1","key":"271_CR5","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1145\/1327452.1327492","volume":"51","author":"J Dean","year":"2008","unstructured":"Dean J, Ghemawat S. Mapreduce: simplified data processing on large clusters. Commun ACM. 2008;51(1):107\u201313.","journal-title":"Commun ACM"},{"key":"271_CR6","doi-asserted-by":"crossref","unstructured":"Belle A, Thiagarajan R, Soroushmehr S, Navidi F, Beard DA, Najarian K. Big data analytics in healthcare. BioMed Res Int. 2015; 2015.","DOI":"10.1155\/2015\/370194"},{"key":"271_CR7","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1016\/j.procs.2015.04.188","volume":"48","author":"J Anuradha","year":"2015","unstructured":"Anuradha J, et al. A brief introduction on big data 5vs characteristics and hadoop technology. Procedia Comput Sci. 2015;48:319\u201324.","journal-title":"Procedia Comput Sci"},{"issue":"12","key":"271_CR8","doi-asserted-by":"publisher","first-page":"17472","DOI":"10.3390\/s131217472","volume":"13","author":"H Banaee","year":"2013","unstructured":"Banaee H, Ahmed MU, Loutfi A. Data mining for wearable sensors in health monitoring systems: a review of recent trends and challenges. Sensors. 2013;13(12):17472\u2013500.","journal-title":"Sensors"},{"key":"271_CR9","doi-asserted-by":"publisher","first-page":"543","DOI":"10.1007\/978-3-319-28031-8_48","volume-title":"Innovations in bio-inspired computing and applications","author":"PS Mathew","year":"2016","unstructured":"Mathew PS, Pillai AS. Big data challenges and solutions in healthcare: a survey. In: Sn\u00e1\u0161el V, Abraham A, Kr\u00f6mer P, Pant M, Muda A, editors. Innovations in bio-inspired computing and applications. Berlin: Springer; 2016. p. 543\u201353."},{"key":"271_CR10","doi-asserted-by":"crossref","unstructured":"Sun J, Reddy CK. Big data analytics for healthcare. In: Proceedings of the 19th ACM SIGKDD International Discovery and Data Mining. New York: ACM; 2013. p. 1525\u20131525.","DOI":"10.1145\/2487575.2506178"},{"key":"271_CR11","first-page":"22","volume":"2","author":"HD Masethe","year":"2014","unstructured":"Masethe HD, Masethe MA. Prediction of heart disease using classification algorithms. Proc World Congress Eng Comput Sci. 2014;2:22\u20134.","journal-title":"Proc World Congress Eng Comput Sci"},{"issue":"10","key":"271_CR12","doi-asserted-by":"publisher","first-page":"4611","DOI":"10.1016\/j.eswa.2015.01.065","volume":"42","author":"A Bhardwaj","year":"2015","unstructured":"Bhardwaj A, Tiwari A. Breast cancer diagnosis using genetically optimized neural network model. Expert Syst Appl. 2015;42(10):4611\u201320.","journal-title":"Expert Syst Appl"},{"issue":"5","key":"271_CR13","doi-asserted-by":"publisher","first-page":"241","DOI":"10.14257\/ijbsbt.2013.5.5.25","volume":"5","author":"D Tomar","year":"2013","unstructured":"Tomar D, Agarwal S. A survey on data mining approaches for healthcare. Int J Bio-Sci Bio-Technol. 2013;5(5):241\u201366.","journal-title":"Int J Bio-Sci Bio-Technol"},{"issue":"1","key":"271_CR14","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1186\/2196-1115-1-2","volume":"1","author":"M Herland","year":"2014","unstructured":"Herland M, Khoshgoftaar TM, Wald R. A review of data mining using big data in health informatics. J Big Data. 2014;1(1):2.","journal-title":"J Big Data"},{"key":"271_CR15","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1007\/978-81-322-2755-7_14","volume-title":"Information systems design and intelligent applications","author":"S Rallapalli","year":"2016","unstructured":"Rallapalli S, Gondkar R, Rao GVM. Cloud based k-means clustering running as a Mapreduce job for big data healthcare analytics using Apache mahout. In: Satapathy S, Mandal J, S Udgata, Bhateja V, editors. Information systems design and intelligent applications. Berlin: Springer; 2016. p. 127\u201335."},{"key":"271_CR16","first-page":"385","volume-title":"Soft Computing Applications","author":"Bidyut Biman Sarkar","year":"2017","unstructured":"Sarkar BB, Paul S, Cornel B, Rohatinovici N, Chaki N. Personal health record management system using Hadoop framework: An application for smarter health care. In: International Workshop Soft Computing Applications. Berlin: Springer; 2016. p. 385\u201393."},{"issue":"2\u20134","key":"271_CR17","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1504\/IJBET.2017.082655","volume":"23","author":"P Sampath","year":"2017","unstructured":"Sampath P, Tamilselvi S, Kumar NS, Lavanya S, Eswari T. Diabetic data analysis in healthcare using Hadoop architecture over big data. Int J Biomed Eng Technol. 2017;23(2\u20134):137\u201347.","journal-title":"Int J Biomed Eng Technol"},{"issue":"1","key":"271_CR18","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1145\/3108936","volume":"18","author":"MM Rathore","year":"2017","unstructured":"Rathore MM, Paul A, Ahmad A, Anisetti M, Jeon G. Hadoop-based intelligent care system (HICS): analytical approach for big data in IoT. ACM Trans Internet Technol (TOIT). 2017;18(1):8.","journal-title":"ACM Trans Internet Technol (TOIT)"},{"key":"271_CR19","first-page":"042056","volume":"263","author":"JA Basco","year":"2017","unstructured":"Basco JA, Senthilkumar N. Real-time analysis of healthcare using big data analytics. Comput Inf Technol. 2017;263:042056.","journal-title":"Comput Inf Technol"},{"key":"271_CR20","doi-asserted-by":"crossref","unstructured":"Yadranjiaghdam B, Pool N, Tabrizi N. A survey on real-time big data analytics: Applications and tools. In: 2016 international conference On computational science and computational intelligence (CSCI). New York: IEEE; 2016. p. 404\u20139.","DOI":"10.1109\/CSCI.2016.0083"},{"key":"271_CR21","doi-asserted-by":"crossref","unstructured":"Hazarika AV, Ram GJSR, Jain E. Performance comparison of hadoop and spark engine. In: 2017 international conference on I-SMAC (IoT in social, mobile, analytics and cloud)(I-SMAC). New York: IEEE; 2017. p. 671\u20134.","DOI":"10.1109\/I-SMAC.2017.8058263"},{"key":"271_CR22","doi-asserted-by":"crossref","unstructured":"Rallapalli S, Suryakanthi T. Predicting the risk of diabetes in big data electronic health records by using scalable random forest classification algorithm. In: 2016 international conference on advances in computing and communication engineering (ICACCE). New York: IEEE; 2016. p. 281\u20134.","DOI":"10.1109\/ICACCE.2016.8073762"},{"key":"271_CR23","doi-asserted-by":"crossref","unstructured":"Feroz MN, Mengel S. Examination of data, rule generation and detection of phishing urls using online logistic regression. In: 2014 IEEE international conference on big data (Big Data). New York: IEEE; 2014. p. 241\u201350.","DOI":"10.1109\/BigData.2014.7004239"},{"key":"271_CR24","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1007\/978-3-319-06269-3_11","volume-title":"Health Information Science","author":"Tingzhi Zhao","year":"2014","unstructured":"Zhao T, Ni H, Zhou X, Qiang L, Zhang D, Yu Z. Detecting abnormal patterns of daily activities for the elderly living alone. In: International conference on health information science. Berlin: Springer; 2014. p. 95\u2013108."},{"issue":"12","key":"271_CR25","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1007\/s10916-016-0647-6","volume":"40","author":"MM Rathore","year":"2016","unstructured":"Rathore MM, Ahmad A, Paul A, Wan J, Zhang D. Real-time medical emergency response system: exploiting IoT and big data for public health. J Med Syst. 2016;40(12):283.","journal-title":"J Med Syst"},{"issue":"2\u20134","key":"271_CR26","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1504\/IJBET.2017.087722","volume":"25","author":"G Manogaran","year":"2017","unstructured":"Manogaran G, Lopez D. A survey of big data architectures and machine learning algorithms in healthcare. Int J Biomed Eng Technol. 2017;25(2\u20134):182\u2013211.","journal-title":"Int J Biomed Eng Technol"},{"key":"271_CR27","doi-asserted-by":"crossref","unstructured":"Lee K, Agrawal A, Choudhary A. Real-time disease surveillance using twitter data: demonstration on flu and cancer. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining. New York: ACM; 2013. p. 1474\u20137.","DOI":"10.1145\/2487575.2487709"},{"key":"271_CR28","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1007\/978-3-319-39601-9_27","volume-title":"Inclusive Smart Cities and Digital Health","author":"Usman Akhtar","year":"2016","unstructured":"Akhtar U, Khattak AM, Lee S. Challenges in managing real-time data in health information system (HIS). In: International conference on smart homes and health telematics. Berlin: Springer; 2016. p. 305\u201313."},{"key":"271_CR29","doi-asserted-by":"crossref","unstructured":"Ed-daoudy A, Maalmi K. Application of machine learning model on streaming health data event in real-time to predict health status using spark. In: 2018 International symposium on advanced electrical and communication technologies (ISAECT). New York: IEEE; 2018. p. 1\u20134.","DOI":"10.1109\/ISAECT.2018.8618860"},{"key":"271_CR30","doi-asserted-by":"crossref","unstructured":"Ed-daoudy A, Maalmi K. Real-time machine learning for early detection of heart disease using big data approach. In: 2019 International conference on wireless technologies, embedded and intelligent systems (WITS). New York: IEEE; 2019. p. 1\u20135.","DOI":"10.1109\/WITS.2019.8723839"},{"key":"271_CR31","unstructured":"Bauer H, Patel M, Veira J. The Internet of Things: sizing up the opportunity. http:\/\/www.mckinsey.com\/. Accessed 15 Dec 2017."},{"key":"271_CR32","unstructured":"Apache kafka. https:\/\/kafka.apache.org. Accessed 15 Dec 2017."},{"key":"271_CR33","unstructured":"Hunt P, Konar M, Junqueira FP, Reed B. Zookeeper: Wait-free coordination for internet-scale systems. In: USENIX Annual technical conference, vol. 8. Boston, MA, USA; 2010."},{"key":"271_CR34","unstructured":"Apache Spark. https:\/\/spark.apache.org. Accessed 15 Dec 2017."},{"key":"271_CR35","unstructured":"Zaharia M, Chowdhury M, Das T, Dave A, Ma J, McCauley M, Franklin MJ, Shenker S, Stoica I. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX conference on networked systems design and implementation. Berkeley: USENIX Association; 2012. p. 2."},{"key":"271_CR36","unstructured":"kaggle. https:\/\/www.kaggle.com\/fmendes\/diabetes-from-dat263x-lab01. Accessed 24 Dec 2018."},{"key":"271_CR37","volume-title":"C4. 5: programs for machine learning","author":"JR Quinlan","year":"2014","unstructured":"Quinlan JR. C4. 5: programs for machine learning. Amsterdam: Elsevier; 2014."},{"key":"271_CR38","unstructured":"Apache cassandra. http:\/\/cassandra.apache.org. Accessed 15 Dec 2017."},{"key":"271_CR39","doi-asserted-by":"crossref","unstructured":"Hassan M, Bansal SK. Semantic data querying over NoSQL databases with Apache Spark. In: 2018 IEEE international conference on information reuse and integration (IRI). New York: IEEE; 2018. p. 364\u201371.","DOI":"10.1109\/IRI.2018.00061"},{"key":"271_CR40","unstructured":"Apache zeppelin. https:\/\/zeppelin.apache.org. Accessed 15 Dec 2017."}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-019-0271-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s40537-019-0271-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-019-0271-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,26]],"date-time":"2020-11-26T00:11:38Z","timestamp":1606349498000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-019-0271-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,27]]},"references-count":40,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["271"],"URL":"https:\/\/doi.org\/10.1186\/s40537-019-0271-7","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,27]]},"assertion":[{"value":"12 July 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 November 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 November 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare that they have no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"104"}}