{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T06:20:46Z","timestamp":1769149246494,"version":"3.49.0"},"reference-count":43,"publisher":"Wiley","license":[{"start":{"date-parts":[[2021,5,23]],"date-time":"2021-05-23T00:00:00Z","timestamp":1621728000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Security and Communication Networks"],"published-print":{"date-parts":[[2021,5,23]]},"abstract":"<jats:p>Modern big data applications tend to prefer a cluster computing approach as they are linked to the distributed computing framework that serves users jobs as per demand. It performs rapid processing of tasks by subdividing them into tasks that execute in parallel. Because of the complex environment, hardware and software issues, tasks might run slowly leading to delayed job completion, and such phenomena are also known as stragglers. The performance improvement of distributed computing framework is a bottleneck by straggling nodes due to various factors like shared resources, heavy system load, or hardware issues leading to the prolonged job execution time. Many state-of-the-art approaches use independent models per node and workload. With increased nodes and workloads, the number of models would increase, and even with large numbers of nodes. Not every node would be able to capture the stragglers as there might not be sufficient training data available of straggler patterns, yielding suboptimal straggler prediction. To alleviate such problems, we propose a novel collaborative learning-based approach for straggler prediction, the alternate direction method of multipliers (ADMM), which is resource-efficient and learns how to efficiently deal with mitigating stragglers without moving data to a centralized location. The proposed framework shares information among the various models, allowing us to use larger training data and bring training time down by avoiding data transfer. We rigorously evaluate the proposed method on various datasets with high accuracy results.<\/jats:p>","DOI":"10.1155\/2021\/8340925","type":"journal-article","created":{"date-parts":[[2021,5,24]],"date-time":"2021-05-24T22:50:10Z","timestamp":1621896610000},"page":"1-9","source":"Crossref","is-referenced-by-count":52,"title":["Collaborative Learning Based Straggler Prevention in Large-Scale Distributed Computing Framework"],"prefix":"10.1155","volume":"2021","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4669-088X","authenticated-orcid":true,"given":"Shyam","family":"Deshmukh","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Vaddeswaram, Guntur 522502, AP, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0907-6824","authenticated-orcid":true,"given":"Komati","family":"Thirupathi Rao","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Vaddeswaram, Guntur 522502, AP, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5106-7609","authenticated-orcid":true,"given":"Mohammad","family":"Shabaz","sequence":"additional","affiliation":[{"name":"Arba Minch University, Arba Minch, Ethiopia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","reference":[{"key":"1","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"issue":"4","key":"2","first-page":"954","article-title":"Improving mapreduce performance using smart speculative execution strategy","volume":"63","author":"Q. Chen","year":"2013","journal-title":"Institute of Electrical and Electronics Engineers Transactions on Computers"},{"key":"3","first-page":"1","volume-title":"Google Cluster-Usage Traces: Format Schema","author":"C. Reiss","year":"2011"},{"key":"4","first-page":"1","article-title":"Wrangler: Predictable and faster jobs using fewer resources.","author":"N. J. Yadwadkar"},{"key":"5","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2018.12.002"},{"key":"6","first-page":"1","article-title":"Apache hadoop yarn: yet another resource negotiator","author":"V. K. Vavilapalli"},{"key":"7","first-page":"261","article-title":"Quincy: fair scheduling for distributed computing clusters","author":"M. Isard"},{"key":"8","doi-asserted-by":"crossref","article-title":"Large-scale cluster management at Google with borg","author":"A. Verma","DOI":"10.1145\/2741948.2741964"},{"issue":"1","key":"9","first-page":"3692","article-title":"Multi-task learning for straggler avoiding predictive job scheduling","volume":"17","author":"N. J. Yadwadkar","year":"2016","journal-title":"The Journal of Machine Learning Research"},{"key":"10","article-title":"Detecting straggler mapreduce tasks in big data processing infrastructure by neural network","volume":"2020","author":"A. Javadpour","year":"2020","journal-title":"The Journal of Supercomputing"},{"key":"11","first-page":"144","article-title":"A straggler diagnosis system for distributed computing using machine learning","author":"H. Shen"},{"key":"12","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-10-3226-4_41"},{"key":"13","volume-title":"Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers","author":"S. Boyd","year":"2011"},{"issue":"7","key":"14","article-title":"Improving mapreduce performance in heterogeneous environments","volume":"8","author":"M. Zaharia","year":"2016","journal-title":"Osdi"},{"key":"15","first-page":"1","article-title":"Big data analysis using Apache hadoop","author":"S. G. Manikandan"},{"key":"16","doi-asserted-by":"crossref","article-title":"Apache hadoop yarn: yet another resource negotiator","author":"V. K. Vavilapalli","DOI":"10.1145\/2523616.2523633"},{"issue":"11","key":"17","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1145\/2934664","article-title":"Apache spark: a unified engine for big data processing","volume":"59","author":"M. Zaharia","year":"2012","journal-title":"Communications of the ACM\u201d"},{"key":"18","first-page":"15","article-title":"Resilient distributed datasets: a fault-tolerant abstraction for in- memory cluster computing","author":"M. Zaharia"},{"key":"19","first-page":"24","article-title":"Reining in the outliers in map-reduce clusters using mantri","volume":"10","author":"G. Ananthanarayanan","year":"2010","journal-title":"Osdi"},{"key":"20","first-page":"185","article-title":"Effective straggler mitigation: attack of the clones","author":"G. Ananthanarayanan"},{"key":"21","doi-asserted-by":"publisher","DOI":"10.1109\/access.2019.2927076"},{"key":"22","first-page":"312","article-title":"Efficient big data analytics with optimized parallel processing","author":"S. Sravanthi"},{"key":"23","first-page":"125","article-title":"An efficient resource allocation strategy based on improved particle swarm optimization (ipso)","volume":"14","author":"T. Naresh","year":"2017","journal-title":"Pakistan Journal of Biotechnology"},{"key":"24","doi-asserted-by":"publisher","DOI":"10.11591\/ijece.v6i3.7943"},{"key":"25","doi-asserted-by":"publisher","DOI":"10.17485\/ijst\/2016\/v9i18\/90697"},{"key":"26","doi-asserted-by":"publisher","DOI":"10.17485\/ijst\/2016\/v9i14\/84797"},{"key":"27","first-page":"8463","article-title":"A review on enhancing map reduce performance with data locality in heterogeneous environment","volume":"9","author":"K. Radha","year":"2016","journal-title":"International Journal of Control Theory and Applications"},{"key":"28","doi-asserted-by":"publisher","DOI":"10.1007\/978-81-322-2752-6_5"},{"key":"29","doi-asserted-by":"publisher","DOI":"10.1007\/s13369-017-2926-z"},{"key":"30","volume-title":"Proactive Straggler Avoidance Using Machine Learning","author":"N. J. Yadwadkar","year":"2012"},{"key":"31","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2019.07.017"},{"key":"32","article-title":"Learning scheduling algorithms for data processing clusters","author":"H. Mao","year":"2018"},{"key":"33","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2013.6691651"},{"key":"34","first-page":"169","article-title":"Cheetah: A dynamic performance optimization approach on heterogeneous big data analytics cluster","author":"H. Du"},{"key":"35","doi-asserted-by":"publisher","DOI":"10.1109\/access.2020.2982320"},{"key":"36","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2012.6425904"},{"key":"37","doi-asserted-by":"publisher","DOI":"10.1007\/s12652-019-01382-3"},{"key":"38","first-page":"5447","article-title":"Decentralized linearized alternating direction method of multipliers","author":"Q. Ling"},{"key":"39","doi-asserted-by":"publisher","DOI":"10.1109\/tsp.2016.2602803"},{"key":"40","doi-asserted-by":"publisher","DOI":"10.1016\/j.parco.2004.04.001"},{"key":"41","first-page":"1174","article-title":"Admm based scalable machine learning on spark","author":"S. Dhar"},{"key":"42","first-page":"2825","article-title":"Scikit- learn: machine learning in python","volume":"12","author":"F. Pedregosa","year":"2011","journal-title":"The Journal of Machine Learning Research"},{"key":"43","first-page":"1871","article-title":"Liblinear: A library for large linear classification","volume":"9","author":"R. E. Fan","year":"2008","journal-title":"Journal of Machine Learning Research"}],"container-title":["Security and Communication Networks"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/scn\/2021\/8340925.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/scn\/2021\/8340925.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/scn\/2021\/8340925.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,24]],"date-time":"2021-05-24T22:50:16Z","timestamp":1621896616000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.hindawi.com\/journals\/scn\/2021\/8340925\/"}},"subtitle":[],"editor":[{"given":"Manjit","family":"Kaur","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2021,5,23]]},"references-count":43,"alternative-id":["8340925","8340925"],"URL":"https:\/\/doi.org\/10.1155\/2021\/8340925","relation":{},"ISSN":["1939-0122","1939-0114"],"issn-type":[{"value":"1939-0122","type":"electronic"},{"value":"1939-0114","type":"print"}],"subject":[],"published":{"date-parts":[[2021,5,23]]}}}