{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,1]],"date-time":"2025-10-01T15:22:05Z","timestamp":1759332125397,"version":"3.37.3"},"reference-count":43,"publisher":"Wiley","license":[{"start":{"date-parts":[[2021,11,24]],"date-time":"2021-11-24T00:00:00Z","timestamp":1637712000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62102087","62173025","20QD22","CXTD12-04"],"award-info":[{"award-number":["62102087","62173025","20QD22","CXTD12-04"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010791","name":"University of International Business and Economics","doi-asserted-by":"publisher","award":["62102087","62173025","20QD22","CXTD12-04"],"award-info":[{"award-number":["62102087","62173025","20QD22","CXTD12-04"]}],"id":[{"id":"10.13039\/100010791","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Scientific Programming"],"published-print":{"date-parts":[[2021,11,24]]},"abstract":"<jats:p>Join operations of data sets play a crucial role in obtaining the relations of massive data in real life. Joining two data sets with MapReduce requires a proper design of the Map and Reduce stages for different scenarios. The factors affecting MapReduce join efficiency include the density of the data sets and data transmission over clusters like Hadoop. This study aims to improve the efficiency of MapReduce join algorithms on Hadoop by leveraging Shannon entropy to measure the information changes of data sets being joined in different MapReduce stages. To reduce the uncertainty of data sets in joins through the network, a novel MapReduce join algorithm with dynamic partition strategies called dynamic partition join (DPJ) is proposed. Leveraging the changes of entropy in the partitions of data sets during the Map and Reduce stages revises the logical partitions by changing the original input of the reduce tasks in the MapReduce jobs. Experimental results indicate that the entropy-based measures can measure entropy changes of join operations. Moreover, the DPJ variant methods achieved lower entropy compared with the existing joins, thereby increasing the feasibility of MapReduce join operations for different scenarios on Hadoop.<\/jats:p>","DOI":"10.1155\/2021\/1602767","type":"journal-article","created":{"date-parts":[[2021,11,24]],"date-time":"2021-11-24T21:06:49Z","timestamp":1637788009000},"page":"1-15","source":"Crossref","is-referenced-by-count":3,"title":["MapReduce-Based Dynamic Partition Join with Shannon Entropy for Data Skewness"],"prefix":"10.1155","volume":"2021","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1786-7551","authenticated-orcid":true,"given":"Donghua","family":"Chen","sequence":"first","affiliation":[{"name":"School of Information Technology and Management, University of International Business and Economics, Beijing 100029, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0246-5058","authenticated-orcid":true,"given":"Runtong","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Economics and Management, Beijing Jiaotong University, Beijing 100044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","reference":[{"key":"1","doi-asserted-by":"publisher","DOI":"10.1109\/jeeit.2019.8717427"},{"key":"2","doi-asserted-by":"publisher","DOI":"10.14778\/2367502.2367563"},{"key":"3","doi-asserted-by":"publisher","DOI":"10.11648\/j.aas.20210603.11"},{"key":"4","doi-asserted-by":"publisher","DOI":"10.1016\/j.aci.2018.11.001"},{"first-page":"137","article-title":"MapReduce: simplified data processing on large clusters","author":"J. Dean","key":"5"},{"first-page":"1788","article-title":"Detecting associations in large dataset on MapReduce","author":"D. Dai","key":"6"},{"key":"7","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2019.03.035"},{"key":"8","doi-asserted-by":"publisher","DOI":"10.1145\/1807167.1807222"},{"key":"9","doi-asserted-by":"publisher","DOI":"10.1016\/j.parco.2021.102790"},{"key":"10","doi-asserted-by":"publisher","DOI":"10.1109\/icde.2012.66"},{"key":"11","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2017.2767596"},{"key":"12","doi-asserted-by":"publisher","DOI":"10.1145\/1247480.1247602"},{"key":"13","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989423"},{"key":"14","doi-asserted-by":"publisher","DOI":"10.1109\/icdm.2010.70"},{"key":"15","doi-asserted-by":"publisher","DOI":"10.1109\/icde.2012.87"},{"issue":"1","key":"16","first-page":"67","article-title":"Bloom join fine-tunes distributed query in Hadoop environment","volume":"4","author":"S. M. Mahajan","year":"2013","journal-title":"International Journal of Advanced Engineering Technology"},{"key":"17","doi-asserted-by":"publisher","DOI":"10.3233\/jifs-201220"},{"key":"18","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3007028"},{"key":"19","doi-asserted-by":"publisher","DOI":"10.1007\/s10707-020-00414-x"},{"key":"20","doi-asserted-by":"publisher","DOI":"10.1145\/1807167.1807273"},{"key":"21","doi-asserted-by":"publisher","DOI":"10.1007\/s42979-021-00738-x"},{"key":"22","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2016.2631599"},{"key":"23","doi-asserted-by":"publisher","DOI":"10.1145\/3459104"},{"author":"S. Migliorini","key":"24","article-title":"A balanced solution for the partition-based spatial merge join in MapReduce"},{"key":"25","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"26","doi-asserted-by":"publisher","DOI":"10.1109\/LISS.2015.7369695"},{"key":"27","doi-asserted-by":"publisher","DOI":"10.1109\/bigdata.2013.6691736"},{"key":"28","doi-asserted-by":"publisher","DOI":"10.1109\/JSYST.2020.3022286"},{"key":"29","doi-asserted-by":"publisher","DOI":"10.1016\/j.jksuci.2020.05.004"},{"volume-title":"Join Algorithms Using Map\/Reduce","year":"2010","author":"J. Chandar","key":"30"},{"key":"31","doi-asserted-by":"publisher","DOI":"10.1145\/2094114.2094118"},{"key":"32","doi-asserted-by":"publisher","DOI":"10.2200\/S00274ED1V01Y201006HLT007"},{"key":"33","doi-asserted-by":"publisher","DOI":"10.1145\/2401603.2401626"},{"key":"34","doi-asserted-by":"publisher","DOI":"10.1145\/362686.362692"},{"key":"35","doi-asserted-by":"publisher","DOI":"10.5220\/0003391105740583"},{"key":"36","doi-asserted-by":"publisher","DOI":"10.1007\/s11227-018-2578-0"},{"issue":"5","key":"37","first-page":"705","article-title":"Indexed map-reduce join algorithm","volume":"6","author":"M. H. Khafagy","year":"2015","journal-title":"International Journal of Scientific Engineering and Research"},{"key":"38","doi-asserted-by":"publisher","DOI":"10.1109\/tkde.2011.47"},{"key":"39","doi-asserted-by":"publisher","DOI":"10.1504\/ijbdi.2019.097395"},{"key":"40","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-018-6586-2"},{"key":"41","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920908"},{"key":"42","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350238"},{"key":"43","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-63924-2_11"}],"container-title":["Scientific Programming"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/sp\/2021\/1602767.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/sp\/2021\/1602767.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/sp\/2021\/1602767.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,11,24]],"date-time":"2021-11-24T21:06:55Z","timestamp":1637788015000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.hindawi.com\/journals\/sp\/2021\/1602767\/"}},"subtitle":[],"editor":[{"given":"Michele","family":"Risi","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2021,11,24]]},"references-count":43,"alternative-id":["1602767","1602767"],"URL":"https:\/\/doi.org\/10.1155\/2021\/1602767","relation":{},"ISSN":["1875-919X","1058-9244"],"issn-type":[{"type":"electronic","value":"1875-919X"},{"type":"print","value":"1058-9244"}],"subject":[],"published":{"date-parts":[[2021,11,24]]}}}