{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:13:31Z","timestamp":1750306411069,"version":"3.41.0"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2015,6,2]],"date-time":"2015-06-02T00:00:00Z","timestamp":1433203200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation","award":["1027854, 0347148, and 0936948"],"award-info":[{"award-number":["1027854, 0347148, and 0936948"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2015,6,2]]},"abstract":"<jats:p>The trend of analyzing big data in artificial intelligence demands highly-scalable machine learning algorithms, among which clustering is a fundamental and arguably the most widely applied method. To extend the applications of regular vector-based clustering algorithms, the Discrete Distribution (D2) clustering algorithm has been developed, aiming at clustering data represented by bags of weighted vectors which are well adopted data signatures in many emerging information retrieval and multimedia learning applications. However, the high computational complexity of D2-clustering limits its impact in solving massive learning problems. Here we present the parallel D2-clustering (PD2-clustering) algorithm with substantially improved scalability. We developed a hierarchical multipass algorithm structure for parallel computing in order to achieve a balance between the individual-node computation and the integration process of the algorithm. Experiments and extensive comparisons between PD2-clustering and other clustering algorithms are conducted on synthetic datasets. The results show that the proposed parallel algorithm achieves significant speed-up with minor accuracy loss. We apply PD2-clustering to image concept learning. In addition, by extending D2-clustering to symbolic data, we apply PD2-clustering to protein sequence clustering. For both applications, we demonstrate the high competitiveness of our new algorithm in comparison with other state-of-the-art methods.<\/jats:p>","DOI":"10.1145\/2700293","type":"journal-article","created":{"date-parts":[[2015,6,2]],"date-time":"2015-06-02T18:19:47Z","timestamp":1433269187000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Parallel Massive Clustering of Discrete Distributions"],"prefix":"10.1145","volume":"11","author":[{"given":"Yu","family":"Zhang","sequence":"first","affiliation":[{"name":"The Pennsylvania State University, University Park, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James Z.","family":"Wang","sequence":"additional","affiliation":[{"name":"The Pennsylvania State University, University Park, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jia","family":"Li","sequence":"additional","affiliation":[{"name":"The Pennsylvania State University, University Park, PA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2015,6,2]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/FOCS.2009.14"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/1046920.1194902"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2432622.2432626"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2072298.2072033"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkg095"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.88"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1646396.1646452"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9939-07-09020-X"},{"key":"e_1_2_2_9_1","unstructured":"T. M. Cover and J. A. Thomas. 2012. Elements of Information Theory. John Wiley &amp; Sons.  T. M. Cover and J. A. Thomas. 2012. Elements of Information Theory. John Wiley &amp; Sons."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1006\/jagm.2000.1090"},{"key":"e_1_2_2_11_1","doi-asserted-by":"crossref","unstructured":"I. Daubechies. 1992. Ten Lectures on Wavelets. SIAM.   I. Daubechies. 1992. Ten Lectures on Wavelets. SIAM.","DOI":"10.1137\/1.9781611970104"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007612920971"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/16.5.451"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2020408.2020516"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-6-56"},{"key":"e_1_2_2_16_1","doi-asserted-by":"crossref","unstructured":"A. Gersho and R. M. Gray. 1992. Vector Quantization and Signal Compression. Springer.   A. Gersho and R. M. Gray. 1992. Vector Quantization and Signal Compression. Springer.","DOI":"10.1007\/978-1-4615-3626-0"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2517151"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2003.1198387"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btq003"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01908075"},{"key":"e_1_2_2_21_1","unstructured":"L. V. Kantorovich. 1942. On the transfer of masses. Dokl. Akad. Nauk. SSSR. 227--229.  L. V. Kantorovich. 1942. On the transfer of masses. Dokl. Akad. Nauk. SSSR. 227--229."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2105-11-544"},{"volume":"2","volume-title":"Proceedings of 8th IEEE International Conference on Computer Vision","author":"Levina E.","key":"e_1_2_2_23_1"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.70847"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCOM.1980.1094577"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:VISI.0000029664.99615.94"},{"key":"e_1_2_2_27_1","volume-title":"Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability","volume":"1","author":"MacQueen J.","year":"1967"},{"key":"e_1_2_2_28_1","unstructured":"G. Monge. 1781. M\u00e9emoire sur la th\u00e9eorie des d\u00e9eblais et des remblais. De l'Imprimerie Royale.  G. Monge. 1781. M\u00e9emoire sur la th\u00e9eorie des d\u00e9eblais et des remblais. De l'Imprimerie Royale."},{"key":"e_1_2_2_29_1","unstructured":"K. G. Murty. 1983. Linear Programming. Vol. 57 Wiley New York.  K. G. Murty. 1983. Linear Programming. Vol. 57 Wiley New York."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/0167-8191(95)00017-I"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkj515"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/0076-6879(90)83007-V"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1093\/molbev\/msp260"},{"volume":"7","volume-title":"Proceedings of the Conference on Empirical Methods on Natural Language Processing and Computational Natural Language Learning","author":"Rosenberg A.","key":"e_1_2_2_34_1"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2037676.2037687"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gks1067"},{"key":"e_1_2_2_37_1","unstructured":"N. X. Vinh J. Epps and J. Bailey. 2010. Information theoretic measures for clusterings comparison: Variants properties normalization and correction for chance. J. Mach. Learn. Research 2837--2854.   N. X. Vinh J. Epps and J. Bailey. 2010. Information theoretic measures for clusterings comparison: Variants properties normalization and correction for chance. J. Mach. Learn. Research 2837--2854."},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2007.02.045"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.129"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-10665-1_71"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1404880.1404887"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2700293","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2700293","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T05:07:43Z","timestamp":1750223263000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2700293"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,6,2]]},"references-count":41,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2015,6,2]]}},"alternative-id":["10.1145\/2700293"],"URL":"https:\/\/doi.org\/10.1145\/2700293","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2015,6,2]]},"assertion":[{"value":"2014-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-12-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2015-06-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}