{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T06:39:08Z","timestamp":1791959948630,"version":"4.3.4"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2019,12,7]],"date-time":"2019-12-07T00:00:00Z","timestamp":1575676800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Data and Information Quality"],"published-print":{"date-parts":[[2020,3,31]]},"abstract":"<jats:p>Duplicate detection algorithms produce clusters of database records, each cluster representing a single real-world entity. As most of these algorithms use pairwise comparisons, the resulting (transitive) clusters can be inconsistent: Not all records within a cluster are sufficiently similar to be classified as duplicate. Thus, one of many subsequent clustering algorithms can further improve the result.<\/jats:p>\n          <jats:p>We explain in detail, compare, and evaluate many of these algorithms and introduce three new clustering algorithms in the specific context of duplicate detection. Two of our three new algorithms use the structure of the input graph to create consistent clusters. Our third algorithm, and many other clustering algorithms, focus on the edge weights, instead. For evaluation, in contrast to related work, we experiment on true real-world datasets, and in addition examine in great detail various pair-selection strategies used in practice. While no overall winner emerges, we are able to identify best approaches for different situations. In scenarios with larger clusters, our proposed algorithm, Extended Maximum Clique Clustering (EMCC), and Markov Clustering show the best results. EMCC especially outperforms Markov Clustering regarding the precision of the results and additionally has the advantage that it can also be used in scenarios where edge weights are not available.<\/jats:p>","DOI":"10.1145\/3352591","type":"journal-article","created":{"date-parts":[[2019,12,9]],"date-time":"2019-12-09T13:35:27Z","timestamp":1575898527000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Transforming Pairwise Duplicates to Entity Clusters for High-quality Duplicate Detection"],"prefix":"10.1145","volume":"12","author":[{"given":"Uwe","family":"Draisbach","sequence":"first","affiliation":[{"name":"Hasso-Plattner-Institute, University of Potsdam, Potsdam, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter","family":"Christen","sequence":"additional","affiliation":[{"name":"Australian National University, Canberra, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felix","family":"Naumann","sequence":"additional","affiliation":[{"name":"Hasso-Plattner-Institute, University of Potsdam, Potsdam, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,12,7]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.7155\/jgaa.00084"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:MACH.0000033116.57574.95"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-008-0098-x"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the ACM SIGKDD International Conference of Knowledge Discovery and Data Mining. 39--48","author":"Bilenko Mikhail","unstructured":"Mikhail Bilenko and Raymond J. Mooney . 2003. Adaptive duplicate detection using learnable string similarity measures . In Proceedings of the ACM SIGKDD International Conference of Knowledge Discovery and Data Mining. 39--48 . Mikhail Bilenko and Raymond J. Mooney. 2003. Adaptive duplicate detection using learnable string similarity measures. In Proceedings of the ACM SIGKDD International Conference of Knowledge Discovery and Data Mining. 39--48."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/362342.362367"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/11508069_15"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2011.127"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-31164-2"},{"key":"e_1_2_1_9_1","volume-title":"Application of advanced record linkage techniques for complex population reconstruction. Arxiv Preprint Arxiv:1612.04286","author":"Christen Peter","year":"2016","unstructured":"Peter Christen . 2016. Application of advanced record linkage techniques for complex population reconstruction. Arxiv Preprint Arxiv:1612.04286 ( 2016 ). Peter Christen. 2016. Application of advanced record linkage techniques for complex population reconstruction. Arxiv Preprint Arxiv:1612.04286 (2016)."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1066157.1066168"},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the International Workshop on Quality in Databases (QDB\u201910)","author":"Draisbach Uwe","year":"2010","unstructured":"Uwe Draisbach and Felix Naumann . 2010 . DuDe: The duplicate detection toolkit . In Proceedings of the International Workshop on Quality in Databases (QDB\u201910) . Uwe Draisbach and Felix Naumann. 2010. DuDe: The duplicate detection toolkit. In Proceedings of the International Workshop on Quality in Databases (QDB\u201910)."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2012.20"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/1191547.1191739"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/1611638.1611641"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDMW.2015.162"},{"key":"e_1_2_1_16_1","volume-title":"197. Computers and Intractability: A Guide to the Theory of NP-Completeness","author":"Garey Michael R.","unstructured":"Michael R. Garey and David S. Johnson . 197. Computers and Intractability: A Guide to the Theory of NP-Completeness . W. H. Freeman and Company , New York . Michael R. Garey and David S. Johnson. 197. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman and Company, New York."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611972887.11"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11222-017-9746-6"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687627.1687771"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-009-0161-2"},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the ACM SIGMOD Workshop on the Web and Databases (WebDB\u201900)","author":"Haveliwala Taher H.","year":"2000","unstructured":"Taher H. Haveliwala , Aristides Gionis , and Piotr Indyk . 2000 . Scalable techniques for clustering the web . In Proceedings of the ACM SIGMOD Workshop on the Web and Databases (WebDB\u201900) . 129--134. Taher H. Haveliwala, Aristides Gionis, and Piotr Indyk. 2000. Scalable techniques for clustering the web. In Proceedings of the ACM SIGMOD Workshop on the Web and Databases (WebDB\u201900). 129--134."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2011.99"},{"key":"e_1_2_1_23_1","volume-title":"Johnson","author":"Horn Roger A.","year":"2012","unstructured":"Roger A. Horn and Charles R . Johnson . 2012 . Matrix Analysis (2nd ed.). Cambridge University Press , New York. Roger A. Horn and Charles R. Johnson. 2012. Matrix Analysis (2nd ed.). Cambridge University Press, New York."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920871"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the SIGMOD Workshop on Research Issues on Data Mining and Knowledge Discovery (DMKD\u201997)","author":"Monge Alvaro","year":"1997","unstructured":"Alvaro Monge and Charles Elkan . 1997 . An efficient domain-independent algorithm for detecting approximately duplicate database records . In Proceedings of the SIGMOD Workshop on Research Issues on Data Mining and Knowledge Discovery (DMKD\u201997) . Alvaro Monge and Charles Elkan. 1997. An efficient domain-independent algorithm for detecting approximately duplicate database records. In Proceedings of the SIGMOD Workshop on Research Issues on Data Mining and Knowledge Discovery (DMKD\u201997)."},{"key":"e_1_2_1_26_1","volume-title":"An Introduction to Duplicate Detection (Synthesis Lectures on Data Management)","author":"Naumann Felix","unstructured":"Felix Naumann and Melanie Herschel . 2010. An Introduction to Duplicate Detection (Synthesis Lectures on Data Management) . Morgan and Claypool Publishers . Felix Naumann and Melanie Herschel. 2010. An Introduction to Duplicate Detection (Synthesis Lectures on Data Management). Morgan and Claypool Publishers."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDMW.2016.0035"},{"key":"e_1_2_1_28_1","doi-asserted-by":"crossref","unstructured":"H. B. Newcombe J. M. Kennedy S. J. Axford and A. P. James. 1959. Automatic linkage of vital records. Science 130 3381 (1959) 954--959.  H. B. Newcombe J. M. Kennedy S. J. Axford and A. P. James. 1959. Automatic linkage of vital records. Science 130 3381 (1959) 954--959.","DOI":"10.1126\/science.130.3381.954"},{"key":"e_1_2_1_29_1","article-title":"Dynamic sorted neighborhood indexing for real-time entity resolution","volume":"6","author":"Ramadan Banda","year":"2015","unstructured":"Banda Ramadan , Peter Christen , Huizhi Liang , Ross W. Gayler , and David Hawking . 2015 . Dynamic sorted neighborhood indexing for real-time entity resolution . J. Data Info. Qual. 6 , 4 (2015), 15:1--15:29. Banda Ramadan, Peter Christen, Huizhi Liang, Ross W. Gayler, and David Hawking. 2015. Dynamic sorted neighborhood indexing for real-time entity resolution. J. Data Info. Qual. 6, 4 (2015), 15:1--15:29.","journal-title":"J. Data Info. Qual."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.3366\/hac.2002.14.1-2.61"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/0196-6774(86)90032-5"},{"key":"e_1_2_1_32_1","first-page":"61","article-title":"Scalable matching and clustering of entities with FAMER","volume":"16","author":"Saeedi Alieh","year":"2018","unstructured":"Alieh Saeedi , Markus Nentwig , Eric Peukert , and Erhard Rahm . 2018 . Scalable matching and clustering of entities with FAMER . Complex Syst. Info. Model. Quart. 16 (2018), 61 -- 83 . Alieh Saeedi, Markus Nentwig, Eric Peukert, and Erhard Rahm. 2018. Scalable matching and clustering of entities with FAMER. Complex Syst. Info. Model. Quart. 16 (2018), 61--83.","journal-title":"Complex Syst. Info. Model. Quart."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-93417-4_37"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1137\/0206038"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-015-0818-7"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350263"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465280"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2723739"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/360715.360746"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/321105.321107"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536336.2536337"}],"container-title":["Journal of Data and Information Quality"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3352591","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3352591","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:13:02Z","timestamp":1750201982000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3352591"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,7]]},"references-count":41,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,3,31]]}},"alternative-id":["10.1145\/3352591"],"URL":"https:\/\/doi.org\/10.1145\/3352591","relation":{},"ISSN":["1936-1955","1936-1963"],"issn-type":[{"value":"1936-1955","type":"print"},{"value":"1936-1963","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,7]]},"assertion":[{"value":"2018-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-12-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}