{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T23:05:46Z","timestamp":1791846346411,"version":"4.3.4"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2021,1,13]],"date-time":"2021-01-13T00:00:00Z","timestamp":1610496000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Data and Information Quality"],"published-print":{"date-parts":[[2021,3,31]]},"abstract":"<jats:p>The integration of multiple data sources is a common problem in a large variety of applications. Traditionally, handcrafted similarity measures are used to discover, merge, and integrate multiple representations of the same entity\u2014duplicates\u2014into a large homogeneous collection of data. Often, these similarity measures do not cope well with the heterogeneity of the underlying dataset. In addition, domain experts are needed to manually design and configure such measures, which is both time-consuming and requires extensive domain expertise.<\/jats:p>\n          <jats:p>We propose a deep Siamese neural network, capable of learning a similarity measure that is tailored to the characteristics of a particular dataset. With the properties of deep learning methods, we are able to eliminate the manual feature engineering process and thus considerably reduce the effort required for model construction. In addition, we show that it is possible to transfer knowledge acquired during the deduplication of one dataset to another, and thus significantly reduce the amount of data required to train a similarity measure. We evaluated our method on multiple datasets and compare our approach to state-of-the-art deduplication methods. Our approach outperforms competitors by up to +26 percent F-measure, depending on task and dataset. In addition, we show that knowledge transfer is not only feasible, but in our experiments led to an improvement in F-measure of up to +4.7 percent.<\/jats:p>","DOI":"10.1145\/3410157","type":"journal-article","created":{"date-parts":[[2021,1,13]],"date-time":"2021-01-13T11:17:11Z","timestamp":1610536631000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Knowledge Transfer for Entity Resolution with Siamese Neural Networks"],"prefix":"10.1145","volume":"13","author":[{"given":"Michael","family":"Loster","sequence":"first","affiliation":[{"name":"Hasso Plattner Institute, University of Potsdam, Potsdam, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ioannis","family":"Koumarelas","sequence":"additional","affiliation":[{"name":"Hasso Plattner Institute, University of Potsdam, Potsdam, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felix","family":"Naumann","sequence":"additional","affiliation":[{"name":"Hasso Plattner Institute, University of Potsdam, Potsdam, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,1,13]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2015.7364061"},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Zohra Bellahsene Angela Bonifati and Erhard Rahm (Eds.). 2011. Schema Matching and Mapping. Springer.  Zohra Bellahsene Angela Bonifati and Erhard Rahm (Eds.). 2011. Schema Matching and Mapping. Springer.","DOI":"10.1007\/978-3-642-16518-4"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/72.279181"},{"key":"e_1_2_1_4_1","volume-title":"Advances in Neural Information Processing Systems","author":"Bergstra James","unstructured":"James Bergstra , R\u00e9mi Bardenet , Yoshua Bengio , and Bal\u00e1zs K\u00e9gl . 2011. Algorithms for hyper-parameter optimization . In Advances in Neural Information Processing Systems . MIT Press , 2546--2554. James Bergstra, R\u00e9mi Bardenet, Yoshua Bengio, and Bal\u00e1zs K\u00e9gl. 2011. Algorithms for hyper-parameter optimization. In Advances in Neural Information Processing Systems. MIT Press, 2546--2554."},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the International Conference on Knowledge Discovery and Data Mining (SIGKDD\u201903)","author":"Bilenko Mikhail","year":"2003","unstructured":"Mikhail Bilenko and Raymond J. Mooney . 2003. Adaptive duplicate detection using learnable string similarity measures . Proceedings of the International Conference on Knowledge Discovery and Data Mining (SIGKDD\u201903) ( 2003 ), 39--48. Mikhail Bilenko and Raymond J. Mooney. 2003. Adaptive duplicate detection using learnable string similarity measures. Proceedings of the International Conference on Knowledge Discovery and Data Mining (SIGKDD\u201903) (2003), 39--48."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1456650.1456651"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218001493000339"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007379606734"},{"key":"e_1_2_1_10_1","first-page":"300","article-title":"Comprehensive survey on distance\/similarity measures between probability density functions","volume":"1","author":"Cha Sung-Hyuk","year":"2007","unstructured":"Sung-Hyuk Cha . 2007 . Comprehensive survey on distance\/similarity measures between probability density functions . Int. J. Math. Models Methods Appl. Sci. 1 (2007), 300 -- 307 . Sung-Hyuk Cha. 2007. Comprehensive survey on distance\/similarity measures between probability density functions. Int. J. Math. Models Methods Appl. Sci. 1 (2007), 300--307.","journal-title":"Int. J. Math. Models Methods Appl. Sci."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.202"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/1378245.1378260"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401913"},{"key":"e_1_2_1_14_1","volume-title":"Entity Resolution, and Duplicate Detection","author":"Christen Peter","unstructured":"Peter Christen . 2012. Data Matching\u2014Concepts and Techniques for Record Linkage , Entity Resolution, and Duplicate Detection . Springer . Peter Christen. 2012. Data Matching\u2014Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection. Springer."},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the International Workshop on Information Integration on the Web (IIWeb\u201903)","author":"Cohen William W.","unstructured":"William W. Cohen , Pradeep Ravikumar , and Stephen E. Fienberg . 2003. A comparison of string distance metrics for name-matching tasks . In Proceedings of the International Workshop on Information Integration on the Web (IIWeb\u201903) . AAAI Press, 73--78. William W. Cohen, Pradeep Ravikumar, and Stephen E. Fienberg. 2003. A comparison of string distance metrics for name-matching tasks. In Proceedings of the International Workshop on Information Integration on the Web (IIWeb\u201903). AAAI Press, 73--78."},{"key":"e_1_2_1_16_1","unstructured":"Sanjib Das AnHai Doan Suganthan G. C. Paul Chaitanya Gokhale and Pradap Konda. [n.d.]. The Magellan Data Repository. Retrieved from https:\/\/sites.google.com\/site\/anhaidgroup\/useful-stuff\/data.  Sanjib Das AnHai Doan Suganthan G. C. Paul Chaitanya Gokhale and Pradap Konda. [n.d.]. The Magellan Data Repository. Retrieved from https:\/\/sites.google.com\/site\/anhaidgroup\/useful-stuff\/data."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2187836.2187900"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.2307\/1932409"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/3236187.3236198"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2002.994694"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2007.250581"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3329859.3329877"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.14778\/2876473.2876474"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS\u201910)","author":"Glorot Xavier","year":"2010","unstructured":"Xavier Glorot and Yoshua Bengio . 2010 . Understanding the difficulty of training deep feedforward neural networks . In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS\u201910) . 249--256. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS\u201910). 249--256."},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the Australasian Conference on Data Mining and Analystics. Australian Computer Society, Inc., 23--31","author":"Goiser Karl","year":"2006","unstructured":"Karl Goiser and Peter Christen . 2006 . Towards automated record linkage . In Proceedings of the Australasian Conference on Data Mining and Analystics. Australian Computer Society, Inc., 23--31 . Karl Goiser and Peter Christen. 2006. Towards automated record linkage. In Proceedings of the Australasian Conference on Data Mining and Analystics. Australian Computer Society, Inc., 23--31."},{"key":"e_1_2_1_26_1","volume-title":"Deep Learning","author":"Goodfellow Ian","unstructured":"Ian Goodfellow , Yoshua Bengio , and Aaron Courville . 2016. Deep Learning . MIT Press . Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press."},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the KDD Workshop on Big Data as a Service (BIGDAS\u201917)","author":"Govind Yash","unstructured":"Yash Govind , Erik Paulson , Mukilan Ashok , Suganthan G. C. Paul , Ali Hitawala , AnHai Doan , Youngchoon Park , Peggy L Peissig , Eric LaRose , and Jonathan C. Badger . 2017. Cloudmatcher: A cloud\/crowd service for entity matching . In Proceedings of the KDD Workshop on Big Data as a Service (BIGDAS\u201917) . Yash Govind, Erik Paulson, Mukilan Ashok, Suganthan G. C. Paul, Ali Hitawala, AnHai Doan, Youngchoon Park, Peggy L Peissig, Eric LaRose, and Jonathan C. Badger. 2017. Cloudmatcher: A cloud\/crowd service for entity matching. In Proceedings of the KDD Workshop on Big Data as a Service (BIGDAS\u201917)."},{"key":"e_1_2_1_28_1","volume-title":"Baxter","author":"Gu Lifang","year":"2006","unstructured":"Lifang Gu and Rohan A . Baxter . 2006 . Decision models for record linkage. In Data Mining\u2014Theory, Methodology, Techniques, and Applications. Springer , 146--160. Lifang Gu and Rohan A. Baxter. 2006. Decision models for record linkage. In Data Mining\u2014Theory, Methodology, Techniques, and Applications. Springer, 146--160."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.100"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_1_31_1","first-page":"241","article-title":"Distribution de la flore alpine dans le bassin des Dranses et dans quelques r\u00e9gions voisines","volume":"37","author":"Jaccard Paul","year":"1901","unstructured":"Paul Jaccard . 1901 . Distribution de la flore alpine dans le bassin des Dranses et dans quelques r\u00e9gions voisines . Bulletin de la Soci\u00e9t\u00e9 Vaudoise des Sciences Naturelles 37 (1901), 241 -- 272 . Paul Jaccard. 1901. Distribution de la flore alpine dans le bassin des Dranses et dans quelques r\u00e9gions voisines. Bulletin de la Soci\u00e9t\u00e9 Vaudoise des Sciences Naturelles 37 (1901), 241--272.","journal-title":"Bulletin de la Soci\u00e9t\u00e9 Vaudoise des Sciences Naturelles"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2009.191"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1586"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the International Conference for Learning Representations (ICLR\u201915)","author":"Diederik","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization . Proceedings of the International Conference for Learning Representations (ICLR\u201915) . Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. Proceedings of the International Conference for Learning Representations (ICLR\u201915)."},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the International Conference on Machine Learning (ICML\u201915)","author":"Koch Gregory","year":"2015","unstructured":"Gregory Koch , Richard Zemel , and Ruslan Salakhutdinov . 2015 . Siamese neural networks for one-shot image recognition . In Proceedings of the International Conference on Machine Learning (ICML\u201915) Deep Learning Workshop. Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. 2015. Siamese neural networks for one-shot image recognition. In Proceedings of the International Conference on Machine Learning (ICML\u201915) Deep Learning Workshop."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1072"},{"key":"e_1_2_1_37_1","first-page":"707","article-title":"Binary codes capable of correcting deletions, insertions, and reversals","volume":"10","author":"Levenshtein Vladimir I.","year":"1966","unstructured":"Vladimir I. Levenshtein . 1966 . Binary codes capable of correcting deletions, insertions, and reversals . Soviet Phys. Doklady 10 , 8 (1966), 707 -- 710 . Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Phys. Doklady 10, 8 (1966), 707--710.","journal-title":"Soviet Phys. Doklady"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2016.7899663"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the Conference on Neural Information Processing Systems (NIPS\u201913)","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov , Ilya Sutskever , Kai Chen , Gregory S. Corrado , and Jeffrey Dean . 2013 . Distributed representations of words and phrases and their compositionality . In Proceedings of the Conference on Neural Information Processing Systems (NIPS\u201913) . 3111--3119. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the Conference on Neural Information Processing Systems (NIPS\u201913). 3111--3119."},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of the International Conference on Knowledge Discovery and Data Mining (SIGKDD\u201996)","author":"Alvaro","unstructured":"Alvaro E. Monge and Charles P. Elkan. 1996. The field matching problem: Algorithms and applications . In Proceedings of the International Conference on Knowledge Discovery and Data Mining (SIGKDD\u201996) . 267--270. Alvaro E. Monge and Charles P. Elkan. 1996. The field matching problem: Algorithms and applications. In Proceedings of the International Conference on Knowledge Discovery and Data Mining (SIGKDD\u201996). 267--270."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3196926"},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of the National Conference on Artificial Intelligence (AAAI\u201916)","author":"Mueller Jonas","year":"2016","unstructured":"Jonas Mueller and Aditya Thyagarajan . 2016 . Siamese recurrent architectures for learning sentence similarity . In Proceedings of the National Conference on Artificial Intelligence (AAAI\u201916) . 2786--2792. Jonas Mueller and Aditya Thyagarajan. 2016. Siamese recurrent architectures for learning sentence similarity. In Proceedings of the National Conference on Artificial Intelligence (AAAI\u201916). 2786--2792."},{"key":"e_1_2_1_43_1","volume-title":"An Introduction to Duplicate Detection","author":"Naumann Felix","unstructured":"Felix Naumann and Melanie Herschel . 2010. An Introduction to Duplicate Detection . Morgan and Claypool Publishers . Felix Naumann and Melanie Herschel. 2010. An Introduction to Duplicate Detection. Morgan and Claypool Publishers."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.14778\/3231751.3231759"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-1617"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2396761.2398606"},{"key":"e_1_2_1_47_1","unstructured":"M. Odell and R. Russell. 1918. The soundex coding system. U.S. Patents 1261167 (1918).  M. Odell and R. Russell. 1918. The soundex coding system. U.S. Patents 1261167 (1918)."},{"key":"e_1_2_1_48_1","unstructured":"Razvan Pascanu Tomas Mikolov and Yoshua Bengio. 2012. Understanding the exploding gradient problem. CoRR abs\/1211.5063. http:\/\/arxiv.org\/abs\/1211.5063.  Razvan Pascanu Tomas Mikolov and Yoshua Bengio. 2012. Understanding the exploding gradient problem. CoRR abs\/1211.5063. http:\/\/arxiv.org\/abs\/1211.5063."},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201914)","author":"Pennington Jeffrey","unstructured":"Jeffrey Pennington , Richard Socher , and Christopher D. Manning . 2014. Glove: Global vectors for word representation . In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201914) . 1532--1543. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201914). 1532--1543."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775087"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/78.650093"},{"key":"e_1_2_1_52_1","volume-title":"Adams","author":"Snoek Jasper","year":"2012","unstructured":"Jasper Snoek , Hugo Larochelle , and Ryan P . Adams . 2012 . Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems. MIT Press , 2960--2968. Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. 2012. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems. MIT Press, 2960--2968."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2670313"},{"key":"e_1_2_1_54_1","volume-title":"Le","author":"Sutskever Ilya","year":"2014","unstructured":"Ilya Sutskever , Oriol Vinyals , and Quoc V . Le . 2014 . Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems. MIT Press , 3104--3112. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems. MIT Press, 3104--3112."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775099"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2015.7113286"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350263"},{"key":"e_1_2_1_58_1","volume-title":"Winkler and Yves Thibaudeau","author":"William","year":"1991","unstructured":"William E. Winkler and Yves Thibaudeau . 1991 . An application of the Fellegi-Sunter model of record linkage to the 1990 U.S. decennial census. U.S. Bureau of the Census (1991), 1--22. William E. Winkler and Yves Thibaudeau. 1991. An application of the Fellegi-Sunter model of record linkage to the 1990 U.S. decennial census. U.S. Bureau of the Census (1991), 1--22."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313578"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.14778\/2994509.2994534"}],"container-title":["Journal of Data and Information Quality"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3410157","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3410157","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:40:59Z","timestamp":1750200059000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3410157"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,13]]},"references-count":60,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,3,31]]}},"alternative-id":["10.1145\/3410157"],"URL":"https:\/\/doi.org\/10.1145\/3410157","relation":{},"ISSN":["1936-1955","1936-1963"],"issn-type":[{"value":"1936-1955","type":"print"},{"value":"1936-1963","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,13]]},"assertion":[{"value":"2019-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-01-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}