{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,22]],"date-time":"2026-03-22T16:03:26Z","timestamp":1774195406505,"version":"3.50.1"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2012,9,1]],"date-time":"2012-09-01T00:00:00Z","timestamp":1346457600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100002428","name":"Austrian Science Fund","doi-asserted-by":"publisher","award":["P20269"],"award-info":[{"award-number":["P20269"]}],"id":[{"id":"10.13039\/501100002428","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2012,9]]},"abstract":"<jats:p>\n            Algorithms for constructing hierarchical structures from user-generated metadata have caught the interest of the academic community in recent years. In social tagging systems, the output of these algorithms is usually referred to as folksonomies (from folk-generated taxonomies). Evaluation of folksonomies and folksonomy induction algorithms is a challenging issue complicated by the lack of golden standards, lack of comprehensive methods and tools as well as a lack of research and empirical\/simulation studies applying these methods. In this article, we report results from a broad comparative study of state-of-the-art folksonomy induction algorithms that we have applied and evaluated in the context of five social tagging systems. In addition to adopting\n            <jats:italic>semantic<\/jats:italic>\n            evaluation techniques, we present and adopt a new technique that can be used to evaluate the usefulness of folksonomies for\n            <jats:italic>navigation<\/jats:italic>\n            . Our work sheds new light on the properties and characteristics of state-of-the-art folksonomy induction algorithms and introduces a new pragmatic approach to folksonomy evaluation, while at the same time identifying some important limitations and challenges of folksonomy evaluation. Our results show that folksonomy induction algorithms specifically developed to capture intuitions of social tagging systems outperform traditional hierarchical clustering techniques. To the best of our knowledge, this work represents the largest and most comprehensive evaluation study of state-of-the-art folksonomy induction algorithms to date.\n          <\/jats:p>","DOI":"10.1145\/2337542.2337559","type":"journal-article","created":{"date-parts":[[2012,10,12]],"date-time":"2012-10-12T20:56:02Z","timestamp":1350075362000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":30,"title":["Evaluation of Folksonomy Induction Algorithms"],"prefix":"10.1145","volume":"3","author":[{"given":"Markus","family":"Strohmaier","sequence":"first","affiliation":[{"name":"Graz University of Technology, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Denis","family":"Helic","sequence":"additional","affiliation":[{"name":"Graz University of Technology, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dominik","family":"Benz","sequence":"additional","affiliation":[{"name":"University of Kassel, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christian","family":"K\u00f6rner","sequence":"additional","affiliation":[{"name":"Graz University of Technology, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Roman","family":"Kern","sequence":"additional","affiliation":[{"name":"Know-Center Graz, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2012,9]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.socnet.2005.01.007"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.64.046135"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the International Semantic Web Conference (ISWC\u201908)","author":"Angeletou S.","year":"2010"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1557914.1557958"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 2nd Web Science Conference (WebSci\u201910)","author":"Benz D."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1038\/nphys1130"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms1063"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 4th Workshop on Evaluating Ontologies for the Web (EON\u201906)","author":"Brank J."},{"key":"e_1_2_1_9_1","first-page":"245","article-title":"Network properties of folksonomies","volume":"20","author":"Cattuto C.","year":"2007","journal-title":"AI Comm."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-88564-1_39"},{"key":"e_1_2_1_11_1","unstructured":"Dellschaft K. 2005. Measuring the similarity of concept hierarchies and its inuence on the evaluation of learning procedures. M.S. thesis Institute for Computer Science University of Koblenz-Landau Germany. Dellschaft K. 2005. Measuring the similarity of concept hierarchies and its inuence on the evaluation of learning procedures. M.S. thesis Institute for Computer Science University of Koblenz-Landau Germany."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/11926078_17"},{"key":"e_1_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Dhillon I. Fan J. and Guan Y. 2001. Efficient clustering of very large document collections. In Data Mining for Scientific and Engineering Applications R. Grossman C. Kamath and R. Naburu Eds. Kluwer. Dhillon I. Fan J. and Guan Y. 2001. Efficient clustering of very large document collections. In Data Mining for Scientific and Engineering Applications R. Grossman C. Kamath and R. Naburu Eds. Kluwer.","DOI":"10.1007\/978-1-4615-1733-7_20"},{"key":"e_1_2_1_14_1","doi-asserted-by":"crossref","unstructured":"Frey B. J. J. and Dueck D. 2007. Clustering by passing messages between data points. Science 315 5814 972--976. Frey B. J. J. and Dueck D. 2007. Clustering by passing messages between data points. Science 315 5814 972--976.","DOI":"10.1126\/science.1136800"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063655"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/SocialCom.2010.31"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1963405.1963465"},{"key":"e_1_2_1_18_1","unstructured":"Heymann P. and Garcia-Molina H. 2006. Collaborative creation of communal hierarchical taxonomies in social tagging systems. Tech. rep. 2006-10 Stanford InfoLab. Heymann P. and Garcia-Molina H. 2006. Collaborative creation of communal hierarchical taxonomies in social tagging systems. Tech. rep. 2006-10 Stanford InfoLab."},{"key":"e_1_2_1_19_1","volume-title":"Bibsonomy: A social bookmark and publication sharing system. In Proceedings of the Conceptual Structures Tool Interoperability Workshop at the 14th International Conference on Conceptual Structures","author":"Hotho A.","year":"2006"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the Special Interest Group on Information Retrieval. Gesellschaft F\u00fcr Informatik","author":"Hotho A."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/335305.335325"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1038\/35022643"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS). MIT Press","author":"Kleinberg J. M.","year":"2001"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the International Congress of Mathematicians. European Mathematical Society Publishing House","author":"Kleinberg J.","year":"2006"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772744"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.82.036106"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.73.026120"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242700"},{"key":"e_1_2_1_29_1","doi-asserted-by":"crossref","unstructured":"Maedche A. 2002. Ontology Learning for the Semantic Web. Kluwer Boston. Maedche A. 2002. Ontology Learning for the Semantic Web . Kluwer Boston.","DOI":"10.1007\/978-1-4615-0925-7"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.212348399"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2006.11.002"},{"key":"e_1_2_1_32_1","first-page":"60","article-title":"The small world problem","volume":"1","author":"Milgram S.","year":"1967","journal-title":"Psych. Today"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/219717.219748"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the AAAI Workshop on Statistical Relational AI. AAAI","author":"Plangprasopchok A."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835804.1835924"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the 22nd Conference on the Advancement of Artificial Intelligence. AAAI Press","author":"Ponzetto S. P."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CSE.2009.93"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718487.1718521"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the 10th IFCS Conference on Studies in Classification, Data Analysis, and Knowledge Organization. Springer, 261--270","author":"Schmitz C."},{"key":"e_1_2_1_40_1","first-page":"7","article-title":"Self-similarity of complex networks and hidden metric spaces","volume":"100","author":"Serrano M. A.","year":"2008","journal-title":"Phys. Rev. Lett."},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of the International AAAI Conference on Weblogs and Social Media (ICWSM\u201910)","author":"Strohmaier M."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242667"},{"key":"e_1_2_1_43_1","unstructured":"Vander Wal T. 2007. Folksonomy coinage and definition. http:\/\/vanderwal.net\/folksonomy.html. Vander Wal T. 2007. Folksonomy coinage and definition. http:\/\/vanderwal.net\/folksonomy.html."},{"key":"e_1_2_1_44_1","doi-asserted-by":"crossref","unstructured":"Watts D. J. and Strogatz S. H. 1998. Collective dynamics of small-world networks. Nature 393 6684 440--442. Watts D. J. and Strogatz S. H. 1998. Collective dynamics of small-world networks. Nature 393 6684 440--442.","DOI":"10.1038\/30918"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1070120"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/WIIAT.2008.269"},{"key":"e_1_2_1_47_1","volume-title":"Proceedings of the IEEE International Joint Conference on Neural Networks.","volume":"5","author":"Zhong S.","year":"2005"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2337542.2337559","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2337542.2337559","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T08:49:00Z","timestamp":1750236540000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2337542.2337559"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,9]]},"references-count":47,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2012,9]]}},"alternative-id":["10.1145\/2337542.2337559"],"URL":"https:\/\/doi.org\/10.1145\/2337542.2337559","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,9]]},"assertion":[{"value":"2011-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-09-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}