{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T13:00:23Z","timestamp":1781614823461,"version":"3.54.5"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,7,31]],"date-time":"2024-07-31T00:00:00Z","timestamp":1722384000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Comput. Cult. Herit."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>\n            The availability of digital music data in various modalities provides opportunities both for music enjoyment and music research. Regarding the latter, the computer-assisted analysis of tonal structures is a central topic. For Western classical music, studies typically rely on machine-readable scores, which are tedious to create for large-scale works and comprehensive corpora. As an alternative, music audio recordings, which are readily available, can be analyzed with computational methods. With this article, we want to bridge the gap between score- and audio-based measurements of tonal structures by leveraging the power of deep neural networks. Such networks are commonly trained in an end-to-end fashion, which introduces biases towards the training repertoire or towards specific annotators. To overcome these problems, we propose a multi-step strategy. First, we compute pitch-class representations of the audio recordings using networks trained on score\u2013audio pairs. Second, we measure the presence of specific tonal structures using a pattern-matching technique that solely relies on music theory knowledge and does not require annotated training data. Third, we highlight these measurements with interactive visualizations, thus leaving the interpretation to the musicological experts. Our experiments on Richard Wagner's large-scale cycle\n            <jats:italic>Der Ring des Nibelungen<\/jats:italic>\n            indicate that deep pitch-class representations lead to a high similarity between score- and audio-based measurements of tonal structures, thus demonstrating how to leverage multi-modal data for application scenarios in the computational humanities, where an explicit and interpretable methodology is essential.\n          <\/jats:p>","DOI":"10.1145\/3659103","type":"journal-article","created":{"date-parts":[[2024,4,23]],"date-time":"2024-04-23T16:13:28Z","timestamp":1713888808000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["From Music Scores to Audio Recordings: Deep Pitch-Class Representations for Measuring Tonal Structures"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2143-4679","authenticated-orcid":false,"given":"Christof","family":"Wei\u00df","sequence":"first","affiliation":[{"name":"Center for Artificial Intelligence and Data Science, Universit\u00e4t W\u00fcrzburg, W\u00fcrzburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6062-7524","authenticated-orcid":false,"given":"Meinard","family":"M\u00fcller","sequence":"additional","affiliation":[{"name":"International Audio Laboratories Erlangen, Erlangen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,7,31]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics10060670"},{"key":"e_1_3_2_3_2","unstructured":"Lei J. Ba Jamie R. Kiros and Geoffrey E. Hinton. 2016. Layer normalization. arXiv:1607.06450. Retrieved from https:\/\/arxiv.org\/abs\/1607.06450"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2018.2869928"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.1417937"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2013.2295926"},{"key":"e_1_3_2_7_2","first-page":"125","volume-title":"Proceedings of the International Conference of Music Perception and Cognition (ICMPC \u201918)","author":"Cuesta Helena","year":"2018","unstructured":"Helena Cuesta, Emilia G\u00f3mez, Agust\u00edn Martorell, and Felipe Lo\u00e1iciga. 2018. Analysis of intonation in unison choir singing. In Proceedings of the International Conference of Music Perception and Cognition (ICMPC \u201918). 125\u2013130."},{"key":"e_1_3_2_8_2","volume-title":"Richard Wagner \u2013 Gesamtausgabe","author":"Dahlhaus Carl","year":"1970","unstructured":"Carl Dahlhaus, Klaus D\u00f6ge, and Egon Voss (Eds.). 1970\u20132020. Richard Wagner \u2013 Gesamtausgabe. Schott Music, Mainz."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.5555\/2209820.2210677"},{"key":"e_1_3_2_10_2","first-page":"167","volume-title":"Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201921)","author":"Edirisooriya Sachinda","year":"2021","unstructured":"Sachinda Edirisooriya, Hao-Wen Dong, Julian J. McAuley, and Taylor Berg-Kirkpatrick. 2021. An empirical evaluation of end-to-end polyphonic optical music recognition. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201921). Online, 167\u2013173."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.5555\/2209814.2210551"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2012.2190047"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2013.6637776"},{"key":"e_1_3_2_14_2","first-page":"464","volume-title":"Proceedings of the International Computer Music Conference (ICMC \u201999)","author":"Fujishima Takuya","year":"1999","unstructured":"Takuya Fujishima. 1999. Realtime chord recognition of musical sound: A system using common lisp music. In Proceedings of the International Computer Music Conference (ICMC \u201999). 464\u2013467."},{"key":"e_1_3_2_15_2","unstructured":"Zsolt G\u00e1rdonyi and Hubert Nordhoff. 2002. Harmonik (2nd ed.). M\u00f6seler Wolfenb\u00fcttel."},{"key":"e_1_3_2_16_2","first-page":"57","volume-title":"Proceedings of the International Computer Music Conference (ICMC \u201905)","author":"G\u00f3mez Emilia","year":"2005","unstructured":"Emilia G\u00f3mez and Jordi Bonada. 2005. Tonality visualization of polyphonic audio. In Proceedings of the International Computer Music Conference (ICMC \u201905). 57\u201360."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2017.4"},{"key":"e_1_3_2_18_2","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR \u201919)","author":"Hawthorne Curtis","year":"2019","unstructured":"Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi A. Huang, Sander Dieleman, Erich Elsen, Jesse H. Engel, and Douglas Eck. 2019. Enabling factorized piano music modeling and generation with the MAESTRO dataset. In Proceedings of the International Conference on Learning Representations (ICLR \u201919)."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_20_2","first-page":"475","volume-title":"Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201916)","author":"Kelz Rainer","year":"2016","unstructured":"Rainer Kelz, Matthias Dorfer, Filip Korzeniowski, Sebastian B\u00f6ck, Andreas Arzt, and Gerhard Widmer. 2016. On the potential of simple framewise approaches to piano transcription. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201916). 475\u2013481."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASL.2007.908129"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1080\/09298215.2019.1613436"},{"key":"e_1_3_2_23_2","first-page":"37","volume-title":"Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201916)","author":"Korzeniowski Filip","year":"2016","unstructured":"Filip Korzeniowski and Gerhard Widmer. 2016. Feature learning for chord recognition: The deep chroma extractor. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201916). 37\u201343."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2018.2856090"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3424911.3425515"},{"key":"e_1_3_2_26_2","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR \u201919)","author":"Loshchilov Ilya","year":"2019","unstructured":"Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In Proceedings of the International Conference on Learning Representations (ICLR \u201919)."},{"key":"e_1_3_2_27_2","first-page":"135","volume-title":"Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201910)","author":"Mauch Matthias","year":"2010","unstructured":"Matthias Mauch and Simon Dixon. 2010. Approximate note transcription for the improved identification of difficult chords. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201910). 135\u2013140."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.1414880"},{"key":"e_1_3_2_29_2","first-page":"8363507:1","article-title":"Score-informed source separation for multichannel orchestral recordings","author":"Miron Marius","year":"2016","unstructured":"Marius Miron, Julio J. Carabias-Orti, Juan J. Bosch, Emilia G\u00f3mez, and Jordi Janer. 2016. Score-informed source separation for multichannel orchestral recordings. Journal of Electrical and Computer Engineering 2016 (2016), 8363507:1\u20138363507:19.","journal-title":"Journal of Electrical and Computer Engineering"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.5555\/2209800.2210174"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASPAA.2005.1540223"},{"issue":"64","key":"e_1_3_2_32_2","first-page":"1","article-title":"Sync toolbox: A python package for efficient, robust, and accurate music synchronization","volume":"6","author":"M\u00fcller Meinard","year":"2021","unstructured":"Meinard M\u00fcller, Yigitcan \u00d6zer, Michael Krause, Thomas Pr\u00e4tzlich, and Jonathan Driedger. 2021. Sync toolbox: A python package for efficient, robust, and accurate music synchronization. Journal of Open Source Software (JOSS) 6, 64 (2021), 3434 1\u20134.","journal-title":"Journal of Open Source Software (JOSS)"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASL.2013.2280218"},{"key":"e_1_3_2_34_2","volume-title":"Proceedings of the International Computer Music Conference (ICMC \u201901)","author":"Purwins Hendrik","year":"2001","unstructured":"Hendrik Purwins, Benjamin Blankertz, and Klaus Obermayer. 2001. Constant Q profiles for tracking modulations in audio data. In Proceedings of the International Computer Music Conference (ICMC \u201901)."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2014.2310706"},{"key":"e_1_3_2_36_2","series-title":"Lecture Notes in Computer Science","first-page":"309","volume-title":"Proceedings of the 11th International Symposium on Computer Music Multidisciplinary Research (CMMR \u201915)","author":"Su Li","year":"2015","unstructured":"Li Su and Yi-Hsuan Yang. 2015. Escaping from the abyss of manual annotation: New methodology of building polyphonic datasets for automatic music transcription. In Proceedings of the 11th International Symposium on Computer Music Multidisciplinary Research (CMMR \u201915). R. Kronland-Martinet, M. Aramaki, and S. Ystad (Eds.), Lecture Notes in Computer Science. Springer, 309\u2013321."},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.2307\/40285812"},{"key":"e_1_3_2_38_2","volume-title":"The Cognition of Basic Musical Structures","author":"Temperley David","year":"2001","unstructured":"David Temperley. 2001. The Cognition of Basic Musical Structures. MIT Press."},{"key":"e_1_3_2_39_2","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR \u201917)","author":"Thickstun John","year":"2017","unstructured":"John Thickstun, Za\u00efd Harchaoui, and Sham M. Kakade. 2017. Learning features of music from scratch. In Proceedings of the International Conference on Learning Representations (ICLR \u201917)."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2020.2973831"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.5334\/tismir.161"},{"key":"e_1_3_2_42_2","first-page":"168","volume-title":"Proceedings of the Conference on Interdisciplinary Musicology (CIM \u201914)","author":"Wei\u00df Christof","year":"2014","unstructured":"Christof Wei\u00df and Julian Habryka. 2014. Chroma-based scale matching for audio tonality analysis. In Proceedings of the Conference on Interdisciplinary Musicology (CIM \u201914). 168\u2013173."},{"key":"e_1_3_2_43_2","first-page":"199","volume-title":"Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201920)","author":"Wei\u00df Christof","year":"2020","unstructured":"Christof Wei\u00df, Stephanie Klauk, Mark Gotham, Meinard M\u00fcller, and Rainer Kleinertz. 2020. Discourse not dualism: An interdisciplinary dialogue on sonata form in Beethoven's early piano sonatas. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201920). 199\u2013206."},{"key":"e_1_3_2_44_2","first-page":"754","volume-title":"Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201921)","author":"Wei\u00df Christof","year":"2021","unstructured":"Christof Wei\u00df and Geoffroy Peeters. 2021. Training deep pitch-class representations with a multi-label CTC loss. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201921). Online, 754\u2013761."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2022.3200547"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2020.3030485"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3429743"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.18420\/in2017_14"},{"key":"e_1_3_2_49_2","first-page":"746","volume-title":"Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201921)","author":"Wei\u00df Christof","year":"2021","unstructured":"Christof Wei\u00df, Johannes Zeitler, Tim Zunner, Florian Schuberth, and Meinard M\u00fcller. 2021. Learning pitch-class representations from score\u2013audio pairs of classical music. In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR \u201921). Online, 746\u2013753."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2018.2879399"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.4245400"},{"key":"e_1_3_2_52_2","first-page":"148","volume-title":"Proceedings of the AES International Conference on Semantic Audio","author":"Zalkow Frank","year":"2017","unstructured":"Frank Zalkow, Christof Wei\u00df, Thomas Pr\u00e4tzlich, Vlora Arifi-M\u00fcller, and Meinard M\u00fcller. 2017. A multi-version approach for transferring measure annotations between music recordings. In Proceedings of the AES International Conference on Semantic Audio. 148\u2013155."}],"container-title":["Journal on Computing and Cultural Heritage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3659103","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3659103","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:44:14Z","timestamp":1750290254000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3659103"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,31]]},"references-count":51,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3659103"],"URL":"https:\/\/doi.org\/10.1145\/3659103","relation":{},"ISSN":["1556-4673","1556-4711"],"issn-type":[{"value":"1556-4673","type":"print"},{"value":"1556-4711","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,31]]},"assertion":[{"value":"2023-01-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-22","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-31","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}