{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,15]],"date-time":"2024-09-15T22:56:19Z","timestamp":1726440979264},"reference-count":42,"publisher":"MIT Press","issue":"4","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The situated spatial presence of musical instruments has been well studied in the fields of acoustics and music perception research, but so far it has not been the focus of human\u2013AI interaction. We respond critically to this trend by seeking to reembody interactive electronics using data derived from natural acoustic phenomena. Two musical works, composed for human soloist and computer-generated live electronics, are intended to situate the listener in an immersive sonic environment in which real and virtual sources blend seamlessly. To do so, we experimented with two contrasting reproduction setups: a surrounding Ambisonic loudspeaker dome and a compact spherical loudspeaker array for radiation synthesis. A large database of measured radiation patterns of orchestral instruments served as a training set for machine learning models to control spatially rich 3-D patterns for electronic sounds. These are exploited during performance in response to live sounds captured with a spherical microphone array and used to train computer models of improvisation and to trigger corpus-based spatial synthesis. We show how AI techniques are useful to utilize complex, multidimensional, spatial data in the context of computer-assisted composition and human\u2013computer interactive improvisation.<\/jats:p>","DOI":"10.1162\/comj_a_00664","type":"journal-article","created":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T18:41:30Z","timestamp":1705516890000},"page":"43-61","update-policy":"http:\/\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":1,"title":["Embodying Spatial Sound Synthesis with AI in Two Compositions for\n                    Instruments and 3-D Electronics"],"prefix":"10.1162","volume":"46","author":[{"given":"Aaron","family":"Einbond","sequence":"first","affiliation":[{"name":"Department of Performing Arts City, University of London Northampton Square, London EC1V 0HB, UK Aaron.Einbond@city.ac.uk"}]},{"given":"Thibaut","family":"Carpentier","sequence":"additional","affiliation":[{"name":"Institut de Recherche et de Coordination Acoustique\/Musique (IRCAM) STMS Lab, CNRS, Sorbonne Universit\u00e9, Minist\u00e8re de la Culture 1, place Igor Stravinsky, 75004 Paris, France thibaut.carpentier@ircam.fr"}]},{"given":"Diemo","family":"Schwarz","sequence":"additional","affiliation":[{"name":"Institut de Recherche et de Coordination Acoustique\/Musique (IRCAM) STMS Lab, CNRS, Sorbonne Universit\u00e9, Minist\u00e8re de la Culture 1, place Igor Stravinsky, 75004 Paris, France diemo.schwarz@ircam.fr"}]},{"given":"Jean","family":"Bresson","sequence":"additional","affiliation":[{"name":"Ableton Sch\u00f6nhauser Allee 6\u20137, 10119 Berlin, Germany jean.bresson@ircam.fr"}]}],"member":"281","published-online":{"date-parts":[[2022,12,1]]},"reference":[{"key":"2024041813192429800_B1","first-page":"291","article-title":"Factor Oracle: A New Structure for\n                        Pattern Matching","author":"Allauzen","year":"1999","journal-title":"Proceedings of\n                            the Annual Conference on Current Trends in Theory and Practice of\n                            Informatics"},{"issue":"9","key":"2024041813192429800_B2","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1007\/s00500-004-0385-4","article-title":"Using Factor Oracles for Machine\n                        Improvisation","volume":"8","author":"Assayag","year":"2004","journal-title":"Soft\n                            Computing"},{"key":"2024041813192429800_B3","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1145\/1178723.1178742","article-title":"OMax Brothers: A Dynamic Topology of\n                        Agents for Improvization Learning","author":"Assayag","year":"2006","journal-title":"Proceedings of the ACM Workshop on Audio and Music\n                            Computing Multimedia"},{"key":"2024041813192429800_B4","first-page":"253","article-title":"Next-Generation Computer-Aided\n                        Composition Environment: A New Implementation of\n                    OpenMusic","author":"Bresson","year":"2017","journal-title":"Proceedings of the\n                            International Computer Music Conference"},{"key":"2024041813192429800_B5","first-page":"184","article-title":"A New Implementation of Spat in\n                        Max","author":"Carpentier","year":"2018","journal-title":"Proceedings of the Sound and\n                            Music Computing Conference"},{"key":"2024041813192429800_B6","doi-asserted-by":"publisher","DOI":"10.1051\/aacus\/2023033","article-title":"Spherical Correlation as a\n                        Similarity Measure for 3D Radiation Patterns of Musical\n                        Instruments","author":"Carpentier","year":"2023","journal-title":"Acta Acustica\n                            7"},{"key":"2024041813192429800_B7","first-page":"223","article-title":"Corpus-Based Transcription as an\n                        Approach to the Compositional Control of Timbre","author":"Einbond","year":"2009","journal-title":"Proceedings of the International Computer Music\n                            Conference"},{"key":"2024041813192429800_B8","first-page":"141","article-title":"Introducing CatOracle: Corpus-based\n                        Concatenative Improvisation with the Audio Oracle\n                    Algorithm","author":"Einbond","year":"2016","journal-title":"Proceedings of the\n                            International Computer Music Conference"},{"key":"2024041813192429800_B9","first-page":"148","article-title":"Instrumental Radiation Patterns as\n                        Models for Corpus-based Spatial Sound Synthesis: \u2018Cosmologies\u2019\n                        for Piano and 3D Electronics","author":"Einbond","year":"2021","journal-title":"Proceedings of the International Computer Music\n                            Conference"},{"key":"2024041813192429800_B10","article-title":"Complementary N-Band IIR\n                        Filterbank Based on 2-Band Complementary Filters","author":"Favrot","year":"2010","journal-title":"Proceedings of the International Workshop on Acoustic\n                            Echo and Noise Control"},{"key":"2024041813192429800_B11","first-page":"287","article-title":"Probabilistic Models for Designing\n                        Motion and Sound Relationships","author":"Fran\u00e7oise","year":"2014","journal-title":"Proceedings of the International Conference on New\n                            Interfaces for Musical Expression"},{"issue":"1","key":"2024041813192429800_B12","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1080\/09298215.2016.1230632","article-title":"Interactive-Compositional\n                        Authoring of Sound Spatialization","volume":"46","author":"Garcia","year":"2016","journal-title":"Journal of New Music Research"},{"issue":"4","key":"2024041813192429800_B13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/comj_a_00658","article-title":"Composing the Assemblage: Probing Aesthetic and\n                        Technical Dimensions of Artistic Creation with Machine\n                        Learning","volume":"46","author":"Gioti","year":"2022","journal-title":"Computer Music\n                            Journal"},{"key":"2024041813192429800_B14","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1145\/3561212.3561232","article-title":"The iOSCahedron: Developing a Hybrid\n                        Spatialization Instrument","author":"Goeschke","year":"2022","journal-title":"Proceedings of the International Audio Mostly\n                            Conference"},{"issue":"5","key":"2024041813192429800_B15","doi-asserted-by":"publisher","first-page":"1270","DOI":"10.1121\/1.381428","article-title":"Multidimensional Perceptual Scaling of\n                        Musical Timbres","volume":"61","author":"Grey","year":"1977","journal-title":"The Journal of the\n                            Acoustical Society of America"},{"issue":"1","key":"2024041813192429800_B16","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1162\/comj_a_00395","article-title":"Textural Composition: Aesthetics,\n                        Techniques, and Spatialization for High-Density Loud-Speaker\n                        Arrays","volume":"41","author":"Hagan","year":"2017","journal-title":"Computer Music\n                            Journal"},{"key":"2024041813192429800_B17","article-title":"Similarity of Musical Instrument\n                        Radiation-Patterns in Pitch and Partial","author":"Hohl","year":"2010","journal-title":"Tagungsband der deutschen Arbeitsgemeinschaft f\u00fcr\n                            Akustik \u201cFortschritte der\n                    Akustik\u201d"},{"key":"2024041813192429800_B18","article-title":"Spatial Transformations for the\n                        Enhancement of Ambisonic Recordings","author":"Kronlachner","year":"2014","journal-title":"Proceedings of the International Conference on Spatial\n                            Audio"},{"issue":"3","key":"2024041813192429800_B19","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1080\/07494469900640381","article-title":"Interacting with Latter-Day Musical\n                        Automata","volume":"18","author":"Lewis","year":"1999","journal-title":"Contemporary Music\n                            Review"},{"issue":"3","key":"2024041813192429800_B20","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1162\/014892699559797","article-title":"Perspectives on the Contribution of\n                        Timbre to Musical Structure","volume":"23","author":"McAdams","year":"1999","journal-title":"Computer Music Journal"},{"issue":"12","key":"2024041813192429800_B21","first-page":"1115","article-title":"Spatial Impulse Response Rendering\n                        I: Analysis and Synthesis","volume":"53","author":"Merimaa","year":"2005","journal-title":"Journal\n                            of the Audio Engineering Society"},{"key":"2024041813192429800_B22","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-09517-2","volume-title":"Acoustics and the Performance of Music:\n                            Manual for Acousticians, Audio Engineers, Musicians, Architects and\n                            Musical Instruments Makers","author":"Meyer","year":"2009","edition":"5th"},{"key":"2024041813192429800_B23","article-title":"3D Sound Field Recording with\n                        Higher Order Ambisonics: Objective Measurements and Validation of Spherical\n                        Microphone","author":"Moreau","year":"2006","journal-title":"Proceedings of the\n                            120th Convention of the Audio Engineering Society"},{"key":"2024041813192429800_B24","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1142\/9789814299312_0028","article-title":"Reconstructing Sound Source Directivity in Virtual\n                        Acoustic Environments","volume-title":"Principles and Applications of Spatial\n                        Hearing","author":"Noisternig","year":"2011"},{"issue":"3","key":"2024041813192429800_B25","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1017\/S1355771809990094","article-title":"Timbre Spatialisation: The Medium Is\n                        the Space","volume":"14","author":"Normandeau","year":"2009","journal-title":"Organised\n                        Sound"},{"article-title":"A Large Set of Audio Features for\n                        Sound Description (Similarity and Classification) in the Cuidado\n                        Project","year":"2004","author":"Peeters","key":"2024041813192429800_B26"},{"key":"2024041813192429800_B27","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-99561-8","volume-title":"Fundamentals of Spherical Array\n                            Processing","author":"Rafaely","year":"2019","edition":"2nd"},{"key":"2024041813192429800_B28","first-page":"347","article-title":"DIRTI: Dirty Tangible\n                        Interfaces","author":"Savary","year":"2012","journal-title":"Proceedings of the\n                            International Conference on New Interfaces for Musical\n                            Expression"},{"key":"2024041813192429800_B29","article-title":"An Exploration of Design Parameters\n                        for Human-Interactive Systems with Compact Spherical Loudspeaker\n                        Arrays","author":"Schmeder","year":"2009","journal-title":"Proceedings of the\n                            Ambisonics Symposium"},{"key":"2024041813192429800_B30","first-page":"423","article-title":"MuBu and Friends: Assembling Tools for\n                        Content Based Real-Time Interactive Audio Processing in\n                        Max\/MSP","author":"Schnell","year":"2009","journal-title":"Proceedings of the\n                            International Computer Music Conference"},{"key":"2024041813192429800_B31","first-page":"50","article-title":"IAE and IAEOU","volume-title":"Topophonie Research Project: Audiographic Cluster\n                            Navigation (2009\u20132012)","author":"Schnell","year":"2012"},{"issue":"1","key":"2024041813192429800_B32","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1080\/09298210600696857","article-title":"Concatenative Sound Synthesis: The\n                        Early Years","volume":"35","author":"Schwarz","year":"2006","journal-title":"Journal of New Music\n                            Research"},{"issue":"2","key":"2024041813192429800_B33","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1109\/MSP.2007.323274","article-title":"Corpus-Based Concatenative\n                        Synthesis","volume":"24","author":"Schwarz","year":"2007","journal-title":"IEEE Signal Processing\n                            Magazine"},{"key":"2024041813192429800_B34","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.1180593","article-title":"The Sound Space as Musical Instrument:\n                        Playing Corpus-Based Concatenative Synthesis","author":"Schwarz","year":"2012","journal-title":"Proceedings of the International Conference on New\n                            Interfaces for Musical Expression"},{"issue":"2","key":"2024041813192429800_B35","doi-asserted-by":"publisher","first-page":"1246","DOI":"10.1121\/1.4976071","article-title":"Generation and Analysis of an Acoustic\n                        Radiation Pattern Database for Forty-One Musical\n                    Instruments","volume":"141","author":"Shabtai","year":"2017","journal-title":"Journal of the Acoustical\n                            Society of America"},{"key":"2024041813192429800_B36","first-page":"114","article-title":"Feature Selection and Composition\n                        Using PyOracle","author":"Surges","year":"2013","journal-title":"Proceedings of\n                            the Artificial Intelligence and Interactive Digital Entertainment\n                            Conference"},{"key":"2024041813192429800_B37","doi-asserted-by":"publisher","first-page":"84","DOI":"10.5281\/zenodo.3249264","article-title":"OM-AI: A Toolkit to Support\n                        AI-Based Computer-Assisted Composition Workflows in\n                        OpenMusic","author":"Vinjar","year":"2019","journal-title":"Proceedings of the\n                            Sound and Music Computing Conference"},{"key":"2024041813192429800_B38","doi-asserted-by":"publisher","DOI":"10.14279\/depositonce-5861.2","article-title":"A Database of Anechoic Microphone\n                        Array Measurements of Musical Instruments: Recordings, Directivities, and\n                        Audio Features","author":"Weinzierl","year":"2017"},{"issue":"2","key":"2024041813192429800_B39","doi-asserted-by":"publisher","first-page":"45","DOI":"10.2307\/3680283","article-title":"Timbre Space as a Musical Control\n                        Structure","volume":"3","author":"Wessel","year":"1979","journal-title":"Computer Music\n                            Journal"},{"article-title":"Analysis and Synthesis of\n                        Sound-Radiation with Spherical Arrays","year":"2009","author":"Zotter","key":"2024041813192429800_B40"},{"key":"2024041813192429800_B41","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-17207-7","volume-title":"Ambisonics: A Practical 3D Audio Theory for\n                            Recording, Studio Production, Sound Reinforcement, and Virtual\n                            Reality","author":"Zotter","year":"2019"},{"issue":"3","key":"2024041813192429800_B42","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1162\/comj_a_00429","article-title":"A Beamformer to Play with Wall\n                        Reflections: The Icosahedral Loudspeaker","volume":"41","author":"Zotter","year":"2017","journal-title":"Computer Music Journal"}],"container-title":["Computer Music Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/comj\/article-pdf\/46\/4\/43\/2364570\/comj_a_00664.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/comj\/article-pdf\/46\/4\/43\/2364570\/comj_a_00664.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,18]],"date-time":"2024-04-18T13:20:08Z","timestamp":1713446408000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/comj\/article\/46\/4\/43\/119128\/Embodying-Spatial-Sound-Synthesis-with-AI-in-Two"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022]]},"references-count":42,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,12,1]]},"published-print":{"date-parts":[[2022,12,1]]}},"URL":"https:\/\/doi.org\/10.1162\/comj_a_00664","relation":{},"ISSN":["0148-9267","1531-5169"],"issn-type":[{"type":"print","value":"0148-9267"},{"type":"electronic","value":"1531-5169"}],"subject":[],"published-other":{"date-parts":[[2022]]},"published":{"date-parts":[[2022]]}}}