{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T18:33:16Z","timestamp":1772908396067,"version":"3.50.1"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,7,30]],"date-time":"2018-07-30T00:00:00Z","timestamp":1532908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,8,31]]},"abstract":"<jats:p>Although 360\u00b0 cameras ease the capture of panoramic footage, it remains challenging to add realistic 360\u00b0 audio that blends into the captured scene and is synchronized with the camera motion. We present a method for adding scene-aware spatial audio to 360\u00b0 videos in typical indoor scenes, using only a conventional mono-channel microphone and a speaker. We observe that the late reverberation of a room's impulse response is usually diffuse spatially and directionally. Exploiting this fact, we propose a method that synthesizes the directional impulse response between any source and listening locations by combining a synthesized early reverberation part and a measured late reverberation tail. The early reverberation is simulated using a geometric acoustic simulation and then enhanced using a frequency modulation method to capture room resonances. The late reverberation is extracted from a recorded impulse response, with a carefully chosen time duration that separates out the late reverberation from the early reverberation. In our validations, we show that our synthesized spatial audio matches closely with recordings using ambisonic microphones. Lastly, we demonstrate the strength of our method in several applications.<\/jats:p>","DOI":"10.1145\/3197517.3201391","type":"journal-article","created":{"date-parts":[[2018,7,31]],"date-time":"2018-07-31T15:56:23Z","timestamp":1533052583000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":33,"title":["Scene-aware audio for 360\u00b0 videos"],"prefix":"10.1145","volume":"37","author":[{"given":"Dingzeyu","family":"Li","sequence":"first","affiliation":[{"name":"Columbia University"}]},{"given":"Timothy R.","family":"Langlois","sequence":"additional","affiliation":[{"name":"Adobe Research"}]},{"given":"Changxi","family":"Zheng","sequence":"additional","affiliation":[{"name":"Columbia University"}]}],"member":"320","published-online":{"date-parts":[[2018,7,30]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980257"},{"key":"e_1_2_2_2_1","volume-title":"Numerical Sound Synthesis","author":"Bilbao Stefan","unstructured":"Stefan Bilbao . 2009. Numerical Sound Synthesis . John Wiley & Sons, Ltd. Stefan Bilbao. 2009. Numerical Sound Synthesis. John Wiley & Sons, Ltd."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982431"},{"key":"e_1_2_2_4_1","first-page":"640","article-title":"Room sizing and optimization at low frequencies","volume":"52","author":"Cox Trevor J","year":"2004","unstructured":"Trevor J Cox , Peter D'Antonio , and Mark R Avis . 2004 . Room sizing and optimization at low frequencies . Journal of the Audio Engineering Society 52 , 6 (2004), 640 -- 651 . Trevor J Cox, Peter D'Antonio, and Mark R Avis. 2004. Room sizing and optimization at low frequencies. Journal of the Audio Engineering Society 52, 6 (2004), 640--651.","journal-title":"Journal of the Audio Engineering Society"},{"key":"e_1_2_2_5_1","volume-title":"Springer handbook of acoustics","author":"Dunn F","unstructured":"F Dunn , WM Hartmann , DM Campbell , and Neville H Fletcher . 2015. Springer handbook of acoustics . Springer . F Dunn, WM Hartmann, DM Campbell, and Neville H Fletcher. 2015. Springer handbook of acoustics. Springer."},{"key":"e_1_2_2_6_1","volume-title":"Audio Engineering Society Convention 108","author":"Farina Angelo","year":"2000","unstructured":"Angelo Farina . 2000 . Simultaneous measurement of impulse response and distortion with a swept-sine technique . In Audio Engineering Society Convention 108 . Audio Engineering Society. Angelo Farina. 2000. Simultaneous measurement of impulse response and distortion with a swept-sine technique. In Audio Engineering Society Convention 108. Audio Engineering Society."},{"key":"e_1_2_2_7_1","volume-title":"Audio Engineering Society Convention 122","author":"Farina Angelo","year":"2007","unstructured":"Angelo Farina . 2007 . Advancements in Impulse Response Measurements by Sine Sweeps . In Audio Engineering Society Convention 122 . Audio Engineering Society. Angelo Farina. 2007. Advancements in Impulse Response Measurements by Sine Sweeps. In Audio Engineering Society Convention 122. Audio Engineering Society."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280818"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.1910974"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2016.7471747"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.1894636"},{"key":"e_1_2_2_12_1","doi-asserted-by":"crossref","unstructured":"J. Huang Z. Chen D. Ceylan and H. Jin. 2017. 6-DOF VR videos with a single 360-camera. In 2017 IEEE Virtual Reality (VR). 37--44.  J. Huang Z. Chen D. Ceylan and H. Jin. 2017. 6-DOF VR videos with a single 360-camera. In 2017 IEEE Virtual Reality (VR). 37--44.","DOI":"10.1109\/VR.2017.7892229"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073702"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.2307\/3680992"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982405"},{"key":"e_1_2_2_16_1","volume-title":"Room Acoustics","author":"Kuttruff Heinrich","unstructured":"Heinrich Kuttruff . 2017. Room Acoustics ( sixth ed.). CRC Press . Heinrich Kuttruff. 2017. Room Acoustics (sixth ed.). CRC Press."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925983"},{"key":"e_1_2_2_18_1","volume-title":"Inverse rendering for computer graphics","author":"Marschner Stephen Robert","unstructured":"Stephen Robert Marschner and Donald P Greenberg . 1998. Inverse rendering for computer graphics . Cornell University . Stephen Robert Marschner and Donald P Greenberg. 1998. Inverse rendering for computer graphics. Cornell University."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073645"},{"key":"e_1_2_2_20_1","volume-title":"Signal analysis","author":"Papoulis Athanasios","unstructured":"Athanasios Papoulis . 1977. Signal analysis . Vol. 191 . McGraw-Hill New York . Athanasios Papoulis. 1977. Signal analysis. Vol. 191. McGraw-Hill New York."},{"key":"e_1_2_2_21_1","volume-title":"Realtime Room Acoustics Using Ambisonics. In Audio Engineering Society Conference: 16th International Conference: Spatial Sound Reproduction.","author":"Pope Jackson","year":"1999","unstructured":"Jackson Pope , David Creasey , and Alan Chalmers . 1999 . Realtime Room Acoustics Using Ambisonics. In Audio Engineering Society Conference: 16th International Conference: Spatial Sound Reproduction. Jackson Pope, David Creasey, and Alan Chalmers. 1999. Realtime Room Acoustics Using Ambisonics. In Audio Engineering Society Conference: 16th International Conference: Spatial Sound Reproduction."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2009.27"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601184"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778805"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2421636.2421637"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2501988.2501993"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.4926438"},{"key":"e_1_2_2_28_1","volume-title":"Acoustic Classification and Optimization for Multi-Modal Rendering of Real-World Scenes","author":"Schissler Carl","year":"2017","unstructured":"Carl Schissler , Christian Loftin , and Dinesh Manocha . 2017a. Acoustic Classification and Optimization for Multi-Modal Rendering of Real-World Scenes . IEEE Transactions on Visualization and Computer Graphics ( 2017 ). Carl Schissler, Christian Loftin, and Dinesh Manocha. 2017a. Acoustic Classification and Optimization for Multi-Modal Rendering of Real-World Scenes. IEEE Transactions on Visualization and Computer Graphics (2017)."},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601216"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2016.2518134"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2017.7892239"},{"key":"e_1_2_2_32_1","volume-title":"Topological Sound Propagation with Reverberation Graphs. Acta Acustica\/Acustica - the Journal of the European Acoustics Association (EAA)","author":"Stavrakis Efstathios","year":"2008","unstructured":"Efstathios Stavrakis , Nicolas Tsingos , and Paul Calamia . 2008. Topological Sound Propagation with Reverberation Graphs. Acta Acustica\/Acustica - the Journal of the European Acoustics Association (EAA) ( 2008 ). Efstathios Stavrakis, Nicolas Tsingos, and Paul Calamia. 2008. Topological Sound Propagation with Reverberation Graphs. Acta Acustica\/Acustica - the Journal of the European Acoustics Association (EAA) (2008)."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1612524113"},{"key":"e_1_2_2_34_1","volume-title":"Precomputing Geometry-Based Reverberation Effects for Games. In Audio Engineering Society Conference: 35th International Conference: Audio for Games. http:\/\/www.aes.org\/e-lib\/browse.cfm?elib=15164","author":"Tsingos Nicolas","year":"2009","unstructured":"Nicolas Tsingos . 2009 . Precomputing Geometry-Based Reverberation Effects for Games. In Audio Engineering Society Conference: 35th International Conference: Audio for Games. http:\/\/www.aes.org\/e-lib\/browse.cfm?elib=15164 Nicolas Tsingos. 2009. Precomputing Geometry-Based Reverberation Effects for Games. In Audio Engineering Society Conference: 35th International Conference: Audio for Games. http:\/\/www.aes.org\/e-lib\/browse.cfm?elib=15164"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383323"},{"key":"e_1_2_2_36_1","volume-title":"Auralization: Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality (RWTHedition) (2008 ed.)","author":"Vorl\u00e4nder Michael","year":"2008","unstructured":"Michael Vorl\u00e4nder . 2008 . Auralization: Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality (RWTHedition) (2008 ed.) . Springer . Michael Vorl\u00e4nder. 2008. Auralization: Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality (RWTHedition) (2008 ed.). Springer."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/279232.279236"},{"key":"e_1_2_2_38_1","volume-title":"Audio Engineering Society Convention 126","author":"Zotter Franz","year":"2009","unstructured":"Franz Zotter , Hannes Pomberger , and Matthias Frank . 2009 . An alternative ambisonics formulation: Modal source strength matching and the effect of spatial aliasing . In Audio Engineering Society Convention 126 . Audio Engineering Society. Franz Zotter, Hannes Pomberger, and Matthias Frank. 2009. An alternative ambisonics formulation: Modal source strength matching and the effect of spatial aliasing. In Audio Engineering Society Convention 126. Audio Engineering Society."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201391","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3197517.3201391","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:06:59Z","timestamp":1750212419000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201391"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,30]]},"references-count":38,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,8,31]]}},"alternative-id":["10.1145\/3197517.3201391"],"URL":"https:\/\/doi.org\/10.1145\/3197517.3201391","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,30]]},"assertion":[{"value":"2018-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}