{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T14:29:42Z","timestamp":1780064982651,"version":"3.54.0"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"9","license":[{"start":{"date-parts":[[2024,8,16]],"date-time":"2024-08-16T00:00:00Z","timestamp":1723766400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"UCA JEDI and EUR DS4H Investments","award":["ANR-15-IDEX-0001, ANR-17-EURE-0004"],"award-info":[{"award-number":["ANR-15-IDEX-0001, ANR-17-EURE-0004"]}]},{"name":"EU Horizon 2020","award":["951911"],"award-info":[{"award-number":["951911"]}]},{"name":"GENCI-IDRIS","award":["2020-AD011012209"],"award-info":[{"award-number":["2020-AD011012209"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>\n            Prediction of head movements in immersive media is key to designing efficient streaming systems able to focus the bandwidth budget on visible areas of the content. However, most of the numerous proposals made to predict user head motion in 360\u00b0 images and videos do not explicitly consider a prominent characteristic of the head motion data: its intrinsic uncertainty. In this article, we present an approach to generate multiple plausible futures of head motion in 360\u00b0 videos, given a common past trajectory. To our knowledge, this is the first work that considers the problem of multiple head motion prediction for 360\u00b0 video streaming. We introduce our discrete variational multiple sequence (DVMS) learning framework, which builds on deep latent variable models. We design a training procedure to obtain a flexible, lightweight stochastic prediction model compatible with sequence-to-sequence neural architectures. Experimental results on four different datasets show that DVMS outperforms competitors adapted from the self-driving domain by up to 41% on prediction horizons up to 5 s, at lower computational and memory costs. To understand how the learned features account for the motion uncertainty, we analyze the structure of the learned latent space and connect it with the physical properties of the trajectories. We also introduce a method to estimate the likelihood of each generated trajectory, enabling the integration of DVMS in a streaming system. We hence deploy an extensive evaluation of the interest of our DVMS proposal for a streaming system. To do so, we first introduce a new Python-based 360\u00b0 streaming simulator that we make available to the community. On real-world user, video, and networking data, we show that predicting multiple trajectories yields higher fairness between the traces, the gains for 20\u201330% of the users reaching up to 10% in visual quality for the best number\n            <jats:italic>K<\/jats:italic>\n            of trajectories to generate.\n          <\/jats:p>","DOI":"10.1145\/3643031","type":"journal-article","created":{"date-parts":[[2024,1,25]],"date-time":"2024-01-25T12:42:06Z","timestamp":1706186526000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Deep Variational Learning for 360\u00b0 Adaptive Streaming"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1744-5079","authenticated-orcid":false,"given":"Quentin","family":"Guimard","sequence":"first","affiliation":[{"name":"Universit\u00e9 C\u00f4te d\u2019Azur, CNRS, I3S, Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1232-1787","authenticated-orcid":false,"given":"Lucile","family":"Sassatelli","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d\u2019Azur, CNRS, I3S and Institut Universitaire de France, Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8884-2728","authenticated-orcid":false,"given":"Francesco","family":"Marchetti","sequence":"additional","affiliation":[{"name":"Universit\u00e0 degli Studi di Firenze, MICC, Firenze, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2537-2700","authenticated-orcid":false,"given":"Federico","family":"Becattini","sequence":"additional","affiliation":[{"name":"Universit\u00e0 degli Studi di Firenze, MICC, Firenze, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4816-0268","authenticated-orcid":false,"given":"Lorenzo","family":"Seidenari","sequence":"additional","affiliation":[{"name":"Universit\u00e0 degli Studi di Firenze, MICC, Firenze, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1052-8322","authenticated-orcid":false,"given":"Alberto Del","family":"Bimbo","sequence":"additional","affiliation":[{"name":"Universit\u00e0 degli Studi di Firenze, MICC, Firenze, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,8,16]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3204949.3204970"},{"key":"e_1_3_3_3_2","volume-title":"Proceedings of the 6th International Conference on Learning Representations (ICLR\u201918)","author":"Babaeizadeh Mohammad","year":"2018","unstructured":"Mohammad Babaeizadeh, Chelsea Finn, Dumitru Erhan, Roy H. Campbell, and Sergey Levine. 2018. Stochastic variational video prediction. In Proceedings of the 6th International Conference on Learning Representations (ICLR\u201918)."},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR48806.2021.9412158"},{"key":"e_1_3_3_5_2","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201918)","author":"Bhattacharyya Apratim","year":"2018","unstructured":"Apratim Bhattacharyya, Mario Fritz, and Bernt Schiele. 2018. Accurate and diverse sampling of sequences based on a \u201cbest of many\u201d sample objective. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201918)."},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","unstructured":"Eli Brookner. 1998. Tracking and Kalman filtering made easy. 10.1002\/0471224197","DOI":"10.1002\/0471224197"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00895"},{"key":"e_1_3_3_8_2","volume-title":"Proceedings of the IEEE 23nd International Workshop on Multimedia Signal Processing (MMSP\u201921)","author":"Chao Fang-Yi","year":"2021","unstructured":"Fang-Yi Chao, Cagri Ozcinar, and Aljosa Smolic. 2021. Transformer-based long-term viewport prediction in 360 \\(^\\circ\\) video: Scanpath is all you need. In Proceedings of the IEEE 23nd International Workshop on Multimedia Signal Processing (MMSP\u201921)."},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3204949.3208139"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.vrih.2021.08.007"},{"key":"e_1_3_3_11_2","unstructured":"Stanislav Fort Clara Huiyi Hu and Balaji Lakshminarayanan. 2020. Deep Ensembles: A Loss Landscape Perspective. Retrieved from https:\/\/openreview.net\/forum?id=r1xZAkrFPr"},{"key":"e_1_3_3_12_2","first-page":"1050","volume-title":"Proceedings of the 33rd International Conference on Machine Learning (ICML\u201916)","author":"Gal Yarin","year":"2016","unstructured":"Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (ICML\u201916). 1050\u20131059."},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3534086.3534335"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587819.3592547"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524273.3528176"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00240"},{"key":"e_1_3_3_17_2","volume-title":"Advances in Neural Information Processing Systems 31 (NeurIPS)","author":"Ha David","year":"2018","unstructured":"David Ha and J\u00fcrgen Schmidhuber. 2018. Recurrent world models facilitate policy evolution. In Advances in Neural Information Processing Systems 31 (NeurIPS)."},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2009.5164354"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICC.2019.8761189"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00246"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/LCN.2018.8638092"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458306.3458872"},{"key":"e_1_3_3_23_2","volume-title":"Proceedings of the 2nd International Conference on Learning Representations (ICLR\u201914)","author":"Kingma Diederik P.","year":"2014","unstructured":"Diederik P. Kingma and Max Welling. 2014. Auto-encoding variational Bayes. In Proceedings of the 2nd International Conference on Learning Representations (ICLR\u201914)."},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3098822.3098843"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00717"},{"key":"e_1_3_3_26_2","article-title":"Multiple trajectory prediction of moving agents with memory augmented networks","author":"Marchetti Francesco","year":"2020","unstructured":"Francesco Marchetti, Federico Becattini, Lorenzo Seidenari, and Alberto Del Bimbo. 2020. Multiple trajectory prediction of moving agents with memory augmented networks. IEEE Trans. Pattern Anal. Mach. Intell. 45, 6 (2020), 6688\u20136702.","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"e_1_3_3_27_2","volume-title":"Bayesian Learning for Neural Networks","author":"Neal Radford M.","year":"2012","unstructured":"Radford M. Neal. 2012. Bayesian Learning for Neural Networks. Vol. 118. Springer, New York, NY."},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240508.3240669"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW54120.2021.00259"},{"key":"e_1_3_3_30_2","first-page":"II\u20131278\u2013II\u20131286","volume-title":"Proceedings of the 31st International Conference on Machine Learning (ICML\u201914)","author":"Rezende Danilo Jimenez","year":"2014","unstructured":"Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31st International Conference on Machine Learning (ICML\u201914). II\u20131278\u2013II\u20131286."},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3414541"},{"key":"e_1_3_3_32_2","volume-title":"Beyond the Kalman Filter: Particle Filters for Tracking Applications","author":"Ristic Branko","year":"2003","unstructured":"Branko Ristic, Sanjeev Arulampalam, and Neil Gordon. 2003. Beyond the Kalman Filter: Particle Filters for Tracking Applications."},{"key":"e_1_3_3_33_2","first-page":"279","volume-title":"Proceedings of the 11th ACM Multimedia Systems Conference (MMSys\u201920)","author":"Romero-Rond\u00f3n Miguel Fabi\u00e1n","year":"2020","unstructured":"Miguel Fabi\u00e1n Romero-Rond\u00f3n, Lucile Sassatelli, Ram\u00f3n Aparicio-Pardo, and Fr\u00e9d\u00e9ric Precioso. 2020. A unified evaluation framework for head motion prediction methods in 360 \\(^\\circ\\) videos. In Proceedings of the 11th ACM Multimedia Systems Conference (MMSys\u201920). 279\u2013284."},{"issue":"9","key":"e_1_3_3_34_2","first-page":"5681","article-title":"TRACK: A new method from a re-examination of deep architectures for head motion prediction in 360 \\(^\\circ\\)  videos","volume":"44","author":"Romero-Rond\u00f3n Miguel Fabi\u00e1n","year":"2022","unstructured":"Miguel Fabi\u00e1n Romero-Rond\u00f3n, Lucile Sassatelli, Ram\u00f3n Aparicio-Pardo, and Fr\u00e9d\u00e9ric Precioso. 2022. TRACK: A new method from a re-examination of deep architectures for head motion prediction in 360 \\(^\\circ\\) videos. IEEE Trans. Pattern Anal. Mach. Intell. 44, 9 (2022), 5681\u20135699.","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.388"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3304112.3325610"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.5555\/645315.649172"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2018.2793599"},{"key":"e_1_3_3_39_2","volume-title":"Video Adaptation for High-Quality Content Delivery","author":"Spiteri Kevin","year":"2021","unstructured":"Kevin Spiteri. 2021. Video Adaptation for High-Quality Content Delivery. Ph. D. Dissertation."},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3204949.3204953"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS40897.2019.8968553"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.01005"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/LCOMM.2016.2601087"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2012.6232277"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458306.3460995"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2858783"},{"key":"e_1_3_3_47_2","first-page":"5333","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201918)","author":"Xu Yanyu","year":"2018","unstructured":"Yanyu Xu, Yanbing Dong, Junru Wu, Zhengzhong Sun, Zhiru Shi, Jingyi Yu, and Shenghua Gao. 2018. Gaze prediction in dynamic 360 \\(^\\circ\\) immersive videos. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201918). 5333\u20135342."},{"key":"e_1_3_3_48_2","first-page":"495","volume-title":"Proceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201920)","author":"Yan Francis Y.","year":"2020","unstructured":"Francis Y. Yan, Hudson Ayers, Chenzhi Zhu, Sadjad Fouladi, James Hong, Keyi Zhang, Philip Levis, and Keith Winstein. 2020. Learning in situ: A randomized experiment in video streaming. In Proceedings of the 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201920). 495\u2013511."},{"key":"e_1_3_3_49_2","article-title":"Hierarchical Bayesian LSTM for head trajectory prediction on omnidirectional images","author":"Yang Li","year":"2021","unstructured":"Li Yang, Mai Xu, Yichen Guo, Xin Deng, Fangyuan Gao, and Zhenyu Guan. 2021. Hierarchical Bayesian LSTM for head trajectory prediction on omnidirectional images. IEEE Trans. Pattern Anal. Mach. Intell. 44, 11 (2021), 7563\u20137580.","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00967"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3073283"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643031","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3643031","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T16:31:21Z","timestamp":1750264281000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643031"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,16]]},"references-count":50,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3643031"],"URL":"https:\/\/doi.org\/10.1145\/3643031","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,16]]},"assertion":[{"value":"2023-04-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}