{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T12:20:28Z","timestamp":1767183628642,"version":"3.41.0"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2019,11,30]],"date-time":"2019-11-30T00:00:00Z","timestamp":1575072000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"\u201cOptimized source coding for multiple terminals in self-organizing networks\u201d"},{"name":"Scientific Research-Flanders"},{"name":"Research Foundation - Flanders"},{"name":"CAPES, CNPq, FAPERGS, and IFSul"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2019,11,30]]},"abstract":"<jats:p>To cope with the massive bandwidth demands of Virtual Reality (VR) video streaming, both the scientific community and the industry have been proposing optimization techniques such as viewport-aware streaming and tile-based adaptive bitrate heuristics. As most of the VR video traffic is expected to be delivered through mobile networks, a major problem arises: both the network performance and VR video optimization techniques have the potential to influence the video playout performance and the Quality of Experience (QoE). However, the interplay between them is neither trivial nor has it been properly investigated. To bridge this gap, in this article, we introduce VR-EXP, an open-source platform for carrying out VR video streaming performance evaluation. Furthermore, we consolidate a set of relevant VR video streaming techniques and evaluate them under variable network conditions, contributing to an in-depth understanding of what to expect when different combinations are employed. To the best of our knowledge, this is the first work to propose a systematic approach, accompanied by a software toolkit, which allows one to compare different optimization techniques under the same circumstances. Extensive evaluations carried out using realistic datasets demonstrate that VR-EXP is instrumental in providing valuable insights regarding the interplay between network performance and VR video streaming optimization techniques.<\/jats:p>","DOI":"10.1145\/3360286","type":"journal-article","created":{"date-parts":[[2019,12,16]],"date-time":"2019-12-16T13:12:30Z","timestamp":1576501950000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":19,"title":["Dissecting the Performance of VR Video Streaming through the VR-EXP Experimentation Platform"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4804-8954","authenticated-orcid":false,"given":"Roberto Iraja Tavares Da Costa","family":"Filho","sequence":"first","affiliation":[{"name":"Institute of Informatics - UFRGS, Brazil"}]},{"given":"Marcelo Caggiani","family":"Luizelli","sequence":"additional","affiliation":[{"name":"Federal University of Pampa, Brazil"}]},{"given":"Stefano","family":"Petrangeli","sequence":"additional","affiliation":[{"name":"Adobe Research, San Jose, CA, United States"}]},{"given":"Maria Torres","family":"Vega","sequence":"additional","affiliation":[{"name":"Ghent University - imec, Gent, Belgium"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9416-9661","authenticated-orcid":false,"given":"Jeroen Van der","family":"Hooft","sequence":"additional","affiliation":[{"name":"Ghent University - imec, Gent, Belgium"}]},{"given":"Tim","family":"Wauters","sequence":"additional","affiliation":[{"name":"Ghent University - imec, Gent, Belgium"}]},{"given":"Filip De","family":"Turck","sequence":"additional","affiliation":[{"name":"Ghent University - imec, Gent, Belgium"}]},{"given":"Luciano Paschoal","family":"Gaspary","sequence":"additional","affiliation":[{"name":"Institute of Informatics - UFRGS, Brazil"}]}],"member":"320","published-online":{"date-parts":[[2019,12,16]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3230543.3230558"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3204949.3204970"},{"key":"e_1_2_1_3_1","volume-title":"JVET common test conditions and evaluation procedures for 360 video. Joint Video Exploration Team of ITU-T SG 16","author":"Boyce Jill","year":"2017","unstructured":"Jill Boyce , Elena Alshina , Adeel Abbas , and Yan Ye. 2017. JVET common test conditions and evaluation procedures for 360 video. Joint Video Exploration Team of ITU-T SG 16 ( 2017 ). ITU - International Telecommunication Union . Retrieved on 21 June, 2010 from http:\/\/phenix.it sudparis.eu\/jvet\/doc_end_user\/documents\/16_Geneva\/wg11\/JVET-P0006-v1.zip. Jill Boyce, Elena Alshina, Adeel Abbas, and Yan Ye. 2017. JVET common test conditions and evaluation procedures for 360 video. Joint Video Exploration Team of ITU-T SG 16 (2017). ITU - International Telecommunication Union. Retrieved on 21 June, 2010 from http:\/\/phenix.it sudparis.eu\/jvet\/doc_end_user\/documents\/16_Geneva\/wg11\/JVET-P0006-v1.zip."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.sigpro.2018.01.004"},{"volume-title":"Cisco Visual Networking Index: Forecast and Trends","year":"2017","key":"e_1_2_1_5_1","unstructured":"Cisco. 2019. Cisco Visual Networking Index: Forecast and Trends , 2017 --2022. Technical Report. Cisco Systems . Cisco. 2019. Cisco Visual Networking Index: Forecast and Trends, 2017--2022. Technical Report. Cisco Systems."},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the IEEE International Conference on Communications (ICC\u201917)","author":"Corbillon X.","year":"2017","unstructured":"X. Corbillon , G. Simon , A. Devlic , and J. Chakareski . 2017. Viewport-adaptive navigable 360-degree video delivery . In Proceedings of the IEEE International Conference on Communications (ICC\u201917) . 1--7. DOI:https:\/\/doi.org\/10.1109\/ICC. 2017 .7996611 10.1109\/ICC.2017.7996611 X. Corbillon, G. Simon, A. Devlic, and J. Chakareski. 2017. Viewport-adaptive navigable 360-degree video delivery. In Proceedings of the IEEE International Conference on Communications (ICC\u201917). 1--7. DOI:https:\/\/doi.org\/10.1109\/ICC.2017.7996611"},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the IEEE Global Communications Conference (GLOBECOM\u201916)","author":"R. I.","year":"2016","unstructured":"R. I. T. da Costa Filho, W. Lautenschlager, N. Kagami, V. Roesler, and L. P. Gaspary. 2016. Network fortune cookie: Using network measurements to predict video streaming performance and QoE . In Proceedings of the IEEE Global Communications Conference (GLOBECOM\u201916) . 1--6. DOI:https:\/\/doi.org\/10.1109\/GLOCOM. 2016 .7842022 10.1109\/GLOCOM.2016.7842022 R. I. T. da Costa Filho, W. Lautenschlager, N. Kagami, V. Roesler, and L. P. Gaspary. 2016. Network fortune cookie: Using network measurements to predict video streaming performance and QoE. In Proceedings of the IEEE Global Communications Conference (GLOBECOM\u201916). 1--6. DOI:https:\/\/doi.org\/10.1109\/GLOCOM.2016.7842022"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 9th ACM Multimedia Systems Conference (MMSys\u201918)","author":"Tavares da Costa Filho Roberto Iraj\u00e1","year":"2049","unstructured":"Roberto Iraj\u00e1 Tavares da Costa Filho , Marcelo Caggiani Luizelli , Maria Torres Vega , Jeroen van der Hooft, Stefano Petrangeli, Tim Wauters, Filip De Turck, and Luciano Paschoal Gaspary. 2018. Predicting the performance of virtual reality video streaming in mobile networks . In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys\u201918) . ACM, New York, NY, 270--283. DOI:https:\/\/doi.org\/10.1145\/3 2049 49.3204966 10.1145\/3204949.3204966 Roberto Iraj\u00e1 Tavares da Costa Filho, Marcelo Caggiani Luizelli, Maria Torres Vega, Jeroen van der Hooft, Stefano Petrangeli, Tim Wauters, Filip De Turck, and Luciano Paschoal Gaspary. 2018. Predicting the performance of virtual reality video streaming in mobile networks. In Proceedings of the 9th ACM Multimedia Systems Conference (MMSys\u201918). ACM, New York, NY, 270--283. DOI:https:\/\/doi.org\/10.1145\/3204949.3204966"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2987443.2987459"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/INM.2007.374789"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3083165.3083180"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3083187.3084016"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3204949.3204957"},{"key":"e_1_2_1_14_1","volume-title":"Proceedings of the IEEE International Symposium on Multimedia (ISM\u201916)","author":"Hosseini M.","year":"2016","unstructured":"M. Hosseini and V. Swaminathan . 2016. Adaptive 360 VR video streaming: Divide and conquer . In Proceedings of the IEEE International Symposium on Multimedia (ISM\u201916) . 107--110. DOI:https:\/\/doi.org\/10.1109\/ISM. 2016 .0028 10.1109\/ISM.2016.0028 M. Hosseini and V. Swaminathan. 2016. Adaptive 360 VR video streaming: Divide and conquer. In Proceedings of the IEEE International Symposium on Multimedia (ISM\u201916). 107--110. DOI:https:\/\/doi.org\/10.1109\/ISM.2016.0028"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3229625.3229629"},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the IEEE 20th International Workshop on Multimedia Signal Processing (MMSP\u201918)","author":"Hristova H.","year":"2018","unstructured":"H. Hristova , X. Corbillon , G. Simon , V. Swaminathan , and A. Devlic . 2018. Heterogeneous spatial quality for omnidirectional video . In Proceedings of the IEEE 20th International Workshop on Multimedia Signal Processing (MMSP\u201918) . 1--6. DOI:https:\/\/doi.org\/10.1109\/MMSP. 2018 .8547114 10.1109\/MMSP.2018.8547114 H. Hristova, X. Corbillon, G. Simon, V. Swaminathan, and A. Devlic. 2018. Heterogeneous spatial quality for omnidirectional video. In Proceedings of the IEEE 20th International Workshop on Multimedia Signal Processing (MMSP\u201918). 1--6. DOI:https:\/\/doi.org\/10.1109\/MMSP.2018.8547114"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 10th International Conference on Ubiquitous and Future Networks (ICUFN\u201918)","author":"Jeong E.","year":"2018","unstructured":"E. Jeong , D. You , C. Hyun , B. Seo , N. Kim , D. H. Kim , and Y. H. Lee . 2018. Viewport prediction method of 360 VR video using sound localization information . In Proceedings of the 10th International Conference on Ubiquitous and Future Networks (ICUFN\u201918) . 679--681. DOI:https:\/\/doi.org\/10.1109\/ICUFN. 2018 .8436981 10.1109\/ICUFN.2018.8436981 E. Jeong, D. You, C. Hyun, B. Seo, N. Kim, D. H. Kim, and Y. H. Lee. 2018. Viewport prediction method of 360 VR video using sound localization information. In Proceedings of the 10th International Conference on Ubiquitous and Future Networks (ICUFN\u201918). 679--681. DOI:https:\/\/doi.org\/10.1109\/ICUFN.2018.8436981"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI\u201995)","volume":"14","author":"Ron","unstructured":"Ron Kohavi et al. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection . In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI\u201995) , Vol. 14 . 1137--1145. Ron Kohavi et al. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI\u201995), Vol. 14. 1137--1145."},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB\u201918)","author":"Ma L.","year":"2018","unstructured":"L. Ma , Y. Xu , J. Sun , W. Huang , S. Xie , Y. Li , and N. Liu . 2018. Buffer control in VR video transmission over MMT system . In Proceedings of the IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB\u201918) . 1--5. DOI:https:\/\/doi.org\/10.1109\/BMSB. 2018 .8436817 10.1109\/BMSB.2018.8436817 L. Ma, Y. Xu, J. Sun, W. Huang, S. Xie, Y. Li, and N. Liu. 2018. Buffer control in VR video transmission over MMT system. In Proceedings of the IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB\u201918). 1--5. DOI:https:\/\/doi.org\/10.1109\/BMSB.2018.8436817"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098822.3098843"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/LCOMM.2018.2848915"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818361"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR\u201918)","author":"Petrangeli S.","year":"2018","unstructured":"S. Petrangeli , G. Simon , and V. Swaminathan . 2018. Trajectory-based viewport prediction for 360-degree virtual reality videos . In Proceedings of the IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR\u201918) . 157--160. DOI:https:\/\/doi.org\/10.1109\/AIVR. 2018 .00033 10.1109\/AIVR.2018.00033 S. Petrangeli, G. Simon, and V. Swaminathan. 2018. Trajectory-based viewport prediction for 360-degree virtual reality videos. In Proceedings of the IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR\u201918). 157--160. DOI:https:\/\/doi.org\/10.1109\/AIVR.2018.00033"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3123453"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980055.2980056"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2011.71"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3204949.3204953"},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of the 35th IEEE International Conference on Computer Communications (INFOCOM\u201916)","author":"Spiteri K.","year":"2016","unstructured":"K. Spiteri , R. Urgaonkar , and R. K. Sitaraman . 2016. BOLA: Near-optimal bitrate adaptation for online videos . In Proceedings of the 35th IEEE International Conference on Computer Communications (INFOCOM\u201916) . 1--9. DOI:https:\/\/doi.org\/10.1109\/INFOCOM. 2016 .7524428 10.1109\/INFOCOM.2016.7524428 K. Spiteri, R. Urgaonkar, and R. K. Sitaraman. 2016. BOLA: Near-optimal bitrate adaptation for online videos. In Proceedings of the 35th IEEE International Conference on Computer Communications (INFOCOM\u201916). 1--9. DOI:https:\/\/doi.org\/10.1109\/INFOCOM.2016.7524428"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the IEEE Wireless Communications and Networking Conference. 1044--1049","author":"Stangherlin K.","year":"2011","unstructured":"K. Stangherlin , R. C. Filho , W. Lautenschl\u00e4ger , V. Guadagnin , L. Balbinot , R. Balbinot , and V. Roesler . 2011. One-way delay measurement in wired and wireless mobile full-mesh networks . In Proceedings of the IEEE Wireless Communications and Networking Conference. 1044--1049 . DOI:https:\/\/doi.org\/10.1109\/WCNC. 2011 .5779279 10.1109\/WCNC.2011.5779279 K. Stangherlin, R. C. Filho, W. Lautenschl\u00e4ger, V. Guadagnin, L. Balbinot, R. Balbinot, and V. Roesler. 2011. One-way delay measurement in wired and wireless mobile full-mesh networks. In Proceedings of the IEEE Wireless Communications and Networking Conference. 1044--1049. DOI:https:\/\/doi.org\/10.1109\/WCNC.2011.5779279"},{"key":"e_1_2_1_31_1","unstructured":"TELCO. 2018. Mobile Operators Market Share in Brazil. Retrieved from: http:\/\/www.teleco.com.br\/en\/en_mshare.asp.  TELCO. 2018. Mobile Operators Market Share in Brazil. Retrieved from: http:\/\/www.teleco.com.br\/en\/en_mshare.asp."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/LCOMM.2016.2601087"},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS\u201915)","author":"Viitanen M.","year":"2015","unstructured":"M. Viitanen , A. Koivula , A. Lemmetti , J. Vanne , and T. D. H\u00e4m\u00e4l\u00e4inen . 2015. Kvazaar HEVC encoder for efficient intra coding . In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS\u201915) . 1662--1665. DOI:https:\/\/doi.org\/10.1109\/ISCAS. 2015 .7168970 10.1109\/ISCAS.2015.7168970 M. Viitanen, A. Koivula, A. Lemmetti, J. Vanne, and T. D. H\u00e4m\u00e4l\u00e4inen. 2015. Kvazaar HEVC encoder for efficient intra coding. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS\u201915). 1662--1665. DOI:https:\/\/doi.org\/10.1109\/ISCAS.2015.7168970"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3083187.3083210"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2785956.2787486"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2829988.2787486"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209660"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3360286","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3360286","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:44:32Z","timestamp":1750203872000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3360286"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,30]]},"references-count":36,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,11,30]]}},"alternative-id":["10.1145\/3360286"],"URL":"https:\/\/doi.org\/10.1145\/3360286","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2019,11,30]]},"assertion":[{"value":"2017-02-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-12-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}