{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T18:28:07Z","timestamp":1787509687167,"version":"build-2736575974"},"reference-count":56,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,6,12]],"date-time":"2021-06-12T00:00:00Z","timestamp":1623456000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The Knowledge Foundation","award":["20170056"],"award-info":[{"award-number":["20170056"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Due to the advances in head-mounted displays (HMDs), hardware and software technologies, and mobile connectivity, virtual reality (VR) applications such as viewing 360\u00b0 videos on HMDs have seen an increased interest in a wide range of consumer and vertical markets. Quality assessment of digital media systems and services related to immersive visual stimuli has been one of the challenging problems of multimedia signal processing. Specifically, subjective quality assessment of 360\u00b0 videos presented on HMDs is needed to obtain a ground truth on the visual quality as perceived by humans. Standardized test methodologies to assess the subjective quality of 360\u00b0 videos on HMDs are currently not as developed as for conventional videos and are subject to further study. In addition, subjective tests related to quality assessment of 360\u00b0 videos are commonly conducted with participants seated on a chair but neglect other options of consumption such as standing viewing. In this paper, we compare the effect that standing and seated viewing of 360\u00b0 videos on an HMD has on subjective quality assessment. A pilot study was conducted to obtain psychophysical and psychophysiological data that covers explicit and implicit responses of the participants to the shown 360\u00b0 video stimuli with different quality levels. The statistical analysis of the data gathered in the pilot study is reported in terms of average rating times, mean opinion scores, standard deviation of opinion scores, head movements, pupil diameter, galvanic skin response (GSR), and simulator sickness scores. The results indicate that the average rating times consumed for 360\u00b0 video quality assessment are similar for standing and seated viewing. Further, the participants showed higher resolving power among different 360\u00b0 video quality levels and were more confident about the given opinion scores for seated viewing. On the other hand, a larger scene exploration of 360\u00b0 videos was observed for standing viewing which appears to distract from the quality assessment task. A slightly higher pupil dilation was recorded for standing viewing which suggests a slightly more immersed experience compared to seated viewing. GSR data indicate a lower degree of emotional arousal in seated viewing which seems to allow the participants to better conduct the quality assessment task. Similarly, simulator sickness symptoms are kept significantly lower when seated. The pilot study also contributes to a holistic view of subjective quality assessment and provides indicative ground truth that can guide the design of large-scale subjective tests.<\/jats:p>","DOI":"10.3390\/computers10060080","type":"journal-article","created":{"date-parts":[[2021,6,14]],"date-time":"2021-06-14T10:30:25Z","timestamp":1623666625000},"page":"80","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["On the Effect of Standing and Seated Viewing of 360\u00b0 Videos on Subjective Quality Assessment: A Pilot Study"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3283-2819","authenticated-orcid":false,"given":"Yan","family":"Hu","sequence":"first","affiliation":[{"name":"Blekinge Institute of Technology, Valhallav\u00e4gen 1, SE-37179 Karlskrona, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7550-5818","authenticated-orcid":false,"given":"Majed","family":"Elwardy","sequence":"additional","affiliation":[{"name":"Blekinge Institute of Technology, Valhallav\u00e4gen 1, SE-37179 Karlskrona, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3604-2766","authenticated-orcid":false,"given":"Hans-J\u00fcrgen","family":"Zepernick","sequence":"additional","affiliation":[{"name":"Blekinge Institute of Technology, Valhallav\u00e4gen 1, SE-37179 Karlskrona, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,12]]},"reference":[{"key":"ref_1","unstructured":"Huawei iLab (2017). VR Big Data Report, Huawei Technologies Co., Ltd."},{"key":"ref_2","unstructured":"Huawei iLab (2018). Cloud VR Network Solution White Paper, Huawei Technologies Co., Ltd."},{"key":"ref_3","unstructured":"Yrj\u00f6l\u00e4, S., Ahokangas, P., and Matinmikko-Blue, M. (2020). White Paper on Business of 6G, University of Oulu. 6G Research Visions, No. 3."},{"key":"ref_4","unstructured":"(2021, March 31). 3rd Generation Partnership Project. Extended Reality in 5G (Release 16). Available online: https:\/\/www.etsi.org\/deliver\/etsi_tr\/126900_126999\/126928\/16.00.00_60\/tr_126928v160000p.pdf."},{"key":"ref_5","unstructured":"Recommendation ITU-T P.910 (2008). Subjective Video Quality Assessment Methods for Multimedia Applications, International Telecommunication Union\u2014Telecommunication Standardization Sector."},{"key":"ref_6","unstructured":"Recommendation ITU-T P.915 (2016). Subjective Assessment Methods for 3D Video Quality, International Telecommunication Union\u2014Telecommunication Standardization Sector."},{"key":"ref_7","unstructured":"Recommendation ITU-R BT.500-13 (2012). Methodology for the Subjective Assessment of the Quality of Television Pictures, International Telecommunication Union\u2014Radiocommunication Standardization Sector."},{"key":"ref_8","unstructured":"Recommendation ITU-T BT.1788 (2007). Methodology for the Subjective Assessment of Video Quality in Multimedia Applications, International Telecommunication Union\u2014Telecommunication Standardization Sector."},{"key":"ref_9","unstructured":"Zhang, B., Zhao, J., Yang, S., Zhang, Y., Wang, J., and Fei, Z. (2017, January 10\u201314). Subjective and Objective Quality Assessment of Panoramic Videos in Virtual Reality Environment. Proceedings of the IEEE International Conference on Multimedia Expo Workshops, Hong Kong, China."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tran, H.T.T., Ngoc, N.P., Pham, C.T., and Jung, Y.J. (2017, January 16\u201318). A Subjective Study on QoE of 360 Video for VR Communication. Proceedings of the IEEE International Workshop on Multimedia Signal Processing, Luton, UK.","DOI":"10.1109\/MMSP.2017.8122249"},{"key":"ref_11","unstructured":"Singla, A., Fremerey, S., Robitza, W., and Raake, A. (June, January 31). Measuring and Comparing QoE and Simulator Sickness of Omnidirectional Videos in Different Head Mounted Displays. Proceedings of the IEEE International Conference on Quality of Multimedia Experience, Erfurt, Germany."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Xu, M., Li, C., Liu, Y., Deng, X., and Lu, J. (2017, January 10\u201314). A Subjective Visual Quality Assessment Method of Panoramic Videos. Proceedings of the IEEE International Conference on Multimedia and Expo, Hong Kong, China.","DOI":"10.1109\/ICME.2017.8019351"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Duan, H., Zhai, G., Yang, X., Li, D., and Zhu, W. (2017, January 22\u201324). IVQAD 2017: An Immersive Video Quality Assessment Database. Proceedings of the International Conference on Systems, Signals and Image Processing, Poznan, Poland.","DOI":"10.1109\/IWSSIP.2017.7965610"},{"key":"ref_14","unstructured":"Lopes, F., Ascenso, J., Rodrigues, A., and Queluz, M.P. (2018, January 17). Subjective and Objective Quality Assessment of Omnidirectional Video. Proceedings of the SPIE Optical Engineering and Applications, San Diego, CA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Fremerey, S., Singla, A., Meseberg, K., and Raake, A. (2018, January 12\u201315). AVTrack360: An Open Dataset and Software Recoding People\u2019s Head Rotations Watching 360\u00b0 Videos on an HMD. Proceedings of the Proceedings of the ACM Multimedia Systems Conference, Amsterdam, The Netherlands.","DOI":"10.1145\/3204949.3208134"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yao, S.H., Fan, C.L., and Hsu, C.H. (2019, January 5\u20137). Towards Quality-of-Experience Models for Watching 360\u00b0 Videos in Head-Mounted Virtual Reality. Proceedings of the International Conference on Quality of Multimedia Experience, Berlin, Germany.","DOI":"10.1109\/QoMEX.2019.8743198"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"T. T. Tran, H., Ngoc, N.P., Pham, C.T., Jung, Y.J., and Thang, T.C. (2019). A Subjective Study on User Perception Aspects in Virtual Reality. Appl. Sci., 9.","DOI":"10.3390\/app9163384"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/TBC.2018.2811627","article-title":"Subjective Panoramic Video Quality Assessment Database for Coding Applications","volume":"64","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Broadcast."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Alexiou, E., Upenik, E., and Ebrahimi, T. (2017, January 16\u201318). Towards Subjective Quality Assessment of Point Cloud Imaging in Augmented Reality. Proceedings of the IEEE International Workshop on Multimedia Signal Processing, Luton, UK.","DOI":"10.1109\/MMSP.2017.8122237"},{"key":"ref_20","unstructured":"Singla, A., Robitza, W., and Raake, A. (February, January 28). Comparison of Subjective Quality Evaluation Methods for Omnidirectional Videos with DSIS and Modified ACR. Proceedings of the Human Vision and Electronic Imaging, Burlingame, CA, USA."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhang, B., Yan, Z., Wang, J., Luo, Y., Yang, S., and Fei, Z. (2018, January 23\u201327). An Audio-Visual Quality Assessment Methodology in Virtual Reality Environment. Proceedings of the IEEE International Conference on Multimedia and Expo Workshops, San Diego, CA, USA.","DOI":"10.1109\/ICMEW.2018.8551522"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Singla, A., Fremerey, S., Robitza, W., Lebreton, P., and Raake, A. (2017, January 23\u201327). Comparison of Subjective Quality Evaluation for HEVC Encoded Omnidirectional Videos at Different Bit-rates for UHD and FHD Resolution. Proceedings of the Thematic Workshops of ACM Multimedia, Mountain View, CA, USA.","DOI":"10.1145\/3126686.3126768"},{"key":"ref_23","unstructured":"Recommendation ITU-T P.360-VR (2020). Subjective Test Methodologies for 360 Degree Video on HMD, International Telecommunication Union\u2014Telecommunication Standardization Sector."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"P\u00e9rez, P., and Escobar, J. (2019, January 5\u20137). MIRO360: A Tool for Subjective Assessment of 360 Degree Video for ITU-T P.360-VR. Proceedings of the International Conference on Quality of Multimedia Experience, Berlin, Germany.","DOI":"10.1109\/QoMEX.2019.8743216"},{"key":"ref_25","unstructured":"Hong, Y., MacQuarrie, A., and Steed, A. (December, January 28). The Effect of Chair Type on Users\u2019 Viewing Experience for 360-degree Video. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, Tokyo, Japan."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"MacQuarrie, A., and Steed, A. (2017, January 18\u201322). Cinematic Virtual Reality: Evaluating the Effect of Display Type on the Viewing Experience for Panoramic Video. Proceedings of the IEEE Virtual Reality, Los Angeles, CA, USA.","DOI":"10.1109\/VR.2017.7892230"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zielasko, D., Weyers, B., Bellgardt, M., Pick, S., Meisner, A., Verjahn, T., and Kuhlen, T.W. (2017, January 19). Remain Seated: Towards Fully-Immersive Desktop VR. Proceedings of the IEEE Workshop on Everyday Virtual Reality, Los Angeles, CA, USA.","DOI":"10.1109\/WEVR.2017.7957707"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zielasko, D., Weyers, B., and Kuhlen, T.W. (2019, January 23\u201327). A Non-Stationary Office Desk Substitution for Desk-Based and HMD-Projected Virtual Reality. Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces, Osaka, Japan.","DOI":"10.1109\/VR.2019.8797837"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Jeong, H., Singh, A., Kim, M., and Johnson, A. (2020, January 22\u201326). Using Augmented Reality to Assist Seated Office Workers\u2019 Data Entry Tasks. Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, Atlanta, GA, USA.","DOI":"10.1109\/VRW50115.2020.00065"},{"key":"ref_30","unstructured":"Nguyen-Vo, T., Riecke, B.E., Stuerzlinger, W., Pham, D.M., and Kruijff, E. (2019). NaviBoard and NaviChair: Limited Translation Combined with Full Rotation for Efficient Virtual Locomotion. IEEE Trans. Vis. Comput. Graph."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Rieke, B.E., and Feuereissen, D. (2012, January 3\u20134). To Move or Not to Move: Can Active Control and User-Driven Motion Cueing Enhance Self-Motion Perception (\u201cVection\u201d) in Virtual Reality?. Proceedings of the ACM Symposium on Applied Perception, Los Angeles, CA, USA.","DOI":"10.1145\/2338676.2338680"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kruijff, E., Rieke, B.E., Trepkowski, C., and Kitson, A. (2015, January 8\u20139). Upper Body Leaning Can Affect Forward Self-Motion Perception in Virtual Environments. Proceedings of the ACM Symposium on Spatial User Interaction, Los Angeles, CA, USA.","DOI":"10.1145\/2788940.2788943"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kitson, A., Hashemian, A.M., Stepanova, E.R., Kruijff, E., and Riecke, B.E. (2017, January 18\u201319). Comparing Leaning-Based Motion Cueing Interfaces for Virtual Reality Locomotion. Proceedings of the IEEE Symposium on 3D User Interfaces, Los Angeles, CA, USA.","DOI":"10.1109\/3DUI.2017.7893320"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Riecke, B.E., Schiphorst, T., and Neustaedter, C. (2019, January 23\u201328). Perch to Fly: Embodied Virtual Reality Flying Locomotion with a Flexible Perching Stance. Proceedings of the ACM Designing Interactive Systems Conference, San Diego, CA, USA.","DOI":"10.1145\/3322276.3322357"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"920","DOI":"10.1518\/001872007X230262","article-title":"Motion Sickness, Console Video Games, and Head-mounted Displays","volume":"49","author":"Merhi","year":"2007","journal-title":"Hum. Factors"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hu, Y., Elwardy, M., and Zepernick, H.J. (2020, January 22\u201326). On the Effect of Standing and Seated Viewing of 360\u00b0 Videos on Subjective Quality Assessment. Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, Atlanta, GA, USA.","DOI":"10.1109\/VRW50115.2020.00061"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Feil-Seifer, D., Haring, K.S., Rossi, S., Wagner, A.R., and Williams, T. (2010). Where to next? The Impact of COVID-19 on Human-Robot Interaction Research. ACM Trans. Hum. Robot Interact., 10.","DOI":"10.1145\/3405450"},{"key":"ref_38","unstructured":"N\u00e5bo, A., Hadi, M., Johansson, E., Krupenia, S., Larsson, A., Muthumani, A., and S\u00f6derman, M. (2020). SAFER Pre-Studies Final Report FP08, SAFER Vehicle and Traffic Safety Centre. Task Force: Hygiene Procedures in Test with Research Persons."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1207\/s15327108ijap0303_3","article-title":"Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness","volume":"3","author":"Kennedy","year":"1993","journal-title":"Int. J. Aviat. Psychol."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Elwardy, M., Zepernick, H.J., Sundstedt, V., and Hu, Y. (2019, January 16\u201318). Impact of Participants\u2019 Experiences with Immersive Multimedia on 360\u00b0 Video Quality Assessment. Proceedings of the IEEE International Conference on Signal Processing and Communication Systems, Gold Coast, Australia.","DOI":"10.1109\/ICSPCS47537.2019.9008739"},{"key":"ref_41","unstructured":"Beihang University, School of Electronic and Information Engineering, Beijing, China (2020, September 12). VQA-ODV. Available online: https:\/\/github.com\/Archer-Tatsu\/VQA-ODV."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Li, C., Xu, M., and Wang, Z. (2018, January 22\u201326). Bridge the Gap Between VQA and Human Behavior on Omnidirectional Video: A Large-Scale Dataset and a Deep Learning Model. Proceedings of the ACM International Conference on Multimedia, Seoul, Korea.","DOI":"10.1145\/3240508.3240581"},{"key":"ref_43","unstructured":"FFmpeg (2020, September 12). H.264 Video Encoding Guide. Available online: https:\/\/trac.ffmpeg.org\/wiki\/Encode\/H.264#crf."},{"key":"ref_44","unstructured":"FFmpeg (2020, September 12). FFmpeg and H.265 Encoding Guide. Available online: https:\/\/trac.ffmpeg.org\/wiki\/Encode\/H.265."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Ho\u00dffeld, T., Schatz, R., and Egger, S. (2011, January 7\u20139). SOS: The MOS is Not Enough!. Proceedings of the International Workshop on Quality of Multimedia Experience, Mechelen, Belgium.","DOI":"10.1109\/QoMEX.2011.6065690"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s41233-016-0002-1","article-title":"QoE Beyond the MOS: An In-depth Look at QoE via Better Metrics and Their Relation to MOS","volume":"1","author":"Hossfeld","year":"2016","journal-title":"Qual. User Exp."},{"key":"ref_47","unstructured":"Engelke, U. (2010). Modelling Perceptual Quality and Visual Saliency for Image and Video Communications. [Ph.D. Thesis, Blekinge Institute of Technology]."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"De Simone, F., Naccari, M., Tagliasachi, M., Dufaux, F., Tubaro, S., and Ebrahimi, T. (2009, January 29\u201331). Subjective Assessment of H.264\/AVC Video Sequences Transmitted Over a Noisy Channel. Proceedings of the International Workshop on Quality of Multimedia Experience, San Diego, CA, USA.","DOI":"10.1109\/QOMEX.2009.5246952"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1037\/0033-2909.91.2.276","article-title":"Task-evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources","volume":"91","author":"Beatty","year":"1982","journal-title":"Psychol. Bull."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1080\/17470210600673818","article-title":"Effort During Visual Search and Counting: Insights from Pupillometry","volume":"60","author":"Porter","year":"2007","journal-title":"Q. J. Exp. Psychol."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1126\/science.132.3423.349","article-title":"Pupil Size as Related to Interest Value of Visual Stimuli","volume":"132","author":"Hess","year":"1960","journal-title":"Science"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fneur.2018.01029","article-title":"Arousal Effects on Pupil Size, Heart Rate, and Skin Conductance in an Emotional Face Task","volume":"9","author":"Wang","year":"2018","journal-title":"Front. Neurol."},{"key":"ref_53","unstructured":"Walker, H.K., Hall, W.D., and Hurst, J.W. (1990). Clinical Methods: The History, Physical, and Laboratory Examinations, Butterworths."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1109\/JSTSP.2016.2609843","article-title":"Psychophysiology-Based QoE Assessment: A Survey","volume":"11","author":"Engelke","year":"2017","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Boucsein, W. (2012). Electrodermal Activity, Springer Science & Business Media. [2nd ed.].","DOI":"10.1007\/978-1-4614-1126-0"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1207\/s15326969eco0303_2","article-title":"An Ecological Theory of Motion Sickness and Postural Instability","volume":"3","author":"Riccio","year":"1991","journal-title":"Ecol. Psychol."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/10\/6\/80\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:13:48Z","timestamp":1760163228000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/10\/6\/80"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,12]]},"references-count":56,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["computers10060080"],"URL":"https:\/\/doi.org\/10.3390\/computers10060080","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,12]]}}}