{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T09:17:07Z","timestamp":1760347027684,"version":"3.41.0"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2022,1,27]],"date-time":"2022-01-27T00:00:00Z","timestamp":1643241600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2022,1,31]]},"abstract":"<jats:p>\n            The media quality assessment research community has traditionally been focusing on developing objective algorithms to predict the result of a typical subjective experiment in terms of\n            <jats:bold>Mean Opinion Score (MOS)<\/jats:bold>\n            value. However, the MOS, being a single value, is insufficient to model the complexity and diversity of human opinions encountered in an actual subjective experiment. In this work we propose a complementary approach for objective media quality assessment that attempts to more closely model what happens in a subjective experiment in terms of single observers and, at the same time, we perform a qualitative analysis of the proposed approach while highlighting its suitability. More precisely, we propose to model, using\n            <jats:bold>neural networks (NNs)<\/jats:bold>\n            , the way single observers perceive media quality. Once trained, these NNs, one for each observer, are expected to mimic the corresponding observer in terms of quality perception. Then, similarly to a subjective experiment, such NNs can be used to simulate the users\u2019 single opinions, which can be later aggregated by means of different statistical indicators such as average, standard deviation, quantiles, etc. Unlike previous approaches that consider subjective experiments as a black box providing reliable ground truth data for training, the proposed approach is able to consider human factors by analyzing and weighting individual observers. Such a model may therefore implicitly account for users\u2019 expectations and tendencies, that have been shown in many studies to significantly correlate with visual quality perception. Furthermore, our proposal also introduces and investigates an index measuring how much inconsistency there would be if an observer was asked to rate many times the same stimulus. Simulation experiments conducted on several datasets demonstrate that the proposed approach can be effectively implemented in practice and thus yielding a more complete objective assessment of end users\u2019 quality of experience.\n          <\/jats:p>","DOI":"10.1145\/3464393","type":"journal-article","created":{"date-parts":[[2022,1,27]],"date-time":"2022-01-27T19:44:21Z","timestamp":1643312661000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Mimicking Individual Media Quality Perception with Neural Network based Artificial Observers"],"prefix":"10.1145","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5127-9935","authenticated-orcid":false,"given":"Lohic Fotio","family":"Tiotsop","sequence":"first","affiliation":[{"name":"Politecnico di Torino, Torino, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tomas","family":"Mizdos","sequence":"additional","affiliation":[{"name":"University of Zilina, Zilina, Slovakia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marcus","family":"Barkowsky","sequence":"additional","affiliation":[{"name":"Deggendorf Institute of Technology, University of Applied Sciences, Deggendorf, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Pocta","sequence":"additional","affiliation":[{"name":"University of Zilina, Zilina, Slovakia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Antonio","family":"Servetti","sequence":"additional","affiliation":[{"name":"Politecnico di Torino, Torino, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8906-354X","authenticated-orcid":false,"given":"Enrico","family":"Masala","sequence":"additional","affiliation":[{"name":"Politecnico di Torino, Torino, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,1,27]]},"reference":[{"doi-asserted-by":"publisher","key":"e_1_3_1_2_2","DOI":"10.1109\/TCSVT.2018.2868262"},{"doi-asserted-by":"publisher","key":"e_1_3_1_3_2","DOI":"10.1587\/transcom.E98.B.2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_4_2","DOI":"10.1111\/j.1475-1313.1997.tb00072.x"},{"unstructured":"Kjell Brunnstr\u00f6m et\u00a0al. 2012. Qualinet white paper on definitions of Quality of Experience. (2012). European Network on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003).","key":"e_1_3_1_5_2"},{"unstructured":"ITU-R Rec. BT.500-11. 2002. Methodology for the subjective assessment of the quality of television pictures. (June 2002).","key":"e_1_3_1_6_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_7_2","DOI":"10.1109\/TBC.2011.2104671"},{"unstructured":"Cisco. 2020. Annual Internet Report: Growth in Internet users (2018-2023). (2020). https:\/\/www.cisco.com\/c\/en\/us\/solutions\/collateral\/executive-perspectives\/annual-internet-report\/white-paper-c11-741490.html.","key":"e_1_3_1_8_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_9_2","DOI":"10.1145\/3407190"},{"doi-asserted-by":"publisher","key":"e_1_3_1_10_2","DOI":"10.1109\/QoMEX.2019.8743303"},{"key":"e_1_3_1_11_2","volume-title":"Proceedings of Human Vision and Electronic Imaging Conference (HVEI)","author":"Tiotsop Lohic Fotio","year":"2020","unstructured":"Lohic Fotio Tiotsop, Tomas Mizdos, Miroslav Uhrina, Peter Pocta, Marcus Barkowsky, and Enrico Masala. 2020. Predicting single observer\u2019s votes from objective measures using neural networks. In Proceedings of Human Vision and Electronic Imaging Conference (HVEI). Society for Imaging Science and Technology (IS&T), Burlingame, CA, USA."},{"key":"e_1_3_1_12_2","first-page":"2737","volume-title":"Proc. Intl. Conf. Acoustics, Speech, and Signal Processing (ICASSP)","author":"Tiotsop Lohic Fotio","year":"2020","unstructured":"Lohic Fotio Tiotsop, Antonio Servetti, and Enrico Masala. 2020. Full reference video quality measure improvement using neural networks. In Proc. Intl. Conf. Acoustics, Speech, and Signal Processing (ICASSP). IEEE, Barcelona, Spain, 2737\u20132741. DOI:https:\/\/doi.org\/10.1109\/ICASSP40776.2020.9053739"},{"doi-asserted-by":"publisher","key":"e_1_3_1_13_2","DOI":"10.1007\/s11042-016-3360-z"},{"unstructured":"Internet Media Group. 2019. Extension of the ITS4S Dataset. (Jan 2019). http:\/\/media.polito.it\/its4s.","key":"e_1_3_1_14_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_15_2","DOI":"10.1007\/s41233-016-0002-1"},{"key":"e_1_3_1_16_2","first-page":"131","volume-title":"Third International Workshop on Quality of Multimedia Experience (QoMEX)","author":"Ho\u00dffeld Tobias","year":"2011","unstructured":"Tobias Ho\u00dffeld, Raimund Schatz, and Sebastian Egger. 2011. SOS: The MOS is not enough!. In Third International Workshop on Quality of Multimedia Experience (QoMEX). IEEE, Mechelen, Belgium, 131\u2013136. DOI:https:\/\/doi.org\/10.1109\/QoMEX.2011.6065690"},{"doi-asserted-by":"publisher","key":"e_1_3_1_17_2","DOI":"10.1145\/2598779"},{"doi-asserted-by":"publisher","key":"e_1_3_1_18_2","DOI":"10.1007\/978-3-642-28962-0_20"},{"unstructured":"ITU-T Rec. G.100 Amd. 1. 2007. Definition of Quality of Experience (QoE). (Jan 2007).","key":"e_1_3_1_19_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_20_2","DOI":"10.1109\/TBC.2018.2823914"},{"key":"e_1_3_1_21_2","first-page":"35","volume-title":"International Workshop on Quality of Multimedia Experience (QoMEX)","author":"Janowski Lucjan","year":"2009","unstructured":"Lucjan Janowski and Zdzislaw Papir. 2009. Modeling subjective tests of quality of experience with a generalized linear model. In International Workshop on Quality of Multimedia Experience (QoMEX). IEEE, San Diego, CA, USA, 35\u201340. DOI:https:\/\/doi.org\/10.1109\/QOMEX.2009.5246979"},{"doi-asserted-by":"publisher","key":"e_1_3_1_22_2","DOI":"10.1109\/TMM.2015.2484963"},{"doi-asserted-by":"publisher","key":"e_1_3_1_23_2","DOI":"10.1145\/1556134.1556137"},{"doi-asserted-by":"publisher","key":"e_1_3_1_24_2","DOI":"10.1109\/CVPR.2019.00836"},{"doi-asserted-by":"publisher","key":"e_1_3_1_25_2","DOI":"10.1109\/TMM.2019.2935687"},{"doi-asserted-by":"publisher","key":"e_1_3_1_26_2","DOI":"10.1109\/TNN.2006.879766"},{"doi-asserted-by":"publisher","key":"e_1_3_1_27_2","DOI":"10.1007\/s11042-014-2229-2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_28_2","DOI":"10.1109\/DCC.2017.26"},{"doi-asserted-by":"publisher","key":"e_1_3_1_29_2","DOI":"10.1016\/j.jesp.2018.08.009"},{"doi-asserted-by":"publisher","key":"e_1_3_1_30_2","DOI":"10.1109\/TMC.2013.155"},{"key":"e_1_3_1_31_2","volume-title":"IHM-HCI Tutorial","author":"Mullin Jim","year":"2001","unstructured":"Jim Mullin, Lucy Smallwood, Anna Watson, and Gillian Wilson. 2001. New techniques for assessing audio and video quality in real-time interactive communications. In IHM-HCI Tutorial. Lille, France."},{"doi-asserted-by":"publisher","key":"e_1_3_1_32_2","DOI":"10.1016\/j.intcom.2010.08.005"},{"unstructured":"Netflix. 2019. VMAF - Video Multi-Method Assessment Fusion. https:\/\/github.com\/Netflix\/vmaf. (Jan. 2019).","key":"e_1_3_1_33_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_34_2","DOI":"10.1007\/s10459-010-9222-y"},{"unstructured":"ITU-T Rec. P.910. 2008. Subjective video quality assessment methods for multimedia applications. (Apr 2008).","key":"e_1_3_1_35_2"},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1007\/978-3-642-30422-4_19","volume-title":"Proc. Intl. Conf. on Mobile Networks and Management","author":"Palhais Joana","year":"2012","unstructured":"Joana Palhais, Rui S. Cruz, and M\u00e1rio S. Nunes. 2012. Quality of experience assessment in internet TV. In Proc. Intl. Conf. on Mobile Networks and Management. Springer, Aveiro, Portugal, 261\u2013274."},{"unstructured":"Margaret H. Pinson. 2018. ITS4S: A Video Quality Dataset with Four-Second Unrepeated Scenes. (Feb 2018). NTIA Technical Memo TM-18-532.","key":"e_1_3_1_37_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_38_2","DOI":"10.1037\/a0029315"},{"key":"e_1_3_1_39_2","volume-title":"2007 2nd International Symposium on Wireless Pervasive Computing","author":"Ries M.","year":"2007","unstructured":"M. Ries, O. Nemethova, and M. Rupp. 2007. Motion based reference-free quality estimation for H.264\/AVC video streaming. In 2007 2nd International Symposium on Wireless Pervasive Computing. IEEE, San Juan, Puerto Rico. DOI:https:\/\/doi.org\/10.1109\/ISWPC.2007.342629"},{"doi-asserted-by":"publisher","key":"e_1_3_1_40_2","DOI":"10.1109\/TMM.2016.2574623"},{"doi-asserted-by":"publisher","key":"e_1_3_1_41_2","DOI":"10.1109\/QoMEX.2019.8743296"},{"doi-asserted-by":"publisher","key":"e_1_3_1_42_2","DOI":"10.1109\/TIP.2005.859378"},{"doi-asserted-by":"publisher","key":"e_1_3_1_43_2","DOI":"10.1007\/s00530-014-0446-1"},{"issue":"63","key":"e_1_3_1_44_2","first-page":"52","article-title":"Test distance vision using a Snellen chart.","volume":"20","author":"Sue Stevens","year":"2007","unstructured":"Stevens Sue. 2007. Test distance vision using a Snellen chart.Community Eye Health 20, 63 (2007), 52.","journal-title":"Community Eye Health"},{"doi-asserted-by":"publisher","key":"e_1_3_1_45_2","DOI":"10.1007\/s11063-019-10036-6"},{"doi-asserted-by":"publisher","key":"e_1_3_1_46_2","DOI":"10.1109\/ICME.2018.8486528"},{"unstructured":"VQEG. 2010. Report on the Validation of Video Quality Models for High Definition Video Content (v. 2.0). (Jun 2010). http:\/\/bit.ly\/2Z7GWDI","key":"e_1_3_1_47_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_48_2","DOI":"10.1109\/ACSSC.2003.1292216"},{"key":"e_1_3_1_49_2","first-page":"175","volume-title":"Proc. IWSDS Workshop on Paralinguistic Information and its Integration in Spoken Dialogue Systems","author":"Wechsung Ina","year":"2011","unstructured":"Ina Wechsung, Matthias Schulz, Klaus-Peter Engelbrecht, Julia Niemann, and Sebastian M\u00f6ller. 2011. All users are (not) equal - the influence of user characteristics on perceived quality, modality choice and performance. In Proc. IWSDS Workshop on Paralinguistic Information and its Integration in Spoken Dialogue Systems. Springer, New York, NY, USA, 175\u2013186. DOI:https:\/\/doi.org\/10.1007\/978-1-4614-1335-6_19"},{"doi-asserted-by":"publisher","key":"e_1_3_1_50_2","DOI":"10.5555\/3304222.3304393"},{"doi-asserted-by":"publisher","key":"e_1_3_1_51_2","DOI":"10.1109\/TBC.2008.2000733"},{"doi-asserted-by":"publisher","key":"e_1_3_1_52_2","DOI":"10.1109\/TMM.2016.2525004"},{"doi-asserted-by":"publisher","key":"e_1_3_1_53_2","DOI":"10.1109\/TMM.2019.2904879"},{"doi-asserted-by":"publisher","key":"e_1_3_1_54_2","DOI":"10.4304\/jcp.7.1.161-168"},{"doi-asserted-by":"publisher","key":"e_1_3_1_55_2","DOI":"10.1109\/ICIP.2019.8803395"},{"unstructured":"Hui Zeng Lei Zhang and Alan C Bovik. 2017. A probabilistic quality representation approach to deep blind image quality prediction. (2017). arXiv:arXiv:1708.08190v2","key":"e_1_3_1_56_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_57_2","DOI":"10.1109\/TNNLS.2018.2890310"},{"doi-asserted-by":"publisher","key":"e_1_3_1_58_2","DOI":"10.1109\/TIP.2003.819861"},{"doi-asserted-by":"publisher","key":"e_1_3_1_59_2","DOI":"10.1145\/3183512"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3464393","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3464393","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:10Z","timestamp":1750191430000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3464393"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,27]]},"references-count":58,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1,31]]}},"alternative-id":["10.1145\/3464393"],"URL":"https:\/\/doi.org\/10.1145\/3464393","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2022,1,27]]},"assertion":[{"value":"2020-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-01-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}