{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:48:16Z","timestamp":1760240896424,"version":"build-2065373602"},"reference-count":51,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2019,10,2]],"date-time":"2019-10-02T00:00:00Z","timestamp":1569974400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In past years, several visual saliency algorithms have been proposed to extract salient regions from multimedia content in view of practical applications. Entropy is one of the important measures to extract salient regions, as these regions have high randomness and attract more visual attention. In the context of perceptual video coding (PVC), computational visual saliency models that utilize the charactertistics of the human visual system to improve the compression ratio are of paramount importance. To date, only a few PVC schemes have been reported that use the visual saliency model. In this paper, we conduct the first attempt to utilize entropy based visual saliency models within the high efficiency video coding (HEVC) framework. The visual saliency map generated for each input video frame is optimally thresholded to generate a binary saliency mask. The proposed HEVC compliant PVC scheme adjusts the quantization parameter according to visual saliency relevance at the coding tree unit (CTU) level. Efficient CTU level rate control is achieved by allocating bits to salient and non-salient CTUs by adjusting the quantization parameter values according to their perceptual weighted map. The attention based on information maximization has shown the best performance on newly created ground truth dataset, which is then incorporated in a HEVC framework. An average bitrate reduction of     6.57 %     is achieved by the proposed HEVC compliant PVC scheme with the same perceptual quality and a nominal increase in coding complexity of     3.34 %     when compared with HEVC reference software. Moreover, the proposed PVC scheme performs better than other HEVC based PVC schemes when encoded at low data rates.<\/jats:p>","DOI":"10.3390\/e21100964","type":"journal-article","created":{"date-parts":[[2019,10,2]],"date-time":"2019-10-02T08:17:54Z","timestamp":1570004274000},"page":"964","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["High Efficiency Video Coding Compliant Perceptual Video Coding Using Entropy Based Visual Saliency Model"],"prefix":"10.3390","volume":"21","author":[{"given":"Muhammad","family":"Zeeshan","sequence":"first","affiliation":[{"name":"Department of Computer Engineering, University of Engineering and Technology, Taxila 47050, Pakistan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3662-2525","authenticated-orcid":false,"given":"Muhammad","family":"Majid","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, University of Engineering and Technology, Taxila 47050, Pakistan"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,2]]},"reference":[{"key":"ref_1","first-page":"40","article-title":"High efficiency video coding (HEVC)","volume":"Volume 39","author":"Sze","year":"2014","journal-title":"Integrated Circuit and Systems, Algorithms and Architectures"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1109\/TCSVT.2015.2428571","article-title":"Algorithm and architecture design of the H. 265\/HEVC intra encoder","volume":"26","author":"Pastuszak","year":"2015","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pastuszak, G. (2016, January 5\u20137). High-speed architecture of the CABAC probability modeling for H. 265\/HEVC encoders. Proceedings of the 2016 International Conference on Signals and Electronic Systems (ICSES), Krakow, Poland.","DOI":"10.1109\/ICSES.2016.7593839"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1544","DOI":"10.1109\/TCSVT.2018.2830126","article-title":"A Highly Parallel Hardware Architecture of Table-Based CABAC Bit Rate Estimator in an HEVC Intra Encoder","volume":"29","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhang, Y., and Lu, C. (2019). High-Performance Algorithm Adaptations and Hardware Architecture for HEVC Intra Encoders. IEEE Trans. Circuits Syst. Video Technol.","DOI":"10.1109\/TCSVT.2019.2913504"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"684","DOI":"10.1109\/JSTSP.2012.2215006","article-title":"Perceptual video compression: A survey","volume":"6","author":"Lee","year":"2012","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1526","DOI":"10.1109\/TIP.2012.2233485","article-title":"Video processing for human perceptual visual quality-oriented video coding","volume":"22","author":"Oh","year":"2013","journal-title":"IEEE Trans. Image Process."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1727","DOI":"10.1109\/TCSVT.2008.2005798","article-title":"Bayesian integration of face and low-level cues for foveated video coding","volume":"18","author":"Boccignone","year":"2008","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1142\/S0129065707001147","article-title":"An embedded saliency map estimator scheme: Application to video encoding","volume":"17","author":"Tsapatsoulis","year":"2007","journal-title":"Int. J. Neural Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1109\/TCSVT.2010.2045912","article-title":"Perceptually-friendly H.264\/AVC video coding based on foveated just-noticeable-distortion model","volume":"20","author":"Chen","year":"2010","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Rehman, A., and Wang, Z. (2012, January 9\u201313). SSIM-inspired perceptual video coding for HEVC. Proceedings of the 2012 IEEE International Conference on Multimedia and Expo (ICME), Melbourne, VIC, Australia.","DOI":"10.1109\/ICME.2012.175"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1109\/30.234634","article-title":"An adaptive perceptual quantization algorithm for video coding","volume":"39","author":"Chun","year":"1993","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1109\/TCE.2007.381759","article-title":"Macroblock-level adaptive frequency weighting for perceptual video coding","volume":"53","author":"Chen","year":"2007","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1006","DOI":"10.1016\/j.image.2013.07.003","article-title":"Visual saliency guided video compression algorithm","volume":"28","author":"Gupta","year":"2013","journal-title":"Signal Process. Image Commun."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.image.2015.04.011","article-title":"Weight-based R-\u03bb rate control for perceptual HEVC coding on conversational videos","volume":"38","author":"Li","year":"2015","journal-title":"Signal Process. Image Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1109\/JSTSP.2014.2314864","article-title":"Region-of-interest based conversational HEVC coding with hierarchical perception model of face","volume":"8","author":"Xu","year":"2014","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3343","DOI":"10.1109\/TIP.2016.2568459","article-title":"HEVC-Based Perceptually Adaptive Video Coding Using a DCT-Based Local Distortion Detection Probability Model","volume":"25","author":"Bae","year":"2016","journal-title":"IEEE Trans. Image Process."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yang, T., Yu, M., Peng, Z., Jiang, G., Jiang, H., Zeng, X., and Zhang, Q. (2017, January 2\u20134). New rate distortion optimization method for video coding. Proceedings of the 2017 2nd International Conference on Image, Vision and Computing (ICIVC), Chengdu, China.","DOI":"10.1109\/ICIVC.2017.7984600"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3841","DOI":"10.1109\/TIP.2014.2336550","article-title":"lambda Domain Rate Control Algorithm for High Efficiency Video Coding","volume":"23","author":"Li","year":"2014","journal-title":"IEEE Trans. Image Process."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Jiang, X., Song, T., Zhu, D., Katayama, T., and Wang, L. (2019). Quality-Oriented Perceptual HEVC Based on the Spatiotemporal Saliency Detection Model. Entropy, 21.","DOI":"10.3390\/e21020165"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"53","DOI":"10.4236\/jcc.2018.64005","article-title":"A Perceptual Video Coding Based on JND Model","volume":"6","author":"Yi","year":"2018","journal-title":"J. Comput. Commun."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"12777","DOI":"10.1007\/s11042-017-4914-4","article-title":"A fast and HEVC-compatible perceptual video coding scheme using a transform-domain Multi-Channel JND model","volume":"77","author":"Wang","year":"2018","journal-title":"Multimed. Tools Appl."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1145\/1658349.1658355","article-title":"Computational visual attention systems and their cognitive foundations: A survey","volume":"7","author":"Frintrop","year":"2010","journal-title":"ACM Trans. Appl. Percept."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yan, L., Zhu, R., Liu, Y., and Mo, N. (2018). Color-Boosted Saliency-Guided Rotation Invariant Bag of Visual Words Representation with Parameter Transfer for Cross-Domain Scene-Level Classification. Remote Sens., 10.","DOI":"10.3390\/rs10040610"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wang, C., Zhang, T., Wan, W., Han, X., and Xu, M. (2017). A Novel STDM Watermarking Using Visual Saliency-Based JND Model. Information, 8.","DOI":"10.3390\/info8030103"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"24332","DOI":"10.1109\/ACCESS.2017.2767622","article-title":"A Patch-Based Saliency Detection Method for Assessing the Visual Privacy Levels of Objects in Photos","volume":"5","author":"Li","year":"2017","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Gao, R., Uchida, S., Shahab, A., Shafait, F., and Frinken, V. (2014). Visual saliency models for text detection in real world. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0114539"},{"key":"ref_28","first-page":"1250","article-title":"Visual saliency detection based object recognition","volume":"6","author":"Gu","year":"2015","journal-title":"J. Inf. Hiding Multimed. Signal Process."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.optlaseng.2019.06.011","article-title":"High-accuracy multi-camera reconstruction enhanced by adaptive point cloud correction algorithm","volume":"122","author":"Chen","year":"2019","journal-title":"Opt. Lasers Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"372","DOI":"10.1016\/j.measurement.2018.08.026","article-title":"Binocular vision measurement and its application in full-field convex deformation of concrete-filled steel tubular columns","volume":"130","author":"Tang","year":"2018","journal-title":"Measurement"},{"key":"ref_31","unstructured":"Ogasawara, K., Miyazaki, T., Sugaya, Y., and Omachi, S. (2017). Object-Based Video Coding by Visual Saliency and Temporal Correlation. IEEE Trans. Emerg. Top. Comput."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Judd, T., Ehinger, K., Durand, F., and Torralba, A. (October, January 29). Learning to predict where humans look. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.","DOI":"10.1109\/ICCV.2009.5459462"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1254","DOI":"10.1109\/34.730558","article-title":"A model of saliency-based visual attention for rapid scene analysis","volume":"20","author":"Itti","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1038\/nrn1411","article-title":"What attributes guide the deployment of visual attention and how do they do it?","volume":"5","author":"Wolfe","year":"2004","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/0010-0285(80)90005-5","article-title":"A feature-integration theory of attention","volume":"12","author":"Treisman","year":"1980","journal-title":"Cogn. Psychol."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Koch, C., and Ullman, S. (1987). Shifts in selective visual attention: Towards the underlying neural circuitry. Matters of Intelligence, Springer.","DOI":"10.1007\/978-94-009-3833-5_5"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"5.1","DOI":"10.1167\/9.3.5","article-title":"Saliency, attention, and visual search: An information theoretic approach","volume":"9","author":"Bruce","year":"2009","journal-title":"J. Vis."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"3848","DOI":"10.3390\/e16073848","article-title":"Hierarchical geometry verification via maximum entropy saliency in image retrieval","volume":"16","author":"Zhao","year":"2014","journal-title":"Entropy"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Harel, J., Koch, C., and Perona, P. (2007, January 3\u20136). Graph-based visual saliency. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.","DOI":"10.7551\/mitpress\/7503.003.0073"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wang, W., Wang, Y., Huang, Q., and Gao, W. (2010, January 13\u201318). Measuring visual saliency by site entropy rate. Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539927"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1023\/A:1012460413855","article-title":"Saliency, scale and image description","volume":"45","author":"Kadir","year":"2001","journal-title":"Int. J. Comput. Vis."},{"key":"ref_42","unstructured":"Lin, Y., Fang, B., and Tang, Y. (2010, January 11\u201315). A Computational Model for Saliency Maps by Using Local Entropy. Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, Atlanta, GA, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Sarkar, S., Paul, S., Burman, R., Das, S., and Chaudhuri, S.S. (2014, January 18\u201320). A fuzzy entropy based multi-level image thresholding using differential evolution. Proceedings of the International Conference on Swarm, Evolutionary, and Memetic Computing, Bhubaneswar, India.","DOI":"10.1007\/978-3-319-20294-5_34"},{"key":"ref_44","unstructured":"Suzuki, T., Baroncini, V., Fran\u00e7ois, E., Topiwala, P., Wenger, S., and Yu, H. (2019, September 20). JCT-VC AHG Report: Test Sequence Material (AHG5). Technical Report, Joint Collaborative Team on Video Coding (JCT-VC). Available online: https:\/\/www.itu.int\/en\/ITU-T\/studygroups\/2017-2020\/16\/Pages\/video\/jctvc.aspx."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1109\/TPAMI.2014.2345401","article-title":"Global contrast based salient region detection","volume":"37","author":"Cheng","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_46","unstructured":"HM (2018, January 25). HM Reference Software 16.11. Available online: https:\/\/hevc.hhi.fraunhofer.de\/svn\/svn_HEVCSoftware\/."},{"key":"ref_47","unstructured":"Bossen, F. (2019, September 23). JCTVC-L1100: Common HM Test Conditions and Software Reference Configurations. JCT-VC Document Management System (April 2013). Available online: https:\/\/www.itu.int\/wftp3\/av-arch\/jctvc-site\/2013_01_L_Geneva\/."},{"key":"ref_48","unstructured":"Wang, Z., Simoncelli, E.P., and Bovik, A.C. (2003, January 9\u201312). Multiscale structural similarity for image quality assessment. Proceedings of the Thirty-Seventh Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"20955","DOI":"10.1007\/s11042-017-5499-7","article-title":"Visual saliency based redundancy allocation in HEVC compatible multiple description video coding","volume":"77","author":"Majid","year":"2018","journal-title":"Multimed. Tools Appl."},{"key":"ref_50","unstructured":"Series, B. (2019, September 20). Methodology for the Subjective Assessment of the Quality of Television Pictures. Recommendation ITU-R BT. Available online: http:\/\/www.itu.int\/dms_pubrec\/itu-r\/rec\/bt\/R-REC-BT.500-12-200909-S!!PDF-E.pdf."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"3178","DOI":"10.1109\/TIP.2018.2818439","article-title":"Learning-based just-noticeable-quantization-distortion modeling for perceptual video coding","volume":"27","author":"Ki","year":"2018","journal-title":"IEEE Trans. Image Process."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/10\/964\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:27:00Z","timestamp":1760189220000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/10\/964"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,2]]},"references-count":51,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2019,10]]}},"alternative-id":["e21100964"],"URL":"https:\/\/doi.org\/10.3390\/e21100964","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2019,10,2]]}}}