{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:21:52Z","timestamp":1750220512669,"version":"3.41.0"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,11,12]],"date-time":"2021-11-12T00:00:00Z","timestamp":1636675200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation (NSF) Center for Big Learning","award":["1747751"],"award-info":[{"award-number":["1747751"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2021,11,30]]},"abstract":"<jats:p>The block-based coding structure in the hybrid video coding framework inevitably introduces compression artifacts such as blocking, ringing, and so on. To compensate for those artifacts, extensive filtering techniques were proposed in the loop of video codecs, which are capable of boosting the subjective and objective qualities of reconstructed videos. Recently, neural network-based filters were presented with the power of deep learning from a large magnitude of data. Though the coding efficiency has been improved from traditional methods in High-Efficiency Video Coding (HEVC), the rich features and information generated by the compression pipeline have not been fully utilized in the design of neural networks. Therefore, in this article, we propose the Residual-Reconstruction-based Convolutional Neural Network (RRNet) to further improve the coding efficiency to its full extent, where the compression features induced from bitstream in form of prediction residual are fed into the network as an additional input to the reconstructed frame. In essence, the residual signal can provide valuable information about block partitions and can aid reconstruction of edge and texture regions in a picture. Thus, more adaptive parameters can be trained to handle different texture characteristics. The experimental results show that our proposed RRNet approach presents significant BD-rate savings compared to HEVC and the state-of-the-art CNN-based schemes, indicating that residual signal plays a significant role in enhancing video frame reconstruction.<\/jats:p>","DOI":"10.1145\/3460820","type":"journal-article","created":{"date-parts":[[2021,11,12]],"date-time":"2021-11-12T21:16:06Z","timestamp":1636751766000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Residual-guided In-loop Filter Using Convolution Neural Network"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0053-6959","authenticated-orcid":false,"given":"Wei","family":"Jia","sequence":"first","affiliation":[{"name":"University of Missouri-Kansas City, MO, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li","family":"Li","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, He Fei, An Hui, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhu","family":"Li","sequence":"additional","affiliation":[{"name":"University of Missouri-Kansas City, MO, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiang","family":"Zhang","sequence":"additional","affiliation":[{"name":"Tencent America, Palo Alto, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shan","family":"Liu","sequence":"additional","affiliation":[{"name":"Tencent America, Palo Alto, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,11,12]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2017.150"},{"key":"e_1_3_1_3_2","unstructured":"Fabrice Bellard. 2019. FFmpeg software A complete cross-platform solution to record convert and stream audio and video. Retrieved from http:\/\/ffmpeg.org\/."},{"key":"e_1_3_1_4_2","unstructured":"Gisle Bjontegaard. 2001. Calculation of average PSNR differences between RD-curves. Document VCEG-M33 Austin Texas."},{"key":"e_1_3_1_5_2","unstructured":"Benjamin Bross Jianle Chen and Shan Liu. 2019. Versatile video coding (draft 4). Document ITU-T SG 16 WP 3 and ISO\/IEC JTC 1\/SC 29\/WG 11 JVET-M1001-v6 Marrakech MA."},{"key":"e_1_3_1_6_2","article-title":"Variational lossy autoencoder","author":"Chen Xi","year":"2016","unstructured":"Xi Chen, Diederik P. Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. Variational lossy autoencoder. arXiv preprint arXiv:1611.02731 (2016).","journal-title":"arXiv preprint arXiv:1611.02731"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1631\/FITEE.1700789"},{"key":"e_1_3_1_8_2","article-title":"A survey of model compression and acceleration for deep neural networks","author":"Cheng Yu","year":"2017","unstructured":"Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang. 2017. A survey of model compression and acceleration for deep neural networks. arXiv preprint arXiv:1710.09282 (2017).","journal-title":"arXiv preprint arXiv:1710.09282"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2765695"},{"key":"e_1_3_1_10_2","unstructured":"W.-J. Chien and M. Karczewicz. 2009. Adaptive filter based on combination of sum-modified Laplacian filter indexing and quadtree partitioning. ITU-T\/ISO\/IEC JCT-VC Document VCEG-AL27."},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-51811-4_3"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.73"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2012.2221529"},{"key":"e_1_3_1_14_2","unstructured":"C.-M. Fu C.-Y. Chen and Y.-W. Huang. 2010. TE10 subtest 3: Quadtree-based adaptive offset. ITU-T\/ISO\/IEC JCT-VC Document JCTVC-C147."},{"key":"e_1_3_1_15_2","unstructured":"C.-M. Fu C.-Y. Chen and Y.-W. Huang. 2011. CE8 subset 3: Picture quadtree adaptive offset. ITU-T\/ISO\/IEC JCT-VC Document JCTVC-D122."},{"key":"e_1_3_1_16_2","unstructured":"C.-M. Fu C.-Y. Chen and C.-Y. Tsai. 2011. CE13: Sample adaptive offset with LCU-independent decoding. ITU-T\/ISO\/IEC JCT-VC Document JCTVC-E049."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvcir.2014.03.001"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.123"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2018.8451086"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.1127647"},{"key":"e_1_3_1_22_2","unstructured":"Y.-W. Huang C.-M. Fu and C.-Y. Chen. 2010. In-loop adaptive restoration. ITU-T\/ISO\/IEC JCT-VC Document JCTVC-B077."},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-11021-5_20"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2896489"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/VCIP.2017.8305149"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP40778.2020.9191106"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2019.2938340"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2015.2434216"},{"key":"e_1_3_1_29_2","article-title":"Adam: A method for stochastic optimization","author":"Kingma Diederik P.","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).","journal-title":"arXiv preprint arXiv:1412.6980"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2017.2766889"},{"key":"e_1_3_1_31_2","unstructured":"Yiming Li Shan Liu and Kei Kawamura. 2019. Methodology and reporting template for neural network coding tool testing. Document ITU-T SG 16 WP 3 and ISO\/IEC JTC 1\/SC 29\/WG 11 JVET-M1006-v1 Marrakech MA."},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2017.151"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2011.2133250"},{"key":"e_1_3_1_34_2","volume-title":"IEEE Trans. Multimedia","author":"Lin Weiyao","year":"2019","unstructured":"Weiyao Lin, Xiaoyi He, Xintong Han, Dong Liu, John See, Junni Zou, Hongkai Xiong, and Feng Wu. 2019. Partition-aware adaptive switching neural networks for post-processing in HEVC. IEEE Trans. Multimedia 22, 11 (2019)."},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2003.815175"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2007.897467"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01264-9_35"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2016.16"},{"key":"e_1_3_1_39_2","unstructured":"Ken McCann Woo-Jin Han and Il-Koo Kim. 2010. Samsung\u2019s response to the call for proposals on video compression technology. ITU-T SG16 WP3 and ISO\/IEC JTC1\/SC29\/WG11 JCT-VC Document JCTVC-A124 Dresden DE."},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2012.2223053"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/IVMSPW.2016.7528223"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_3_1_43_2","unstructured":"Karl Sharman and Karsten Suehring. 2018. Common test conditions. Document ITU-T SG 16 WP 3 and ISO\/IEC JTC 1\/SC 29\/WG 11 JCTVC-AE1100 San Diego US."},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2012.2221191"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.479"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2013.2271974"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.5555\/3157096.3157158"},{"key":"e_1_3_1_48_2","volume-title":"Mathematical Statistics with Applications","author":"Wackerly Dennis","year":"2014","unstructured":"Dennis Wackerly, William Mendenhall, and Richard L. Scheaffer. 2014. Mathematical Statistics with Applications. Cengage Learning."},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/DCC.2017.42"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/VCIP.2018.8698740"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.302"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2003.815165"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-018-01144-2"},{"key":"e_1_3_1_54_2","volume-title":"IEEE Trans. Circ. Syst. Vid. Technol.","author":"Yang Ren","year":"2018","unstructured":"Ren Yang, Mai Xu, Tie Liu, Zulin Wang, and Zhenyu Guan. 2018. Enhancing quality for HEVC compressed videos. IEEE Trans. Circ. Syst. Vid. Technol. 29, 7 (2018), 2039\u20132054."},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2017.8019299"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539957"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2016.2581618"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2815841"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2012.2190391"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460820","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3460820","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:24:28Z","timestamp":1750195468000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460820"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,12]]},"references-count":58,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,11,30]]}},"alternative-id":["10.1145\/3460820"],"URL":"https:\/\/doi.org\/10.1145\/3460820","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2021,11,12]]},"assertion":[{"value":"2020-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-11-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}