{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T02:27:22Z","timestamp":1775010442901,"version":"3.50.1"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T00:00:00Z","timestamp":1646352000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62101007, 62031013, 62088102, 62071449"],"award-info":[{"award-number":["62101007, 62031013, 62088102, 62071449"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100012152","name":"National Postdoctoral Program for Innovative Talents","doi-asserted-by":"crossref","award":["BX2021009"],"award-info":[{"award-number":["BX2021009"]}],"id":[{"id":"10.13039\/501100012152","id-type":"DOI","asserted-by":"crossref"}]},{"name":"High performance Computing Platform of Peking University"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2022,11,30]]},"abstract":"<jats:p>Recently, deep learning for video coding, such as deep predictive coding, deep transform coding, and deep in-loop filtering, has been an emerging research area. The coding gain of hybrid coding framework could be extensively promoted by the data-driven models. However, previous deep coding tools especially deep in-loop filtering mainly consider the performance improvement while pay less attention to the reliability, usability, and adaptivity of the networks. In this article, a nested-residual guided convolutional neural network (NR-CNN) structure with cascaded global shortcut and configurable residual blocks is proposed for in-loop filtering. By taking advantage of the correlation between different color components, we further extend the NR-CNN by utilizing luminance as textural and structural guidance for chrominance filtering, which significantly improves the filtering performance. To fully exploit the proposed network into codec integration, we subsequently introduce an efficient and adaptive framework consisting of an adaptive granularity optimization and a parallel inference pipeline for deep learning based filtering. The former contributes to the coding performance improvement through an adaptive decision-making based on rate-distortion analysis at various granularities. The latter reduces the running time of network inference. The extensive experimental results show the superiority of the proposed method, achieving 8.2%, 14.9%, and 13.2% BD-rate savings on average under random access (RA) configuration. Meanwhile, the proposed method also obtains better subjective quality.<\/jats:p>","DOI":"10.1145\/3502723","type":"journal-article","created":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T10:31:58Z","timestamp":1646389918000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["NR-CNN: Nested-Residual Guided CNN In-loop Filtering for Video Coding"],"prefix":"10.1145","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0898-0771","authenticated-orcid":false,"given":"Kai","family":"Lin","sequence":"first","affiliation":[{"name":"Peking University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chuanmin","family":"Jia","sequence":"additional","affiliation":[{"name":"Peking University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xinfeng","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shanshe","family":"Wang","sequence":"additional","affiliation":[{"name":"Peking University, Information Technology R&amp;D InnovationCenter of Peking University, Peng Cheng Laboratory, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Siwei","family":"Ma","sequence":"additional","affiliation":[{"name":"Peking University, Information Technology R&amp;D InnovationCenter of Peking University, Peng Cheng Laboratory, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wen","family":"Gao","sequence":"additional","affiliation":[{"name":"Peking University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,3,4]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2017.150"},{"key":"e_1_3_2_3_2","article-title":"The common test conditions of AVS3","author":"Chen Jie","year":"2019","unstructured":"Jie Chen and Kui Fan. 2019. The common test conditions of AVS3. AVS-Doc, N2727.","journal-title":"AVS-Doc, N2727"},{"key":"e_1_3_2_4_2","article-title":"Algorithm description for versatile video coding and test model 10 (VTM\u201910)","author":"Chen Jianle","year":"2020","unstructured":"Jianle Chen, Yan Ye, and Seung Hwan Kim. 2020. Algorithm description for versatile video coding and test model 10 (VTM\u201910). JVET-S2002.","journal-title":"JVET-S2002"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-51811-4_3"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.73"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/tcsvt.2012.2221529"},{"key":"e_1_3_2_8_2","article-title":"Adaptive clipping in JEM2.0","author":"Galpin Fu","year":"2016","unstructured":"Fu Galpin, Pu Bordes, and F. Le L\u00e9annec. 2016. Adaptive clipping in JEM2.0. JVET-C0040.","journal-title":"JVET-C0040"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2021.3104305"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.5555\/1886063.1886065"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2018.8451086"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/DCC.2018.00066"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.243"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS.2018.8351609"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2896489"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/VCIP.2017.8305149"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460820"},{"key":"e_1_3_2_19_2","article-title":"An enhanced sample adaptive offset filtering method for AVS3","author":"Jian Yunrui","year":"2020","unstructured":"Yunrui Jian, Jiaqi Zhang, Falei Luo, Shanshe Wang, Siwei Ma, Lin Li, Yi Su, and Yanan Feng. 2020. An enhanced sample adaptive offset filtering method for AVS3. AVS-M5373.","journal-title":"AVS-M5373"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/PCS.2016.7906346"},{"key":"e_1_3_2_21_2","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR (Poster) ."},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/PCS.2018.8456278"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00034-019-01110-4"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.image.2021.116409"},{"key":"e_1_3_2_25_2","article-title":"Cross-component sample adaptive offset","author":"Kuo Che-Wei","year":"2020","unstructured":"Che-Wei Kuo, Xiaoyu Xiu, Wei Chen, Yi-Wen Chen, Hong-Jheng Jhu, and Xianglin Wang. 2020. Cross-component sample adaptive offset. AVS-M5800.","journal-title":"AVS-M5800"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS.2019.8702443"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2817044"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/DCC47342.2020.00028"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/PCS48520.2019.8954561"},{"key":"e_1_3_2_30_2","doi-asserted-by":"crossref","unstructured":"Di Ma Fan Zhang and David R Bull. 2021. BVI-DVC: A training database for deep video compression. IEEE Transactions on Multimedia .","DOI":"10.1109\/TMM.2021.3108943"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2014.2371951"},{"key":"e_1_3_2_32_2","article-title":"Image and video compression with neural networks: A review","author":"Ma Siwei","year":"2019","unstructured":"Siwei Ma, Xinfeng Zhang, Chuanmin Jia, Zhenghui Zhao, Shiqi Wang, and Shanshe Wanga. 2019. Image and video compression with neural networks: A review. IEEE Transactions on Circuits and Systems for Video Technology 30, 6 (2019), 1683\u20131698.","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/PCS48520.2019.8954547"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.5555\/3104322.3104425"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2012.2223053"},{"key":"e_1_3_2_36_2","article-title":"In-loop reshaping for SDR video","author":"Pu Fangjun","year":"2018","unstructured":"Fangjun Pu, Taoran Lu, Yin Peng, Walt Husak, Sean McCarthy, and Tao Chen. 2018. In-loop reshaping for SDR video. JVET-K0309.","journal-title":"JVET-K0309"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2018.8451589"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2012.2221191"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2013.2271974"},{"key":"e_1_3_2_40_2","article-title":"Extended coding unit partitioning for future video coding","author":"Wang Meng","year":"2019","unstructured":"Meng Wang, Junru Li, Li Zhang, Kai Zhang, Hongbin Liu, Shiqi Wang, Sam Kwong, and Siwei Ma. 2019. Extended coding unit partitioning for future video coding. IEEE Transactions on Image Processing 29, (2019), 2931\u20132946","journal-title":"IEEE Transactions on Image Processing"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/VCIP.2018.8698740"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/VCIP.2017.8305038"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2003.815168"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS.2019.8702522"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2847035"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/PCS48520.2019.8954503"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISM.2015.90"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2830640"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2662206"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/PCS.2012.6213380"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2815841"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2913545"},{"key":"e_1_3_2_53_2","article-title":"Enhanced bi-prediction with convolutional neural network for high efficiency video coding","author":"Zhao Zhenghui","year":"2018","unstructured":"Zhenghui Zhao, Shiqi Wang, Shanshe Wang, Xinfeng Zhang, Siwei Ma, and Jiansheng Yang. 2018. Enhanced bi-prediction with convolutional neural network for high efficiency video coding. IEEE Transactions on Circuits and Systems for Video Technology 29, 11 (2018), 3291\u20133301.","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICMEW46912.2020.9105970"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3502723","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3502723","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:47Z","timestamp":1750183787000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3502723"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,4]]},"references-count":53,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,11,30]]}},"alternative-id":["10.1145\/3502723"],"URL":"https:\/\/doi.org\/10.1145\/3502723","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,4]]},"assertion":[{"value":"2021-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-03-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}