{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T19:31:24Z","timestamp":1785526284115,"version":"3.56.0"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,7,30]],"date-time":"2018-07-30T00:00:00Z","timestamp":1532908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,8,31]]},"abstract":"<jats:p>\n            We propose the first deep learning approach for exemplar-based local colorization. Given a reference color image, our convolutional neural network directly maps a grayscale image to an output colorized image. Rather than using hand-crafted rules as in traditional exemplar-based methods, our end-to-end colorization network learns how to\n            <jats:italic>select, propagate<\/jats:italic>\n            , and\n            <jats:italic>predict<\/jats:italic>\n            colors from the large-scale data. The approach performs robustly and generalizes well even when using reference images that are unrelated to the input grayscale image. More importantly, as opposed to other learning-based colorization methods, our network allows the user to achieve customizable results by simply feeding different references. In order to further reduce manual effort in selecting the references, the system automatically recommends references with our proposed image retrieval algorithm, which considers both semantic and luminance information. The colorization can be performed fully automatically by simply picking the top reference suggestion. Our approach is validated through a user study and favorable quantitative comparisons to the-state-of-the-art methods. Furthermore, our approach can be naturally extended to video colorization. Our code and models are freely available for public use.\n          <\/jats:p>","DOI":"10.1145\/3197517.3201365","type":"journal-article","created":{"date-parts":[[2018,7,31]],"date-time":"2018-07-31T15:56:23Z","timestamp":1533052583000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":192,"title":["Deep exemplar-based colorization"],"prefix":"10.1145","volume":"37","author":[{"given":"Mingming","family":"He","sequence":"first","affiliation":[{"name":"Hong Kong UST"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dongdong","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jing","family":"Liao","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pedro V.","family":"Sander","sequence":"additional","affiliation":[{"name":"Hong Kong UST"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lu","family":"Yuan","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,7,30]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.150"},{"key":"e_1_2_2_2_1","volume-title":"Proc","author":"Babenko Artem","unstructured":"Artem Babenko , Anton Slesarev , Alexandr Chigorin , and Victor Lempitsky . 2014. Neural codes for image retrieval . In Proc . ECCV. Springer , 584--599. Artem Babenko, Anton Slesarev, Alexandr Chigorin, and Victor Lempitsky. 2014. Neural codes for image retrieval. In Proc. ECCV. Springer, 584--599."},{"key":"e_1_2_2_3_1","volume-title":"Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv preprint arXiv:1511.00561","author":"Badrinarayanan Vijay","year":"2015","unstructured":"Vijay Badrinarayanan , Alex Kendall , and Roberto Cipolla . 2015 . Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv preprint arXiv:1511.00561 (2015). Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. 2015. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv preprint arXiv:1511.00561 (2015)."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818107"},{"key":"e_1_2_2_5_1","volume-title":"Pattern Recognition (ICPR), 2012 21st International Conference on. IEEE, 3058--3061","author":"Bugeau Aur\u00e9lie","year":"2012","unstructured":"Aur\u00e9lie Bugeau and Vinh-Thong Ta . 2012 . Patch-based image colorization . In Pattern Recognition (ICPR), 2012 21st International Conference on. IEEE, 3058--3061 . Aur\u00e9lie Bugeau and Vinh-Thong Ta. 2012. Patch-based image colorization. In Pattern Recognition (ICPR), 2012 21st International Conference on. IEEE, 3058--3061."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2013.2288929"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-88690-7_10"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.126"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.296"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00696"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.168"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.55"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024190"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.72"},{"key":"e_1_2_2_15_1","volume-title":"Proc","author":"Gong Yunchao","unstructured":"Yunchao Gong , Liwei Wang , Ruiqi Guo , and Svetlana Lazebnik . 2014. Multi-scale orderless pooling of deep convolutional activation features . In Proc . ECCV. Springer , 392--407. Yunchao Gong, Liwei Wang, Ruiqi Guo, and Svetlana Lazebnik. 2014. Multi-scale orderless pooling of deep convolutional activation features. In Proc. ECCV. Springer, 392--407."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2393347.2393402"},{"key":"e_1_2_2_17_1","volume-title":"Deep Residual Learning for Image Recognition. arXiv preprint arXiv:1512.03385","author":"He Kaiming","year":"2015","unstructured":"Kaiming He , Xiangyu Zhang , Shaoqing Ren , and Jian Sun . 2015. Deep Residual Learning for Image Recognition. arXiv preprint arXiv:1512.03385 ( 2015 ). Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep Residual Learning for Image Recognition. arXiv preprint arXiv:1512.03385 (2015)."},{"key":"e_1_2_2_18_1","volume-title":"Neural Color Transfer between Images. arXiv preprint arXiv:1710.00756","author":"He Mingming","year":"2017","unstructured":"Mingming He , Jing Liao , Lu Yuan , and Pedro V Sander . 2017. Neural Color Transfer between Images. arXiv preprint arXiv:1710.00756 ( 2017 ). Mingming He, Jing Liao, Lu Yuan, and Pedro V Sander. 2017. Neural Color Transfer between Images. arXiv preprint arXiv:1710.00756 (2017)."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1101149.1101223"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925974"},{"key":"e_1_2_2_21_1","volume-title":"International Conference on Machine Learning. 448--456","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy . 2015 . Batch normalization: Accelerating deep network training by reducing internal covariate shift . In International Conference on Machine Learning. 448--456 . Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning. 448--456."},{"key":"e_1_2_2_22_1","unstructured":"Revital Irony Daniel Cohen-Or and Dani Lischinski. 2005. Colorization by Example. In Rendering Techniques. 201--210.   Revital Irony Daniel Cohen-Or and Dani Lischinski. 2005. Colorization by Example. In Rendering Techniques. 201--210."},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.632"},{"key":"e_1_2_2_24_1","volume-title":"Proc","author":"Johnson Justin","unstructured":"Justin Johnson , Alexandre Alahi , and Li Fei-Fei . 2016. Perceptual losses for real-time style transfer and super-resolution . In Proc . ECCV. Springer , 694--711. Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In Proc. ECCV. Springer, 694--711."},{"key":"e_1_2_2_25_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik","year":"2014","unstructured":"Diederik Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_2_26_1","unstructured":"Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.   Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_35"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015780"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073683"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.147"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409060.1409105"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.5555\/2383847.2383887"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1142017"},{"key":"e_1_2_2_34_1","volume-title":"Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434","author":"Radford Alec","year":"2015","unstructured":"Alec Radford , Luke Metz , and Soumith Chintala . 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 ( 2015 ). Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015)."},{"key":"e_1_2_2_35_1","volume-title":"A baseline for visual instance retrieval with deep convolutional networks. arXiv preprint arXiv:1412.6574","author":"Razavian Ali Sharif","year":"2016","unstructured":"Ali Sharif Razavian , Josephine Sullivan , Stefan Carlsson , and Atsuto Maki . 2016. A baseline for visual instance retrieval with deep convolutional networks. arXiv preprint arXiv:1412.6574 ( 2016 ). Ali Sharif Razavian, Josephine Sullivan, Stefan Carlsson, and Atsuto Maki. 2016. A baseline for visual instance retrieval with deep convolutional networks. arXiv preprint arXiv:1412.6574 (2016)."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/38.946629"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.481"},{"key":"e_1_2_2_39_1","volume-title":"Proc. CVPR.","author":"Sangkloy Patsorn","year":"2016","unstructured":"Patsorn Sangkloy , Jingwan Lu , Chen Fang , Fisher Yu , and James Hays . 2016 . Scribbler: Controlling deep image synthesis with sketch and color . In Proc. CVPR. Patsorn Sangkloy, Jingwan Lu, Chen Fang, Fisher Yu, and James Hays. 2016. Scribbler: Controlling deep image synthesis with sketch and color. In Proc. CVPR."},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.215"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.77"},{"key":"e_1_2_2_43_1","volume-title":"Particular object retrieval with integral max-pooling of CNN activations. arXiv preprint arXiv:1511.05879","author":"Tolias Giorgos","year":"2015","unstructured":"Giorgos Tolias , Ronan Sicre , and Herv\u00e9 J\u00e9gou . 2015. Particular object retrieval with integral max-pooling of CNN activations. arXiv preprint arXiv:1511.05879 ( 2015 ). Giorgos Tolias, Ronan Sicre, and Herv\u00e9 J\u00e9gou. 2015. Particular object retrieval with integral max-pooling of CNN activations. arXiv preprint arXiv:1511.05879 (2015)."},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.175"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566576"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2005.864231"},{"key":"e_1_2_2_47_1","volume-title":"Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122","author":"Yu Fisher","year":"2015","unstructured":"Fisher Yu and Vladlen Koltun . 2015. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 ( 2015 ). Fisher Yu and Vladlen Koltun. 2015. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 (2015)."},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_40"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073703"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201365","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3197517.3201365","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:39:45Z","timestamp":1750210785000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201365"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,30]]},"references-count":49,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,8,31]]}},"alternative-id":["10.1145\/3197517.3201365"],"URL":"https:\/\/doi.org\/10.1145\/3197517.3201365","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,30]]},"assertion":[{"value":"2018-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}