{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T10:30:35Z","timestamp":1780482635533,"version":"3.54.1"},"reference-count":74,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2022,5,13]],"date-time":"2022-05-13T00:00:00Z","timestamp":1652400000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000185","name":"DARPA","doi-asserted-by":"crossref","award":["HR00112090017"],"award-info":[{"award-number":["HR00112090017"]}],"id":[{"id":"10.13039\/100000185","id-type":"DOI","asserted-by":"crossref"}]},{"name":"NSF","award":["CCF-1723445"],"award-info":[{"award-number":["CCF-1723445"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,10,31]]},"abstract":"<jats:p>We present a method to automatically synthesize efficient, high-quality demosaicking algorithms, across a range of computational budgets, given a loss function and training data. It performs a multi-objective, discrete-continuous optimization which simultaneously solves for the program structure and parameters that best tradeoff computational cost and image quality. We design the method to exploit domain-specific structure for search efficiency. We apply it to several tasks, including demosaicking both Bayer and Fuji X-Trans color filter patterns, as well as joint demosaicking and super-resolution. In a few days on 8 GPUs, it produces a family of algorithms that significantly improves image quality relative to the prior state-of-the-art across a range of computational budgets from 10 s to 1000\u00a0s of operations per pixel (1 dB\u20133 dB higher quality at the same cost, or 8.5\u2013200\u00d7 higher throughput at same or better quality). The resulting programs combine features of both classical and deep learning-based demosaicking algorithms into more efficient hybrid combinations, which are bandwidth-efficient and vectorizable by construction. Finally, our method automatically schedules and compiles all generated programs into optimized SIMD code for modern processors.<\/jats:p>","DOI":"10.1145\/3508461","type":"journal-article","created":{"date-parts":[[2022,3,11]],"date-time":"2022-03-11T02:09:37Z","timestamp":1646964577000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Searching for Fast Demosaicking Algorithms"],"prefix":"10.1145","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4180-6433","authenticated-orcid":false,"given":"Karima","family":"Ma","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Gharbi","sequence":"additional","affiliation":[{"name":"Adobe Systems Inc, Seattle, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Adams","sequence":"additional","affiliation":[{"name":"Adobe Systems Inc, Seattle, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shoaib","family":"Kamil","sequence":"additional","affiliation":[{"name":"Adobe Systems Inc, Seattle, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tzu-Mao","family":"Li","sequence":"additional","affiliation":[{"name":"University of California San Diego, La Jolla, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Connelly","family":"Barnes","sequence":"additional","affiliation":[{"name":"Adobe Systems Inc, Seattle, WA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonathan","family":"Ragan-Kelley","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,5,13]]},"reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322967"},{"key":"e_1_3_2_3_1","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1117\/12.204825","volume-title":"Proceedings of the Cameras and Systems for Electronic Photography and Scientific Imaging","author":"Jr James E. Adams","year":"1995","unstructured":"James E. Adams Jr. 1995. Interactions between color plane interpolation and other image processing functions in electronic photography. In Proceedings of the Cameras and Systems for Electronic Photography and Scientific Imaging. 144\u2013151."},{"key":"e_1_3_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2004.841200"},{"key":"e_1_3_2_5_1","first-page":"177","article-title":"Performance-oriented neural architecture search","author":"Anderson A.","year":"2019","unstructured":"A. Anderson, J. Su, Rozenn Dahyot, and D. Gregg. 2019. Performance-oriented neural architecture search. In Proceedings of theInternational Conference on High Performance Computing & Simulation.177\u2013184.","journal-title":"International Conference on High Performance Computing & Simulation."},{"key":"e_1_3_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.3021088"},{"key":"e_1_3_2_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/11744023_40"},{"key":"e_1_3_2_8_1","unstructured":"Davis Blalock Jose Javier Gonzalez Ortiz Jonathan Frankle and John Guttag. 2020. What is the state of neural network pruning? In Proceedings of Machine Learning and Systems 2020 MLSys 2020 Austin TX USA March 2-4 2020 Inderjit S. Dhillon Dimitris S. Papailiopoulos and Vivienne Sze (Eds.). mlsys.org. https:\/\/proceedings.mlsys.org\/book\/296.pdf."},{"key":"e_1_3_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2009.2017171"},{"key":"e_1_3_2_10_1","unstructured":"CarVac. 2021. librtprocess Github Page. Retrieved 31 March 2022 from https:\/\/github.com\/CarVac\/librtprocess."},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.image.2015.10.003"},{"key":"e_1_3_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.273"},{"key":"e_1_3_2_13_1","first-page":"59","article-title":"Fast, accurate and lightweight super-resolution with neural architecture search","author":"Chu Xiangxiang","year":"2021","unstructured":"Xiangxiang Chu, Bo Zhang, Hailong Ma, R. Xu, Jixiang Li, and Qingyuan Li. 2021b. Fast, accurate and lightweight super-resolution with neural architecture search. In Proceedings of the 2020 25th International Conference on Pattern Recognition.59\u201364.","journal-title":"In Proceedings of the 2020 25th International Conference on Pattern Recognition."},{"key":"e_1_3_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR48806.2021.9413080"},{"key":"e_1_3_2_15_1","unstructured":"David R. Cok. 1987. Signal processing method and apparatus for producing interpolated chrominance values in a sampled color image signal. US Patent 4 642 678. Accessed 31 March 2022."},{"key":"e_1_3_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2012.6467476"},{"key":"e_1_3_2_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10593-2_13"},{"key":"e_1_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2005.859503"},{"key":"e_1_3_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2341928"},{"key":"e_1_3_2_20_1","unstructured":"Thomas Elsken Jan Hendrik Metzen and Frank Hutter. 2018. Efficient multi-objective neural architecture search via lamarckian evolution. In Proceedings of the 7th International Conference on Learning Representations ICLR 2019 New Orleans LA USA May 6-9 2019 . OpenReview.net. https:\/\/openreview.net\/forum?id=ByME42AqK7."},{"issue":"55","key":"e_1_3_2_21_1","first-page":"1","article-title":"Neural architecture search: A survey.","volume":"20","author":"Elsken Thomas","year":"2019","unstructured":"Thomas Elsken, Jan Hendrik Metzen, Frank Hutter, et\u00a0al. 2019. Neural architecture search: A survey.The Journal of Machine Learning Research 20, 55 (2019), 1\u201321. http:\/\/jmlr.org\/papers\/v20\/18-598.html.","journal-title":"The Journal of Machine Learning Research"},{"key":"e_1_3_2_22_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.538372"},{"key":"e_1_3_2_23_1","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Frankle Jonathan","year":"2019","unstructured":"Jonathan Frankle and Michael Carbin. 2019. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In Proceedings of the International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=rJl-b3RcF7."},{"key":"e_1_3_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDSP.2011.6004885"},{"key":"e_1_3_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982399"},{"key":"e_1_3_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/30.883419"},{"key":"e_1_3_2_27_1","first-page":"1","volume-title":"Proceedings of the International Conference on Computer-Aided Design","author":"Gong Chengyue","year":"2019","unstructured":"Chengyue Gong, Zixuan Jiang, Dilin Wang, Yibo Lin, Qiang Liu, and David Z. Pan. 2019. Mixed precision neural architecture search for energy efficient deep learning. In Proceedings of the International Conference on Computer-Aided Design. IEEE, 1\u20137."},{"key":"e_1_3_2_28_1","unstructured":"John F. Hamilton Jr and James E. Adams Jr. 1997. Adaptive color plan interpolation in single sensor color electronic camera. US Patent 5 629 734. Accessed 31 March 2022."},{"key":"e_1_3_2_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818104"},{"key":"e_1_3_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661260"},{"key":"e_1_3_2_32_1","volume-title":"Computer Graphics Forum","author":"Henz Bernardo","year":"2018","unstructured":"Bernardo Henz, Eduardo S. L. Gastal, and Manuel M. Oliveira. 2018. Deep joint design of color filter arrays and demosaicing. Computer Graphics Forum 37, 7 (2018), 389\u2013399."},{"key":"e_1_3_2_33_1","unstructured":"Robert H. Hibbard. 1995. Apparatus and method for adaptively interpolating a full color image utilizing luminance gradients. US Patent 5 382 976. Accessed 31 March 2022."},{"key":"e_1_3_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2004.838691"},{"key":"e_1_3_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2006.875241"},{"key":"e_1_3_2_36_1","first-page":"112","volume-title":"Proceedings of the Applications of Artificial Neural Networks in Image Processing.","author":"Kapah Oren","year":"2000","unstructured":"Oren Kapah and Hagit Zabrodsky Hel-Or. 2000. Demosaicking using artificial neural networks. In Proceedings of the Applications of Artificial Neural Networks in Image Processing. International Society for Optics and Photonics, 112\u2013120."},{"key":"e_1_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2359774"},{"key":"e_1_3_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2013.6738475"},{"key":"e_1_3_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCPHOT.2016.7492871"},{"key":"e_1_3_2_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01264-9_19"},{"key":"e_1_3_2_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2905991"},{"key":"e_1_3_2_42_1","volume-title":"Genetic Programming: On the Programming of Computers By Means of Natural Selection","author":"Koza John R.","year":"1992","unstructured":"John R. Koza and John R. Koza. 1992. Genetic Programming: On the Programming of Computers By Means of Natural Selection. MIT Press."},{"key":"e_1_3_2_43_1","first-page":"2415","volume-title":"Proceedings of the International Conference on Image Processing","author":"Kwan Cindy","year":"2004","unstructured":"Cindy Kwan and Xiaolin Wu. 2004. A classification approach to color demosaicking. In Proceedings of the International Conference on Image Processing. IEEE, 2415\u20132418."},{"key":"e_1_3_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201383"},{"key":"e_1_3_2_45_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.766768"},{"key":"e_1_3_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2941832"},{"key":"e_1_3_2_47_1","first-page":"2849","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Lin Darryl","year":"2016","unstructured":"Darryl Lin, Sachin Talathi, and Sreekanth Annapureddy. 2016. Fixed point quantization of deep convolutional networks. In Proceedings of the International Conference on Machine Learning. PMLR, 2849\u20132858."},{"key":"e_1_3_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00231"},{"key":"e_1_3_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2904903"},{"key":"e_1_3_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2904903"},{"key":"e_1_3_2_51_1","first-page":"477","volume-title":"Proceedings of the International Conference on Image Processing","author":"Menon Daniele","year":"2009","unstructured":"Daniele Menon and Giancarlo Calvagno. 2009. Joint demosaicking and denoising with space-varying filters. In Proceedings of the International Conference on Image Processing. IEEE, 477\u2013480."},{"key":"e_1_3_2_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2883815"},{"key":"e_1_3_2_53_1","unstructured":"Guocheng Qian Jinjin Gu Jimmy S. J. Ren Chao Dong Furong Zhao and Juan-Ting Lin. 2019. Trinity of pixel enhancement: A joint solution for demosaicking denoising and super-resolution. arXiv:1905.02538. Retrieved from https:\/\/arxiv.org\/abs\/1905.02538."},{"key":"e_1_3_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185528"},{"key":"e_1_3_2_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2019.00480"},{"key":"e_1_3_2_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.207"},{"key":"e_1_3_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024186"},{"key":"e_1_3_2_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2803341"},{"key":"e_1_3_2_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2017.8296823"},{"key":"e_1_3_2_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00293"},{"key":"e_1_3_2_61_1","first-page":"793","volume-title":"Proc. IEEE I International Conference on Multimedia & Expo.","author":"Tan Runjie","year":"2017","unstructured":"Runjie Tan, Kai Zhang, Wangmeng Zuo, and Lei Zhang. 2017b. Color image demosaicking via deep residual learning. In Proc. IEEE I International Conference on Multimedia & Expo.793\u2013798."},{"key":"e_1_3_2_62_1","unstructured":"The RawTherapee Team. 2021. RawTherapee Home Page. Retrieved 31 March 2022 from https:\/\/rawtherapee.com."},{"key":"e_1_3_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661276"},{"key":"e_1_3_2_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2018.00131"},{"key":"e_1_3_2_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.634"},{"key":"e_1_3_2_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00351"},{"key":"e_1_3_2_67_1","doi-asserted-by":"publisher","DOI":"10.1201\/9781439819319"},{"key":"e_1_3_2_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2008.2011384"},{"key":"e_1_3_2_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2005.857260"},{"key":"e_1_3_2_70_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.3600632"},{"key":"e_1_3_2_71_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01234-2_18"},{"key":"e_1_3_2_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00257"},{"key":"e_1_3_2_73_1","first-page":"1","volume-title":"Proceedings of the Conference on SysML","author":"Zhou Yanqi","year":"2018","unstructured":"Yanqi Zhou and Gregory Diamos. 2018. Neural architect: A multi-objective neural architecture search with performance prediction. In Proceedings of the Conference on SysML. 1\u20133."},{"key":"e_1_3_2_74_1","unstructured":"Barret Zoph and Quoc V. Le. 2016. Neural architecture search with reinforcement learning. In Proceedings of the 5th International Conference on Learning Representations ICLR 2017 Toulon France April 24-26 2017 . OpenReview.net. https:\/\/openreview.net\/forum?id=r1Ue8Hcxg."},{"key":"e_1_3_2_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00907"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3508461","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3508461","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3508461","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:40Z","timestamp":1750188640000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3508461"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,13]]},"references-count":74,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2022,10,31]]}},"alternative-id":["10.1145\/3508461"],"URL":"https:\/\/doi.org\/10.1145\/3508461","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,13]]},"assertion":[{"value":"2021-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-05-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}