{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T16:20:32Z","timestamp":1782318032815,"version":"3.54.5"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2023,12,5]],"date-time":"2023-12-05T00:00:00Z","timestamp":1701734400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2023,12,5]]},"abstract":"<jats:p>\n            Neural Radiance Fields (NeRF) can be optimized to obtain high-fidelity 3D scene reconstructions of objects and large-scale scenes. However, NeRFs require accurate camera parameters as input --- inaccurate camera parameters result in blurry renderings. Extrinsic and intrinsic camera parameters are usually estimated using Structure-from-Motion (SfM) methods as a pre-processing step to NeRF, but these techniques rarely yield perfect estimates. Thus, prior works have proposed jointly optimizing camera parameters alongside a NeRF, but these methods are prone to local minima in challenging settings. In this work, we analyze how different camera parameterizations affect this joint optimization problem, and observe that standard parameterizations exhibit large differences in magnitude with respect to small perturbations, which can lead to an ill-conditioned optimization problem. We propose using a proxy problem to compute a whitening transform that eliminates the correlation between camera parameters and normalizes their effects, and we propose to use this transform as a\n            <jats:italic toggle=\"yes\">preconditioner<\/jats:italic>\n            for the camera parameters during joint optimization. Our preconditioned camera optimization significantly improves reconstruction quality on scenes from the Mip-NeRF 360 dataset: we reduce error rates (RMSE) by 67% compared to state-of-the-art NeRF approaches that do not optimize for cameras like Zip-NeRF, and by 29% relative to state-of-the-art joint optimization approaches using the camera parameterization of SCNeRF. Our approach is easy to implement, does not significantly increase runtime, can be applied to a wide variety of camera parameterizations, and can straightforwardly be incorporated into other NeRF-like models.\n          <\/jats:p>","DOI":"10.1145\/3618321","type":"journal-article","created":{"date-parts":[[2023,12,5]],"date-time":"2023-12-05T10:20:48Z","timestamp":1701771648000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":49,"title":["CamP: Camera Preconditioning for Neural Radiance Fields"],"prefix":"10.1145","volume":"42","author":[{"given":"Keunhong","family":"Park","sequence":"first","affiliation":[{"name":"Google Research, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philipp","family":"Henzler","sequence":"additional","affiliation":[{"name":"Google Research, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ben","family":"Mildenhall","sequence":"additional","affiliation":[{"name":"Google Research, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonathan T.","family":"Barron","sequence":"additional","affiliation":[{"name":"Google Research, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ricardo","family":"Martin-Brualla","sequence":"additional","affiliation":[{"name":"Google Research, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,12,5]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Mip-nerf 360: Unbounded anti-aliased neural radiance fields. CVPR","author":"Barron Jonathan T.","year":"2022","unstructured":"Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. 2022. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. CVPR (2022)."},{"key":"e_1_2_1_2_1","volume-title":"Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields. arXiv:2304.06706","author":"Barron Jonathan T.","year":"2023","unstructured":"Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. 2023. Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields. arXiv:2304.06706 (2023)."},{"key":"e_1_2_1_3_1","volume-title":"Neural reflectance fields for appearance acquisition. arXiv:2008.03824","author":"Bi Sai","year":"2020","unstructured":"Sai Bi, Zexiang Xu, Pratul Srinivasan, Ben Mildenhall, Kalyan Sunkavalli, Milo\u0161 Ha\u0161an, Yannick Hold-Geoffroy, David Kriegman, and Ravi Ramamoorthi. 2020. Neural reflectance fields for appearance acquisition. arXiv:2008.03824 (2020)."},{"key":"e_1_2_1_4_1","volume-title":"NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior. arXiv:2212.07388","author":"Bian Wenjing","year":"2022","unstructured":"Wenjing Bian, Zirui Wang, Kejie Li, Jia-Wang Bian, and Victor Adrian Prisacariu. 2022. NoPe-NeRF: Optimising Neural Radiance Field with No Pose Prior. arXiv:2212.07388 (2022)."},{"key":"e_1_2_1_5_1","volume-title":"DBARF: Deep Bundle-Adjusting Generalizable Neural Radiance Fields. CVPR","author":"Chen Yu","year":"2023","unstructured":"Yu Chen and Gim Hee Lee. 2023. DBARF: Deep Bundle-Adjusting Generalizable Neural Radiance Fields. CVPR (2023)."},{"key":"e_1_2_1_6_1","volume-title":"GARF: gaussian activated radiance fields for high fidelity reconstruction and pose estimation. arXiv","author":"Chng Shin-Fang","year":"2022","unstructured":"Shin-Fang Chng, Sameera Ramasinghe, Jamie Sherrah, and Simon Lucey. 2022. GARF: gaussian activated radiance fields for high fidelity reconstruction and pose estimation. arXiv (2022)."},{"key":"e_1_2_1_7_1","volume-title":"Decentred lens-systems. Monthly notices of the royal astronomical society 79, 5","author":"Conrady Alexander Eugen","year":"1919","unstructured":"Alexander Eugen Conrady. 1919. Decentred lens-systems. Monthly notices of the royal astronomical society 79, 5 (1919), 384--390."},{"key":"e_1_2_1_8_1","unstructured":"Christopher De Sa. 2019. Lecture notes in CS4787 Principles of Large-Scale Machine Learning: Lecture 8."},{"key":"e_1_2_1_9_1","volume-title":"The raxel imaging model and ray-based calibration. IJCV","author":"Grossberg Michael D","year":"2005","unstructured":"Michael D Grossberg and Shree K Nayar. 2005. The raxel imaging model and ray-based calibration. IJCV (2005)."},{"key":"e_1_2_1_10_1","volume-title":"Deep blending for free-viewpoint image-based rendering. ACM TOG","author":"Hedman Peter","year":"2018","unstructured":"Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, and Gabriel Brostow. 2018. Deep blending for free-viewpoint image-based rendering. ACM TOG (2018)."},{"key":"e_1_2_1_11_1","volume-title":"Robust Camera Pose Refinement for Multi-Resolution Hash Encoding. arXiv:2302.01571","author":"Heo Hwan","year":"2023","unstructured":"Hwan Heo, Taekyung Kim, Jiyoung Lee, Jaewon Lee, Soohyun Kim, Hyunwoo J Kim, and Jin-Hwa Kim. 2023. Robust Camera Pose Refinement for Multi-Resolution Hash Encoding. arXiv:2302.01571 (2023)."},{"key":"e_1_2_1_12_1","volume-title":"Sape: Spatially-adaptive progressive encoding for neural optimization. NeurIPS","author":"Hertz Amir","year":"2021","unstructured":"Amir Hertz, Or Perel, Raja Giryes, Olga Sorkine-Hornung, and Daniel Cohen-Or. 2021. Sape: Spatially-adaptive progressive encoding for neural optimization. NeurIPS (2021)."},{"key":"e_1_2_1_13_1","volume-title":"Self-calibrating neural radiance fields. ICCV","author":"Jeong Yoonwoo","year":"2021","unstructured":"Yoonwoo Jeong, Seokjun Ahn, Christopher Choy, Anima Anandkumar, Minsu Cho, and Jaesik Park. 2021. Self-calibrating neural radiance fields. ICCV (2021)."},{"key":"e_1_2_1_14_1","volume-title":"Optimal whitening and decorrelation. The American Statistician","author":"Kessy Agnan","year":"2018","unstructured":"Agnan Kessy, Alex Lewin, and Korbinian Strimmer. 2018. Optimal whitening and decorrelation. The American Statistician (2018)."},{"key":"e_1_2_1_15_1","volume-title":"Adam: A method for stochastic optimization. arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980 (2014)."},{"key":"e_1_2_1_16_1","volume-title":"3D-RCNN: Instance-level 3d object reconstruction via render-and-compare. CVPR","author":"Kundu Abhijit","year":"2018","unstructured":"Abhijit Kundu, Yin Li, and James M Rehg. 2018. 3D-RCNN: Instance-level 3d object reconstruction via render-and-compare. CVPR (2018)."},{"key":"e_1_2_1_17_1","volume-title":"Visibility based preconditioning for bundle adjustment. CVPR","author":"Kushal Avanish","year":"2012","unstructured":"Avanish Kushal and Sameer Agarwal. 2012. Visibility based preconditioning for bundle adjustment. CVPR (2012)."},{"key":"e_1_2_1_18_1","volume-title":"Deepim: Deep iterative matching for 6d pose estimation. ECCV","author":"Li Yi","year":"2018","unstructured":"Yi Li, Gu Wang, Xiangyang Ji, Yu Xiang, and Dieter Fox. 2018. Deepim: Deep iterative matching for 6d pose estimation. ECCV (2018)."},{"key":"e_1_2_1_19_1","volume-title":"Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes. arXiv:2011.13084","author":"Li Zhengqi","year":"2020","unstructured":"Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. 2020. Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes. arXiv:2011.13084 (2020)."},{"key":"e_1_2_1_20_1","volume-title":"Barf: Bundle-adjusting neural radiance fields. ICCV","author":"Lin Chen-Hsuan","year":"2021","unstructured":"Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey. 2021. Barf: Bundle-adjusting neural radiance fields. ICCV (2021)."},{"key":"e_1_2_1_21_1","volume-title":"Soft rasterizer: A differentiable renderer for image-based 3d reasoning. ICCV","author":"Liu Shichen","year":"2019","unstructured":"Shichen Liu, Tianye Li, Weikai Chen, and Hao Li. 2019. Soft rasterizer: A differentiable renderer for image-based 3d reasoning. ICCV (2019)."},{"key":"e_1_2_1_22_1","volume":"201","author":"Loper Matthew M","unstructured":"Matthew M Loper and Michael J Black. 2014. OpenDR: An approximate differentiable renderer. ECCV (2014).","journal-title":"Michael J Black."},{"key":"e_1_2_1_23_1","volume-title":"Modern Robotics","author":"Lynch Kevin M","unstructured":"Kevin M Lynch and Frank C Park. 2017. Modern Robotics. Cambridge University Press."},{"key":"e_1_2_1_24_1","volume-title":"NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections. CVPR","author":"Martin-Brualla Ricardo","year":"2021","unstructured":"Ricardo Martin-Brualla, Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, and Daniel Duckworth. 2021. NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections. CVPR (2021)."},{"key":"e_1_2_1_25_1","volume-title":"Real-Fusion: 360\u00b0 Reconstruction of Any Object from a Single Image. arXiv:2302.10663","author":"Melas-Kyriazi Luke","year":"2023","unstructured":"Luke Melas-Kyriazi, Christian Rupprecht, Iro Laina, and Andrea Vedaldi. 2023. Real-Fusion: 360\u00b0 Reconstruction of Any Object from a Single Image. arXiv:2302.10663 (2023)."},{"key":"e_1_2_1_26_1","volume-title":"Gnerf: Gan-based neural radiance field without posed camera. ICCV","author":"Meng Quan","year":"2021","unstructured":"Quan Meng, Anpei Chen, Haimin Luo, Minye Wu, Hao Su, Lan Xu, Xuming He, and Jingyi Yu. 2021. Gnerf: Gan-based neural radiance field without posed camera. ICCV (2021)."},{"key":"e_1_2_1_27_1","volume-title":"Neural rerendering in the wild. CVPR","author":"Meshry Moustafa","year":"2019","unstructured":"Moustafa Meshry, Dan B Goldman, Sameh Khamis, Hugues Hoppe, Rohit Pandey, Noah Snavely, and Ricardo Martin-Brualla. 2019. Neural rerendering in the wild. CVPR (2019)."},{"key":"e_1_2_1_28_1","volume-title":"Progressively Optimized Local Radiance Fields for Robust View Synthesis. CVPR","author":"Meuleman Andreas","year":"2023","unstructured":"Andreas Meuleman, Yu-Lun Liu, Chen Gao, Jia-Bin Huang, Changil Kim, Min H Kim, and Johannes Kopf. 2023. Progressively Optimized Local Radiance Fields for Robust View Synthesis. CVPR (2023)."},{"key":"e_1_2_1_29_1","volume-title":"NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV","author":"Mildenhall Ben","year":"2020","unstructured":"Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. ECCV (2020)."},{"key":"e_1_2_1_30_1","volume-title":"Instant neural graphics primitives with a multiresolution hash encoding. ACM TOG","author":"M\u00fcller Thomas","year":"2022","unstructured":"Thomas M\u00fcller, Alex Evans, Christoph Schied, and Alexander Keller. 2022. Instant neural graphics primitives with a multiresolution hash encoding. ACM TOG (2022)."},{"key":"e_1_2_1_31_1","volume-title":"Nerfies: Deformable neural radiance fields. ICCV","author":"Park Keunhong","year":"2021","unstructured":"Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M Seitz, and Ricardo Martin-Brualla. 2021. Nerfies: Deformable neural radiance fields. ICCV (2021)."},{"key":"e_1_2_1_32_1","volume-title":"Radiance Field Gradient Scaling for Unbiased Near-Camera Training. arXiv:2305.02756","author":"Philip Julien","year":"2023","unstructured":"Julien Philip and Valentin Deschaintre. 2023. Radiance Field Gradient Scaling for Unbiased Near-Camera Training. arXiv:2305.02756 (2023)."},{"key":"e_1_2_1_33_1","volume-title":"Focal length and object pose estimation via render and compare. CVPR","author":"Ponimatkin Georgy","year":"2022","unstructured":"Georgy Ponimatkin, Yann Labb\u00e9, Bryan Russell, Mathieu Aubry, and Josef Sivic. 2022. Focal length and object pose estimation via render and compare. CVPR (2022)."},{"key":"e_1_2_1_34_1","volume-title":"De l'attraction des sph\u00e9ro\u00efdes. Correspondence Sur l'\u00c9cole Imp\u00e9riale Polytechnique","author":"Rodrigues Olinde","year":"1816","unstructured":"Olinde Rodrigues. 1816. De l'attraction des sph\u00e9ro\u00efdes. Correspondence Sur l'\u00c9cole Imp\u00e9riale Polytechnique (1816)."},{"key":"e_1_2_1_35_1","volume-title":"NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields. arXiv preprint arXiv:2210.13641","author":"Rosinol Antoni","year":"2022","unstructured":"Antoni Rosinol, John J Leonard, and Luca Carlone. 2022. NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields. arXiv preprint arXiv:2210.13641 (2022)."},{"key":"e_1_2_1_36_1","doi-asserted-by":"crossref","unstructured":"Johannes Lutz Sch\u00f6nberger and Jan-Michael Frahm. 2016. Structure-from-Motion Revisited. CVPR (2016).","DOI":"10.1109\/CVPR.2016.445"},{"key":"e_1_2_1_37_1","volume-title":"Why having 10,000 parameters in your camera model is better than twelve. CVPR","author":"Schops Thomas","year":"2020","unstructured":"Thomas Schops, Viktor Larsson, Marc Pollefeys, and Torsten Sattler. 2020. Why having 10,000 parameters in your camera model is better than twelve. CVPR (2020)."},{"key":"e_1_2_1_38_1","volume-title":"Computer vision: algorithms and applications","author":"Szeliski Richard","unstructured":"Richard Szeliski. 2022. Computer vision: algorithms and applications. Springer Nature."},{"key":"e_1_2_1_39_1","volume-title":"Block-nerf: Scalable large scene neural view synthesis. CVPR","author":"Tancik Matthew","year":"2022","unstructured":"Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben Mildenhall, Pratul P. Srinivasan, Jonathan T. Barron, and Henrik Kretzschmar. 2022. Block-nerf: Scalable large scene neural view synthesis. CVPR (2022)."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323035"},{"key":"e_1_2_1_41_1","volume-title":"SPARF: Neural Radiance Fields from Sparse and Noisy Poses. CVPR","author":"Truong Prune","year":"2023","unstructured":"Prune Truong, Marie-Julie Rakotosaona, Fabian Manhardt, and Federico Tombari. 2023. SPARF: Neural Radiance Fields from Sparse and Noisy Poses. CVPR (2023)."},{"key":"e_1_2_1_42_1","volume-title":"Ibrnet: Learning multi-view image-based rendering. CVPR","author":"Wang Qianqian","year":"2021","unstructured":"Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul P. Srinivasan, Howard Zhou, Jonathan T. Barron, Ricardo Martin-Brualla, Noah Snavely, and Thomas Funkhouser. 2021a. Ibrnet: Learning multi-view image-based rendering. CVPR (2021)."},{"key":"e_1_2_1_43_1","volume-title":"NeRF-: Neural radiance fields without known camera parameters. arXiv:2102.07064","author":"Wang Zirui","year":"2021","unstructured":"Zirui Wang, Shangzhe Wu, Weidi Xie, Min Chen, and Victor Adrian Prisacariu. 2021b. NeRF-: Neural radiance fields without known camera parameters. arXiv:2102.07064 (2021)."},{"key":"e_1_2_1_44_1","volume-title":"Critical configurations for radial distortion self-calibration. CVPR","author":"Changchang Wu.","year":"2014","unstructured":"Changchang Wu. 2014. Critical configurations for radial distortion self-calibration. CVPR (2014)."},{"key":"e_1_2_1_45_1","volume-title":"Sinerf: Sinusoidal neural radiance fields for joint pose estimation and scene reconstruction. arXiv:2210.04553","author":"Xia Yitong","year":"2022","unstructured":"Yitong Xia, Hao Tang, Radu Timofte, and Luc Van Gool. 2022. Sinerf: Sinusoidal neural radiance fields for joint pose estimation and scene reconstruction. arXiv:2210.04553 (2022)."},{"key":"e_1_2_1_46_1","volume-title":"pixelnerf: Neural radiance fields from one or few images. CVPR","author":"Yu Alex","year":"2021","unstructured":"Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. 2021. pixelnerf: Neural radiance fields from one or few images. CVPR (2021)."},{"key":"e_1_2_1_47_1","volume-title":"Color map optimization for 3d reconstruction with consumer depth cameras. ACM TOG","author":"Zhou Qian-Yi","year":"2014","unstructured":"Qian-Yi Zhou and Vladlen Koltun. 2014. Color map optimization for 3d reconstruction with consumer depth cameras. ACM TOG (2014)."},{"key":"e_1_2_1_48_1","volume-title":"On the continuity of rotation representations in neural networks. CVPR","author":"Zhou Yi","year":"2019","unstructured":"Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li. 2019. On the continuity of rotation representations in neural networks. CVPR (2019)."},{"key":"e_1_2_1_49_1","volume-title":"Nice-slam: Neural implicit scalable encoding for slam. CVPR","author":"Zhu Zihan","year":"2022","unstructured":"Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R Oswald, and Marc Pollefeys. 2022. Nice-slam: Neural implicit scalable encoding for slam. CVPR (2022)."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3618321","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3618321","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T10:49:57Z","timestamp":1755773397000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3618321"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,5]]},"references-count":49,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,12,5]]}},"alternative-id":["10.1145\/3618321"],"URL":"https:\/\/doi.org\/10.1145\/3618321","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,5]]},"assertion":[{"value":"2023-12-05","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}