{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T06:33:05Z","timestamp":1762929185604,"version":"3.45.0"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"11","license":[{"start":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T00:00:00Z","timestamp":1756425600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T00:00:00Z","timestamp":1756425600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2025,11]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>We propose depth from coupled optical differentiation, a low-computation passive-lighting 3D sensing mechanism. It is based on our discovery that per-pixel object distance can be rigorously determined by a coupled pair of optical derivatives of a defocused image using a simple, closed-form relationship. Unlike previous depth-from-defocus (DfD) methods that leverage higher-order spatial derivatives of the image to estimate scene depths, the proposed mechanism\u2019s use of only first-order optical derivatives makes it significantly more robust to noise. Furthermore, unlike many previous DfD algorithms with requirements on aperture code, this relationship is proved to be universal to a broad range of aperture codes. We build the first 3D sensor based on depth from coupled optical differentiation. Its optical assembly includes a deformable lens and a motorized iris, which enables dynamic adjustments to the optical power and aperture radius. The sensor captures two pairs of images: one pair with a differential change of optical power and the other with a differential change of aperture scale. From the four images, a depth and confidence map can be generated with only 36 floating point operations per output pixel (FLOPOP), more than ten times lower than the previous lowest passive-lighting depth sensing solution to our knowledge. Additionally, the depth map generated by the proposed sensor demonstrates more than twice the working range of previous DfD methods while using significantly lower computation.<\/jats:p>","DOI":"10.1007\/s11263-025-02534-z","type":"journal-article","created":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T18:55:45Z","timestamp":1756493745000},"page":"8109-8126","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Depth from Coupled Optical Differentiation"],"prefix":"10.1007","volume":"133","author":[{"given":"Junjie","family":"Luo","sequence":"first","affiliation":[]},{"given":"Yuxuan","family":"Liu","sequence":"additional","affiliation":[]},{"given":"Emma","family":"Alexander","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8329-7668","authenticated-orcid":false,"given":"Qi","family":"Guo","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,8,29]]},"reference":[{"key":"2534_CR1","unstructured":"Alexander, E. (2019). A theory of depth from differential defocus. PhD thesis, Harvard University"},{"key":"2534_CR2","doi-asserted-by":"crossref","unstructured":"Alexander, E., Guo, Q., Koppal, S., Gortler, S. J., & Zickler, T. (2018). Focal flow: Velocity and depth from differential defocus through motion. International Journal of Computer Vision,126, pp. 1062\u20131083.","DOI":"10.1007\/s11263-017-1051-5"},{"key":"2534_CR3","doi-asserted-by":"crossref","unstructured":"Chang J., & Wetzstein, G. (2019). Deep optics for monocular depth estimation and 3d object detection. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision, pp. 10193\u201310202","DOI":"10.1109\/ICCV.2019.01029"},{"key":"2534_CR4","doi-asserted-by":"crossref","unstructured":"Chen, W., Mirdehghan, P., Fidler, S., & Kutulakos, K.N. (2020). Auto-tuning structured light by optical stochastic gradient descent. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 5970\u20135980","DOI":"10.1109\/CVPR42600.2020.00601"},{"key":"2534_CR5","doi-asserted-by":"crossref","unstructured":"Dana, K. J., Ginneken, V., Bram, N., Shree, K., & Koenderink, J. J. (1999). Reflectance and texture of real-world surfaces. ACM Transactions On Graphics (TOG),18(1), pp. 1\u201334.","DOI":"10.1145\/300776.300778"},{"key":"2534_CR6","doi-asserted-by":"crossref","unstructured":"Ding, X., Xu, L., Wang, H., Wang, X., & Lv, G. (2011). Stereo depth estimation under different camera calibration and alignment errors. Applied Optics,50(10), pp. 1289\u20131301.","DOI":"10.1364\/AO.50.001289"},{"key":"2534_CR7","unstructured":"Fan, R., Wang, L., Bocus, M.J., & Pitas, I. (2020). Computer stereo vision for autonomous driving. arXiv preprint arXiv:2012.03194"},{"key":"2534_CR8","doi-asserted-by":"crossref","unstructured":"Farid, H., & Simoncelli, E. P. (1998). Range estimation by optical differentiation. JOSA A,15(7), pp. 1777\u20131786.","DOI":"10.1364\/JOSAA.15.001777"},{"key":"2534_CR9","doi-asserted-by":"crossref","unstructured":"Foix, S., Alenya, G., & Torras, C. (2011). Lock-in time-of-flight (tof) cameras: A survey. IEEE Sensors Journal,11(9), pp. 1917\u20131926.","DOI":"10.1109\/JSEN.2010.2101060"},{"key":"2534_CR10","unstructured":"Guo, Q. (2022). Efficient passive ranging with computational optics. PhD thesis, Harvard University"},{"key":"2534_CR11","doi-asserted-by":"crossref","unstructured":"Guo, Q., Alexander, E., & Zickler, T. (2017). Focal track: Depth and accommodation with oscillating lens deformation. In: Proceedings of the IEEE international conference on computer vision, pp. 966\u2013974","DOI":"10.1109\/ICCV.2017.110"},{"key":"2534_CR12","doi-asserted-by":"crossref","unstructured":"Guo, Q., Frosio, I., Gallo, O., Zickler, T., & Kautz, J. (2018). Tackling 3d tof artifacts through learning and the flat dataset. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 368\u2013383","DOI":"10.1007\/978-3-030-01246-5_23"},{"key":"2534_CR13","doi-asserted-by":"crossref","unstructured":"Guo, Q., Shi, Z., Huang, Y.-W., Alexander, E., Qiu, C.-W., Capasso, F., & Zickler, T. (2019). Compact single-shot metalens depth sensors inspired by eyes of jumping spiders. Proceedings of the National Academy of Sciences,116(46), pp. 22959\u201322965.","DOI":"10.1073\/pnas.1912154116"},{"key":"2534_CR14","doi-asserted-by":"crossref","unstructured":"Gur, S., & Wolf, L. (2019). Single image depth estimation trained via depth from defocus cues. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 7683\u20137692","DOI":"10.1109\/CVPR.2019.00787"},{"key":"2534_CR15","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1007\/978-3-030-63416-2_482","volume-title":"Computer vision: a reference guide","author":"SW Hasinoff","year":"2021","unstructured":"Hasinoff, S. W. (2021). Photon, poisson noise. Computer vision: a reference guide (pp. 980\u2013982). London, UK: Springer."},{"key":"2534_CR16","doi-asserted-by":"crossref","unstructured":"Horaud, R., Hansard, M., Evangelidis, G., & M\u00e9nier, C. (2016). An overview of depth cameras and range scanners based on time-of-flight technologies. Machine vision and applications,27(7), pp. 1005\u20131020.","DOI":"10.1007\/s00138-016-0784-4"},{"key":"2534_CR17","doi-asserted-by":"crossref","unstructured":"Ishihara, S., Sulc, A., & Sato, I. (2019). Depth from spectral defocus blur. In: 2019 IEEE International Conference on Image Processing (ICIP), IEEE, pp. 1980\u20131984","DOI":"10.1109\/ICIP.2019.8803191"},{"key":"2534_CR18","doi-asserted-by":"crossref","unstructured":"Ishihara, S., Sulc, A., & Sato, I. (2021). Depth estimation using spectrally varying defocus blur. JOSA A,38(8), pp. 1140\u20131149.","DOI":"10.1364\/JOSAA.422059"},{"key":"2534_CR19","unstructured":"Joshi, N., & Zitnick, C.L. (2014). Micro-baseline stereo. Microsoft Research Technical Report"},{"key":"2534_CR20","doi-asserted-by":"crossref","unstructured":"Koschan, A., & Rodehorst, V. (1997). Dense depth maps by active color illumination and image pyramids. In: Advances in Computer Vision. Springer, pp. 137\u2013148","DOI":"10.1007\/978-3-7091-6867-7_15"},{"key":"2534_CR21","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780199581139.001.0001","volume-title":"Animal eyes","author":"MF Land","year":"2012","unstructured":"Land, M. F., & Nilsson, D. E. (2012). Animal eyes (2nd ed.). Kettering, UK: OUP Oxford.","edition":"2"},{"key":"2534_CR22","doi-asserted-by":"crossref","unstructured":"Levin, A., Fergus, R., Durand, F., & Freeman, W.T. (2007). Image and depth from a conventional camera with a coded aperture. ACM transactions on graphics (TOG) 26(3): pp. 70\u2013es","DOI":"10.1145\/1276377.1276464"},{"key":"2534_CR23","unstructured":"Levin, A., Fergus, R., Durand, F., & Freeman, W.T. (2019). Physical cue based depth-sensing by color coding with deaberration network. arXiv preprint arXiv:1908.00329"},{"key":"2534_CR24","doi-asserted-by":"crossref","unstructured":"Luo, W., Schwing, A.G., & Urtasun, R. (2016). Efficient deep learning for stereo matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5695\u20135703","DOI":"10.1109\/CVPR.2016.614"},{"key":"2534_CR25","doi-asserted-by":"crossref","unstructured":"Mirdehghan, P., Chen, W., & Kutulakos, K.N. (2018). Optimal structured light a la carte. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6248\u20136257","DOI":"10.1109\/CVPR.2018.00654"},{"key":"2534_CR26","unstructured":"Nakamura, M., & Fukushima, N. (2017). Fast implementation of box filtering. In: Proc. International Workshop on Advanced Image Technology (IWAIT)"},{"key":"2534_CR27","doi-asserted-by":"crossref","unstructured":"Park, Y.-H., Cho, Y.-C., You, J.-W., Park, C.-Y., Yoon, H., Lee, S.-H., Kwon, J.-O., & Lee, S.-W. (2012) Micro-optical system based 3d imaging for full hd depth image capturing. In: MOEMS and Miniaturized Systems XI, SPIE, pp. 258\u2013272","DOI":"10.1117\/12.905946"},{"key":"2534_CR28","doi-asserted-by":"crossref","unstructured":"Pentland, A. P. (1987). A new sense for depth of field. IEEE transactions on pattern analysis and machine intelligence,4, pp. 523\u2013531.","DOI":"10.1109\/TPAMI.1987.4767940"},{"key":"2534_CR29","doi-asserted-by":"crossref","unstructured":"P\u00e9rez-Yus, A., L\u00f3pez-Nicol\u00e1s, G., & Guerrero, J.J. (2015). Detection and modelling of staircases using a wearable depth sensor. In: Computer Vision-ECCV 2014 Workshops: Zurich, Switzerland, September 6-7 and 12, 2014, Proceedings, Part III 13, Springer, pp. 449\u2013463","DOI":"10.1007\/978-3-319-16199-0_32"},{"key":"2534_CR30","doi-asserted-by":"crossref","unstructured":"Ploumpis, S., Amanatiadis, A., & Gasteratos, A. (2015). A stereo matching approach based on particle filters and scattered control landmarks. Image and Vision Computing,38, pp. 13\u201323.","DOI":"10.1016\/j.imavis.2015.04.001"},{"key":"2534_CR31","unstructured":"Rotheneder, S. (2018). Performance analysis of a stereo matching implementation in opencl. PhD thesis, Wien"},{"key":"2534_CR32","doi-asserted-by":"crossref","unstructured":"Schechner, Y. Y., & Kiryati, N. (2000). Depth from defocus vs. stereo: How different really are they? International Journal of Computer Vision,39, pp. 141\u2013162.","DOI":"10.1023\/A:1008175127327"},{"key":"2534_CR33","doi-asserted-by":"crossref","unstructured":"Sheinin, M., & Schechner, Y.Y. (2019). Depth from texture integration. In: 2019 IEEE International Conference on Computational Photography (ICCP), IEEE, pp. 1\u201310","DOI":"10.1109\/ICCPHOT.2019.8747333"},{"key":"2534_CR34","doi-asserted-by":"crossref","unstructured":"Subbarao, M., & Surya, G. (1994). Depth from defocus: A spatial domain approach. International Journal of Computer Vision,13(3), pp. 271\u2013294.","DOI":"10.1007\/BF02028349"},{"key":"2534_CR35","unstructured":"Supreeth, A., Joseph, R. B., William, L. W., Kiriakos, N. K., & Srinivasa, G. N. (2017). Epipo-lar time-of-flight imaging. ACM Transactions on Graphics (TOG),36(4), pp. 37."},{"key":"2534_CR36","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-34372-9","volume-title":"Computer vision: algorithms and applications","author":"R Szeliski","year":"2022","unstructured":"Szeliski, R. (2022). Computer vision: algorithms and applications. London, UK: Springer Nature."},{"key":"2534_CR37","doi-asserted-by":"crossref","unstructured":"Tang, H., Cohen, S., Price, B., Schiller, S., & Kutulakos, K.N. (2017). Depth from defocus in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2740\u20132748","DOI":"10.1109\/CVPR.2017.507"},{"key":"2534_CR38","doi-asserted-by":"crossref","unstructured":"Tan, S., Yang, F., Boominathan, V., Veeraraghavan, A., & Naik, G. V. (2021). 3d imaging using extreme dispersion in optical metasurfaces. ACS Photonics,8(5), pp. 1421\u20131429.","DOI":"10.1021\/acsphotonics.1c00110"},{"key":"2534_CR39","doi-asserted-by":"crossref","unstructured":"Wadhwa, N., Garg, R., Jacobs, D. E., Feldman, B. E., Kanazawa, N., Carroll, R., Movshovitz-Attias, Y., Barron, J. T., Pritch, Y., & Levoy, M. (2018). Synthetic depth-of-field with a single-camera mobile phone. ACM Transactions on Graphics (ToG),37(4), pp. 1\u201313.","DOI":"10.1145\/3197517.3201329"},{"key":"2534_CR40","doi-asserted-by":"crossref","unstructured":"Watanabe, M., & Nayar, S. K. (1998). Rational filters for passive depth from defocus. International Journal of Computer Vision,27, pp. 203\u2013225.","DOI":"10.1023\/A:1007905828438"},{"key":"2534_CR41","doi-asserted-by":"crossref","unstructured":"Wood, R., Nagpal, R., & Wei, G. Y. (2013). Flight of the robobees. Scientific American,308(3), pp. 60\u201365.","DOI":"10.1038\/scientificamerican0313-60"},{"key":"2534_CR42","doi-asserted-by":"crossref","unstructured":"Wu, Y., Boominathan, V., Chen, H., Sankaranarayanan, A., & Veeraraghavan, A. (2019). Phasecam3d-learning phase masks for passive single view depth estimation. In: 2019 IEEE International Conference on Computational Photography (ICCP), IEEE, pp. 1\u201312","DOI":"10.1109\/ICCPHOT.2019.8747330"},{"key":"2534_CR43","doi-asserted-by":"crossref","unstructured":"Zhang, S. (2018). High-speed 3d shape measurement with structured light methods: A review. Optics and lasers in engineering,106, pp. 119\u2013131.","DOI":"10.1016\/j.optlaseng.2018.02.017"},{"key":"2534_CR44","doi-asserted-by":"crossref","unstructured":"Zhou, C., Lin, S., & Nayar, S. K. (2011). Coded aperture pairs for depth from defocus and defocus deblurring. International journal of computer vision,93, pp. 53\u201372.","DOI":"10.1007\/s11263-010-0409-8"}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-025-02534-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11263-025-02534-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-025-02534-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T06:29:27Z","timestamp":1762928967000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11263-025-02534-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,29]]},"references-count":44,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2025,11]]}},"alternative-id":["2534"],"URL":"https:\/\/doi.org\/10.1007\/s11263-025-02534-z","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"type":"print","value":"0920-5691"},{"type":"electronic","value":"1573-1405"}],"subject":[],"published":{"date-parts":[[2025,8,29]]},"assertion":[{"value":"16 September 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 July 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 August 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of Interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Materials availability"}},{"value":"All code used in this paper are available at\n                      \n                      .","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code availability"}}]}}