{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T16:04:18Z","timestamp":1740153858490,"version":"3.37.3"},"reference-count":21,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,7,17]],"date-time":"2019-07-17T00:00:00Z","timestamp":1563321600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,7,17]],"date-time":"2019-07-17T00:00:00Z","timestamp":1563321600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["16H02849"],"award-info":[{"award-number":["16H02849"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000646","name":"Japan Society for the Promotion of Science London","doi-asserted-by":"publisher","award":["15H02758"],"award-info":[{"award-number":["15H02758"]}],"id":[{"id":"10.13039\/501100000646","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000646","name":"Japan Society for the Promotion of Science London","doi-asserted-by":"publisher","award":["15H02779"],"award-info":[{"award-number":["15H02779"]}],"id":[{"id":"10.13039\/501100000646","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["IPSJ T Comput Vis Appl"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The combination of a pattern projector and a camera is widely used for 3D measurement. To recover shape from a captured image, various kinds of depth cues are extracted from projected patterns in the image, such as disparities from active stereo or blurriness for depth from defocus. Recently, several techniques have been proposed to improve 3D quality using multiple depth cues by installing coded apertures in projectors or by increasing the number of projectors. However, superposition of projected patterns forms a complicated light field in 3D space, which makes the process of analyzing captured images challenging. In this paper, we propose a learning-based technique to extract depth information from such a light field, which includes multiple depth cues. In the learning phase, prior to the 3D measurement of unknown scenes, projected patterns as they appear at various depths are prepared from not only actual images but also ones generated virtually using computer graphics and geometric calibration results. Then, we use principal component analysis (PCA) to extract features of small patches. In the 3D measurement (reconstruction) phase, the same features of patches are extracted from a captured image of a target scene and compared with the learned data. By using the dimensional reduction by feature extraction, an efficient search algorithm, such as an approximated nearest neighbor (ANN), can be used for the matching process. Another important advantage of our learning-based approach is that we can use most known projection patterns without changing the algorithm.<\/jats:p>","DOI":"10.1186\/s41074-019-0058-y","type":"journal-article","created":{"date-parts":[[2019,7,17]],"date-time":"2019-07-17T14:02:52Z","timestamp":1563372172000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Learning-based active 3D measurement technique using light field created by video projectors"],"prefix":"10.1186","volume":"11","author":[{"given":"Yuki","family":"Shiba","sequence":"first","affiliation":[]},{"given":"Satoshi","family":"Ono","sequence":"additional","affiliation":[]},{"given":"Ryo","family":"Furukawa","sequence":"additional","affiliation":[]},{"given":"Shinsaku","family":"Hiura","sequence":"additional","affiliation":[]},{"given":"Hiroshi","family":"Kawasaki","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,7,17]]},"reference":[{"key":"58_CR1","first-page":"15","volume-title":"Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 IEEE Computer Society Conference On","author":"J Jurik","year":"2011","unstructured":"Jurik J, Jones A, Bolas M, Debevec P (2011) Prototyping a light field display involving direct observation of a video projector array In: Computer Vision and Pattern Recognition Workshops (CVPRW), 2011 IEEE Computer Society Conference On, 15\u201320.. IEEE, Colorado Springs."},{"key":"58_CR2","first-page":"113","volume-title":"Pacific-Rim Symposium on Image and Video Technology","author":"M Visentini-Scarzanella","year":"2015","unstructured":"Visentini-Scarzanella M, Hirukawa T, Kawasaki H, Furukawa R, Hiura S (2015) Two plane volumetric display for simultaneous independent images at multiple depths In: Pacific-Rim Symposium on Image and Video Technology, 113\u2013126.. Springer-Verlag, New York."},{"key":"58_CR3","doi-asserted-by":"publisher","first-page":"5074","DOI":"10.1109\/ICRA.2017.7989592","volume-title":"2017 IEEE International Conference on Robotics and Automation (ICRA)","author":"R Sagawa","year":"2017","unstructured":"Sagawa R, Furukawa R, Matsumoto A, Kawasaki H (2017) Learning-based feature extraction for active 3d scan with reducing color crosstalk of multiple pattern projections In: 2017 IEEE International Conference on Robotics and Automation (ICRA), 5074\u20135080.. IEEE, Singapore."},{"issue":"4","key":"58_CR4","doi-asserted-by":"publisher","first-page":"827","DOI":"10.1016\/j.patcog.2003.10.002","volume":"37","author":"J Salvi","year":"2004","unstructured":"Salvi J, Pages J, Batlle J (2004) Pattern codification strategies in structured light systems. Pattern Recog 37(4):827\u2013849.","journal-title":"Pattern Recog"},{"key":"58_CR5","doi-asserted-by":"publisher","first-page":"88","DOI":"10.2197\/ipsjtcva.6.88","volume":"6","author":"H Masuyama","year":"2014","unstructured":"Masuyama H, Kawasaki H, Furukawa R (2014) Depth from projector\u2019s defocus based on multiple focus pattern projection. IPSJ Trans Comput Vision Appl 6:88\u201392.","journal-title":"IPSJ Trans Comput Vision Appl"},{"issue":"3","key":"58_CR6","doi-asserted-by":"publisher","first-page":"907","DOI":"10.1145\/1141911.1141974","volume":"25","author":"Li Zhang","year":"2006","unstructured":"Zhang L, Nayar S (2006) Projection defocus analysis for scene capture and image display, 907\u2013915.. ACM Transactions on Graphics (TOG), New York.","journal-title":"ACM Transactions on Graphics"},{"key":"58_CR7","first-page":"205","volume-title":"European Conference on Computer Vision","author":"S Achar","year":"2014","unstructured":"Achar S, Narasimhan SG (2014) Multi focus structured light for recovering scene shape and global illumination In: European Conference on Computer Vision, 205\u2013219.. Springer, Cham."},{"key":"58_CR8","first-page":"209","volume-title":"1989 Advances in Intelligent Robotics Systems Conference","author":"B Girod","year":"1990","unstructured":"Girod B, Scherock S (1990) Depth from defocus of structured light In: 1989 Advances in Intelligent Robotics Systems Conference, 209\u2013215.. International Society for Optics and Photonics, Philadelphia."},{"issue":"12","key":"58_CR9","doi-asserted-by":"publisher","first-page":"1186","DOI":"10.1109\/34.546256","volume":"18","author":"SK Nayar","year":"1996","unstructured":"Nayar SK, Watanabe M, Noguchi M (1996) Real-time focus range sensor. IEEE Trans Pattern Anal Mach Intell 18(12):1186\u20131198.","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"58_CR10","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1109\/3DV.2013.26","volume-title":"3D Vision-3DV 2013, 2013 International Conference On","author":"H Kawasaki","year":"2013","unstructured":"Kawasaki H, Horita Y, Masuyama H, Ono S, Kimura M, Takane Y (2013) Optimized aperture for estimating depth from projector\u2019s defocus In: 3D Vision-3DV 2013, 2013 International Conference On, 135\u2013142.. IEEE, Seattle."},{"key":"58_CR11","volume-title":"IEEE Conference on Image Processing (ICIP)","author":"H Kawasaki","year":"2012","unstructured":"Kawasaki H, Horita Y, Morinaga H, Matugano Y, Ono S, Kimura M, Takane Y (2012) Structured light with coded aperture for wide range 3D measurement In: IEEE Conference on Image Processing (ICIP).. Orlando, IEEE."},{"key":"58_CR12","first-page":"3568","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"H Kawasaki","year":"2015","unstructured":"Kawasaki H, Ono S, Horita Y, Shiba Y, Furukawa R, Hiura S (2015) Active one-shot scan for wide depth range using a light field projector based on coded aperture In: Proceedings of the IEEE International Conference on Computer Vision, 3568\u20133576.. IEEE, Santiago."},{"issue":"4","key":"58_CR13","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1145\/2601097.2601223","volume":"33","author":"SR Fanello","year":"2014","unstructured":"Fanello SR, Keskin C, Izadi S, Kohli P, Kim D, Sweeney D, Criminisi A, Shotton J, Kang SB, Paek T (2014) Learning to be a depth camera for close-range human capture and interaction. ACM Trans Graph 33(4):86\u201318611.","journal-title":"ACM Trans Graph"},{"issue":"1","key":"58_CR14","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1162\/jocn.1991.3.1.71","volume":"3","author":"M Turk","year":"1991","unstructured":"Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71\u201386.","journal-title":"J Cogn Neurosci"},{"issue":"1","key":"58_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.cviu.2005.05.005","volume":"101","author":"KW Bowyer","year":"2006","unstructured":"Bowyer KW, Chang K, Flynn P (2006) A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition. Comput Vision Image Underst 101(1):1\u201315.","journal-title":"Comput Vision Image Underst"},{"issue":"1","key":"58_CR16","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/BF01421486","volume":"14","author":"H Murase","year":"1995","unstructured":"Murase H, Nayar SK (1995) Visual learning and recognition of 3-d objects from appearance. Int J Comput Vision 14(1):5\u201324.","journal-title":"Int J Comput Vision"},{"issue":"8","key":"58_CR17","doi-asserted-by":"publisher","first-page":"891","DOI":"10.1109\/34.608291","volume":"19","author":"M Uenohara","year":"1997","unstructured":"Uenohara M, Kanade T (1997) Use of Fourier and Karhunen-Loeve decomposition for fast pattern matching with a large set of templates. IEEE Trans Pattern Anal Mach Intell 19(8):891\u2013898.","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"58_CR18","unstructured":"Muja M, Lowe DG (2009) Fast approximate nearest neighbors with automatic algorithm configuration In: International Conference on Computer Vision Theory and Application, 331\u2013340.. INSTICC."},{"key":"58_CR19","first-page":"41","volume":"70","author":"P Felzenszwalb","year":"2006","unstructured":"Felzenszwalb P, Huttenlocher D (2006) Efficient belief propagation for early vision. IJCV. Washington, 70:41\u201354.","journal-title":"IJCV. Washington,"},{"key":"58_CR20","first-page":"363","volume-title":"Proc. 2012 Second Joint 3DIM\/3DPVT Conference","author":"R Sagawa","year":"2012","unstructured":"Sagawa R, Sakashita K, Kasuya N, Kawasaki H, Furukawa R, Yagi Y (2012) Grid-based active stereo with single-colored wave pattern for dense one-shot 3D scan In: Proc. 2012 Second Joint 3DIM\/3DPVT Conference, 363\u2013370.. IEEE, Zurich."},{"issue":"2","key":"58_CR21","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1109\/34.44402","volume":"12","author":"P Vuylsteke","year":"1990","unstructured":"Vuylsteke P, Oosterlinck A (1990) Range image acquisition with a single binary-encoded light pattern. IEEE Trans PAMI 12(2):148\u2013164.","journal-title":"IEEE Trans PAMI"}],"container-title":["IPSJ Transactions on Computer Vision and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s41074-019-0058-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s41074-019-0058-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s41074-019-0058-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,30]],"date-time":"2021-07-30T12:14:38Z","timestamp":1627647278000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s41074-019-0058-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,17]]},"references-count":21,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["58"],"URL":"https:\/\/doi.org\/10.1186\/s41074-019-0058-y","relation":{},"ISSN":["1882-6695"],"issn-type":[{"type":"electronic","value":"1882-6695"}],"subject":[],"published":{"date-parts":[[2019,7,17]]},"assertion":[{"value":"6 April 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 May 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 July 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare that they have no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"6"}}