{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T14:06:00Z","timestamp":1770905160016,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2020,9,21]],"date-time":"2020-09-21T00:00:00Z","timestamp":1600646400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2019YFB1312400"],"award-info":[{"award-number":["2019YFB1312400"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The capsule endoscopy robot can only use monocular vision due to the dimensional limit. To improve the depth perception of the monocular capsule endoscopy robot, this paper proposes a photometric stereo-based depth map reconstruction method. First, based on the characteristics of the capsule endoscopy robot system, a photometric stereo framework is established. Then, by combining the specular property and Lambertian property of the object surface, the depth of the specular highlight point is estimated, and the depth map of the whole object surface is reconstructed by a forward upwind scheme. To evaluate the precision of the depth estimation of the specular highlight region and the depth map reconstruction of the object surface, simulations and experiments are implemented with synthetic images and pig colon tissue, respectively. The results of the simulations and experiments show that the proposed method provides good precision for depth map reconstruction in monocular capsule endoscopy.<\/jats:p>","DOI":"10.3390\/s20185403","type":"journal-article","created":{"date-parts":[[2020,9,21]],"date-time":"2020-09-21T08:18:01Z","timestamp":1600676281000},"page":"5403","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Photometric Stereo-Based Depth Map Reconstruction for Monocular Capsule Endoscopy"],"prefix":"10.3390","volume":"20","author":[{"given":"Yang","family":"Hao","sequence":"first","affiliation":[{"name":"Intelligent Robotics Institute, School of Mechatronical Engineering, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jing","family":"Li","sequence":"additional","affiliation":[{"name":"Beijing Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fei","family":"Meng","sequence":"additional","affiliation":[{"name":"Key Laboratory of Biomimetic Robots and Systems, Beijing Institute of Technology, Ministry of Education, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peisen","family":"Zhang","sequence":"additional","affiliation":[{"name":"Intelligent Robotics Institute, School of Mechatronical Engineering, Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gastone","family":"Ciuti","sequence":"additional","affiliation":[{"name":"Beijing Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology, Beijing 100081, China"},{"name":"The BioRobotics Institute, Scuola Superiore Sant\u2019Anna, 56025 Pisa, Italy"},{"name":"Department of Excellence in Robotics &amp; AI, Scuola Superiore Sant\u2019Anna, 56025 Pisa, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paolo","family":"Dario","sequence":"additional","affiliation":[{"name":"Beijing Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology, Beijing 100081, China"},{"name":"The BioRobotics Institute, Scuola Superiore Sant\u2019Anna, 56025 Pisa, Italy"},{"name":"Department of Excellence in Robotics &amp; AI, Scuola Superiore Sant\u2019Anna, 56025 Pisa, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qiang","family":"Huang","sequence":"additional","affiliation":[{"name":"Beijing Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology, Beijing 100081, China"},{"name":"Key Laboratory of Biomimetic Robots and Systems, Beijing Institute of Technology, Ministry of Education, Beijing 100081, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,21]]},"reference":[{"key":"ref_1","unstructured":"Fan, Y., Meng, M.Q.H., and Li, B. (2010, January 1\u20134). 3D reconstruction of wireless capsule endoscopy images. Proceedings of the 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, Buenos Aires, Argentina."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ciuti, G., Visentini-Scarzanella, M., Dore, A., Menciassi, A., Dario, P., and Yang, G.Z. (2012, January 24\u201327). Intra-operative monocular 3D reconstruction for image-guided navigation in active locomotion capsule endoscopy. Proceedings of the 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Rome, Italy.","DOI":"10.1109\/BioRob.2012.6290771"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Prasath, V.S., Figueiredo, I.N., Figueiredo, P.N., and Palaniappan, K. (September, January 28). Mucosal region detection and 3D reconstruction in wireless capsule endoscopy videos using active contours. Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA.","DOI":"10.1109\/EMBC.2012.6346847"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Karargyris, A., Karargyris, O., and Bourbakis, N. (June, January 31). 3D representation of the digestive tract surface in Wireless Capsule Endoscopy videos. Proceedings of the 2010 IEEE International Conference on BioInformatics and BioEngineering, Philadelphia, PA, USA.","DOI":"10.1109\/BIBE.2010.9"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2732","DOI":"10.1137\/140968100","article-title":"Near field photometric stereo with point light sources","volume":"7","author":"Mecca","year":"2014","journal-title":"SIAM J. Imaging Sci."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Agrawal, A., Raskar, R., and Chellappa, R. (2006, January 7\u201313). What is the range of surface reconstructions from a gradient field?. Proceedings of the European Conference on Computer Vision, Graz, Austria.","DOI":"10.1007\/11744023_45"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Stoyanov, D., Scarzanella, M.V., Pratt, P., and Yang, G.Z. (2010, January 20\u201324). Real-time stereo reconstruction in robotically assisted minimally invasive surgery. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Beijing, China.","DOI":"10.1007\/978-3-642-15705-9_34"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Newcombe, R.A., and Davison, A.J. (2010, January 13\u201318). Live dense reconstruction with a single moving camera. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539794"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1447","DOI":"10.1109\/LRA.2019.2897145","article-title":"Improved human\u2014Robot collaborative control of redundant robot for teleoperated minimally invasive surgery","volume":"4","author":"Su","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2943","DOI":"10.1109\/LRA.2020.2974445","article-title":"Deep neural network approach in robot tool dynamics identification for bilateral teleoperation","volume":"5","author":"Su","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1109\/THMS.2020.2984181","article-title":"A smartphone-based adaptive recognition and real-time monitoring system for human activities","volume":"5","author":"Qi","year":"2020","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1006\/cviu.1997.0613","article-title":"Shape reconstruction from an endoscope image by shape from shading technique for a point light source at the projection center","volume":"66","author":"Okatani","year":"1997","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1117\/12.956740","article-title":"Photometric stereo: A reflectance map technique for determining surface orientation from image intensity","volume":"Volume 155","author":"Woodham","year":"1979","journal-title":"Image Understanding Systems and Industrial Applications I"},{"key":"ref_14","first-page":"167","article-title":"Constraining object features using a polarization reflectance model","volume":"1","author":"Wolff","year":"1993","journal-title":"Phys. Based Vis. Princ. Pract. Radiom."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"076017","DOI":"10.1117\/1.JBO.18.7.076017","article-title":"Photometric stereo endoscopy","volume":"18","author":"Parot","year":"2013","journal-title":"J. Biomed. Opt."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1239","DOI":"10.1109\/TPAMI.2003.1233898","article-title":"The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows","volume":"25","author":"Barsky","year":"2003","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yang, Q., Wang, S., and Ahuja, N. (2010, January 5\u201311). Real-time specular highlight removal using bilateral filtering. Proceedings of the European Conference on Computer Vision, Crete, Greece.","DOI":"10.1007\/978-3-642-15561-1_7"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kim, H., Jin, H., Hadap, S., and Kweon, I. (2013, January 23\u201328). Specular reflection separation using dark channel prior. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.192"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Tan, R.T., and Ikeuchi, K. (2008). Separating reflection components of textured surfaces using a single image. Digitally Archiving Cultural Objects, Springer.","DOI":"10.1007\/978-0-387-75807"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1016\/j.cviu.2012.03.001","article-title":"Surface reflectance and normal estimation from photometric stereo","volume":"116","author":"Yang","year":"2012","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ikehata, S., and Aizawa, K. (2014, January 23\u201328). Photometric stereo using constrained bivariate regression for general isotropic surfaces. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.280"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Blinn, J.F. (1977, January 20\u201322). Models of light reflection for computer synthesized pictures. Proceedings of the 4th Annual Conference on Computer Graphics and Interactive Techniques, San Jose, CA, USA.","DOI":"10.1145\/563858.563893"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1007\/s41095-017-0101-9","article-title":"Photometric stereo for strong specular highlights","volume":"4","author":"Khanian","year":"2018","journal-title":"Comput. Vis. Media"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.cag.2015.05.020","article-title":"Realistic photometric stereo using partial differential irradiance equation ratios","volume":"51","author":"Mecca","year":"2015","journal-title":"Comput. Graph."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1016\/j.dld.2013.01.025","article-title":"Experimental assessment of a novel robotically-driven endoscopic capsule compared to traditional colonoscopy","volume":"45","author":"Arezzo","year":"2013","journal-title":"Dig. Liver Dis."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"10528","DOI":"10.3748\/wjg.v21.i37.10528","article-title":"Capsule endoscopy of the future: What\u2019s on the horizon?","volume":"21","author":"Slawinski","year":"2015","journal-title":"World J. Gastroenterol. WJG"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1016\/j.jmmm.2017.12.085","article-title":"Magnetically-driven medical robots: An analytical magnetic model for endoscopic capsules design","volume":"452","author":"Li","year":"2018","journal-title":"J. Magn. Magn. Mater."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1586\/17434440.2014.941809","article-title":"Flexible and capsule endoscopy for screening, diagnosis and treatment","volume":"11","author":"Sliker","year":"2014","journal-title":"Expert Rev. Med. Devices"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Collins, T., and Bartoli, A. (2012, January 1\u20135). 3d reconstruction in laparoscopy with close-range photometric stereo. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Nice, France.","DOI":"10.1007\/978-3-642-33418-4_78"},{"key":"ref_30","unstructured":"Quarteroni, A., and Valli, A. (2008). Numerical Approximation of Partial Differential Equations, Springer Science & Business Media."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Strikwerda, J.C. (2004). Finite Difference Schemes and Partial Differential Equations, SIAM.","DOI":"10.1137\/1.9780898717938"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1137\/120902458","article-title":"A direct differential approach to photometric stereo with perspective viewing","volume":"7","author":"Mecca","year":"2014","journal-title":"SIAM J. Imaging Sci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Zhang","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Hao, Y., Visentini-Scarzanella, M., Li, J., Zhang, P., Ciuti, G., Dario, P., and Huang, Q. (2020). Light source position calibration method for photometric stereo in capsule endoscopy. Advanced Robotics, Taylor & Francis.","DOI":"10.1080\/01691864.2020.1757203"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Visentini-Scarzanella, M., and Kawasaki, H. (2015). Simultaneous camera, light position and radiant intensity distribution calibration. Image and Video Technology, Springer.","DOI":"10.1007\/978-3-319-29451-3_44"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1016\/j.mri.2015.12.010","article-title":"Innovative anisotropic phantoms for calibration of diffusion tensor imaging sequences","volume":"34","year":"2016","journal-title":"Magn. Reson. Imaging"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.neuroimage.2018.06.046","article-title":"Physical and numerical phantoms for the validation of brain microstructural MRI: A cookbook","volume":"182","author":"Fieremans","year":"2018","journal-title":"Neuroimage"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s00723-008-0095-7","article-title":"Quantitative assessment of injury in rat spinal cords in vivo by MRI of water diffusion tensor","volume":"34","author":"Adamek","year":"2008","journal-title":"Appl. Magn. Reson."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Sulaiman, S., and Marciani, L. (2019). MRI of the Colon in the Pharmaceutical Field: The Future before us. Pharmaceutics, 11.","DOI":"10.3390\/pharmaceutics11040146"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/18\/5403\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:12:00Z","timestamp":1760177520000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/18\/5403"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,21]]},"references-count":39,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["s20185403"],"URL":"https:\/\/doi.org\/10.3390\/s20185403","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,21]]}}}