{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T03:46:11Z","timestamp":1762055171306,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2022,6,13]],"date-time":"2022-06-13T00:00:00Z","timestamp":1655078400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"New Energy and Industrial Technology Development Organization (NEDO)","award":["JPNP20006"],"award-info":[{"award-number":["JPNP20006"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Estimation of muscle activity is very important as it can be a cue to assess a person\u2019s movements and intentions. If muscle activity states can be obtained through non-contact measurement, through visual measurement systems, for example, muscle activity will provide data support and help for various study fields. In the present paper, we propose a method to predict human muscle activity from skin surface strain. This requires us to obtain a 3D reconstruction model with a high relative accuracy. The problem is that reconstruction errors due to noise on raw data generated in a visual measurement system are inevitable. In particular, the independent noise between each frame on the time series makes it difficult to accurately track the motion. In order to obtain more precise information about the human skin surface, we propose a method that introduces a temporal constraint in the non-rigid registration process. We can achieve more accurate tracking of shape and motion by constraining the point cloud motion over the time series. Using surface strain as input, we build a multilayer perceptron artificial neural network for inferring muscle activity. In the present paper, we investigate simple lower limb movements to train the network. As a result, we successfully achieve the estimation of muscle activity via surface strain.<\/jats:p>","DOI":"10.3390\/jimaging8060168","type":"journal-article","created":{"date-parts":[[2022,6,13]],"date-time":"2022-06-13T06:31:59Z","timestamp":1655101919000},"page":"168","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Estimating Muscle Activity from the Deformation of a Sequential 3D Point Cloud"],"prefix":"10.3390","volume":"8","author":[{"given":"Hui","family":"Niu","sequence":"first","affiliation":[{"name":"National Institute of Advanced Industrial Science and Technology, Tsukuba 305-8560, Japan"},{"name":"CNRS-AIST JRL (Joint Robotics Laboratory), IRL, Tsukuba 305-8560, Japan"},{"name":"Graduate School of Science and Technology, University of Tsukuba, Tsukuba 305-0006, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Takahiro","family":"Ito","sequence":"additional","affiliation":[{"name":"National Institute of Advanced Industrial Science and Technology, Tsukuba 305-8560, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Damien","family":"Desclaux","sequence":"additional","affiliation":[{"name":"CNRS-AIST JRL (Joint Robotics Laboratory), IRL, Tsukuba 305-8560, Japan"},{"name":"ISAE-SUPAERO, University of Toulouse, 31055 Toulouse, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ko","family":"Ayusawa","sequence":"additional","affiliation":[{"name":"National Institute of Advanced Industrial Science and Technology, Tsukuba 305-8560, Japan"},{"name":"CNRS-AIST JRL (Joint Robotics Laboratory), IRL, Tsukuba 305-8560, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yusuke","family":"Yoshiyasu","sequence":"additional","affiliation":[{"name":"National Institute of Advanced Industrial Science and Technology, Tsukuba 305-8560, Japan"},{"name":"CNRS-AIST JRL (Joint Robotics Laboratory), IRL, Tsukuba 305-8560, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ryusuke","family":"Sagawa","sequence":"additional","affiliation":[{"name":"National Institute of Advanced Industrial Science and Technology, Tsukuba 305-8560, Japan"},{"name":"CNRS-AIST JRL (Joint Robotics Laboratory), IRL, Tsukuba 305-8560, Japan"},{"name":"Graduate School of Science and Technology, University of Tsukuba, Tsukuba 305-0006, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eiichi","family":"Yoshida","sequence":"additional","affiliation":[{"name":"National Institute of Advanced Industrial Science and Technology, Tsukuba 305-8560, Japan"},{"name":"CNRS-AIST JRL (Joint Robotics Laboratory), IRL, Tsukuba 305-8560, Japan"},{"name":"Department of Applied Electronics, Faculty of Advanced Engineering, Tokyo University of Science, Tokyo 125-8585, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Sagawa, R., Ayusawa, K., Yoshiyasu, Y., and Murai, A. (2018, January 8\u201314). Predicting Muscle Activity and Joint Angle from Skin Shape. Proceedings of the ECCV 2018 Workshops, Munich, Germany.","DOI":"10.1007\/978-3-030-11009-3_30"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sagawa, R., Yoshiyasu, Y., Alspach, A., Ayusawa, K., Yamane, K., and Hilton, A. (2015, January 25\u201327). Analyzing muscle activity and force with skin shape captured by non-contact visual sensor. Proceedings of the Image and Video Technology, (PSIVT2015), Auckland, New Zealand.","DOI":"10.1007\/978-3-319-29451-3_39"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MMUL.2012.24","article-title":"Microsoft kinect sensor and its effect","volume":"19","author":"Zhang","year":"2012","journal-title":"IEEE Multimed."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2333","DOI":"10.1109\/TPAMI.2019.2915841","article-title":"Intel\u00ae RealSense\u2122 SR300 Coded light depth Camera","volume":"42","author":"Zabatani","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1109\/34.121791","article-title":"A method for registration of 3-D shapes","volume":"14","author":"Besl","year":"1992","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1123\/jab.13.2.135","article-title":"The use of surface electromyography in biomechanics","volume":"13","year":"1997","journal-title":"J. Appl. Biomech."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1113\/jphysiol.1952.sp004764","article-title":"A quantitative description of membrane current and its application to conduction and excitation in nerve","volume":"117","author":"Hodgkin","year":"1952","journal-title":"J. Physiol."},{"key":"ref_8","first-page":"136","article-title":"The heat of shortening and the dynamic constants of muscle","volume":"126","author":"Hill","year":"1938","journal-title":"Proc. R. Soc. Lond."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1007\/s004220050577","article-title":"Impedance characteristics of a neuro-musculoskeletal model of the human arm I: Posture control","volume":"81","author":"Stroeve","year":"1999","journal-title":"J. Biol. Cybern"},{"key":"ref_10","first-page":"3","article-title":"Standards for reporting EMG data","volume":"9","author":"Merletti","year":"1999","journal-title":"J. Electromyogr. Kinesiol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Neugebauer, P. (1997, January 6). Geometrical cloning of 3D objects via simultaneous registration of multiple range images. Proceedings of the 1997 International Conference on Shape Modeling and Applications, Aizu-Wakamatsu, Japan.","DOI":"10.1109\/SMA.1997.634890"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1111\/j.1467-8659.2008.01282.x","article-title":"Global correspondence optimization for non-rigid registration of depth scans","volume":"Volume 27","author":"Li","year":"2008","journal-title":"Computer Graphics Forum"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Sumner, R.W., Schmid, J., and Pauly, M. (2007). Embedded deformation for shape manipulation. SIGGRAPH \u201907: ACM SIGGRAPH 2007 Papers, Association for Computing Machinery.","DOI":"10.1145\/1275808.1276478"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1145\/882262.882311","article-title":"The space of human body shapes: Reconstruction and parameterization from range scans","volume":"22","author":"Allen","year":"2003","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1145\/1015706.1015736","article-title":"Deformation transfer for triangle meshes","volume":"23","author":"Sumner","year":"2004","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Amberg, B., Romdhani, S., and Vetter, T. (2007, January 17\u201322). Optimal step nonrigid ICP algorithms for surface registration. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.","DOI":"10.1109\/CVPR.2007.383165"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1449","DOI":"10.1111\/j.1467-8659.2008.01285.x","article-title":"Non-rigid registration under isometric deformations","volume":"Volume 27","author":"Huang","year":"2008","journal-title":"Computer Graphics Forum"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Sagawa, R., Akasaka, K., Yagi, Y., Hamer, H., and Van Gool, L. (October, January 27). Elastic convolved ICP for the registration of deformable objects. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision Workshops, Kyoto, Japan.","DOI":"10.1109\/ICCVW.2009.5457428"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Weise, T., Li, H., Van Gool, L., and Pauly, M. (2009, January 1\u20132). Face\/off: Live facial puppetry. Proceedings of the 2009 ACM SIGGRAPH\/Eurographics Symposium on Computer Animation, New Orleans, LA, USA.","DOI":"10.1145\/1599470.1599472"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.cag.2012.03.014","article-title":"As-conformal-as-possible discrete volumetric mapping","volume":"36","author":"Poulin","year":"2012","journal-title":"Comput. Graph."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Martinez Esturo, J., R\u00f6ssl, C., and Theisel, H. (2014). Generalized Metric Energies for Continuous Shape Deformation. Mathematical Methods for Curves and Surfaces, Springer.","DOI":"10.1007\/978-3-642-54382-1_8"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1111\/cgf.12451","article-title":"As-conformal-as-possible surface registration","volume":"Volume 33","author":"Yoshiyasu","year":"2014","journal-title":"Computer Graphics Forum"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2897824.2925969","article-title":"Fusion4d: Real-time performance capture of challenging scenes","volume":"35","author":"Dou","year":"2016","journal-title":"ACM Trans. Graph. (ToG)"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Slavcheva, M., Baust, M., Cremers, D., and Ilic, S. (2017, January 21\u201326). Killingfusion: Non-rigid 3d reconstruction without correspondences. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.581"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2366145.2366213","article-title":"Temporally Coherent Completion of Dynamic Shapes","volume":"31","author":"Li","year":"2012","journal-title":"ACM Trans. Graph."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Newcombe, R.A., Fox, D., and Seitz, S.M. (2015, January 7\u201312). Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298631"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/j.media.2004.06.025","article-title":"4D deformable models with temporal constraints: Application to 4D cardiac image segmentation","volume":"9","author":"Montagnat","year":"2005","journal-title":"Med. Image Anal."},{"key":"ref_28","unstructured":"Sagawa, R., Oishi, T., Nakazawa, A., Kurazume, R., and Ikeuchi, K. (October, January 30). Iterative refinement of range images with anisotropic error distribution. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Lausanne, Switzerland."},{"key":"ref_29","unstructured":"Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., and Schilling, A. (2007, January 4\u20136). Reconstruction of deforming geometry from time-varying point clouds. Proceedings of the Symposium on Geometry Processing, Barcelona, Spain."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yu, T., Guo, K., Xu, F., Dong, Y., Su, Z., Zhao, J., Li, J., Dai, Q., and Liu, Y. (2017, January 22\u201329). Bodyfusion: Real-time capture of human motion and surface geometry using a single depth camera. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.104"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Yu, T., Zheng, Z., Guo, K., Zhao, J., Dai, Q., Li, H., Pons-Moll, G., and Liu, Y. (2018, January 18\u201322). Doublefusion: Real-time capture of human performances with inner body shapes from a single depth sensor. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00761"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/S0262-8856(98)00117-6","article-title":"Registration and integration of textured 3D data","volume":"17","author":"Johnson","year":"1999","journal-title":"Image Vis. Comput."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Godin, G., Rioux, M., and Baribeau, R. (1994, January 2\u20134). Three-dimensional registration using range and intensity information. Proceedings of the Videometrics III. International Society for Optics and Photonics, Boston, MA, USA.","DOI":"10.1117\/12.189139"},{"key":"ref_34","unstructured":"Godin, G., Laurendeau, D., and Bergevin, R. (June, January 28). A method for the registration of attributed range images. Proceedings of the Third International Conference on 3-D Digital Imaging and Modeling, Quebec City, QC, Canada."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s00138-011-0340-1","article-title":"Three-dimensional point cloud alignment detecting fiducial markers by structured light stereo imaging","volume":"23","author":"Barone","year":"2012","journal-title":"Mach. Vis. Appl."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Sidorov, K.A., Richmond, S., and Marshall, D. (2011, January 20\u201325). Efficient groupwise non-rigid registration of textured surfaces. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995632"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Park, J., Zhou, Q.Y., and Koltun, V. (2017, January 22\u201329). Colored point cloud registration revisited. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.25"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Sagawa, R., Sakashita, K., Kasuya, N., Kawasaki, H., Furukawa, R., and Yagi, Y. (2012, January 13\u201315). Grid-based active stereo with single-colored wave pattern for dense one-shot 3D scan. Proceedings of the 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, Zurich, Switzerland.","DOI":"10.1109\/3DIMPVT.2012.41"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Sagawa, R., and Satoh, Y. (2017, January 21\u201326). Illuminant-camera communication to observe moving objects under strong external light by spread spectrum modulation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.249"},{"key":"ref_40","unstructured":"Kazhdan, M., Bolitho, M., and Hoppe, H. (2006, January 26\u201328). Poisson surface reconstruction. Proceedings of the Fourth Eurographics Symposium on Geometry Processing, Sardinia, Italy."},{"key":"ref_41","unstructured":"Rusinkiewicz, S., and Levoy, M. (June, January 28). Efficient variants of the ICP algorithm. Proceedings of the Third International Conference on 3-D Digital Imaging and Modeling, Quebec City, QC, Canada."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Ayusawa, K., Suleiman, W., and Yoshida, E. (2019, January 3\u20138). Predictive Inverse Kinematics: Optimizing Future Trajectory through Implicit Time Integration and Future Jacobian Estimation. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Macau, China.","DOI":"10.1109\/IROS40897.2019.8968110"},{"key":"ref_43","unstructured":"Alves, J., Yamamura, N., Oda, T., and Teodosiu, C. (2010, January 24\u201327). Numerical simulation of musculo-skeletal systems by V-biomech. Proceedings of the CMBBE2010 Symposium, Valencia, Spain."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/6\/168\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:28:35Z","timestamp":1760138915000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/6\/168"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,13]]},"references-count":43,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["jimaging8060168"],"URL":"https:\/\/doi.org\/10.3390\/jimaging8060168","relation":{},"ISSN":["2313-433X"],"issn-type":[{"type":"electronic","value":"2313-433X"}],"subject":[],"published":{"date-parts":[[2022,6,13]]}}}