{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T11:44:29Z","timestamp":1783597469585,"version":"3.55.0"},"reference-count":22,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2024,8,27]],"date-time":"2024-08-27T00:00:00Z","timestamp":1724716800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Seeding To Accelerate Research Themes (START) program at the University of Michigan College of Engineering"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Pose estimation is crucial for ensuring passenger safety and better user experiences in semi- and fully autonomous vehicles. Traditional methods relying on pose estimation from regular color images face significant challenges due to a lack of three-dimensional (3D) information and the sensitivity to occlusion and lighting conditions. Depth images, which are invariant to lighting issues and provide 3D information about the scene, offer a promising alternative. However, there is a lack of strong work in 3D pose estimation from such images due to the time-consuming process of annotating depth images with 3D postures. In this paper, we present a novel approach to 3D human posture estimation using depth and infrared (IR) images. Our method leverages a three-stage fine-tuning process involving simulation data, approximated data, and a limited set of manually annotated samples. This approach allows us to effectively train a model capable of accurate 3D pose estimation with a median error of under 10 cm across all joints, using fewer than 100 manually annotated samples. To the best of our knowledge, this is the first work focusing on vehicle occupant posture detection utilizing only depth and IR data. Our results demonstrate the feasibility and efficacy of this approach, paving the way for enhanced passenger safety in autonomous vehicle systems.<\/jats:p>","DOI":"10.3390\/s24175530","type":"journal-article","created":{"date-parts":[[2024,8,27]],"date-time":"2024-08-27T03:51:06Z","timestamp":1724730666000},"page":"5530","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Three-Dimensional Posture Estimation of Vehicle Occupants Using Depth and Infrared Images"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7955-8573","authenticated-orcid":false,"given":"Anuj","family":"Tambwekar","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, University of Michigan, Ann Arbor, MI 48109, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5445-6826","authenticated-orcid":false,"given":"Byoung-Keon D.","family":"Park","sequence":"additional","affiliation":[{"name":"University of Michigan Transportation Research Institute, Ann Arbor, MI 48109, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arpan","family":"Kusari","sequence":"additional","affiliation":[{"name":"University of Michigan Transportation Research Institute, Ann Arbor, MI 48109, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3811-5853","authenticated-orcid":false,"given":"Wenbo","family":"Sun","sequence":"additional","affiliation":[{"name":"University of Michigan Transportation Research Institute, Ann Arbor, MI 48109, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"696","DOI":"10.1177\/1541931218621157","article-title":"How does relaxing posture influence take-over performance in an automated vehicle?","volume":"62","author":"Yang","year":"2018","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1874","DOI":"10.1177\/1071181319631327","article-title":"Sitting Postures and Activities in Autonomous Vehicles\u2013New Requirements towards Occupant Safety","volume":"63","author":"Pelzer","year":"2019","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wan, J., and Wang, N. (2018, January 26\u201329). A method of motion-based immersive design system for vehicle occupant package. Proceedings of the Volume 1B: 38th Computers and Information in Engineering Conference, Quebec City, QC, Canada.","DOI":"10.1115\/DETC2018-85054"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/TPAMI.2019.2929257","article-title":"OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields","volume":"43","author":"Cao","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., and Doll\u00e1r, P. (2015). Microsoft COCO: Common Objects in Context. arXiv.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Bartol, K., Bojani\u0107, D., Petkovi\u0107, T., D\u2019Apuzzo, N., and Pribanic, T. (2020, January 17\u201318). A Review of 3D Human Pose Estimation from 2D Images. Proceedings of the 3DBODY. TECH 2020-11th International Conference and Exhibition on 3D Body Scanning and Processing Technologies, Online.","DOI":"10.15221\/20.29"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"108375","DOI":"10.1016\/j.sigpro.2021.108375","article-title":"Predicting high-fidelity human body models from impaired point clouds","volume":"192","author":"Hu","year":"2022","journal-title":"Signal Process."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1180","DOI":"10.1007\/s11263-018-1081-7","article-title":"Depth-Based Hand Pose Estimation: Methods, Data, and Challenges","volume":"126","author":"Rogez","year":"2018","journal-title":"Int. J. Comput. Vis."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"402","DOI":"10.1109\/TITS.2023.3306314","article-title":"Pose and Shape Estimation of Humans in Vehicles","volume":"25","author":"Ko","year":"2024","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1325","DOI":"10.1109\/TPAMI.2013.248","article-title":"Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments","volume":"36","author":"Ionescu","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1590","DOI":"10.1177\/1541931214581331","article-title":"Creating Custom Human Avatars for Ergonomic Analysis using Depth Cameras","volume":"58","author":"Reed","year":"2014","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1504\/IJVD.2020.114780","article-title":"Model-based characterisation of vehicle occupants using a depth camera","volume":"83","author":"Park","year":"2020","journal-title":"Int. J. Veh. Des."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_15","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on International Conference on Machine Learning, Madison, WI, USA. ICML\u201910."},{"key":"ref_16","unstructured":"Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., and Garnett, R. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Proceedings of the Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_17","unstructured":"Dai, J., Li, Y., He, K., and Sun, J. (2016, January 5\u201310). R-FCN: Object detection via region-based fully convolutional networks. Proceedings of the NIPS\u201916: 30th International Conference on Neural Information Processing Systems, Barcelona Spain."},{"key":"ref_18","unstructured":"Xu, B., Wang, N., Chen, T., and Li, M. (2015). Empirical Evaluation of Rectified Activations in Convolutional Network. arXiv."},{"key":"ref_19","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), San Diego, CA, USA."},{"key":"ref_20","unstructured":"Gastaldi, X. (2017, January 24\u201326). Shake-Shake regularization of 3-branch residual networks. Proceedings of the International Conference on Learning Representations, Toulon, France."},{"key":"ref_21","unstructured":"Reddi, S.J., Kale, S., and Kumar, S. (May, January 30). On the Convergence of Adam and Beyond. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_22","unstructured":"Hoffman, J., Wang, D., Yu, F., and Darrell, T. (2016). FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/17\/5530\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:43:18Z","timestamp":1760110998000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/17\/5530"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,27]]},"references-count":22,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2024,9]]}},"alternative-id":["s24175530"],"URL":"https:\/\/doi.org\/10.3390\/s24175530","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,27]]}}}