{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T12:23:43Z","timestamp":1788351823792,"version":"build-2803163510"},"reference-count":31,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T00:00:00Z","timestamp":1788307200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["32560931"],"award-info":[{"award-number":["32560931"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"award":["32560931"],"award-info":[{"award-number":["32560931"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004763","name":"Natural Science Foundation of Inner Mongolia Autonomous Region","doi-asserted-by":"publisher","award":["2024MS06023"],"award-info":[{"award-number":["2024MS06023"]}],"id":[{"id":"10.13039\/501100004763","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Animals"],"abstract":"<jats:p>Intelligent sheep behavior monitoring is vital for modern husbandry, but faces severe challenges in natural pastures due to high-density flock occlusion. Traditional 2D skeleton-based networks often suffer from depth ambiguity and feature collapse, misclassifying static tremors as dynamic displacement. To overcome this, we propose a robust multimodal dual-stream framework using skeletal and local visual fusion. The architecture features an upstream spatial perception stage utilizing YOLOv11m-Pose. To reduce annotation costs and improve robustness, we introduce an Active Hard-Example Mining mechanism, explicitly retaining difficult samples with severe overlapping or edge truncation. For downstream behavioral decisions, a multimodal dual-stream architecture processes the targets. The Spatio\u2013Temporal Kinematic Stream employs a Kinematic Denoising Engine, incorporating a 1D Gaussian filter and displacement dead-zone gate to purify 2D coordinates before feeding them into a BiLSTM network. Concurrently, the Spatial Visual Stream uses a ResNet-50 backbone on cropped RGB patches to capture essential spatial context, addressing the limitations of pure coordinates. Finally, a weighted Softmax layer integrates both streams. Experiments on a complex real-world dataset validate this approach. A baseline kinematic-only model achieved just 69.05% overall accuracy and 68.18% walking precision. In contrast, our dual-stream fusion network achieved 93.26% overall accuracy, elevating walking precision to 97.14% and the eating F1-score to 94.29%. By effectively decoupling similar static and dynamic behaviors, this study demonstrates the indispensability of local visual features, establishing a high-precision baseline for smart livestock monitoring.<\/jats:p>","DOI":"10.3390\/ani16172759","type":"journal-article","created":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T12:04:16Z","timestamp":1788350656000},"page":"2759","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Multimodal Dual-Stream Framework for Sheep Behavior Recognition Using Skeletal and Local Visual Fusion"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7605-9330","authenticated-orcid":false,"given":"Chuanzhong","family":"Xuan","sequence":"first","affiliation":[{"name":"College of Mechanical and Electrical Engineering, Inner Mongolia Agricultural University, Hohhot 010018, China"},{"name":"Inner Mongolia Autonomous Region Engineering Research Center for Intelligent Equipment in Forage and Feed Production, Inner Mongolia Agricultural University, Hohhot 010018, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"}]},{"given":"Junze","family":"Jia","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electrical Engineering, Inner Mongolia Agricultural University, Hohhot 010018, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"conceptualization","vocabulary":"credit"}]},{"given":"Suhui","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electrical Engineering, Inner Mongolia Agricultural University, Hohhot 010018, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"software","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-1630-3767","authenticated-orcid":false,"given":"Zhaohui","family":"Tang","sequence":"additional","affiliation":[{"name":"College of Mechanical and Electrical Engineering, Inner Mongolia Agricultural University, Hohhot 010018, China"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"},{"role":"funding-acquisition","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Tzanidakis, C., Tzamaloukas, O., Simitzis, P., and Panagakis, P. (2023). Precision Livestock Farming Applications (PLF) for Grazing Animals. Agriculture, 13.","DOI":"10.3390\/agriculture13020288"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"107010","DOI":"10.1016\/j.compag.2022.107010","article-title":"Application of deep learning in sheep behaviors recognition and influence analysis of training data characteristics on the recognition effect","volume":"198","author":"Cheng","year":"2022","journal-title":"Comput. Electron. Agric."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"572","DOI":"10.1007\/s11263-022-01716-3","article-title":"Going Deeper than Tracking: A Survey of Computer-Vision Based Recognition of Animal Pain and Emotions","volume":"131","author":"Feighelstein","year":"2023","journal-title":"Int. J. Comput. Vis."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Guo, X.R., Ma, C.Y., Wang, C., Cui, X.C., Xu, G.D., Wang, R.M., Liu, Y.Q., Sun, B., Wang, Z.J., and Guo, X.C. (2025). A Sheep Behavior Recognition Approach Based on Improved FESS-YOLOv8n Neural Network. Animals, 15.","DOI":"10.3390\/ani15060893"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"106713","DOI":"10.1016\/j.compag.2022.106713","article-title":"Biometric identification of sheep via a machine-vision system","volume":"194","author":"Hitelman","year":"2022","journal-title":"Comput. Electron. Agric."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"108143","DOI":"10.1016\/j.compag.2023.108143","article-title":"A two-stage recognition method based on deep learning for sheep behavior","volume":"212","author":"Gu","year":"2023","journal-title":"Comput. Electron. Agric."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhao, H.K., Mao, R., Li, M., Li, B., and Wang, M.L. (2023). SheepInst: A High-Performance Instance Segmentation of Sheep Images Based on Deep Learning. Animals, 13.","DOI":"10.3390\/ani13081338"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Mansbridge, N., Mitsch, J., Bollard, N., Ellis, K., Miguel-Pacheco, G.G., Dottorini, T., and Kaler, J. (2018). Feature Selection and Comparison of Machine Learning Algorithms in Classification of Grazing and Rumination Behaviour in Sheep. Sensors, 18.","DOI":"10.3390\/s18103532"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ding, L.Y., Zhang, C.X., Yue, Y.X., Yao, C.X., Li, Z., Hu, Y.T., Yang, B.Z., Ma, W.H., Yu, L.G., and Gao, R.H. (2025). Wearable Sensors-Based Intelligent Sensing and Application of Animal Behaviors: A Comprehensive Review. Sensors, 25.","DOI":"10.20944\/preprints202506.1631.v1"},{"key":"ref_10","first-page":"1420","article-title":"A Comprehensive Review of Deep Learning Approaches for Animal Detection on Video Data","volume":"14","author":"Kumar","year":"2023","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"109635","DOI":"10.1016\/j.compag.2024.109635","article-title":"Improved composite deep learning and multi-scale signal features fusion enable intelligent and precise behaviors recognition of fattening Hu sheep","volume":"227","author":"Zhang","year":"2024","journal-title":"Comput. Electron. Agric."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"106255","DOI":"10.1016\/j.compag.2021.106255","article-title":"Behaviour recognition of pigs and cattle: Journey from computer vision to deep learning","volume":"187","author":"Chen","year":"2021","journal-title":"Comput. Electron. Agric."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Hu, T.C., Yan, R.R., Jiang, C.X., Chand, N.V., Bai, T., Guo, L.F., and Qi, J.W. (2023). Grazing Sheep Behaviour Recognition Based on Improved YOLOV5. Sensors, 23.","DOI":"10.3390\/s23104752"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"He, R.A., Wang, X.M., Chen, H.Z., and Liu, C. (2023). VHR-BirdPose: Vision Transformer-Based HRNet for Bird Pose Estimation with Attention Mechanism. Electronics, 12.","DOI":"10.3390\/electronics12173643"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Liu, S.C., Fan, Q.C., Liu, S.H., and Zhao, C.J. (2022). DepthFormer: A High-Resolution Depth-Wise Transformer for Animal Pose Estimation. Agriculture, 12.","DOI":"10.3390\/agriculture12081280"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/s12204-025-2815-7","article-title":"ListPose: Lightweight and Implicit Spatial-Temporal Modeling with TokenPose for Video-Based Pose Estimation","volume":"31","author":"Wu","year":"2025","journal-title":"J. Shanghai Jiaotong Univ. (Sci.)"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sun, C.J., Hu, J.G., Wang, Q.Y., Zhu, C., Chen, L., and Shi, C.M. (2025). YOLO-BCD: A Lightweight Multi-Module Fusion Network for Real-Time Sheep Pose Estimation. Sensors, 25.","DOI":"10.3390\/s25092687"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1719","DOI":"10.4081\/jae.2025.1719","article-title":"Sheep pose estimation via image analysis and body measurements derived from key points","volume":"56","author":"Bati","year":"2025","journal-title":"J. Agric. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"110283","DOI":"10.1016\/j.compag.2025.110283","article-title":"Detection of estrous ewes\u2019 tail-wagging behavior in group-housed environments using Temporal-Boost 3D convolution","volume":"234","author":"Shi","year":"2025","journal-title":"Comput. Electron. Agric."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Psota, E.T., Schmidt, T., Mote, B., and P\u00e9rez, L.C. (2020). Long-Term Tracking of Group-Housed Livestock Using Keypoint Detection and MAP Estimation for Individual Animal Identification. Sensors, 20.","DOI":"10.3390\/s20133670"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Br\u00fcnger, J., Gentz, M., Traulsen, I., and Koch, R. (2020). Panoptic Segmentation of Individual Pigs for Posture Recognition. Sensors, 20.","DOI":"10.3390\/s20133710"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"496","DOI":"10.1038\/s41592-022-01443-0","article-title":"Multi-animal pose estimation, identification and tracking with DeepLabCut","volume":"19","author":"Lauer","year":"2022","journal-title":"Nat. Methods"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"106559","DOI":"10.1016\/j.compag.2021.106559","article-title":"T-LEAP: Occlusion-robust pose estimation of walking cows using temporal information","volume":"192","author":"Russello","year":"2022","journal-title":"Comput. Electron. Agric."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1145\/3603618","article-title":"Deep Learning-based Human Pose Estimation: A Survey","volume":"56","author":"Zheng","year":"2024","journal-title":"ACM Comput. Surv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"117521","DOI":"10.1109\/ACCESS.2025.3585267","article-title":"2D Animal Skeletons Keypoint Detection: Research Progress and Future Trends","volume":"13","author":"Ma","year":"2025","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"103483","DOI":"10.1016\/j.cviu.2022.103483","article-title":"Animal pose estimation: A closer look at the state-of-the-art, existing gaps and opportunities","volume":"222","author":"Jiang","year":"2022","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Yang, Y.X., Deng, Y.F., Li, J.Z., Liu, M.Q., Yao, Y., Peng, Z.Y., Gu, L.H., and Peng, Y.Q. (2024). An Effective Yak Behavior Classification Model with Improved YOLO-Pose Network Using Yak Skeleton Key Points Images. Agriculture, 14.","DOI":"10.3390\/agriculture14101796"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"105863","DOI":"10.1016\/j.compag.2020.105863","article-title":"Pose estimation and behavior classification of broiler chickens based on deep neural networks","volume":"180","author":"Fang","year":"2021","journal-title":"Comput. Electron. Agric."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Nwokedi, E.I., Bains, R.S., Bidaut, L., Ye, X.J., Wells, S., and Brown, J.M. (2023). Dual-Stream Spatiotemporal Networks with Feature Sharing for Monitoring Animals in the Home Cage. Sensors, 23.","DOI":"10.3390\/s23239532"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Jahn, S., D\u00fcpjan, S., R\u00f6ttgen, V., Jordt, J.N., Albrecht, S., and Homeier-Bachmann, T. (2026). Integrating YOLOv8-pose for localization and behavior tracking of calves in precision livestock farming. Front. Anim. Sci., 6.","DOI":"10.3389\/fanim.2025.1718641"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1007\/978-981-96-1535-3_15","article-title":"Identification of Rumination Patterns in Cattle Through Optical Flow Analysis and Machine Learning Techniques","volume":"Volume 1322","author":"Ishikawa","year":"2025","journal-title":"Genetic and Evolutionary Computing"}],"container-title":["Animals"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-2615\/16\/17\/2759\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T12:09:56Z","timestamp":1788350996000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-2615\/16\/17\/2759"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,2]]},"references-count":31,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["ani16172759"],"URL":"https:\/\/doi.org\/10.3390\/ani16172759","relation":{},"ISSN":["2076-2615"],"issn-type":[{"value":"2076-2615","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,2]]}}}