{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T15:41:07Z","timestamp":1792856467194,"version":"4.5.0"},"reference-count":47,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2025,3,14]],"date-time":"2025-03-14T00:00:00Z","timestamp":1741910400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["FR 2069\/8\u20101"],"award-info":[{"award-number":["FR 2069\/8\u20101"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Autism Research"],"published-print":{"date-parts":[[2025,4]]},"abstract":"<jats:title>ABSTRACT<\/jats:title><jats:p>Mannerisms describe repetitive or unconventional body movements like arm flapping. These movements are early markers of restricted and repetitive behaviors (RRBs) in autism spectrum disorder (ASD). However, assessing mannerisms reliably is challenging. Even after extensive training in behavioral observations, inter\u2010rater agreements for mannerism items remain insufficient. The current study used machine learning (ML) to classify mannerisms from videotaped behavioral observations in children with ASD. We developed a classification scheme for mannerisms as ground truth and applied it to videotaped behavioral observations from an early intervention study. ML was used in two steps: First, the OpenPose algorithm post hoc extracted features based on body movements in the videos. Second, a long short\u2010term memory (LSTM) neural network classified the features in a multi\u2010label approach to distinguish between the absence of mannerisms, flapping, jumping, and both flapping + jumping. The trained models achieved 70.2% accuracy (<jats:italic>F<\/jats:italic>1 score: 31.8%) using nested cross\u2010validation. The analysis improves on previous videotaped ML classification studies by splitting training and test data subject\u2010wise, highlighting its clinical applicability. The LSTM models are made publicly available for use with other video datasets. Our results show that ML\u2010based classification of mannerisms is a promising tool for enhancing objective diagnostic methods of behavioral observations.<\/jats:p>","DOI":"10.1002\/aur.70020","type":"journal-article","created":{"date-parts":[[2025,3,14]],"date-time":"2025-03-14T09:59:38Z","timestamp":1741946378000},"page":"833-844","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Semi\u2010Automated Multi\u2010Label Classification of Autistic Mannerisms by Machine Learning on Post Hoc Skeletal Tracking"],"prefix":"10.1002","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-1911-859X","authenticated-orcid":false,"given":"Christian","family":"Lemler","sequence":"first","affiliation":[{"name":"Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Autism Research and Intervention Center of Excellence University Hospital Frankfurt, Goethe\u2010University  Frankfurt am Main Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3055-1751","authenticated-orcid":false,"given":"Solvejg K.","family":"Kleber","sequence":"additional","affiliation":[{"name":"Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Autism Research and Intervention Center of Excellence University Hospital Frankfurt, Goethe\u2010University  Frankfurt am Main Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leonie","family":"Polzer","sequence":"additional","affiliation":[{"name":"Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Autism Research and Intervention Center of Excellence University Hospital Frankfurt, Goethe\u2010University  Frankfurt am Main Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9221-162X","authenticated-orcid":false,"given":"Naisan","family":"Raji","sequence":"additional","affiliation":[{"name":"Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Autism Research and Intervention Center of Excellence University Hospital Frankfurt, Goethe\u2010University  Frankfurt am Main Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Janina","family":"Kitzerow\u2010Cleven","sequence":"additional","affiliation":[{"name":"Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Autism Research and Intervention Center of Excellence University Hospital Frankfurt, Goethe\u2010University  Frankfurt am Main Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ziyon","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Autism Research and Intervention Center of Excellence University Hospital Frankfurt, Goethe\u2010University  Frankfurt am Main Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-3429-8306","authenticated-orcid":false,"given":"Simeon","family":"Platte","sequence":"additional","affiliation":[{"name":"Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Autism Research and Intervention Center of Excellence University Hospital Frankfurt, Goethe\u2010University  Frankfurt am Main Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christine M.","family":"Freitag","sequence":"additional","affiliation":[{"name":"Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Autism Research and Intervention Center of Excellence University Hospital Frankfurt, Goethe\u2010University  Frankfurt am Main Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5721-207X","authenticated-orcid":false,"given":"Nico","family":"Bast","sequence":"additional","affiliation":[{"name":"Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Autism Research and Intervention Center of Excellence University Hospital Frankfurt, Goethe\u2010University  Frankfurt am Main Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2025,3,14]]},"reference":[{"key":"e_1_2_11_2_1","doi-asserted-by":"publisher","DOI":"10.1176\/appi.books.9780890425596"},{"key":"e_1_2_11_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dajour.2022.100071"},{"key":"e_1_2_11_4_1","unstructured":"Barami T.2020.\u201cSet_person_id [Computer Software].\u201dhttps:\/\/github.com\/CMU\u2010Perceptual\u2010Computing\u2010Lab\/openpose\/issues\/1448#issuecomment\u2010575936689."},{"key":"e_1_2_11_5_1","volume-title":"Diagnostisches Interview f\u00fcr Autismus\u2010Revidiert (ADI\u2010R)","author":"B\u00f6lte S.","year":"2006"},{"key":"e_1_2_11_6_1","doi-asserted-by":"publisher","DOI":"10.1002\/aur.49"},{"key":"e_1_2_11_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10803\u2010016\u20102716\u20106"},{"key":"e_1_2_11_8_1","doi-asserted-by":"crossref","unstructured":"Cao Z. G.Hidalgo T.Simon S.\u2010E.Wei andY.Sheikh.2018.\u201cOpenPose: Realtime Multi\u2010Person 2D Pose Estimation Using Part Affinity Fields.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.1812.08008.","DOI":"10.1109\/CVPR.2017.143"},{"key":"e_1_2_11_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/aur.2449"},{"key":"e_1_2_11_10_1","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2022.986876"},{"key":"e_1_2_11_11_1","doi-asserted-by":"crossref","unstructured":"Cook A. B.Mandal D.Berry andM.Johnson.2019.\u201cTowards Automatic Screening of Typical and Atypical Behaviors in Children With Autism.\u201dIEEE International Conference on Data Science and Advanced Analytics.https:\/\/doi.org\/10.1109\/DSAA.2019.00065.","DOI":"10.1109\/DSAA.2019.00065"},{"key":"e_1_2_11_12_1","doi-asserted-by":"publisher","DOI":"10.3390\/s19225026"},{"key":"e_1_2_11_13_1","doi-asserted-by":"crossref","unstructured":"Donahue J. L. A.Hendricks M.Rohrbach et\u00a0al.2015.\u201cLong\u2010Term Recurrent Convolutional Networks for Visual Recognition and Description.\u201dInProceedings of the IEEE Conference on Computer Vision and Pattern Recognition.https:\/\/doi.org\/10.48550\/arXiv.1411.4389.","DOI":"10.1109\/CVPR.2015.7298878"},{"key":"e_1_2_11_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jaac.2014.08.004"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00787\u2010013\u20100375\u20100"},{"key":"e_1_2_11_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-05318-5_1"},{"key":"e_1_2_11_17_1","doi-asserted-by":"crossref","unstructured":"Fourie M. andD.van derHaar.2020.\u201cA Feature Importance Study in Ballet Pose Recognition With OpenPose. Artificial Intelligence.\u201dIn International Conference on Human\u2010Computer Interaction. 243\u2013254.https:\/\/doi.org\/10.1007\/978\u20103\u2010030\u201050334\u20105_16.","DOI":"10.1007\/978-3-030-50334-5_16"},{"key":"e_1_2_11_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/aur.2077"},{"key":"e_1_2_11_19_1","volume-title":"Deep Learning","author":"Goodfellow I.","year":"2016"},{"key":"e_1_2_11_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10803\u2010010\u20101102\u2010z"},{"key":"e_1_2_11_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10803\u2010016\u20102782\u20109"},{"key":"e_1_2_11_22_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_11_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40489\u2010019\u201000158\u2010x"},{"key":"e_1_2_11_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10803\u2010021\u201004952\u20107"},{"key":"e_1_2_11_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00787\u2010018\u20101143\u2010y"},{"key":"e_1_2_11_26_1","doi-asserted-by":"publisher","DOI":"10.1177\/1362361318793253"},{"key":"e_1_2_11_27_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13063\u2010019\u20103881\u20107"},{"key":"e_1_2_11_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/aur.1588"},{"key":"e_1_2_11_29_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598\u2010021\u201094378\u2010z"},{"key":"e_1_2_11_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcm.2016.02.012"},{"key":"e_1_2_11_31_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1603.06560"},{"key":"e_1_2_11_32_1","doi-asserted-by":"publisher","DOI":"10.1002\/pits.22681"},{"key":"e_1_2_11_33_1","volume-title":"INTERACT Benutzerhandbuch","author":"Mangold P.","year":"2020"},{"key":"e_1_2_11_34_1","doi-asserted-by":"crossref","unstructured":"Pak M. andS.Kim.2017.\u201cA Review of Deep Learning in Image Recognition.\u201dInternational Conference on Computer Applications and Information Processing Technology (CAIPT).https:\/\/doi.org\/10.1109\/CAIPT.2017.8320684.","DOI":"10.1109\/CAIPT.2017.8320684"},{"key":"e_1_2_11_35_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1201.0490"},{"key":"e_1_2_11_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ridd.2012.04.006"},{"key":"e_1_2_11_37_1","doi-asserted-by":"publisher","DOI":"10.1177\/1362361316669235"},{"key":"e_1_2_11_38_1","volume-title":"ADOS\u20102: Diagnostische Beobachtungsskala f\u00fcr Autistische St\u00f6rungen\u20142","author":"Poustka L.","year":"2015"},{"key":"e_1_2_11_39_1","volume-title":"Bayley Scales of Infant and Toddler Development Third Edition. German Version","author":"Reuner G.","year":"2015"},{"key":"e_1_2_11_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10803\u2010020\u201004430\u20106"},{"key":"e_1_2_11_41_1","doi-asserted-by":"publisher","DOI":"10.5765\/jkacap.190027"},{"key":"e_1_2_11_42_1","doi-asserted-by":"publisher","DOI":"10.1080\/17538157.2017.1399132"},{"key":"e_1_2_11_43_1","doi-asserted-by":"publisher","DOI":"10.1155\/2018\/7068349"},{"key":"e_1_2_11_44_1","doi-asserted-by":"crossref","unstructured":"Vyas K. R.Ma B.Rezaei et\u00a0al.2019.\u201cRecognition of Atypical Behavior in Autism Diagnosis From Video Using Pose Estimation Over Time.\u201dIEEE Workshop on Machine Learning for Signal Processing.https:\/\/doi.org\/10.1109\/MLSP.2019.8918863.","DOI":"10.1109\/MLSP.2019.8918863"},{"key":"e_1_2_11_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00521\u2010019\u201004232\u20107"},{"key":"e_1_2_11_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-68763-2_42"},{"key":"e_1_2_11_47_1","doi-asserted-by":"publisher","DOI":"10.3390\/s21020411"},{"key":"e_1_2_11_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijdevneu.2004.05.001"}],"container-title":["Autism Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aur.70020","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,4,23]],"date-time":"2025-04-23T09:56:18Z","timestamp":1745402178000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/aur.70020"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,14]]},"references-count":47,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,4]]}},"alternative-id":["10.1002\/aur.70020"],"URL":"https:\/\/doi.org\/10.1002\/aur.70020","archive":["Portico"],"relation":{},"ISSN":["1939-3792","1939-3806"],"issn-type":[{"value":"1939-3792","type":"print"},{"value":"1939-3806","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,14]]},"assertion":[{"value":"2024-01-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-14","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}