{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,6]],"date-time":"2025-12-06T05:24:59Z","timestamp":1764998699420,"version":"3.46.0"},"reference-count":58,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T00:00:00Z","timestamp":1764633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"crossref","award":["R01LM014342","R01LM013364","DP2EB035858"],"award-info":[{"award-number":["R01LM014342","R01LM013364","DP2EB035858"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Stanford Center for Digital Health"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Early detection of Autism Spectrum Disorder (ASD), a neurodevelopmental condition characterized by social communication challenges, is essential for timely intervention. Naturalistic home videos collected via mobile applications offer scalable opportunities for digital diagnostics. We leveraged GuessWhat, a mobile game designed to engage parents and children, which has generated over 3000 structured videos from 382 children. From this collection, we curated a final analytic sample of 688 feature-rich videos centered on a single dyad, enabling more consistent modeling. We developed a two-step pipeline: (1) filtering to isolate high-quality videos, and (2) feature engineering to extract interpretable behavioral signals. Unimodal LSTM-based models trained on eye gaze, head position, and facial expression achieved test AUCs of 86% (95% CI: 0.79\u20130.92), 78% (95% CI: 0.69\u20130.86), and 67% (95% CI: 0.55\u20130.78), respectively. Late-stage fusion of unimodal outputs significantly improved predictive performance, yielding a test AUC of 90% (95% CI: 0.84\u20130.95). Our findings demonstrate the complementary value of distinct behavioral channels and support the feasibility of using mobile-captured videos for detecting clinically relevant signals. While further work is needed to improve generalizability and inclusivity, this study highlights the promise of real-time, scalable autism phenotyping for early interventions.<\/jats:p>","DOI":"10.3390\/a18120764","type":"journal-article","created":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T18:42:02Z","timestamp":1764960122000},"page":"764","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Ensemble Modeling of Multiple Physical Indicators to Dynamically Phenotype Autism Spectrum Disorder"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-0994-1296","authenticated-orcid":false,"given":"Marie Amale","family":"Huynh","sequence":"first","affiliation":[{"name":"Department of Biomedical Data Science and Department of Pediatrics, Stanford University, Stanford, CA 94305, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0077-5485","authenticated-orcid":false,"given":"Aaron","family":"Kline","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Science and Department of Pediatrics, Stanford University, Stanford, CA 94305, USA"}]},{"given":"Saimourya","family":"Surabhi","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Science and Department of Pediatrics, Stanford University, Stanford, CA 94305, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4423-5269","authenticated-orcid":false,"given":"Kaitlyn","family":"Dunlap","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Science and Department of Pediatrics, Stanford University, Stanford, CA 94305, USA"}]},{"given":"Onur Cezmi","family":"Mutlu","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Science and Department of Pediatrics, Stanford University, Stanford, CA 94305, USA"}]},{"given":"Mohammadmahdi","family":"Honarmand","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Science and Department of Pediatrics, Stanford University, Stanford, CA 94305, USA"}]},{"given":"Parnian","family":"Azizian","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Science and Department of Pediatrics, Stanford University, Stanford, CA 94305, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3276-4411","authenticated-orcid":false,"given":"Peter","family":"Washington","sequence":"additional","affiliation":[{"name":"Division of Clinical Informatics & Digital Transformation, Department of Medicine, University of California, San Francisco, CA 94143, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7889-9146","authenticated-orcid":false,"given":"Dennis P.","family":"Wall","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Science and Department of Pediatrics, Stanford University, Stanford, CA 94305, USA"}]}],"member":"1968","published-online":{"date-parts":[[2025,12,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.15585\/mmwr.ss7202a1","article-title":"Prevalence and characteristics of autism spectrum disorder among children aged 8 years\u2014Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2020","volume":"72","author":"Maenner","year":"2023","journal-title":"MMWR Surveill. Summ."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1183","DOI":"10.1542\/peds.2007-2361","article-title":"Identification and evaluation of children with autism spectrum disorders","volume":"120","author":"Johnson","year":"2007","journal-title":"Pediatrics"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"American Psychiatric Association (2013). DSM-5 Task Force. Diagnostic and Statistical Manual of Mental Disorders: DSM-5, American Psychiatric Association. [5th ed.].","DOI":"10.1176\/appi.books.9780890425596"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"101502","DOI":"10.1016\/j.rasd.2019.101502","article-title":"The lifetime social cost of autism: 1990\u20132029","volume":"72","author":"Cakir","year":"2020","journal-title":"Res. Autism Spectr. Disord."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1002\/0470869380.ch16","article-title":"Early intervention and brain plasticity in autism","volume":"Volume 251","author":"Dawson","year":"2003","journal-title":"Autism: Neural Basis and Treatment Possibilities: Novartis Foundation Symposium 251"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1001\/archpsyc.63.6.694","article-title":"Autism from 2 to 9 years of age","volume":"63","author":"Lord","year":"2006","journal-title":"Arch. Gen. Psychiatry"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.cca.2019.12.009","article-title":"Biomarkers in autism spectrum disorders: Current progress","volume":"502","author":"Shen","year":"2020","journal-title":"Clin. Chim. Acta"},{"key":"ref_8","first-page":"851","article-title":"Whittling down the wait time: Exploring models to minimize the delay from initial concern to diagnosis and treatment of autism spectrum disorder","volume":"63","author":"Foster","year":"2016","journal-title":"Pediatr. Clin."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1097\/NMD.0b013e3182a2168a","article-title":"Diagnostic and statistical manual of mental disorders, and clinical utility","volume":"201","author":"First","year":"2013","journal-title":"J. Nerv. Ment. Dis."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1177\/1362361306068505","article-title":"The discriminative ability and diagnostic utility of the ADOS-G, ADI-R, and GARS for children in a clinical setting","volume":"10","author":"Mazefsky","year":"2006","journal-title":"Autism"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2770","DOI":"10.1007\/s10803-017-3176-3","article-title":"Examining the role of race, ethnicity, and gender on social and behavioral ratings within the Autism Diagnostic Observation Schedule","volume":"47","author":"Harrison","year":"2017","journal-title":"J. Autism Dev. Disord."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1177\/1362361317722030","article-title":"Race influences parent report of concerns about symptoms of autism spectrum disorder","volume":"23","author":"Donohue","year":"2019","journal-title":"Autism"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"e229498","DOI":"10.1001\/jamanetworkopen.2022.9498","article-title":"Analysis of race and sex bias in the autism diagnostic observation schedule (ADOS-2)","volume":"5","author":"Kalb","year":"2022","journal-title":"JAMA Netw. Open"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kalantarian, H., Washington, P., Schwartz, J., Daniels, J., Haber, N., and Wall, D. (2018, January 4\u20137). A gamified mobile system for crowdsourcing video for autism research. Proceedings of the 2018 IEEE International Conference on Healthcare Informatics (ICHI), New York, NY, USA.","DOI":"10.1109\/ICHI.2018.00052"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.artmed.2019.06.004","article-title":"Labeling images with facial emotion and the potential for pediatric healthcare","volume":"98","author":"Kalantarian","year":"2019","journal-title":"Artif. Intell. Med."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1030","DOI":"10.1055\/s-0041-1736626","article-title":"A mobile game platform for improving social communication in children with autism: A feasibility study","volume":"12","author":"Penev","year":"2021","journal-title":"Appl. Clin. Inform."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3038","DOI":"10.1007\/s10803-021-05179-2","article-title":"Identifying autism with head movement features by implementing machine learning algorithms","volume":"52","author":"Zhao","year":"2022","journal-title":"J. Autism Dev. Disord."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Lakkapragada, A., Kline, A., Mutlu, O.C., Paskov, K., Chrisman, B., Stockham, N., Washington, P., and Wall, D.P. (2022). The classification of abnormal hand movement to aid in autism detection: Machine learning study. JMIR Biomed. Eng., 7.","DOI":"10.2196\/33771"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.cobme.2018.12.002","article-title":"Computer vision and behavioral phenotyping: An autism case study","volume":"9","author":"Sapiro","year":"2019","journal-title":"Curr. Opin. Biomed. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Dawson, G., Campbell, K., Hashemi, J., Lippmann, S.J., Smith, V., Carpenter, K., Egger, H., Espinosa, S., Vermeer, S., and Baker, J. (2018). Atypical postural control can be detected via computer vision analysis in toddlers with autism spectrum disorder. Sci. Rep., 8.","DOI":"10.1038\/s41598-018-35215-8"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"827","DOI":"10.1001\/jamapediatrics.2021.0530","article-title":"Computational methods to measure patterns of gaze in toddlers with autism spectrum disorder","volume":"175","author":"Chang","year":"2021","journal-title":"JAMA Pediatr."},{"key":"ref_22","first-page":"113938","article-title":"Machine learning-based prediction of autism spectrum disorder from eye tracking data: A systematic review","volume":"164","author":"Akan","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"116198","DOI":"10.1016\/j.image.2021.116198","article-title":"Predicting ASD diagnosis in children with synthetic and image-based eye gaze data","volume":"94","author":"Liaqat","year":"2021","journal-title":"Signal Process. Image Commun."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e31830","DOI":"10.2196\/31830","article-title":"Identification of Social Engagement Indicators Associated with Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods","volume":"24","author":"Varma","year":"2022","journal-title":"J. Med Internet Res."},{"key":"ref_25","first-page":"1234","article-title":"Advanced Variations in Eye-Tracking Methodologies for Autism Diagnostics","volume":"49","author":"Takahashi","year":"2019","journal-title":"J. Autism Dev. Disord."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1186\/s13229-018-0198-4","article-title":"Objective measurement of head movement differences in children with and without autism spectrum disorder","volume":"9","author":"Martin","year":"2018","journal-title":"Mol. Autism"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Singh, P., Pathak, A., Ganai, U.J., Bhushan, B., and Subramanian, V.K. (2024, January 17\u201318). Video Based Computational Coding of Movement Anomalies in ASD Children. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Seattle, WA, USA.","DOI":"10.1109\/CVPRW63382.2024.00045"},{"key":"ref_28","first-page":"919","article-title":"Exploring complexity of facial dynamics in autism spectrum disorder","volume":"14","author":"Krishnappababu","year":"2021","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Jiang, M., Francis, S.M., Srishyla, D., Conelea, C., Zhao, Q., and Jacob, S. (2019, January 23\u201327). Classifying individuals with ASD through facial emotion recognition and eye-tracking. Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany.","DOI":"10.1109\/EMBC.2019.8857005"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"e39917","DOI":"10.2196\/39917","article-title":"Training and profiling a pediatric facial expression classifier for children on mobile devices: Machine learning study","volume":"7","author":"Banerjee","year":"2023","journal-title":"JMIR Form. Res."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Song, C., Li, J., and Ouyang, G. (2022, January 9\u201312). Early diagnosis of asd based on facial expression recognition and head pose estimation. Proceedings of the 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Prague, Czech Republic.","DOI":"10.1109\/SMC53654.2022.9945448"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Simeoli, R., Milano, N., Rega, A., and Marocco, D. (2021). Using technology to identify children with autism through motor abnormalities. Front. Psychol., 12.","DOI":"10.3389\/fpsyg.2021.635696"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Marzi, C., Narzisi, A., Milone, A., Masi, G., and Pirrelli, V. (2022). Reading Behaviors through Patterns of Finger-Tracking in Italian Children with Autism Spectrum Disorder. Brain Sci., 12.","DOI":"10.3390\/brainsci12101316"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Vabalas, A., Gowen, E., Poliakoff, E., and Casson, A.J. (2020). Applying machine learning to kinematic and eye movement features of a movement imitation task to predict autism diagnosis. Sci. Rep., 10.","DOI":"10.1038\/s41598-020-65384-4"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"e35406","DOI":"10.2196\/35406","article-title":"Classifying autism from crowdsourced semistructured speech recordings: Machine learning model comparison study","volume":"5","author":"Chi","year":"2022","journal-title":"JMIR Pediatr. Parent."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Plank, I.S., Koehler, J.C., Nelson, A.M., Koutsouleris, N., and Falter-Wagner, C. (2023). Automated extraction of speech and turn-taking parameters in autism allows for diagnostic classification using a multivariable prediction model. Front. Psychiatry, 14.","DOI":"10.3389\/fpsyt.2023.1257569"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"e13174","DOI":"10.2196\/13174","article-title":"The performance of emotion classifiers for children with parent-reported autism: Quantitative feasibility study","volume":"7","author":"Kalantarian","year":"2020","journal-title":"JMIR Ment. Health"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"e26760","DOI":"10.2196\/26760","article-title":"Improved digital therapy for developmental pediatrics using domain-specific artificial intelligence: Machine learning study","volume":"5","author":"Washington","year":"2022","journal-title":"JMIR Pediatr. Parent."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2489","DOI":"10.1038\/s41591-023-02574-3","article-title":"Early detection of autism using digital behavioral phenotyping","volume":"29","author":"Perochon","year":"2023","journal-title":"Nat. Med."},{"key":"ref_40","first-page":"AIcs2400510","article-title":"Validation of a Mobile App for Remote Autism Screening in Toddlers","volume":"1","author":"Aiello","year":"2024","journal-title":"NEJM AI"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2003","DOI":"10.1109\/TNSRE.2022.3192431","article-title":"A multimodal approach for identifying autism spectrum disorders in children","volume":"30","author":"Han","year":"2022","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1007\/s13246-023-01350-4","article-title":"Multimodal Kernel-based discriminant correlation analysis data-fusion approach: An automated autism spectrum disorder diagnostic system","volume":"47","author":"Wadhera","year":"2024","journal-title":"Phys. Eng. Sci. Med."},{"key":"ref_43","first-page":"759","article-title":"Data-driven diagnostics and the potential of mobile artificial intelligence for digital therapeutic phenotyping in computational psychiatry","volume":"5","author":"Washington","year":"2020","journal-title":"Biol. Psychiatry Cogn. Neurosci. Neuroimaging"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1146\/annurev-biodatasci-020722-125454","article-title":"A review of and roadmap for data science and machine learning for the neuropsychiatric phenotype of autism","volume":"6","author":"Washington","year":"2023","journal-title":"Annu. Rev. Biomed. Data Sci."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1007\/s41666-018-0034-9","article-title":"Guess What? Towards Understanding Autism from Structured Video Using Facial Affect","volume":"3","author":"Kalantarian","year":"2019","journal-title":"J. Healthc. Inform. Res."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"3383","DOI":"10.1007\/s10803-021-05218-y","article-title":"Validation of autism diagnosis and clinical data in the SPARK cohort","volume":"52","author":"Fombonne","year":"2022","journal-title":"J. Autism Dev. Disord."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Sparrow, S.S. (2011). Vineland adaptive behavior scales. Encyclopedia of Clinical Neuropsychology, Springer.","DOI":"10.1007\/978-0-387-79948-3_1602"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Constantino, J.N. (2013). Social responsiveness scale. Encyclopedia of Autism Spectrum Disorders, Springer.","DOI":"10.1007\/978-1-4419-1698-3_296"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1953","DOI":"10.1007\/s10803-016-2718-4","article-title":"Clinical evaluation of a novel and mobile autism risk assessment","volume":"46","author":"Duda","year":"2016","journal-title":"J. Autism Dev. Disord."},{"key":"ref_50","unstructured":"Rutter, M., Bailey, A., and Lord, C. (2003). The Social Communication Questionnaire, Western Psychological Services."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"758","DOI":"10.1111\/dmcn.16200","article-title":"Psychometric evaluation of the Autism Symptom Dimensions Questionnaire","volume":"67","author":"Frazier","year":"2025","journal-title":"Dev. Med. Child Neurol."},{"key":"ref_52","unstructured":"(2023, June 04). Amazon Rekognition Developer Guide. Available online: https:\/\/docs.aws.amazon.com\/rekognition\/latest\/APIReference\/Welcome.html."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Hou, C., Kalantarian, H., Washington, P., Dunlap, K., and Wall, D.P. (2021). Leveraging video data from a digital smartphone autism therapy to train an emotion detection classifier. medRxiv, 2021.","DOI":"10.1101\/2021.07.28.21260646"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1109\/TG.2018.2877325","article-title":"A mobile game for automatic emotion-labeling of images","volume":"12","author":"Kalantarian","year":"2018","journal-title":"IEEE Trans. Games"},{"key":"ref_55","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Adv. Neural Inf. Process. Syst., 32."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019, January 4\u20138). Optuna: A Next-generation Hyperparameter Optimization Framework. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330701"},{"key":"ref_58","unstructured":"Google (2025, November 11). ML Kit: Machine Learning for Mobile Developers. Available online: https:\/\/developers.google.com\/ml-kit."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/12\/764\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,6]],"date-time":"2025-12-06T05:21:22Z","timestamp":1764998482000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/12\/764"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,2]]},"references-count":58,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["a18120764"],"URL":"https:\/\/doi.org\/10.3390\/a18120764","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2025,12,2]]}}}