{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T20:12:30Z","timestamp":1778616750093,"version":"3.51.4"},"reference-count":62,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,9,19]],"date-time":"2025-09-19T00:00:00Z","timestamp":1758240000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:p>Understanding human behavior is a fundamental goal of social sciences, yet conventional methodologies are often limited by labor-intensive data collection and complex analyses. Computational models offer a promising alternative for analyzing large datasets and identifying key behavioral indicators, but their adoption is hindered by technical complexity and substantial computational requirements. To address these barriers, we introduce <jats:italic>DISCOVER<\/jats:italic>, a modular and user-friendly software framework designed to streamline computational data exploration for human behavior analysis. <jats:italic>DISCOVER<\/jats:italic> democratizes access to state-of-the-art models, enabling researchers across disciplines to conduct detailed behavioral analyses without extensive technical expertise. In this paper, we are showcasing <jats:italic>DISCOVER<\/jats:italic> using four modular data exploration workflows that build on each other: Semantic Content Exploration, Visual Inspection, Aided Annotation, and Multimodal Scene Search. Finally, we report initial findings from a user study. The study examined <jats:italic>DISCOVER<\/jats:italic>\u2019s potential to support prospective psychotherapists in structuring information for treatment planning, i.e. case conceptualizations.<\/jats:p>","DOI":"10.3389\/fdgth.2025.1638539","type":"journal-article","created":{"date-parts":[[2025,9,19]],"date-time":"2025-09-19T13:59:21Z","timestamp":1758290361000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["DISCOVER: a Data-driven Interactive System for Comprehensive Observation, Visualization, and ExploRation of human behavior"],"prefix":"10.3389","volume":"7","author":[{"given":"Tobias","family":"Hallmen","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dominik","family":"Schiller","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Antonia","family":"Vehlen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Steffen","family":"Eberhardt","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tobias","family":"Baur","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daksitha","family":"Withanage Don","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wolfgang","family":"Lutz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elisabeth","family":"Andr\u00e9","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,9,19]]},"reference":[{"key":"B1","article-title":"ELAN: a professional framework for multimodality research","author":"Wittenburg","year":""},{"key":"B2","article-title":"Anvil: the video annotation research tool","author":"Kipp","year":""},{"key":"B3","article-title":"Transcribing and annotating spoken language with exmaralda","author":"Schmidt","year":""},{"key":"B4","article-title":"Cooperative and transparent machine learning for the context-sensitive analysis of social interactions (dissertation)","author":"Baur","year":""},{"key":"B5","article-title":"\u201cFeeltrace\u201d: an instrument for recording perceived emotion in real time","author":"Cowie","year":""},{"key":"B6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4018\/jse.2012010101","article-title":"Tracing emotion: an overview","volume":"3","author":"Cowie","year":"2012","journal-title":"IJSE"},{"key":"B7","doi-asserted-by":"publisher","first-page":"e5","DOI":"10.5334\/jors.ar","article-title":"CARMA: software for continuous affect rating and media annotation","volume":"2","author":"Girard","year":"2014","journal-title":"J Open Res Softw"},{"key":"B8","doi-asserted-by":"crossref","DOI":"10.3758\/s13428-017-0915-5","article-title":"DARMA: dual axis rating and media annotation","author":"Girard","year":""},{"key":"B9","doi-asserted-by":"publisher","first-page":"102411","DOI":"10.1016\/j.ijhcs.2020.102411","article-title":"Emodash: a dashboard supporting retrospective awareness of emotions in online learning","volume":"139","author":"Ez-Zaouia","year":"2020","journal-title":"Int J Hum Comput Stud"},{"key":"B10","doi-asserted-by":"crossref","DOI":"10.1145\/3290605.3300802","article-title":"Rescue: a framework for real-time feedback on behavioral cues using multimodal anomaly detection","author":"Arakawa","year":""},{"key":"B11","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1109\/TAFFC.2016.2614300","article-title":"Multisense\u2014context-aware nonverbal behavior analysis framework: a psychological distress use case","volume":"8","author":"Stratou","year":"2017","journal-title":"IEEE Trans Affect Comput"},{"key":"B12","doi-asserted-by":"crossref","DOI":"10.1145\/3411764.3445615","article-title":"Meetingcoach: an intelligent dashboard for supporting effective & inclusive meetings","author":"Samrose","year":""},{"key":"B13","doi-asserted-by":"crossref","DOI":"10.1145\/2493432.2493502","article-title":"Mach: my automated conversation coach","author":"Hoque","year":""},{"key":"B14","doi-asserted-by":"crossref","DOI":"10.1145\/3462244.3479886","article-title":"ConAn: a usable tool for multimodal conversation analysis","author":"Penzkofer","year":""},{"key":"B15","doi-asserted-by":"crossref","DOI":"10.1145\/2502081.2502223","article-title":"The social signal interpretation (SSI) framework: multimodal signal processing and recognition in real-time","author":"Wagner","year":""},{"key":"B16","doi-asserted-by":"crossref","DOI":"10.1145\/3382507.3418832","article-title":"Opensense: a platform for multimodal data acquisition and behavior perception","author":"Stefanov","year":""},{"key":"B17","article-title":"Platform for situated intelligence","author":"Bohus","year":""},{"key":"B18","doi-asserted-by":"crossref","DOI":"10.1145\/3461615.3485432","article-title":"Multisensor-pipeline: a lightweight, flexible, and extensible framework for building multimodal-multisensor interfaces","author":"Barz","year":""},{"key":"B19","article-title":"Supporting experts with a multimodal machine-learning-based tool for human behavior analysis of conversational videos","author":"Arakawa","year":""},{"key":"B20","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1007\/s13218-020-00632-3","article-title":"eXplainable cooperative machine learning with NOVA","volume":"34","author":"Baur","year":"2020","journal-title":"K\u00fcnstliche Intell"},{"key":"B21","article-title":"LLaMA: Open and efficient foundation language models","author":"Touvron","year":""},{"key":"B22","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1186\/s12911-024-02481-8","article-title":"Exploring the potential of chatgpt in medical dialogue summarization: a study on consistency with human preferences","volume":"24","author":"Liu","year":"2024","journal-title":"BMC Med Inform Decis Mak"},{"key":"B23","article-title":"ChatGPT as a factual inconsistency evaluator for abstractive text summarization","author":"Luo","year":""},{"key":"B24","article-title":"Comparing abstractive summaries generated by chatgpt to real summaries through blinded reviewers and text classification algorithms","author":"Soni","year":""},{"key":"B25","doi-asserted-by":"publisher","first-page":"100508","DOI":"10.48550\/arXiv.2308.07935","article-title":"Transforming sentiment analysis in the financial domain with chatgpt","volume":"14","author":"Fatouros","year":"2023","journal-title":"Mach Learn Appl"},{"key":"B26","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2024.findings-naacl.246","article-title":"Sentiment analysis in the era of large language models: a reality check","author":"Zhang","year":""},{"key":"B27","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-78541-2_21","article-title":"WIBA: what is being argued? a comprehensive approach to argument mining","author":"Irani","year":""},{"key":"B28","article-title":"Argument-mining from podcasts using ChatGPT","author":"Pojoni","year":""},{"key":"B29","doi-asserted-by":"crossref","DOI":"10.21437\/Interspeech.2023-78","article-title":"Whisperx: time-accurate speech transcription of long-form audio","author":"Bain","year":""},{"key":"B30","article-title":"Robust speech recognition via large-scale weak supervision","author":"Radford","year":""},{"key":"B31","doi-asserted-by":"crossref","DOI":"10.21437\/Interspeech.2021-560","article-title":"End-to-end speaker segmentation for overlap-aware resegmentation","author":"Bredin","year":""},{"key":"B32","doi-asserted-by":"crossref","DOI":"10.1109\/ICASSP40776.2020.9052974","article-title":"pyannote.audio: neural building blocks for speaker diarization","author":"Bredin","year":""},{"key":"B33","article-title":"SpeechBrain: a general-purpose speech toolkit","author":"Ravanelli","year":""},{"key":"B34","article-title":"XLM-T: multilingual language models in Twitter for sentiment analysis and beyond","author":"Barbieri","year":""},{"key":"B35","article-title":"Training a broad-coverage German sentiment classification model for dialog systems","author":"Guhr","year":""},{"key":"B36","doi-asserted-by":"publisher","first-page":"10745","DOI":"10.1109\/TPAMI.2023.3263585","article-title":"Dawn of the transformer era in speech emotion recognition: closing the valence gap","volume":"45","author":"Wagner","year":"2023","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B37","article-title":"Blazepose: on-device real-time body pose tracking","author":"Bazarevsky","year":""},{"key":"B38","article-title":"Blazeface: sub-millisecond neural face detection on mobile gpus","author":"Bazarevsky","year":""},{"key":"B39","doi-asserted-by":"crossref","DOI":"10.1109\/WACV57701.2024.00802","article-title":"Libreface: an open-source toolkit for deep facial expression analysis","author":"Chang","year":""},{"key":"B40","article-title":"Real-time facial surface geometry from monocular video on mobile gpus","author":"Kartynnik","year":""},{"key":"B41","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1007\/BF01115465","article-title":"Measuring facial movement","volume":"1","author":"Ekman","year":"1976","journal-title":"Environ Psychol Nonverbal Behav"},{"key":"B42","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1038\/s42256-020-00280-0","article-title":"Estimation of continuous valence and arousal levels from faces in naturalistic conditions","volume":"3","author":"Toisoul","year":"2021","journal-title":"Nat Mach Intell"},{"key":"B43","doi-asserted-by":"publisher","first-page":"6","DOI":"10.3389\/fcomp.2020.00006","article-title":"Relevance-based data masking: a model-agnostic transfer learning approach for facial expression recognition","volume":"2","author":"Schiller","year":"2020","journal-title":"Front Comput Sci"},{"key":"B44","article-title":"Vision transformers need registers","author":"Darcet","year":""},{"key":"B45","article-title":"DINOv2: learning robust visual features without supervision","author":"Oquab","year":""},{"key":"B46","article-title":"Seamless: multilingual expressive and streaming speech translation","author":"Barrault","year":""},{"key":"B47","doi-asserted-by":"crossref","DOI":"10.1145\/1873951.1874246","article-title":"Opensmile: the Munich versatile and fast open-source audio feature extractor","author":"Eyben","year":""},{"key":"B48","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1109\/TAFFC.2015.2457417","article-title":"The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing","volume":"7","author":"Eyben","year":"2015","journal-title":"IEEE Trans Affect Comput"},{"key":"B49","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2020.acl-main.747","article-title":"Unsupervised cross-lingual representation learning at scale","author":"Conneau","year":""},{"key":"B50","article-title":"AI-based feedback in counselling competence training of prospective teachers","author":"Hallmen","year":""},{"key":"B51","doi-asserted-by":"publisher","first-page":"841","DOI":"10.1080\/10503307.2023.2181114","article-title":"Routine outcome monitoring (rom) and feedback: research review and recommendations","volume":"33","author":"Barkham","year":"2023","journal-title":"Psychother Res"},{"key":"B52","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1037\/ccp0000904","article-title":"Data-informed psychological therapy, measurement-based care, and precision mental health","volume":"92","author":"Lutz","year":"2024","journal-title":"J Consult Clin Psychol"},{"key":"B53","doi-asserted-by":"crossref","DOI":"10.1026\/02912-000","volume-title":"Evaluation und Effekterfassung in der Psychotherapie","author":"Lutz","year":"2019"},{"key":"B54","doi-asserted-by":"crossref","DOI":"10.4324\/9780203380574-9","article-title":"Introduction to formulation","author":"Johnstone","year":""},{"key":"B55","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1002\/jclp.22516","article-title":"Case conceptualization research in cognitive behavior therapy: a state of the science review","volume":"74","author":"Easden","year":"2018","journal-title":"J Clin Psychol"},{"key":"B56","doi-asserted-by":"publisher","first-page":"579","DOI":"10.1037\/0022-006X.73.4.579","article-title":"The quality of psychotherapy case formulations: a comparison of expert, experienced, and novice cognitive-behavioral and psychodynamic therapists","volume":"73","author":"Eells","year":"2005","journal-title":"J Consult Clin Psychol"},{"key":"B57","doi-asserted-by":"publisher","first-page":"e12421","DOI":"10.32872\/cpe.12421","article-title":"From theory to practice: a transtheoretical treatment and training model (4TM)","volume":"6","author":"Lutz","year":"2024","journal-title":"Clin Psychol Eur"},{"key":"B58","doi-asserted-by":"publisher","first-page":"104443","DOI":"10.1016\/j.brat.2023.104443","article-title":"Implementing precision methods in personalizing psychological therapies: barriers and possible ways forward","volume":"172","author":"Deisenhofer","year":"2024","journal-title":"Behav Res Ther"},{"key":"B59","volume-title":"Bergin and Garfield\u2019s Handbook of Psychotherapy and Behavior Change","author":"Barkham","year":"2021"},{"key":"B60","article-title":"Training and supervision in psychotherapy: what we know and where we need to go","author":"Knox","year":""},{"key":"B61","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1080\/10503307.2024.2322522","article-title":"Decoding emotions: exploring the validity of sentiment analysis in psychotherapy","volume":"35","author":"Eberhardt","year":"2025","journal-title":"Psychother Res"},{"key":"B62","article-title":"Mistral NeMo 12B: a large language model","author":"Mistral","year":""}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2025.1638539\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,19]],"date-time":"2025-09-19T13:59:25Z","timestamp":1758290365000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2025.1638539\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,19]]},"references-count":62,"alternative-id":["10.3389\/fdgth.2025.1638539"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2025.1638539","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,19]]},"article-number":"1638539"}}