{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T10:19:20Z","timestamp":1776939560313,"version":"3.51.4"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643686318","type":"electronic"}],"license":[{"start":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T00:00:00Z","timestamp":1761004800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,10,21]]},"abstract":"<jats:p>Digital health interventions often require structured, protocol-driven dialogues delivered with high fidelity. Evaluating whether an agent employing a Large Language Model (LLM) can meet these requirements remains challenging, especially in early development stages. In this work, we present VALISE (Virtual Agent Laboratory for Instruction-Following Simulation and Evaluation), a modular framework for simulating and evaluating LLM agent behavior in delivering structured health interventions. VALISE enables configurable agent\u2013patient simulations using synthetic personas and evaluates protocol adherence through a customizable, automated grid assessed by ensembles of LLM-based judges. We demonstrate its use with Brief Action Planning (BAP), a short intervention promoting behavior change in sedentary individuals. Our results strongly align LLM-based and expert annotations, supporting VALISE\u2019s effectiveness for early-stage evaluations. VALISE offers a reproducible, extensible platform for testing instruction-following capabilities of LLM agents in digital health.<\/jats:p>","DOI":"10.3233\/faia251428","type":"book-chapter","created":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T10:02:23Z","timestamp":1761127343000},"source":"Crossref","is-referenced-by-count":1,"title":["VALISE: A Virtual Agent Laboratory for Instruction-Following Simulation and Evaluation of LLM-Powered Digital Health Interventions"],"prefix":"10.3233","author":[{"given":"Marco","family":"Bolpagni","sequence":"first","affiliation":[{"name":"University of Padova"},{"name":"Fondazione Bruno Kessler"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Simone","family":"De Carli","sequence":"additional","affiliation":[{"name":"Fondazione Bruno Kessler"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Leonardo","family":"Sanna","sequence":"additional","affiliation":[{"name":"Fondazione Bruno Kessler"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mauro","family":"Dragoni","sequence":"additional","affiliation":[{"name":"Fondazione Bruno Kessler"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Silvia","family":"Gabrielli","sequence":"additional","affiliation":[{"name":"Fondazione Bruno Kessler"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","ECAI 2025"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA251428","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T10:02:23Z","timestamp":1761127343000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA251428"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,21]]},"ISBN":["9781643686318"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia251428","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"value":"0922-6389","type":"print"},{"value":"1879-8314","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,21]]}}}