{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T02:13:49Z","timestamp":1771467229983,"version":"3.50.1"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2025,7,8]],"date-time":"2025-07-08T00:00:00Z","timestamp":1751932800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,7,8]],"date-time":"2025-07-08T00:00:00Z","timestamp":1751932800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Instituto Polit\u00e9cnico de Leiria"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Virtual Reality"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Substance abuse has undeniable effects on several cognitive domains, with these associated deficits contributing to poorer treatment outcomes in substance use disorders (SUD). Cognitive training programs (CTPs) address these cognitive deficits, contributing to improved treatment outcomes. The main goal of this study was to validate the content of VRainSUD, a CTP virtual reality-based platform and a mobile follow-up application that proposes to reduce cognitive deficits in SUD. The Delphi method was used to reach a consensus on each of the program\u2019s characteristics (e.g., structure, cognitive training tasks). A panel of experts was invited to participate in the content validation by answering two rounds of questions (scale and free-text boxes) regarding the program. Changes were made between the rounds according to the results of the first round. The consensus was defined as at least 70% of the experts agreeing on the validity of a characteristic. From 39 invited experts, 11 completed the first round, and 6 completed both rounds. The structure of the program reached a consensus on the first round, along with most tasks (with the exception of two mobile application tasks). Significant changes were made following the feedback received, namely turning the program personalizable, and adding positive feedback to the tasks in order to improve adherence and engagement. The Delphi method was a valuable tool to improve VRainSUD. The experts\u2019 opinions validated main features and informed important additional changes, highlighting the importance of expert feedback in the development of cognitive interventions.<\/jats:p>","DOI":"10.1007\/s10055-025-01185-2","type":"journal-article","created":{"date-parts":[[2025,7,8]],"date-time":"2025-07-08T10:15:23Z","timestamp":1751969723000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["VRainSUD: content validation of a cognitive training program using the Delphi method"],"prefix":"10.1007","volume":"29","author":[{"given":"T\u00e2nia","family":"Caetano","sequence":"first","affiliation":[]},{"given":"Maria Salom\u00e9","family":"Pinho","sequence":"additional","affiliation":[]},{"given":"Hugo","family":"Freire","sequence":"additional","affiliation":[]},{"given":"Dany","family":"Mota","sequence":"additional","affiliation":[]},{"given":"Eduardo","family":"Ramadas","sequence":"additional","affiliation":[]},{"given":"Jessica","family":"Lopes","sequence":"additional","affiliation":[]},{"given":"Maria dos Anjos","family":"Dixe","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,7,8]]},"reference":[{"key":"1185_CR1","doi-asserted-by":"publisher","first-page":"1551","DOI":"10.9728\/dcs.2017.18.8.1551","volume":"18","author":"NF Amista","year":"2017","unstructured":"Amista NF, Kim JJ, Kim N (2017) Trend and future of virtual reality for addiction treatment of substance use disorders: a systematic review. J Digit Contents Soc 18:1551\u20131560. https:\/\/doi.org\/10.9728\/dcs.2017.18.8.1551","journal-title":"J Digit Contents Soc"},{"key":"1185_CR2","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1111\/j.1460-2466.1992.tb00810.x","volume":"42","author":"F Biocca","year":"1992","unstructured":"Biocca F (1992) Communication within virtual reality: creating a space for research. J Commun 42:5\u201322. https:\/\/doi.org\/10.1111\/j.1460-2466.1992.tb00810.x","journal-title":"J Commun"},{"key":"1185_CR3","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1177\/193229681100500210","volume":"5","author":"PS Bordnick","year":"2011","unstructured":"Bordnick PS, Carter BL, Traylor AC (2011) What virtual reality research in addictions can tell us about the future of obesity assessment and treatment. J Diabetes Sci Technol 5:265\u2013271. https:\/\/doi.org\/10.1177\/193229681100500210","journal-title":"J Diabetes Sci Technol"},{"key":"1185_CR4","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1146\/annurev.clinpsy.3.022806.091455","volume":"3","author":"TH Brandon","year":"2007","unstructured":"Brandon TH, Vidrine JI, Litvin EB (2007) Relapse and relapse prevention. Annu Rev Clin Psychol 3:257\u2013284. https:\/\/doi.org\/10.1146\/annurev.clinpsy.3.022806.091455","journal-title":"Annu Rev Clin Psychol"},{"key":"1185_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fpsyt.2020.512761","volume":"11","author":"SJ Brooks","year":"2020","unstructured":"Brooks SJ, Mackenzie-Phelan R, Tully J, Schi\u00f6th HB (2020) Review of the neural processes of working memory training: Controlling the impulse to throw the baby out with the bathwater. Front Psychiatry 11:1\u201315. https:\/\/doi.org\/10.3389\/fpsyt.2020.512761","journal-title":"Front Psychiatry"},{"key":"1185_CR6","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1590\/s1980-57642009dn30200010","volume":"3","author":"PS Brum","year":"2009","unstructured":"Brum PS, Forlenza OV, Yassuda MS (2009) Cognitive training in older adults with Mild Cognitive Impairment: Impact on cognitive and functional performance. Dement Neuropsychol 3:124\u2013131. https:\/\/doi.org\/10.1590\/s1980-57642009dn30200010","journal-title":"Dement Neuropsychol"},{"key":"1185_CR7","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1016\/j.chb.2015.01.038","volume":"48","author":"C Burgers","year":"2015","unstructured":"Burgers C, Eden A, Van Engelenburg MD, Buningh S (2015) How feedback boosts motivation and play in a brain-training game. Comput Human Behav 48:94\u2013103. https:\/\/doi.org\/10.1016\/j.chb.2015.01.038","journal-title":"Comput Human Behav"},{"key":"1185_CR8","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2021.730165","author":"T Caetano","year":"2021","unstructured":"Caetano T, Pinho MS, Ramadas E et al (2021) Cognitive training effectiveness on memory, executive functioning, and processing speed in individuals with substance use disorders: a systematic review. Front Psychol. https:\/\/doi.org\/10.3389\/fpsyg.2021.730165","journal-title":"Front Psychol"},{"key":"1185_CR9","doi-asserted-by":"publisher","DOI":"10.1016\/j.ctim.2020.102480","author":"B Cie\u015blik","year":"2020","unstructured":"Cie\u015blik B, Mazurek J, Rutkowski S et al (2020) Virtual reality in psychiatric disorders: a systematic review of reviews. Complem Ther Med. https:\/\/doi.org\/10.1016\/j.ctim.2020.102480","journal-title":"Complem Ther Med"},{"key":"1185_CR10","doi-asserted-by":"publisher","first-page":"987","DOI":"10.1016\/S2215-0366(18)30337-7","volume":"5","author":"L Degenhardt","year":"2018","unstructured":"Degenhardt L, Charlson F, Ferrari A et al (2018) The global burden of disease attributable to alcohol and drug use in 195 countries and territories, 1990\u20132016: a systematic analysis for the global burden of disease study 2016. Lancet Psychiatry 5:987\u20131012. https:\/\/doi.org\/10.1016\/S2215-0366(18)30337-7","journal-title":"Lancet Psychiatry"},{"key":"1185_CR11","doi-asserted-by":"publisher","first-page":"93","DOI":"10.3414\/ME14-02-0021","volume":"55","author":"P Gamito","year":"2016","unstructured":"Gamito P, Oliveira J, Brito R et al (2016) Evaluation of cognitive functions through the systemic lisbon battery: normative data. Methods Inf Med 55:93\u201397. https:\/\/doi.org\/10.3414\/ME14-02-0021","journal-title":"Methods Inf Med"},{"key":"1185_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2196\/18482","volume":"23","author":"P Gamito","year":"2021","unstructured":"Gamito P, Oliveira J, Matias M et al (2021) Virtual reality cognitive training among individuals with alcohol use disorder undergoing residential treatment: pilot randomized controlled trial. J Med Internet Res 23:1\u201310. https:\/\/doi.org\/10.2196\/18482","journal-title":"J Med Internet Res"},{"key":"1185_CR13","doi-asserted-by":"publisher","DOI":"10.1016\/j.arr.2022.101671","author":"HM Gavelin","year":"2022","unstructured":"Gavelin HM, Domell\u00f6f ME, Leung I et al (2022) Computerized cognitive training in Parkinson\u2019s disease: a systematic review and meta-analysis. Ageing Res Rev. https:\/\/doi.org\/10.1016\/j.arr.2022.101671","journal-title":"Ageing Res Rev"},{"key":"1185_CR14","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2021.543586","author":"A Ghi\u0163\u0103","year":"2021","unstructured":"Ghi\u0163\u0103 A, Hern\u00e1ndez-Serrano O, Fern\u00e1ndez-Ruiz J et al (2021) Attentional bias, alcohol craving, and anxiety implications of the virtual reality cue-exposure therapy in severe alcohol use disorder: a case report. Front Psychol. https:\/\/doi.org\/10.3389\/fpsyg.2021.543586","journal-title":"Front Psychol"},{"key":"1185_CR15","doi-asserted-by":"publisher","DOI":"10.3389\/fnhum.2016.00537","author":"H Hallock","year":"2016","unstructured":"Hallock H, Collins D, Lampit A et al (2016) Cognitive training for post-acute traumatic brain injury: a systematic review and meta-analysis. Front Hum Neurosci. https:\/\/doi.org\/10.3389\/fnhum.2016.00537","journal-title":"Front Hum Neurosci"},{"key":"1185_CR16","doi-asserted-by":"publisher","first-page":"1008","DOI":"10.1046\/j.1365-2648.2000.t01-1-01567.x","volume":"32","author":"F Hasson","year":"2000","unstructured":"Hasson F, Keeney S, McKenna H (2000) Research guidelines for the Delphi survey technique. J Adv Nurs 32:1008\u20131015. https:\/\/doi.org\/10.1046\/j.1365-2648.2000.t01-1-01567.x","journal-title":"J Adv Nurs"},{"key":"1185_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fnhum.2014.00844","volume":"8","author":"A Hone-Blanchet","year":"2014","unstructured":"Hone-Blanchet A, Wensing T, Fecteau S (2014) The use of virtual reality in craving assessment and cue-exposure therapy in substance use disorders. Front Hum Neurosci 8:1\u201315. https:\/\/doi.org\/10.3389\/fnhum.2014.00844","journal-title":"Front Hum Neurosci"},{"key":"1185_CR18","doi-asserted-by":"crossref","unstructured":"Keeney S, Hasson F, Mckenna H (2010) The Delphi technique in nursing and health research","DOI":"10.1002\/9781444392029"},{"key":"1185_CR19","doi-asserted-by":"publisher","first-page":"760","DOI":"10.1016\/S2215-0366(16)00104-8","volume":"3","author":"GF Koob","year":"2016","unstructured":"Koob GF, Volkow ND (2016) Neurobiology of addiction: a neurocircuitry analysis. Lancet Psychiatry 3:760\u2013773. https:\/\/doi.org\/10.1016\/S2215-0366(16)00104-8","journal-title":"Lancet Psychiatry"},{"key":"1185_CR20","doi-asserted-by":"publisher","first-page":"1843","DOI":"10.1212\/WNL.0000000000002145","volume":"85","author":"IHK Leung","year":"2015","unstructured":"Leung IHK, Walton CC, Hallock H et al (2015) Cognitive training in Parkinson disease: a systematic review and meta-analysis. Neurology 85:1843\u20131851. https:\/\/doi.org\/10.1212\/WNL.0000000000002145","journal-title":"Neurology"},{"key":"1185_CR21","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1016\/j.learninstruc.2017.12.007","volume":"60","author":"G Makransky","year":"2019","unstructured":"Makransky G, Terkildsen TS, Mayer RE (2019) Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learn Instr 60:225\u2013236. https:\/\/doi.org\/10.1016\/j.learninstruc.2017.12.007","journal-title":"Learn Instr"},{"key":"1185_CR22","doi-asserted-by":"publisher","first-page":"719","DOI":"10.1037\/edu0000473","volume":"113","author":"G Makransky","year":"2021","unstructured":"Makransky G, Andreasen NK, Baceviciute S, Mayer RE (2021) Immersive virtual reality increases liking but not learning with a science simulation and generative learning strategies promote learning in immersive virtual reality. J Educ Psychol 113:719\u2013735. https:\/\/doi.org\/10.1037\/edu0000473","journal-title":"J Educ Psychol"},{"key":"1185_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/09602011.2018.1468271","volume":"8","author":"DWK Man","year":"2018","unstructured":"Man DWK (2018) Virtual reality-based cognitive training for drug abusers: a randomised controlled trial. Neuropsychol Rehabil 8:1\u201318. https:\/\/doi.org\/10.1080\/09602011.2018.1468271","journal-title":"Neuropsychol Rehabil"},{"key":"1185_CR24","doi-asserted-by":"publisher","first-page":"207","DOI":"10.3233\/RNN-170754","volume":"36","author":"N Mendoza Laiz","year":"2018","unstructured":"Mendoza Laiz N, Del Valle DS, Rioja Collado N et al (2018) Potential benefits of a cognitive training program in mild cognitive impairment (MCI). Restor Neurol Neurosci 36:207\u2013213. https:\/\/doi.org\/10.3233\/RNN-170754","journal-title":"Restor Neurol Neurosci"},{"key":"1185_CR25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fpubh.2020.00457","volume":"8","author":"M Niederberger","year":"2020","unstructured":"Niederberger M, Spranger J (2020) Delphi technique in health sciences: a map. Front Public Heal 8:1\u201310. https:\/\/doi.org\/10.3389\/fpubh.2020.00457","journal-title":"Front Public Heal"},{"key":"1185_CR26","doi-asserted-by":"publisher","DOI":"10.1016\/j.drugalcdep.2022.109319","volume":"232","author":"MA Parvaz","year":"2022","unstructured":"Parvaz MA, Rabin RA, Adams F et al (2022) Structural and functional brain recovery in individuals with substance use disorders during abstinence: a review of longitudinal neuroimaging studies. Drug Alcohol Depend 232:109319. https:\/\/doi.org\/10.1016\/j.drugalcdep.2022.109319","journal-title":"Drug Alcohol Depend"},{"key":"1185_CR27","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2020.01008","author":"C Peifer","year":"2020","unstructured":"Peifer C, Sch\u00f6nfeld P, Wolters G et al (2020) Well done! Effects of positive feedback on perceived self-efficacy, flow and performance in a mental arithmetic task. Front Psychol. https:\/\/doi.org\/10.3389\/fpsyg.2020.01008","journal-title":"Front Psychol"},{"key":"1185_CR28","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1159\/000323950","volume":"36","author":"C Peretz","year":"2011","unstructured":"Peretz C, Korczyn AD, Shatil E, Aharonson V, Birnboim S, Giladi N (2011) Computer-based, personalized cognitive training versus classical computer games: a randomized double-blind prospective trial of cognitive stimulation. Neuroepidemiology 36:91\u201399","journal-title":"Neuroepidemiology"},{"key":"1185_CR29","unstructured":"Ritchie H, Roser M (2022a) Alcohol consumption. In: Our World Data. https:\/\/ourworldindata.org\/alcohol-consumption. Accessed 06 September 2024"},{"key":"1185_CR30","unstructured":"Ritchie H, Roser M (2022b) Opioids, cocaine, cannabis, and illicit drugs. In: Our World Data. https:\/\/ourworldindata.org\/illicit-drug-use. Accessed 06 September 2024"},{"key":"1185_CR31","unstructured":"Roser M, Ritchie H, Spooner F (2021) Burden of disease. In: Our World Data. https:\/\/ourworldindata.org\/burden-of-disease. Accessed 20 May 2022"},{"issue":"1","key":"1185_CR32","first-page":"1","volume":"31","author":"DP Rowland","year":"2022","unstructured":"Rowland DP, Casey LM, Ganapathy A, Cassimatis M, Clough BA (2022) A decade in review: a systematic review of virtual reality interventions for emotional disorders. Psychosoc Interv 31(1):1\u201320","journal-title":"Psychosoc Interv"},{"key":"1185_CR33","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.tics.2018.10.004","volume":"23","author":"G Sala","year":"2019","unstructured":"Sala G, Gobet F (2019) Cognitive training does not enhance general cognition. Trends Cogn Sci 23:9\u201320. https:\/\/doi.org\/10.1016\/j.tics.2018.10.004","journal-title":"Trends Cogn Sci"},{"key":"1185_CR34","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2019.01409","author":"T Segawa","year":"2020","unstructured":"Segawa T, Baudry T, Bourla A et al (2020) Virtual reality (VR) in assessment and treatment of addictive disorders: a systematic review. Front Neurosci. https:\/\/doi.org\/10.3389\/fnins.2019.01409","journal-title":"Front Neurosci"},{"key":"1185_CR35","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1016\/S0377-2217(01)00113-8","volume":"136","author":"J Shanteau","year":"2002","unstructured":"Shanteau J, Weiss DJ, Thomas RP et al (2002) Performance-based assessment of expertise: how to decide if someone is an expert or not. Eur J Oper Res 136:253\u2013263. https:\/\/doi.org\/10.1016\/S0377-2217(01)00113-8","journal-title":"Eur J Oper Res"},{"key":"1185_CR36","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1177\/030802269806100403","volume":"61","author":"T Sumsion","year":"1998","unstructured":"Sumsion T (1998) The Delphi technique: an adaptive research tool. Br J Occup Ther 61:153\u2013156","journal-title":"Br J Occup Ther"},{"key":"1185_CR37","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1111\/ajad.13342","volume":"32","author":"D Taubin","year":"2023","unstructured":"Taubin D, Berger A, Greenwald D et al (2023) A systematic review of virtual reality therapies for substance use disorders: Impact on secondary treatment outcomes. Am J Addict 32:13\u201323. https:\/\/doi.org\/10.1111\/ajad.13342","journal-title":"Am J Addict"},{"key":"1185_CR38","doi-asserted-by":"publisher","first-page":"585","DOI":"10.23736\/S1973-9087.18.05548-X","volume":"55","author":"C Vander Linden","year":"2019","unstructured":"Vander Linden C, Verhelst H, Deschepper E et al (2019) Cognitive training benefit depends on brain injury location in adolescents with traumatic brain injury: a pilot study. Eur J Phys Rehabil Med 55:585\u2013594. https:\/\/doi.org\/10.23736\/S1973-9087.18.05548-X","journal-title":"Eur J Phys Rehabil Med"},{"key":"1185_CR39","doi-asserted-by":"publisher","first-page":"15037","DOI":"10.1073\/pnas.1010654108","volume":"108","author":"ND Volkow","year":"2011","unstructured":"Volkow ND, Wang GJ, Fowler JS et al (2011) Addiction: Beyond dopamine reward circuitry. Proc Natl Acad Sci U S A 108:15037\u201315042. https:\/\/doi.org\/10.1073\/pnas.1010654108","journal-title":"Proc Natl Acad Sci U S A"},{"key":"1185_CR40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1001\/jamanetworkopen.2022.0970","volume":"5","author":"J Wei","year":"2022","unstructured":"Wei J, Hou J, Mu T et al (2022) Evaluation of computerized cognitive training and cognitive and daily function in patients living with HIV: a meta-analysis. JAMA Netw Open 5:1\u201317. https:\/\/doi.org\/10.1001\/jamanetworkopen.2022.0970","journal-title":"JAMA Netw Open"},{"key":"1185_CR41","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2018-027062","author":"H Zhang","year":"2019","unstructured":"Zhang H, Huntley J, Bhome R et al (2019) Effect of computerised cognitive training on cognitive outcomes in mild cognitive impairment: a systematic review and meta-analysis. BMJ Open. https:\/\/doi.org\/10.1136\/bmjopen-2018-027062","journal-title":"BMJ Open"}],"container-title":["Virtual Reality"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10055-025-01185-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10055-025-01185-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10055-025-01185-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,22]],"date-time":"2025-10-22T07:57:06Z","timestamp":1761119826000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10055-025-01185-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,8]]},"references-count":41,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["1185"],"URL":"https:\/\/doi.org\/10.1007\/s10055-025-01185-2","relation":{},"ISSN":["1434-9957"],"issn-type":[{"value":"1434-9957","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,8]]},"assertion":[{"value":"16 January 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 June 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 July 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"105"}}